How to copy files around without anyone seeing them

How to copy files around without anyone seeing them

In a previous article, I showed you how
to backup your mySQL database from one box to another.  In this example I used ftp.
  I now know of a more secure method which is suitable for use across untrusted
networks, particularly the internet.

See also ssh – much more secure
than telnet for a few more ways of copying files files around

How’s it all done then?

It’s all done with ssh (or more correctly,
with OpenSSH).  When I first tried this solution
with ssh, I couldn’t find a way to connect to the other box from within a script.
  I could see no way to securely supply the password.  So I gave up.  Then
I posted a message to the freebsd questions mailing list and I found my answer.

If you
read man ssh, you’ll find a section which talks about RSA based authentication.  This
allows one box to authenticate itself without having to supply a password.  Which is
exactly what is needed in this situation.

What you’ll need first

First, you’ll need a login on both machines.  And both machines will
need to be running ssh (my preference is the OpenSSH implementation of ssh).  I suggest
you connect to both machines now, via ssh of course, and then continue with the
rest of the article.

I’ll refer to one machine as the source machine.  That’s the
box from which you wish to transfer files.  I’ll refer to the other machine as the
destination machine, the box to which you wish to transfer files.

It’s all about keys

WARNING: This section recommends using an empty passphrase which is is risky.
If anyone obtains your private key, they will
be able to login to any machine on which your public key is an authorized_key.

The non-password authentication is done with keys.  And it’s done like this.

• Run ssh-keygen on the source machine.   Accept the default values for all
  questions.  Be sure to supply an empty passphrase.  This will create two files
  under .ssh in your home directory.   These files are ~/.ssh/identity
  and ~/.ssh/identity.pub.   The first file should not be revealed to anyone.
    The second file is public and contains your public key.
• Copy the contents of ~/.ssh/identity.pub to the destination machine and place
  it within ~/.ssh/authorized_keys.  Be sure to name the file correctly.

---

WARNING: There has been an SSH1 exploit and you should
be using ssh2/DSA or ssh2/rsa keys. Such keys go into ~/.ssh/authorized_keys2 but are
generated in a similar way. See about the exploit to learn more.

---

If you are doing a copy.paste with the public key, remember that authorized_keys
contains only one key per line, although this line may be very long.  You should now
be able to connect from the source box to the destination box without a password.  
Like this:

ssh user@destination.box

If that doesn’t work, then something is wrong.  check the above steps and try
again.

You should also read ssh – authorized keys and
chmod to see how I later broke this solution by changing directory permissions.

How does this magic work?

ssh-keygen create two keys, one public, one private.  When
you connect to the remote box, the ssh server on that box sends your ssh
program a challenge in the form of a random number.  This random number challenge is
encrypted with the public key you placed on the destination box.   The challenge
can only be decrypted by the private key, which is on the source box.  The ssh
program decrypts this number and tells the server the answer.  In this method, the
client tells the server that it knows the private key.  It is by this method that one
box proves to another box it is who it says it is.

The backup script

I took the original backup script I created for mySQL and modified it to
use ssh.  Here is the amended script.  You can also obtain this script from xxx.

#!/bin/sh

#
# mysql databse backup
# Copyright 1999, 2000 DVL Software Limited
#
# Available from 
# http://www.freebsddiary.org/samples/dns_fetch.sh
#

#
# the name of the backup file. file name format is 
# backup.2000.01.12.at.22.59.48.tgz
#
BackupFile="forum.backup.`date +%Y.%m.%d.at.%H.%M.%S`.tgz"

#
# dump the database.
# make the following replacements:
#
#     userid     - the user id to use when connecting 
#                  to the database
#     password   - the password for the above user
#     database   - the name of database to dump
#     /pathto/   - the path to the backup file
#
/usr/local/bin/mysqldump -uuserid -ppassword -c 
                                  --add-drop-table database 
                                  > /pathto/forum_backup.txt

#
# compress it
#
tar cfz $BackupFile /pathto/forum_backup.txt

#
# copy it offsite, change user and othersite.org accordingly.
#
scp $BackupFile user@othersite.org:$BackupFile

#
# remove the files we created
#
rm $BackupFile forum_backup.txt

Additions to the above

There are a few nice additions to the above script which work rather
nicely.  I also use this script to backup various directories, but exclude others.
  The additions to do that look like this:

tar cfz $BackupFile                        \
       -X exclude.txt                      \
       /home/freebsddiary/forum_backup.txt \
       /home/freebsddiary/www/*.php3       \
       /home/freebsddiary/www/phorum       \
       /home/freebsddiary/www/phpPolls

As you can see, I backup all the php3 files, and everything in the phorum and phpPolls
directory.  But I also exclude everything specified in the exclude.txt file.  
Here’s what that file contains:

$ more exclude.txt
*/_vti_cnf/*

You can put whatever you want.  In this case, no directories named _vti_cnf will
be included in the backup.

Doing it all from a cron job

This should work flawlessley.  The only thing needed now is a cron
job to start off the above.  Here is what I use:

$ more ~/crontab
#/home/freebsddiary/crontab - dan's crontab for FreeBSDDiary
#
#
SHELL=/bin/sh
#PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
#HOME=/var/log
# mail any output to `dan', no matter whose crontab this is
MAILTO=dan@example.org
#
#minute hour    mday    month   wday    command
#
#
0       5       *       *       *       $HOME/dump_database.sh

This will run the patch job at 5am every day.  Adjust the values as appropriate to
your need.  See man 5 crontab for some very good examples.

The above can be added to the cron jobs by doing this:

crontab ~/crontab

That’s everything!

Michael O Shea wrote in to mention that using rsync over SSH would be
faster as it transfers only changed files.  That’s a very good idea if tranferring
the same group of files on a regular basis.  See rsync –
synchronizing two file trees for more information.

That should be everything. 
Please, if you do follow these instructions, and they work for you, please tell your
friends.  It if doesn’t work, and you can figure out what I’ve left out, please add your comments using the link at the top or bottom of this article.