Lawrence Berkeley National Laboratory

Berkeley Center for Structural Biology

Data Backup at 8.2.1 and 8.2.2

SCP
FTP
RSYNC
Firewire Drive (Linux)
Firewire Drive (PC)
Firewire Drive (Mac)
Manual Linux DVD Backup
DVD Backup (PC)



SCP   

SCP FROM THE ALS:

1. Data may be scp'ed from your user account by typing:
   scp -pr datadirectory username@hostname:/datadirectory
   where hostname is the name of your home institution computer.
   
   
2. EXAMPLE: scp -pr /data/dcsuser/arnold ArnoldS@berkeley.edu:/home/usraccounts

NOTE: The -r is for recursive, so that a directory and all its contents are copied,
while the -p preserves the original date on the file. At home, you can then
type ls -Flart *.img to list the image files in the order in which they were taken.)
 If you want to copy a single file or several files, use
scp -p filename username@hostname:/datadirectory (or *.img in place of the filename)

SCP FROM OFFSITE:

1. Log onto the download machine by typing
   
   ssh username@bcsb-download.als.lbl.gov or
   ssh username@bcsb-download-1.als.lbl.gov
   
   
2. You will be logged into your processing directory so you can scp processed data directly.
   
   
3. For raw images you must go to the directory that corresponds to the beamline at which you
   collected your data. For example, type cd /data/bl8222/dcsuser/yourdata
   
   
4. Once in the correct directory you can scp as indicated above

Note: group accounts such as stanford, ucberkeley, ucsf etc will not be able to log on
to the bcsb-download-1 machine; you must have an individual PI account.

Back to top    


FTP    

1. First tar your image files using the command tar -cvf name.tar directory where directory is
   the directory full of images or processed data that you want tarred
   
   
2. At the prompt, type ftp or sftp sitename (i.e., ftp home.berkeley.edu)
   
   
3. At the ftp prompt, type passive
   
   
4. At the ftp prompt, type prompt
   
   
5. At the ftp prompt, type bin
   
   
6. At the ftp prompt, type cd datadirectory (where directory is the directory at home where you want to store your files)
   
   
7. At the ftp prompt, type put name.tar
   
   
8. When done, type bye to end the ftp session
   
   
9. NOTE: The ftp timeout on your home machine may be set so that you can only send a few images at a time.

Back to top
    

RSYNC    

Go to the /data/dcsuser directory and type:

rsync -aPzv -e ssh dirname/ user@hostname:/finaldir

where dirname is your directory with collected images, on any of the control machines,
and user is your login name to the remote host with the hostname.

If a local copy is required then type:
rsync -avpP /source_directory /destination_directory

Back to top  


Firewire Drive (Linux)      

1. Plug in firewire or usb cable to the linux box. There are both usb and firewire cables available,
   and they are labeled with the computer name (eg, "bl821k4")
   
   
2. Power ON your firewire drive
   
   
3. At this point, you should be able to open windows corresponding to the firewire drive
   and the data drive, and drag and drop files, just like on a PC. If not, then:
   
   
4. First run: sudo /usr/local/bin/gofw script. This may find your drive. If not, continue on:
   
   
5. Linux system will always write to preformatted drives with EXT3 or FAT32/WIN95
   (but not NTFS). Run /usr/local/bin/fdisk -l to list what device is assigned to your firewire drive.
   It will usually be something like /dev/sdb or /dev/sdb1.
   
   
6. If your drive is not formatted and you want to be able to read your data in Windows OS,
   do the low level format: fdisk /dev/sdb or the high level format : /sbin/mkdosfs -v -F 32 /dev/sdb
   
   NOTE: When you run fdisk, you can use all the default options to format your drive as a linux drive.
   If you want to format it as FAT32, then select option b or c when asked which operating system to
   use. Select 1 partition. Use Win95 for drives with size >137Gb and Win95LBA for drives with size < 137 Gb.
   
   
7. Now type mkfs.ext3 /dev/sdb1.
   
   
8. Check the name of the drive by typing cat /etc/fstab. It might be called firewire1, for example.
   
   
9. If the drive does not now automatically appear on the desktop, then run mount /dev/sdb1 /firewire1.
   
   
10. You can see what's on the drive by typing cd /firwire1 and then ls.
    
    
11. You can drag and drop files, or copy data with rsync:
    rsync -auvr /data/dcsuser/myname/mydir /firewire1
    
    
12. Run umount /firewire
    
    
13. Power OFF your device before unplugging it.
    
    
    

Back to top    


Firewire Drive (PC)    

1. Log in to the PC as dcsuser.
   
   
2. Plug in firewire drive to windows PC at back. Plug in power supply (we have power
   supplies available for Maxtor firewire drives).
   
   
3. If the drive is not already formatted then format using NTFS ( Go to START -> settings ->
   control panel -> admin tools -> computer management, then choose disk management).
   
   
4. See the staff if you need to format a LaCie drive.
   
   
5. Right-click on My Computer on the desktop
   
   
6. Select Map Network Drive
   
   
7. For "Choose Drive Letter", select a drive letter that is not already in use
   
   
8. For "Choose Folder", either type in or select from the pull-down list, for example: \\bl821-ccd1.als.lbl.gov\data
   Note: substitute for the beamline you are on. For example, for beamline 5.0.2 use \\bl502-ccd1.als.lbl.gov\data
   
   
9. Click on Finish
   
   
10. Drag and drop data files to the firewire.
    
    
11. To access your processed data, Right-click on My Computer on the desktop
    and select Map Network Drive. Choose a drive letter that is not in use, and either
    type in or select from the pull-down list: \\bcsb-srv.als.lbl.gov\homes
    
    
12. Click on Connect Using a Different User, and enter your username and password.
    
    
13. Drag and drop data files to the firewire.    
    
    
    

Firewire Drive (MAC)       

1. Log in to the Mac using your username. (If you have an option, choose the BCSB domain,
   not LocalPC
   
   
2. Open 'finder' by clicking the mouse with the cursor on some clear space on the desktop.
   
   
3. use Finder-> Go -> Connect to server
   
   
4. Type in
   smb://bl822-ccd1.als.lbl.gov/data
   (substitute beamline you are on) or to connect to your home directories type:
   smb://bcsb-srv.als.lbl.gov/homes
   
   
5. Alternately, use Finder>Go>Connect To Folder> Data
   
   
6. You will then be prompted for user name and password (give 'dcsuser' name for
   data directories and your own name, for example 'taylor' for the home directories).
   Once you supply these, /data and /homes directory structure will appear on the Desktop.
   
   
7. You can then drag and drop data onto your firewire drive

Back to top    

Manual Linux DVD Backup    

1. Log on to the linux box using your account name
   
   
2. Put a DVD in the drive (the k3 and k4 machines can write 16x DVDs;
   the k2 machines can write 8x DVDs)
   
   
3. type writedvd datadirectory (remember to check the total space of your backup directory,
   which shouldn't exceed 4.5Gb)
   
   
4. EXAMPLE: writedvd /data/bl822/dcsuser/arnold
   
   
5. Once the DVD is done writing, you can look at what is on the DVD by first typing
   mount /media/cdrecorder(for bl822k3, bl822k4, and bl821k4machines
   or mount /mnt/cdrom (for bl822k2 and bl821k2 machines)
   or mount /media/cdrecorder1 (for bl821k3)
   
   NOTE: you can list the name of the dvd writer by typing cat /etc/fstab
   
   and then list what is on the DVD by typing ls /media/cdrecorder
   
   
6. Type cd to get back to your home directory.
   
   
7. Type eject to eject the DVD.

Back to top    


DVD Backup on PC    

1. Log on to the beamline PC
   
   
2. Double-click on the Easy CD Creator icon on the desktop
   
   
3. Select Make a Data CD
   
   
4. Select Data DVD
   
   
5. Insert a blank DVD into the drive
   
   
6. Drag and drop files, then click on Record
   
   
7. Note: to see the raw and processed data on the PC desktop, see Using Firewire Drives.