Microsoft builds a better Robocopy
Jun0
Better bloggers than I have already climbed the digital soapbox and touted the usefulness of Microsoft’s Robocopy command-line tool, so I won’t waste a lot of time going into detail how great it is at copying, syncing, or moving files and directories in Windows. And if you manage storage of any kind, you already know how difficult it is to make file migration appear seamless to the user community. You also already have your favorite go-to utility to pull these tasks off.
But if you’re still researching, or if you’re on the fence, or if command line tools aren’t really the first thing you reach for; I would like to take a moment to highlight two new improvements MS made to Robocopy recently which have been INCREDIBLY useful to me:
#1: Copying Directory Timestamps
Prior to Vista and Windows 2008, Robocopy could only copy file timestamps and not directory timestamps. Instead it used the current copy date and time when creating new directories. This was incredibly confusing to users who organize their life by such dates, and also frustrating as a storage admin to see all these old directories suddenly have a new lease on life when we know NO ONE has accessed these directories in YEARS. With Vista and Windows 2008, the version of Robocopy has been updated to include the /DCOPY:T switch, which copies directory timestamps.
#2: Perform Multithreaded File Copies
Multithreading is the ability to execute different parts of a program, or threads, simultaneously. The program must work in such a way that all the threads can run at the same time without interfering with each other. In the case of Robocopy; the goal of this would be faster copy times if you are running multiple threads. In the past there have been several workarounds made through scripting to get Robocopy to perform like this. But with the advent of Windows 7 and Windows 2008 R2, Robocopy now features a new multi-threaded copy option. To perform a multi-threaded copy, you use the /MT[:n] switch, where n indicates the number of threads to be used. By default, n is 8 when you use the /MT option, and can be any value from 1 to 128.
My own favorite Robocopy syntax? Here’s one I use on a regular basis:
Robocopy.exe /R:0 /W:0 “\pathtosourcedirectory” “\pathtotargetdirectory” /E /ZB /COPYALL /DCOPY:T /PURGE /MT:10 /Mot:60 /RH:1700-2359 /V /LOG+:”C:pathtologs.txt” /TEE
ThisĀ single command will:
- Copy all file and directory information, including directory time stamps.
- Copy subdirectories including empty directories.
- Use Restart mode. (If access is denied, this option uses Backup mode.)
- Delete destination files and directories that no longer exist in the source.
- Use multithreaded option (10 threads).
- Monitor the source directory, and runs again in 60 minutes if changes are detected.
- Specify that copy job will run from 17:00-23:59. (Job will be in paused state outside of those times.)
- Produce verbose logs, and shows all skipped files.
- Write the status output to a log file (appends the output to the existing log file).
- Also write the status output to the console window.
For more information on Robocopy visit Microsoft’s Technet site.
Backing up your SAN Switches
May4
Thought I’d share a tool I use to back up my SAN switches. It’s called save-simply-your-san. And although it could probably use a jazzier name, it does what it does very simply and cleanly.
It’s a command-line program released on GNU GPL v2 that connects to your switches via SSH. It works with Cisco, Brocade and McData switches.
Example batch file:
SaveSimplyyourSAN -i 192.168.0.1 -s brocade -u admin -p password -t ftp >brocade-backup.log 2>&1
SaveSimplyyourSAN -i 192.168.0.1 -s cisco -u admin -p password >cisco-backup.log 2>&1
SaveSimplyyourSAN -i 192.168.0.1 -s mcdata -u admin -p password >mcdata-backup.log 2>&1
SaveSimplyyourSAN -i 192.168.0.1 -s mcdata -u admin -p password -c telnet -e “show all”>>mcdata-backup.log 2>&1
I scripted all my switches into batch files and schedule them via Windows Scheduled Tasks. It’s a big help.
Wading into the Perl Scripting Pool
Mar0
This week I wrote my first Perl script.

Quick background – I am working on a project which requires a script to do the following:
- to delete files based on predetermined age of file
- to delete files in multiple locations
- to log the files it has deleted
- has to be able to be scheduled
Pretty simple requirements. Of course my own additional requirement would be – has to run on Linux (no Microsoft fees for this project).
I immediately thought to do this in Perl, because (a) Perl was basically written for file manipulation, and (b) it’s the all purpose programming language.
So my goals are pretty simple. And (hopefully) here is my script (version 1):
#!/usr/bin/perl ############################################################### # This is a script to delete all files that have file creation # dates older than X days from 2 directories. Script will output # separate text files logging which files have been deleted # along with date/time stamp. Logs will be appended and kept. ############################################################### $dir1 = "/path/to/first/directory"; $age1 = X; # age in days $dir2 = "/path/to/second/directory"; $age2 = X; # age in days my $log_file1="/path/to/first/log/log1.txt"; my $log_file2="/path/to/second/log/log2.txt"; my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday, $isdst)=localtime(time); my $date = sprintf ('%4d-%02d-%02d %02d:%02d:%02d', $year+1900,$mon+1,$mday,$hour,$min,$sec); ############################################################### # Begin script .... ############################################################### opendir FH, "$dir1" || die $!; my @filenames = readdir(FH);closedir FH; foreach my $file(@filenames){ if(-e "$dir1/$file"){ if (-M "$dir1/$file" > $age1) { next if ($file =~ m/^\./); open OUTPUT, (">>$log_file1") || die "Couldn't open log, $!"; unlink("$dir1/$file"); print OUTPUT "$file - deleted on $date.\n"; close OUTPUT || die "Couldn't close log, $!";} } } ############################################################### opendir FH, "$dir2" || die $!; my @filenames = readdir(FH);closedir FH; foreach my $file(@filenames){ if(-e "$dir2/$file"){ if (-M "$dir2/$file" > $age2) { next if ($file =~ m/^\./); open OUTPUT, (">>$log_file2") || die "Couldn't open log, $!"; unlink("$dir2/$file"); print OUTPUT "$file - deleted on $date.\n"; close OUTPUT || die "Couldn't close log, $!"} } } ############################################################### # Finish script .... ############################################################### print "Script Completed!\n";
And it works!
I have been testing on my workstation running Centos 5. I mount two CIFS shares which are hosted on NetApp Filers (one on the East Coast, one on the West Coast). It outputs the 2 text files which log all the deleted files.
I have yet to show this script to the Unix folks. And if they don’t laugh at me, I should be able to use in production. (I should be able to schedule this with cron when I roll this out into production. Also plan to use fstab to automate the CIFS mounts.)
Some future goals I have for this are:
- Output to html so Sys Ops can view the logs through browser instead of text file
- Email text files to Sys Ops so that each time script is run they receive an email