Syncing Files Between PC’s and Mac’s

I don’t know how I missed this, but Microsoft has a really great product/service that allows you to synchronize files between multiple computers (PC’s and Mac’s) across the Internet (thanks Mike for pointing this out). In addition the product also allows you to share files with your friends. This communication happens in a private peer-2-peer fashion and is encrypted. The product is called FolderShare, and Microsoft purchased late last year and is offering the service for free.

Quoting from their site you can:

  • Synchronize all your devices – Retrieve work files at home or access photos at work. With your devices in sync, you no longer have to be frustrated that your information is on another computer.
  • Share files, photos, and home videos with your peers – Select the content you want to share, invite members, and they will be able to access the shared files directly from their device.
  • Access your computer or device remotely – FolderShare mobile access allows you to access your computer from any web browser.

The service works as advertised and is something that I plan to share with my Dad and Father-in-Law as a way for them to backup their information between their machines (they both have desktops and laptops) as well as provide them a way to backup to my house as well (if they want).

Initially I had thought that it would be perfect, easy to use solution to my off-site storage problem related to backup. While it works great, it does a have a few limitations that make it unsuitable for use in my particular situation.

The three big limiting factors are that it only supports files up to 2 GB in size (not that big of deal if you take that into account and split your big files), but the real killer for me is a limitation of 10,000 files per “library”. Some of my directories have over 25,000 files (think all the pictures my wife takes). I started thinking about workarounds, but eventually nixed those ideas as just too complex. The third issue is that it runs as a user level application. So that means that you have to be logged in and have the application running in order to it to sync.

This is a great tool for your personal and/or small business use. It is really easy to use, its free (at least for now), and it fills a niche in most people’s backup strategy by providing off-site backups, as well as access to your files (particularly if you are a laptop user), and finally it is cross platform Mac OSX and Windows.

I Want to Backup Both My Mac and PC

So I recently purchased two cheap 250 GB external USB hard drives. I plan to use them solely for backing up both my Mac and Windows PC. I plan to keep one unit at the house and the other off-site. Either at my office or maybe ship it to my Dad’s for safe keeping.

I’ve been looking around for the best ways to backup both systems to the same hard drive. My problem is that I want to the solution to be simple and robust. Ideally, I’d like to be able to plug the hard drive into either computer and back the other up. I’d like to be able to see the files from both OS’s at the same time. I’d like to do daily, weekly, monthly backups with incremental and compression. It would be nice to also mirror the boxes so that I have a bootable device as well.

I know that I won’t get all that with just one solution (and basically 1 drive). I’m going to try a few things and see what works so this post is basically just to document some initial finding to see what I may want to try.

I found the following on the forums at OSXFAQ:

Under Panther, the command line diskutil tool has an option entitled “MBRFormat” for its “partition” verb, which writes the MBR in DOS/Windows format.

In other words, if you have a disk which is accessible through the device node /dev/disk1, and you wish to partition it into two 80GB partitions – one FAT32 and one HFS+J, you do it as follows:

Code:
diskutil partitionDisk /dev/disk1 2 MBRFormat MS-DOS DosDrive 80G "Journaled HFS+" MacDrive 80G

This would allow my to plug the drive into either type of computer and copy the data over using some method. On the Mac I’d be able to see both file system types, but without some commercial software like MacDrive I would only be able to see the FAT32 partition on the PC. I could leave the drive attached to the Mac and copy the files over the network from the PC. I’d then be able to restore directly if I needed to. I’d need some software on the Mac to make this work.

One really neat solution (that is probably overkill for me) is called BackupPC.

BackupPC is a high-performance, enterprise-grade system for backing up Linux and WinXX PCs and laptops to a server’s disk. BackupPC is highly configurable and easy to install and maintain.

Given the ever decreasing cost of disks and raid systems, it is now practical and cost effective to backup a large number of machines onto a server’s local disk or network storage. This is what BackupPC does. For some sites, this might be the complete backup solution …

BackupPC is written in Perl and extracts backup data via SMB using Samba, tar over ssh/rsh/nfs, or rsync. It is robust, reliable, well documented and freely available as Open Source on SourceForge …

BackupPC Features:

  • A clever pooling scheme minimizes disk storage and disk I/O. Identical files across multiple backups of the same or different PCs are stored only once resulting in substantial savings in disk storage and disk I/O.
  • One example of disk use: 95 laptops with each full backup averaging 3.6GB each, and each incremental averaging about 0.3GB. Storing three weekly full backups and six incremental backups per laptop is around 1200GB of raw data, but because of pooling and compression only 150GB is needed.
  • Optional compression support further reducing disk storage. Since only new files (not already pooled) need to be compressed, there is only a modest impact on CPU time…

I’ve also looked at

  • rdiff-backup which uses rsync like methods plus incremental backups (even of binary files) but they don’t have good windows support yet
  • SuperDuper which is a free/shareware Mac application that will make a mirrored bootable drive as well as has other backup modes.
  • An article about HOWTO: Backup Your Mac With rsync which I’ve used before
  • Dirvish which is a set of scripts for rsync based backups
  • A script for copying opened files on Windows XP and 2003 Server (it uses VSS)
  • An article from LifeHacker about how to backup your PC (with a software recommendation).

I will probably start with just partitioning my disk into two sections and doing a basic copy to get things going and experiment from there. Once I finally get a solution that I like I’ll be sure and update everyone, and if you have a particular solution that you like, please leave me a comment.

51 Hours in 3 Days

51 hours, that’s the amount to time I spent at work the first three days of this year. Tuesday morning started out normal enough; I got up and drove to work. The commute was a little more difficult than before the holiday but nothing too bad. I got the morning update from my staff, which was slightly diminished due to a couple of guys being out sick. We had a brief staff meeting at 10:00 AM, then at about 10:30 the proverbial *#@& hit the fan.

We had a major server crash. The crash was similar to one we experienced back in September. It involved our second Dell PowerVault 220. We are not sure why the array initially went offline (Dell tech support want to blame a cable, but that is really just a bunch of hogwash), but we subsequently find three drives with media errors so they ship us three new ones. (After the fact we determine that we are one revision behind in firmware on the array, guess what that update fixes. It corrects problems with the array timing out under heavy loads, just what we are experiencing.) It tried to fail over to the hot spare, but the server crashed completely. Oh great, we’ve been here before. We try to bring it up and do a check disk. That fails (just as we expected). So our only recourse it to rebuild the array and restore from backup.

Given our problems last September, we knew that this was not going to be easy. And it wasn’t. Once we got the array rebuilt (and 3 new disks Dell), we started the restore process. We use a wonderful backup product from Symantec (formally Veritas) BackupExec it works great. All they need to now is release a RestoreExec product to go along with it.

The restore process was horrible. Even though the backups were good, the BackupExec Remote Agent would crash periodically. At times it would go for 2 hours and at others it would go for 5 minutes. So even though we’ve never had good luck we tried to call tech support. After a six hour phone call to India where we talked to Dale or was it Devon (like that was really his name), we decided that their ideas were complete hogwash and they had no clue what the problem was. At this point we just plunged ahead with the restore. We turned on detailed logging and whenever it crashed, we would skip the offending file and start it up again.

The only good thing to come out of all of this was that the firm decided to spend the money to bring in a storage specialist to analyze our entire storage and backup systems as well as our data workflow. Needless to say it was not the kind of start I wanted for the New Year. I crashed really hard when I finally got home on Friday night. I still felt really lousy on Saturday, kind of like a bad hangover with the nausea. We’re still dealing with minor repercussions this week as the users encounter corrupted files.