Backing Up Data: Workflow/Storage

[Cajungaltx]

My work flow right now is:

I take photos
I upload photos to computer.
I burn copy of RAW files, store in sleeve, in manilla folder. I organize by date, then last name.
I use Light Room, go through and pick the 'keepers', edit the keepers. Burn full sized copy to JPEG, burn a 'keeper' CD. Store in manilla Folder.

Sounds nice and professional, right?

Except for the fact that I have a crapload of images on my computer....

So I started deleting everything when I finished a session. But when I want to go back and find 'that' photo that I remember... you know, the one I took in the summer last year, with the car and the kid? Come on.... dang it... pulls out date book, trying to find the date it was on....

Nightmare alley.

I currently have 4 HD's in my computer - 1.5TB, 2 - 750 GB and 60GB I think.

I am looking at setting up a Home Server for backing up data (just to be safe) however I'd really like something so that I could setup a way to access my 'keeper' images without having to go grab disks out of folders.

But at the same time - I don't want to have duplicate copies of 3TB's of data. Yes, I'd freak if I lost it, or a hard drive crashed...but I have my original RAW files saved.

Could SOMEONE with some years of experience in this tell me how they manage the massive data that comes with this addictive interest?

I'm looking at investing in this part of my hobby shortly, and need to get some good direction before making any purchases that are going to just screw the whole thing up.

[EyeBurn]

1. Stop using single external drives
...and
2. switch to either mirrored External drives (ie: Western Digital's 2TB mirror-edition...this is 1TB useable). approx $250
...or...
3. buy a personal NAS (network attached storage) ie: Buffalo TeraStation approx $2000
4. If you're a mac user, buy a timecapsule (1TB)...note: it's a single drive so I purchased two time capsules: 1 sits at home, the 2nd sits in my office...and my laptop (and images) sync to both devices. approx cost: $200 each?

option #2 is very affordable, once you get one...copy all of your master images to it and reassign those single external drives as something else
option #3 will have faster performance and better fault tolerance should one of the drives fail, costs more.
option #4 very easy to do, but if you setup two timecapsules you'd need to have permission and/or help to setup the 2nd one at an office. ** a bonus to using this setup is if you have a disaster at home (act of GOD or theft) you'll still have the other backup at your office. if you work from home, then the scenario doesn't work as well...or if you have IT ppl at the office that are not ok with you keeping a personal backup at the office.

let me know if you want more details...or maybe a less confusing info.

-Benjamin
PS. You can also consider an online service like: mozy, idrive, carbonite for backups.
PSS. The best solution I would recommend would be mirrored external drive + online backup service...or the dual timecapsules if you're a mac user.

[texfoto]

For the past 3 years or so I've been using iTunes and digital SLR's. Once I did that I unknowingly married myself to hard drive space. It's a revolving door of money and keeping up with technology from hard drives to new computers every few years. So, I feel your pain.

Ditto what was said by Eyeburn for the options available. I'll elaborate a bit.

You didn't mention what programs you use to help you search for pictures. What I have done is learned to use programs like Lightroom that will catalogue your pictures in a library. You can have multiple libraries and use key words to search for your pictures or look them up by date. For example, if you save one library a year, you can look through the 2007 library for that picture you know you took in 2007. You can put your "keepers" or 5 star rated photos in your most active library.

That is the application side. Hard drives are a must and they cost money. Just make sure you have something offsite. A good rule to live by for data backups is this; a single copy of a file means it doesn't exist. A photography class teacher told me that for his client pictures he has 37 one Terabyte hard drives backed up in a bank vault. That's RAW file size for you.

LONG TERM
Keep in mind you have to remember to transfer your files maybe every 5 to 10 years to keep up with technology too. Remember zip drives? Let me know if you can find one for your old Zip disk backups.

One thing people fail to realize about CD's and DVD backups is they may not be as long term as you think they are. If you value your photos or any files for that matter DO NOT rely on this media as a sole method for backups. There have been tests done which show how these forms of media are at times not lasting more than 3 or 5 years. We aren't talking about leaving them in the sun either. These were stored in cases in a dark file cabinet. This is due to the way CD and DVD re-writable's are made. The surfaces are made of an organic dye so you can burn files to them. Over time they degrade. There are different types of dye with different reliability rates but they can and do degrade over time.

Enough of my nerd lecture.

How do I do it? Keep in mind this is a hobby for me but I seem to have accumulated so many pictures that I'm having to cut them down like you as well.

- Lightroom on an external drive (I have a laptop only)
- Backup to 2 other Terabyte hard drives and take one off to a friends house and swap them out every once in a while when I get around to it.



mike

[Cajungaltx]

Originally Posted by texfoto

For the past 3 years or so I've been using iTunes and digital SLR's. Once I did that I unknowingly married myself to hard drive space. It's a revolving door of money and keeping up with technology from hard drives to new computers every few years. So, I feel your pain.

Ditto what was said by Eyeburn for the options available. I'll elaborate a bit.

You didn't mention what programs you use to help you search for pictures. What I have done is learned to use programs like Lightroom that will catalogue your pictures in a library. You can have multiple libraries and use key words to search for your pictures or look them up by date. For example, if you save one library a year, you can look through the 2007 library for that picture you know you took in 2007. You can put your "keepers" or 5 star rated photos in your most active library.

That is the application side. Hard drives are a must and they cost money. Just make sure you have something offsite. A good rule to live by for data backups is this; a single copy of a file means it doesn't exist. A photography class teacher told me that for his client pictures he has 37 one Terabyte hard drives backed up in a bank vault. That's RAW file size for you.

LONG TERM
Keep in mind you have to remember to transfer your files maybe every 5 to 10 years to keep up with technology too. Remember zip drives? Let me know if you can find one for your old Zip disk backups.

One thing people fail to realize about CD's and DVD backups is they may not be as long term as you think they are. If you value your photos or any files for that matter DO NOT rely on this media as a sole method for backups. There have been tests done which show how these forms of media are at times not lasting more than 3 or 5 years. We aren't talking about leaving them in the sun either. These were stored in cases in a dark file cabinet. This is due to the way CD and DVD re-writable's are made. The surfaces are made of an organic dye so you can burn files to them. Over time they degrade. There are different types of dye with different reliability rates but they can and do degrade over time.

Enough of my nerd lecture.

How do I do it? Keep in mind this is a hobby for me but I seem to have accumulated so many pictures that I'm having to cut them down like you as well.

- Lightroom on an external drive (I have a laptop only)
- Backup to 2 other Terabyte hard drives and take one off to a friends house and swap them out every once in a while when I get around to it.



mike

How about bluray - one of my friends works for HP and he suggested that.

[Aric C. Hoek]

I have a ton of images on my server as well, so I purchased a Drobo which backs them all up every hour on the hour.

[katiexjo]

Originally Posted by Aric C. Hoek

I have a ton of images on my server as well, so I purchased a Drobo which backs them all up every hour on the hour.

What's a Drobo? I have been using external hard drives and I am thinking of switching to an automatic online backup like Carbonite, does anyone have experience with this?

~KJ

[BEphoto]

I was going to suggest drobo as well. They really are awesome. I don't own one yet but plan to buy one in the near future. Www.drobo.com I think is the site.

[Peacock]

I know Photoshop Elements has a Move / Copy to removeable media, what it does is copy you originals / masters to a removeable harddrive or cd / dvd discs and leaves thumbnails of the images in you catalog so that you can still view low resolution files in your organizer. If you need the original image then you can just pull the disc and move the original back.

I would have thought for sure that Lightroom would have a feature as this also, being a big brother to Elements.

[LoungeLizard]

If you're going to look at Drobo, look at Microsoft's Windows Home Server software (available for about $90) and setup a regular tower desktop with as many HDDs as you can with this software.

I've documented several times what I use - and it came in handy over the Christmas holidays with a couple of HDD failures.

When working on a job, I keep all files on my internal HDD in my desktop and back it up to an external USB HDD. When the jobs are "completed", I archive all the files to two external HDDs (one stays at home, the other is off-site), and the JPEGs go on my Windows Home Server. Once that is all confirmed, the files on the internal HDD are removed.

At all times, I have two copies of my RAW files, album layouts, etc. I have three copies of my JPEGs.

[bolvangar]

Your cheapest option would probably be to setup a linux server with 4-5 1tb drives in it. You could share the drives over your network so you could easily copy and pull photos off the server and setup raid on the server to ensure if a drive fails you don't loose anything. If you want to really be safe you could do off site backup's to another server or even a SAN.

PM me if you want some more guidance. I can also help you with off site storage on a server or SAN if you want to go that route through where I work.

[Peacock]

You can do the same with FreeNAS, install it on a USB stick, on an older workstation, slap a bunch of drives in it! and you have storage, build a second on and you can backup your first to you second for redundancy!

http://www.freenas.org/

[gtakacs]

Originally Posted by Peacock

You can do the same with FreeNAS, install it on a USB stick, on an older workstation, slap a bunch of drives in it! and you have storage, build a second on and you can backup your first to you second for redundancy!

http://www.freenas.org/

If money is less of a factor and smaller form with quiet operation and less power consumption is more your thing a dedicated NAS will work great (which is essentially a linux box with an energy efficient mobo/cpu software RAID controller). You can get an open box QNAP-409 Pro for $369 at newegg and put two 1.5TB drives in it in mirror mode to get 1.5TB storage that is mirrored. Then when you run out of that you can add another 1.5TB drive, turn it into RAID5 and have 3TB total storage with one drive failsafe. Then when you run out of space again, just slap another 1.5TB in it for a total of 4.5TB storage in RAID5.

If network access is not high on your list, the drobo will do it via USB2/Firewire for faster access but more money and less convenience.

You should also get some kind of cataloging software to keep track of all the pictures, I like Imatch and ACDsee.

[toverman]

I still have a Zip drive! Just don't ask me where it is.

[911golfer]

I back up my raw files to the PC and then two external drives. The PC is hooked up to Carbonite. I then back-up to the iMac with time machine and a 1.5 TB Lacie drive for processed pics. After all that it goes onto a small portable hard drives into the fire safe. I gave up on DVD/CD due to failures and lack of space. Blue-Ray is not cost effective and may suffer from the same problems of DVD rot. I believe in on-site and off-site back-ups. Only takes one fire or one tornado to lose it all!

[Hogleg 44]

Besides all of my external HDD's, mine are backed up at Amazon!
At least that's where all of the stuff I have uploaded to my Phanfare.com photo sharing account is backed up. And that's almost everything worth anything to me that I have!
Smugmug too. They also have everything that Phanfare has, and I can retrieve it from either place, anywhere there is a hot spot, whenever I might need it.