|
|

07-23-2006, 01:45 PM
|
|
Member
|
|
Join Date: May 2006
Location: South West of West Australia
Posts: 99
|
|
Managing and storing large files
Hi all
with my purchase of the FZ30 comes the move to large file sizes as I have decided to use the RAW technology. This means files of around 10 MB. Has anyone seen any good software for managing this?
With my 2GB SD card uploading the photos to my computer takes about 50 minutes and chews into the battery. Is there anything that can make the transfer faster?
Once the pictures are loaded storage becomes the issue. Can I zip them or something?
Finally I've never hit on a good way to name images. I take pictures of anything & everything. My file names range from dates, to subject, to mood. Has anyone found a good system that means they can think, what did I do with that picture of a Moaning Frog from the day I stubbed my toe?
Seriously I am such a know nothing.
Thanks
Katheric 
|

07-31-2006, 08:25 PM
|
|
Administrator
|
|
Join Date: Aug 2004
Posts: 176
|
|
Re: Managing and storing large files
Your post is several topics as I see it. Let me start with the "getting it to the computer". (We'll split this message thread if more people hop in and it develops with multiple topics)
You seem to be using the CAMERA itself as the "flash memory reader"... and noting this uses battery power. I'd recommend you get a SD (presumably) flash memory reader for your computer. This will let you remove the SD card from your camera (put another in so the camera is ready to pick up and go. Put a spare card in the camera NOW!!! [from experience: a camera without "film" isn't ready to go!] Too often I took the memory card out and left the camera empty and my wife picks up the camera to take it somewhere and use it... ) Then put the filled SD card into the card reader (many models, some read many formats from xD to MemoryStick to CF to SD/MMC, etc. Should cost between $10 and $30. Make sure you get a USBv2 model) and connect it to your computer via USB2 (and add a USBv2 card to your computer if you only have USBv1.1 currently). The limiting factor in the transfer will be the speed of the memory card itself now (since USBv2 runs at 480Mbps). And you won't be using the batteries in your camcorder (since the "film" flash memory is out of the camera and powered by the computer and flash memory reader.)
|

07-31-2006, 08:29 PM
|
|
Administrator
|
|
Join Date: Aug 2004
Posts: 176
|
|
Re: Managing and storing large files
photo format - there can be much discussion here!
Personally, I prefer "standard" formats, eg. JPEG. Yes, I know it is lossy, and not as "good" theoretically nor practically as a "RAW" file. But it is universal and that is very useful to me. I don't EDIT in JPEG but I keep finding myself saving final photos back to JPEG.
IF you need "the best your camera is capable of", then you will use RAW format (and use plugins for Photoshop if necessary, etc.) and you will benefit from no loss of visual detail as well as a larger colour space (the colour information is usually store in more bits so there's a higher resolution and more colour information). Alas, as you noticed, a far larger file size.
You can ZIP (or RAR or z7, etc.) the files, but this usually won't save much space. Here, I am referring to "compression" which is reversable and "loss-less" = you'd decompress the file and get exactly what you had originally. JPEGs are pretty well compressed already (and indeed lossily compressed to "intelligently throw away data that perceptive studies show isn't as noticable to human eyes/brain as other data, eg. colour info in dark areas of the photo") ZIPping a JPEG won't save you much. I'd expect RAW formats would be somewhat compressible but don't expect as much compression using ZIP as if you saved the photo as a JPEG (again because the JPEG compression outright deletes visual data judged insignificant).
A BitMap "BMP" file format on the other hand is very inefficient with no encoding or "shorthand" at all in the file.... and readily compress greatly using ZIP. But I doubt you are saving your files in BMP format.
|

07-31-2006, 08:44 PM
|
|
Administrator
|
|
Join Date: Aug 2004
Posts: 176
|
|
Re: Managing and storing large files
How to store -
1 - I make sure cameras have the right date and time set. This is usally embedded into the metadata for photos and can be useful.
2 - Although the photos have to be transferred to the computer or something else to be "worked on by the computer", I tend to do it sooner than later. That is, I don't wait for my camera's flash memory to be full but will try to copy soon after events. This way I remember more and can annotate better, and more of the photos "belong together" and I don't have a huge pile of photos from muliple projects to sort all at once (which can be such a large task that I'd put it off and never do it.)
3a - some people store their photos in "one big directory" dumping ground. At least they've got copied off the camera. (Ok, I put SOME photos in one big "to sort.. someday" directory but I try not to let that happen).
3 - I store photos in directories and subdirectories. There are many different "catagories" you can use from chronological to "who's in the photo" to "what event was this photo about". I keep the camera supplied filename and add to this descriptions in the filename, as well as "tags" of metadata in the file (depending upon the file type. Eg. JPeGs have quite a few standard defined fields for comments, caption, keywords, etc.) I find it handy to usually keep the original" filename around though if for no other reason than it is easy to search for similar names to find other photos I took near that time, etc.
4 - data backups! I prefer a directory structure on a "library disk" (hard drives I've dedicated to just holding data) since I can easily back up the photos by copying the entire disk to a DVD or just directories of certain photos, etc. I also copy the disk's files to another hard drive "an offline library disk" occasionally. Whatever you do, you will want to have some backup. If you don't need the backup, you WILL need it and want it right after you lose everything due to a hard disk failure. In general, don't ever have EVERYTHING in just one place or where any single device's failure would cause all to be lost. (Here again I prefer "universal standards" rather than proprietary and "RAW" files, but the principle is the same - don't have any single point of failure.) This includes entire computers! I'm being paranoid and exaggerating here, but I think the best backup system includes an occasionaly burn-to-DVD and off-site storage so that if the entire computer goes or gets a nasty malicious virus/worm on it, it can't go and delete everything by itself. Likewise, a fire.
Large external "USBv2" hard drives are often on sale and pretty cheap - and make a great backup. Connect the drive, copy your data, DISCONNECT AND STORE the drive away somewhere. No, not as rugged as an optical disk, but pretty good and very fast and quite cheap insurance < $US100 for 250GB isn't hard to find.
Ultimately, a backup system has to be comprehensive yet simple/easy to use or it won't get used and that's the same as having no backup.
|

07-31-2006, 08:54 PM
|
|
Administrator
|
|
Join Date: Aug 2004
Posts: 176
|
|
Re: Managing and storing large files
sorting -
If you annotate the photos by filename, then you can use your computer/operating-system Search and file management facilities to um, search for terms in the filename.
Keywords and metadata "not visually visable but human readable text stored INSIDE the photo file depending upon the file format" is where I find the real power in organizing. There are many Windows utilities that access and search this metadata in JPeG files for instance. (Eg. IrfanView for the PC, free!) Adobe has an excellent utility called Bridge for the Mac and PC (comes with Photoshop Elements on the Mac but only with Photoshop for the PC. Photoshop Elements for the PC offers some limited [lame] program which does NOT work uniformly and standardly with the standard metadata in a JPEG file.). Here you can spend hours browsing and annotating, say putting in "day I stubbed my toe" in a comment, as well as Moaning Frog (or Morning?). Heck, put in both keywords "moaning" and "morning" if you think that will help you find it later.
Again, you can spend hours "surfing" your photo collection editing and adding thoughts, comments, and keywords to them or just making searches for keywords of photos you took and annottated long ago and remininsce.
I end up doing somewhat of all above -
1 - I sort the photos by grouping them physically into directories of whatever seems to fit most of the photos I put there - usually an event or the subject. If photos don't "fit", I make a new directory (or abandon them in the "to sort someday" folder")
2 - I append to the filename the most important words I think I'd need to find the photos. THIS ALSO WORKS if you name your directories! Eg. a file search for my daughter's name will return files with her name in them as well as directories named with her name in them.
3 - (takes the most time but is also the most "fun") I use the keywords and metadata sort of randomly (when I randomly have extra time...) to annotate further comments which wouldn't seem right to be as part of the filename.
|

08-07-2006, 03:51 AM
|
|
Member
|
|
Join Date: May 2006
Location: South West of West Australia
Posts: 99
|
|
Re: Managing and storing large files
Wow,
thanks Kerr. Lots to take in & consider. Have been dragging my feet but the new computer & external hard drive is a must at this point I think. This clunker is so old now.
Cheers
Katheric
|

08-07-2006, 09:39 AM
|
|
Mad About Cameras
|
|
Join Date: May 2006
Location: England, UK
Posts: 477
|
|
Re: Managing and storing large files
Ah, RAW, the dark side of digital imaging.... Welcome :-)
Unlike Kerr, I'm not a believer in JPEG when it comes to shooting RAW. About 80% of my photography is done in JPEG but I reserve RAW for when I pull out all the stops, for when image quality is paramount, which is why I prefer converting RAW images to lossless TIFF images.
My RAW workflow - and I don't shoot RAW unless the situation demands it - goes something like this:
1.) Take picture
2.) Transfer to computer (using a speedy card-reader)
3.) Select and Delete unwanted images (currently using Adobe Lightroom)
4.) Convert remaining images to TIFF (but keep original RAW files)
5.) Add keywords (in Lightroom)
6.) Edit in Photoshop
7.) Store without sharpening
8.) Turn into JPEG/sharpen for output when required: web, printing, etc.
What you end up with is, essentially, a library of RAW files and a corresponding library of TIFF images. Ideally, one or both of these libraries shoudl be backed up.
A big hard-drive is essential. I bought a Lacie Brick 250gb hard-drive this week which looks like a big red lego brick... You can buy several of them and stack them. And they come in different colours (white, blue).
Adobe Lightroom is proving very popular when it comes to creating a library of RAW files and converting from RAW. It's also free at the moment for both Windows and Mac users because it's in beta-testing.
Last edited by Hiding_Pup : 08-07-2006 at 09:44 AM.
|

08-07-2006, 06:58 PM
|
|
Slightly Mental Moderator
|
|
Join Date: Jun 2006
Location: Honolulu, Hawai'i - a Brit abroad
Posts: 253
|
|
Re: Managing and storing large files
Just a quick anectode about the law of unintended consequences. Don't worry - it's vaguely relevant...promise!
The missus has a laptop with a widescreen display. While this offers a fair amount of real estate for multiple windows, I've always felt that it feels weird when a single window is maximised, as she tends to do. There just isn't a lot of vertical space, but you definitely get an abundance of horizontal room.
Anyway, a while back, she was complaining that it was sort of hard to find anything on her machine. She's not the greatest with computers - she's good at word processing and surfing and all that fun stuff, but she's less sure about managing files and so forth.
Given the wide screen and this comment, I figured I would install Google Desktop for her. Once it indexed everything, it would help her find stuff, and it would make better use of the horizontal space available (it can be set up to give a big toolbar sort of thing on the side of the screen).
I set it up to show her her gmail, the weather, a little scratchpad area were she could just type random reminders and stuff into and indicators for her battery and wireless strength. For the hell of it I also set up their little photo viewer, pointing it at the copy of our photos on her laptop (there's another good way of maintaining another backup of stuff).
Wouldn't you know it, the photo viewer is her favourite thing about the entire tool. Sod the searches - she gets to see a new random picture every 10 seconds.  The number of times I've been called over to look at a photo that's popped up at random, I cannot even begin to count. She loves the thing.
Anyway, if you want a way of randomly viewing photos from a large collection, Google Desktop is apparently a pretty decent way to do it.
Oh, and being able to google your own stuff isn't half bad, either. 
|

08-08-2006, 12:02 PM
|
|
Administrator
|
|
Join Date: Aug 2004
Posts: 176
|
|
Re: Managing and storing large files
Thanks for the hint about Lightroom. I had been using Adobe Bridge for a group project after finding that Bridge came with PhotoshopElements(MAC) but a really limited and lame program came instead with PhotoshopElements(PC)! On the PC I usually personally use Irfanview as the photo (and metadata) viewer for almost all file-types and use editing programs for editing, etc. Bridge seems to use "Lightroom" sounds a lot like Bridge and I'd like to see how it compares.
I agree with Hiding_Pup's workflow - there are times where you need/want all the visual quality and data possible, and that would be with RAW rather than JPEG indeed. Converting the RAW into a lossless (well, not lossily compressed) and standardized TIFF sounds like an excellent way to go. Archive and use the TIFFs and then you are not searching in the field or in a couple of years for some way to view/use an obscure proprietary RAW on a different or new computer, etc.
|

08-22-2006, 10:29 AM
|
|
Member
|
|
Join Date: May 2006
Location: South West of West Australia
Posts: 99
|
|
Re: Managing and storing large files
Hi all,
have been a bit quiet as I have bought an external hard drive (300gb) & have been debugging, defraging, generally removing a whole heap of junk etc before my first back up. Backup complete, all files deleted from main harddrive & guess what. I think the machine is actually going slower.
System Mechanic advised it needed a deep clean & then spent over 13 hours promising to be finished in a few seconds. I ended up shutting the computer down and going to bed. Do I need to click on anything to safely remove or turn on the external hard drive (can't find anything in the manual). Perhaps if I turn that off it will all speed up?
By the way, thanks for the great feedback above. I'm off to download Adobe Lightroom to make myself feel better.
Katheric 
|
| Thread Tools |
Search this Thread |
|
|
|
| Display Modes |
Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT. The time now is 09:10 PM.
|