In this post I’m going to detail out how I manage my photo library. I have changed my workflow countless times over the years and I have the deep battle scars to prove it. I’ve switched services, storage providers, apps, folder structures, metadata schemes, you-name-it.
In 2018 I decided to set out on a sort of mission; I wanted to create a photo library that would be relatively future proof. Should I decide to use a different app or storage platform, I’d want to be able to do so easily rather than painfully.
Note: This solution is likely not for everyone. The number of hours I’ve spent to come to this solution cannot be overstated. But, I’m very happy with how it is working. Maybe if you’re reading this it can work for you too. Or, perhaps you can pull just one idea from it that will help you.
Here is what I had to accomplish:
- Create a folder structure that I can control and maintain regardless of the photo application I choose to use to view my library or edit my photos
- Store as much metadata about the photos within the files themselves – rather than an external database – so that the metadata would travel with the image across all platforms
- Be highly searchable, glanceable, sharable
- Store my personal photos, my wife’s personal photos, my photography “work” and projects, and my digital film scans.
- Bonus: Be accessible from any of my devices
Tools
- Bulldozer – A simple PHP script that copies all of my images into their proper folders based on date created
- SetEXIFData – an app to set the date taken, GPS coordinates, keywords, and other EXIF information for my digital film scans
- Adobe Photoshop – For editing photos
- Adobe Bridge (Camera Raw) – For Super Resolution (on 35mm film scans)
- OneDrive – For cloud storage of my library
- Photos for Mac – To search, view my library locally, as well as create smart albums.
- Epson V600 – To scan my film negatives, slides from family members, or old photo prints.
- Silverfast – The scanning software that I use. It is a little clunky but by far the best software I’ve used to scan.
- A few external hard drives
Here is what my current storage situation looks like.
My photo library weighs in at over 450GB and growing rapidly. I have no less than 4 complete copies of that library. I have the “working library”, “backup library”, “off-site backup library”, and “cloud backup library”.
Why 4 copies? Restoring from a cloud-based backup when you live in a rural area is… less than fun. I remember when Everpix was shutting down and they sent a notice to all users to download their entire libraries in a very short period of time. I’ll never make the mistake again of relying on a cloud-based storage solution for my photos.
Of course, hard drives go bad. So I have a local backup in my home office at all times should anything go wrong with the working library. The off-site backup goes with me in my bag. Sometimes I’ll leave it at work in case my house burns down. I know, this isn’t easy to think about but alas, I do. Lastly, the cloud-based backup is a failsafe. Cloud-based storage is incredibly inexpensive. In fact, my storage comes with my Microsoft 365 subscription. It also affords me the ability to search my library from any of my devices from anywhere in the world. Something I don’t do very often anymore but it is nice to have when I need it.
Workflow
My workflow for storing photos varies only slightly based on the source of my images. I have a few iPhones, my wife’s iPhone, some DSLRs, a drone, a wildlife camera, and multiple film cameras. I could be importing photos from any one of these devices (or all of them) at the same time. So I have a slightly different way of handling each of them.
Let’s start out with a simple example; imagine that all of the images I’m importing came from a digital camera like the iPhone, DSLR, or drone.
First, I throw all of the images from the devices into a single folder at ~/Pictures/import/digital
Next, I’ll take a few moments to go through those photos and delete anything I’d rather not import into my library. I’m not very strict at this stage. I am of the opinion that photos become more important with age – and again since storage is so inexpensive – so I keep the vast majority of the photos we create.
One quick note here: Since Eliza and I have the same iPhone models, I will quickly rename the photos that we took to be prepended with takenby_colin_FILENAME and takenby_eliza_FILENAME. This makes it dead easy to know who took the photo later even though the device models will be the same.
Once I’m finished culling and renaming the photos in the folder I move them into a waiting room of sorts at ~/Pictures/import/ready_to_be_copied. Let’s pretend I have 500 files in that directory now.
Next, I have a simple PHP script that I’ve called Dozer that will look at the file’s creation date and automatically sort them into their appropriate folders. So, an image taken on March 17, 2021 will be copied into a folder 2021/03/17 that exists on my main working library drive.
Dozer gives a little report in the terminal about how many files were moved, how many folders were created, and even warns me if any of the files were duplicates (it happens sometimes). This way I can be sure that all of the photos that I wanted to be copied were copied.
Once I’m satisfied that the process was done successfully, I’ll run Dozer a second time to throw the same files onto OneDrive in the same folder structure. I’d like to improve this part of my workflow wherein some sort of comparison is run against my entire OneDrive library.
Next, I open the Photos for Mac app and import the new files. When I do so, I make certain to check the “Keep Folder Organization” checkbox on the Import screen. By doing this Photos for Mac will keep the files where they are and will not move them into the main library file. Photos for Mac will, however, make a few different sized thumbnails to keep the app snappy. So your Photos for Mac library file will increase in size.
Using Photos for Mac gives me a quick glance at my entire library quickly, it also allows me to do some other organization like Albums, etc. on top of my date structure. I have Smart Albums for the cameras I own, for different areas of interest (such as Architecture), for people, etc. Though I still feel that Photos for Mac is not a good longterm storage solution I do believe it is an excellent viewing app. In addition, it is getting better and better at facial and object detection which is a nice bonus for searching my library. I can search for “cat” or “sunset” or for a specific person. Also, Photos for Mac respects the EXIF data that is on the file allowing me to sort photos based on the device that created them, any keywords I may have added, etc.
Film scans
I scan my film negatives using an Epson V600 and the Silverfast app. I generally scan at fairly high resolution and rather flatly contrasted – preferring to do my crops, edits, digital dodging and burning in Photoshop. Once I have a satisfactory file I’ll use SetEXIFdata, a rather crude Mac app, to reset the EXIF data of the file to the film camera’s details, GPS coordinates, etc. This is the controversial bit, many people would take offense to this. But I prefer to have this metadata go with the file, rather than using Photos for Mac to tag the photo or something. Because then all of the data is stored in the Photos for Mac database. And believe me, I’ve seen it, and it isn’t pretty.
My film photos come in two general categories; personal and projects. My personal film photos are usually from a vacation or day trip or something. And so I store these in very much the same way as I do photos that came from my iPhone. My projects files, though, are stored in a Photo Projects folder by some sort of name. For instance, I’ve got a multi-year photo project going where I take portraits of my friends in or near their backyard garden. Those go into a Garden Friends folder. I back up these files the same way I do all the others.
I am currently very satisfied with this workflow. There are a few sticking points that I’d like to improve, and I wish I had a UI for searching all of my files on my phone (that wasn’t OneDrive) but at least I have a system that works for me, is future proof to a degree, that I can control, and that is fast.