Menu

Colin Devroe

Photographer. Podcaster. Blogger. Reverse Engineer.

How to move referenced originals in Photos for Mac

Warning!! I’ve only just hacked this solution together and I don’t fully understand the ramifications of my actions yet. If there are any, I will update this post.

First, a bit of context on how I use Photos for Mac (Photos).

I do not allow Photos to store my original files within its “package”. I have my reasons. When I import photos I check the box labeled “Keep Folder Organization”. This way, I can keep my photos in a directory structure of my choice rather than how Photos chooses to organize them.

I wanted to take one of my photo libraries (I have two) on the go with me on a portable external hard drive that I can keep in my bag. After much searching I could not find anything that explained how to move my original photos from one external hard drive to another and have Photos recognize this change.

So finally, I had a few moments to spare, and I figured I would dig under the hood of Photos to see how it kept the references to these files and see if I could update those references to the new location.

Photos uses a SQLite database to store much of the information it needs to do what it does. Things like facial recognition, album names, keywords, etc. are all stored in a heap in this database. In a few locations, it turns out, it also stores the path to each individual original photo in your library.

So far (one night, as of this writing) this solution has seemingly worked for me. I will continue to play around with the results to see if I can uncover some adverse side effect. Until then, here are the steps I took to move an entire original photo library onto a portable external hard drive.

Photos’ SQLite database viewed in Sqlitebrowser
  1. Make a copy of your .photoslibrary file. Just in case.
  2. Copy all the original photos from one drive to the other. For me, this was simple. I keep my photo library originals in separate directories so I can copy a single directory and grab them all. For me, this was /Volumes/Hard Drive 1/Carbonite Photo Storage/Photography Projects/ to /Volumes/Hard Drive 2/Photo Archive/Photography Projects/
  3. Open the Photos.sqlite database found within the Photo Library package contents. Secondary-click on your .photoslibrary file, select Open Package Contents and navigate to database/Photos.sqlite (I used Sqlitebrowser)
  4. Update the ZNAME and ZVOLUMEUUIDSTRING fields in the ZFILESYSTEMVOLUME table. To get the new values, open the System Information app on macOS and find the new values for your hard drive under Hardware > Storage. I could not find ZVOLUMEUUID anywhere so I left it as-is. No idea if this will come back to bite me.
  5. Update the ZFILESYSTEMBOOKMARK table with the relative paths to the originals. To do this, I ran the following SQL – UPDATE ZFILESYSTEMBOOKMARK SET ZPATHRELATIVETOVOLUME = REPLACE(ZPATHRELATIVETOVOLUME, 'Carbonite Photo Storage', 'Photo Archive')
  6. Update the ZGENERICASSET table with the new paths for all photos on the ZDIRECTORY field. To do this, I ran the following SQL – UPDATE ZGENERICASSET SET ZDIRECTORY = REPLACE(ZDIRECTORY, 'Hard Drive 1/Carbonite Photo Storage', 'Hard Drive 2/Photo Archive')
  7. Save the Sqlite database file.
  8. Open Photos!

One way to tell if this worked for you is to open Photos, choose a photo from your library, and select “Show Referenced File in Finder”. This will open a Finder window with the selected file in its location. If it opens to the new hard drive you copied your originals to, it worked.

I’m going to be using this library a fair bit in the coming days and so I hope that if there are any issues with this approach I will find them quickly and can update this post. See also the comments in case others try this and leave some feedback.

Jack Baty gives up on Lightroom

Jack Baty:

I’m here to tell you that I can not make it work for me. There’s too much overhead in having to decide what to add to a synced collection and when. And where to keep any synced originals? Do I do that in both apps? And so on. I seem to end up with duplicates for no reason I can fathom. I’m constantly moving images from the automatic synced folders to their proper place in the filesystem. It often feels like the worst of both worlds. I’ve seen people do it. I’ve watched the videos and read the blog posts. I’ve tried, but nope, it’s all too finicky for me.

I hit a similar corner with Lightroom Classic when I was trying to make it work for me. As an app, photo editor, and manager it is very good. But it is tied to Adobe CC which for me, is a long term deal breaker. And I could not figure out the best way to manage my files for some reason.

I’ve been slowly piecing together my own solution, as you all may know that read this blog on the regular, but it isn’t something I can really share with anyone else.

My current workflow consists of a script or two on my Mac to move files from my digital cameras, film scans, drones, and other devices into their appropriate places and backups and cloud services, combined with two libraries in Photos for Mac* (one for personal photos, one for hobby projects) where the libraries are on my hard drive and the original files are on external storage.

It is working fairly well. But I wouldn’t wish it on anyone. For instance, to edit a photo I have to jump through a fiery hoop or two in order to not end up with a bunch of duplicates. I don’t know how to solve this problem yet but I plan on doing so.

Back to Jack. I’m with him. Some of these apps, especially those he mentions, are almost paralyzing in their commitment levels and features. I just wish all of this photo management was so much easier.

* which I have some issues with.

A tweetstorm about Photos for Mac

I’m old, so I can still call them tweetstorms rather than threads.

I just posted a tweetstorm regarding Photos for Mac on Catalina. I posted it there because I’m sort of hoping that a few Apple people are still lingering on the WWDC hashtag.

Here are my tweets:

  1. I have the second-best computer you sell, and facial recognition is going on weeks to make a dent in my photo library.
  2. How can I gracefully quit ‘photoanalysisd’ when I want to unmount an external drive? Getting sick of “Force Ejecting” (though I do love how that sounds like a Star Wars reference)
  3. Is there any way to ask Photos to start its processes again after mounting an external drive?
  4. Why would Photos just stop “thinking”? How do I “jiggle the handle”? Notice CPU usage. The app is open and in the background – should be using 100% of CPU to work. (See Figure 1)
  5. I created a Smart Album to find unnamed Faces. Maybe you can add this directly to Photos as a feature? It makes it much easier to find photos that have faces but Photos doesn’t know their name. (See Figure 2)
  6. I’d like an option to delete a photo from the hard drive when I delete from the Library. Is this possible and I’m simply missing it? As of right now, I have to “Find referenced file in Finder” and delete both in Finder and in Library.
  7. Can I move a Photo Library from an external drive to the local drive and all of the references will stay in tact?
  8. The Places feature works on individual people or if I search for a location but the map feature under Places in the sidebar shows no photos at all.
  9. Can you add progress indicators throughout the app? Importing shows progress, but it’d be nice if facial and object recognition or other tasks gave some indication of “doneness”. As it stands, it appears as though Photos is broken. But I know it isn’t. It’s just “thinking”.
  10. Exporting original photos should retain all metadata (unless specified to remove in preferences).
  11. Bonus: Adding descriptions or keywords, etc. should be stored directly on the file itself. It can be stored in the Photos database too. But storing on the file itself makes Photos for Mac “future proof” a bit.
Figure 1
Figure 2

Who knows. Maybe someone will read those tweets.

Importing tens of thousands of photos into Photos for Mac, on a maxed out 16-inch MacBook Pro, cripples the machine. It is nearly unusable. Closing the app doesn’t help because it has background processes when the app is closed.

Photos for Mac isn’t a long term photo library option

Bradley Chambers, writing for 9to5Mac, about his photo library backup strategy:

If there is one thing I am obsessed with when it comes to technology, it’s my pictures. I keep them extremely organized and culled.

He then goes on to say, regarding his use of iCloud Photo Library as a sort of backup:

This service puts a copy of all of my media on Apple’s servers, and that means if I lose my iPhone, iPad, or MacBook Pro, I can sign into a new device using my iCloud account, and all my media will be there. One thing to remember is that iCloud Photo Library is a sync service. Syncing means that if you delete a photo on one device, it’ll be deleted elsewhere. For that reason, I don’t consider iCloud Photo Library a true backup.

If you want to use iCloud Photo Library to sync your photos between devices, and even use it as a way to have a full backup of your photos, I suppose you can. However, after doing that for a few years and then wanting to move away from it – I would not recommend Photos on Mac or iCloud Photo Library as a long term photo library solution.

The problem is a few fold, but here are the main points:

  • does not store photo metadata in a readable format or with the individual files at all
  • does not store photos in a directory structure that is human understandable
  • bloats your library’s size dramatically

I have well over 350GB of photos and videos. When I migrated away from Photos for Mac I thought that it must store these in a sane directory structure. When you view the Package Contents of your Photos Library file, it appears as though it does but it does not. Each photo is kept within layers of directories by date within directories by the date they are imported not taken. For me, a huge portion of my library was stored in the 2013 directory, even though most of the photos were not taken in that year. Using various Windows 10 tools I was able to read the file’s metadata to create a sane directory structure and put those files into their proper locations based on when they were taken. Even with automated tools it took me a few weeks to do this.

In addition, all the work you do tagging, face tagging, etc. of photos could end up being for naught. That hard work won’t leave Photos for Mac onto another platform. Perhaps you’re not worried about moving from Mac to Windows or from Photos to another library manager, but you should be. Apple has already killed iPhoto in favor of Photos for Mac and lost a lot of functionality when they did. Who is to say they won’t do that again? Or discontinue the Mac altogether some day?

I still have more work to do before I’m able to share my full workflow for storing, searching, syncing, and backing up my photo library – but this experience has taught me that I always want my library to be future proof, human readable, platform agnostic, and not be locked into any one company’s ecosystem. I’m close and I look forward to sharing my strategy in the near future.