Workflow: Using Google Photos as a Master Archive

A Google Photos master archive can be a game changer for your photography. Master Archives are a powerful tool for breaking out of RAW editor lock in, but they are also the key to unlocking your photos for easy use. A Google Photos master archive puts your entire archive of works at your fingertips to share and show off.

If you aren’t sure what a master archive is, or why you would want one, check out the previous posts in the workflow series here.

Google Photos Master Archive

By keeping a google photos master archive of JPG images, you gain easy access on any enabled device: Android & iOS devices and just about any computer with internet access. That cloud based access makes sharing your work easy, whether it is on social media, through a blog post, face to face, or with partners to promote you with.

A Google Photos master archive doesn’t solve every problem a professional photographer has, but it solves a lot with minimal work.

At a high level, there are 2 options, simply sync your entire existing archive or export a google photos specific subset of images. Both have their pros and cons. Neither adds much work to a well designed workflow.

The Google-verse

Like many people, I have a love hate relationship with Google. Over the years they have created an ecosystem of incredibly powerful cloud based tools:  Drive, Photo, Docs and Gmail just to name a few. Their tools are interoperate, making sharing between them easy. And, all along they’ve made them available for free, with paid premium options. One of the best feature is unlimited storage in Google Photos.

Their intention seems to be to make sure everyone’s cell phone photos are backed up and never lost. Google created Android, the system that runs the vast majority of photos today, but Google Photos works fine with iOS – iPhones – too. They offer unlimited storage for photos, regardless of their source. They can be taken on a phone, or a “real” camera and uploaded from a “real” computer.

Unlimited Storage?

With unlimited storage (assuming you let google scale your images or pay for a truly unlimited account), you can park as many JPGs as you want inside Google Photos. The photos are also accessible through Drive since it is what actually does the storing for you. With scaled images, they won’t chew up your quota.

Clouds & AI Search

Google Photos will also do it’s (somewhat scary) AI magic on your images and create “galleries” for each person, pet and place it identifies. You can search your archive by a person’s name (assuming it identified them and didn’t just group them together) and using general text search against what google things the photo is of. Google Photos does seem to ignore the JPG rating and tags although it might pull them in as hints for the AI based searching. It’s hard to tell. There is no direct way to search by them in any case (at least that I’ve found).

Regardless, your images will become available via the Google Photos app on any mobile device you log into. They are also available on any device with a browser and internet via the Google Photos website. You can share albums (subset of your archive) to other Gmail users easily, or as a link to non-google users.

What to Store in your Google Photos Master Archive

Do you want to store some or all of your archive? Full resolution or scaled? Making sure your archive is available inside Google Photos doesn’t have to be hard. There are a couple of approaches I’ve tried. Which is best will depend on your habits and how you use the photos.

There are a couple of options for software to manage the syncing, and I’ll talk about that below, but for now, lets look at what we might sync and not how.

Option 1: Sync Local Master Archive

If you are like me, you have a local master archive: a directory tree containing exported JPG images that mirrors the structure of your RAW archive. In my case, I export 1 star and higher photos in most cases. They are exported at full resolution, 95% quality and using the AdobeRGB color profile. I have a local folder on my laptop with the previous 12 months of images, and more folders on an external drive with everything else before that (about 10 years worth).

If you sync your entire master archive, you will export your master archive JPGs as normal to their normal directory. Set up Google Backup and Sync to synchronize your local archive folder with your Google Photos master archive.

The Good and the Bad

Pros:

• Doesn’t require extra work, or any extra storage space.
• Google manages what is uploaded and keeping it all in sync.
• The folder structure of your archive is preserved in drive under “Computers” inside Drive.
• Your full resolution and quality archive images are available anywhere making things like ordering prints when not at your main computer super easy.
• Google drive acts as a backup for your master archive.

Cons:

• Takes more time to upload thanks to the full res, high quality nature of your archive images.
• The initial sync can take a very long time for an existing large archive.
• Everything is uploaded, so you can’t filter out lower rating stuff you might not want to see. I.e. if you have albums with a lot of 0 star images you get all the photos not just the highlights.
• If your archive is in AdobeRBG or another wide gamut profile, images shift (sometimes dramatically) when posted to Instagram or other social sites. It appears to be a bug in how they import them (just applying a different profile instead of converting like they should). Since you aren’t reexporting you don’t get a chance to convert them yourself.
• Your full resolution and quality archive images are available on the web making security a possible concern.

Option 2: Sync a Google Photo Specific Export

An alternative to syncing up your local master archive is to perform a Google Photos specific export. The basic idea is that as you finish a given shoot, you export an appropriate version of the JPG files into a Google Photos specific folder.  Since it is only for Google Photos, scale the images to near 12 megapixels and moderate quality. Use the sRGB color profile designed for sharing.

Point Google Backup and Sync to the folder you exported the images. Once the upload completes, you can delete images from the sync folder assuming you have Backup and Sync configured correctly (see below). Manually export new work to the folder as it is finished and it will get automatically get synced to Google Photos.

If you have an existing long lived archive, this approach is a little bothersome. You need to open your archive and select all of the old photos you want in drive. Then you export them. In my case, that might be hundreds of thousands of photos which will take a long time. To complicate things further, if you used a Project Based Workflow but never set up master archive, you would have to revisit each project to perform the export.

The Good and the Bad

Pros:

• You can filter what gets exported.
• Quicker upload due to the smaller initial size.
• Can work in batches.
• You mange converting the color profile to a web safe profile (sRGB) during this step using a good tool instead of Facebook/Instagram mangling them.

Cons:

• You have to export all of it, which takes a lot of processor time, and also hard drive space for large archives.
• You have to manage what goes into google photos. Might be a pro too… but might require a lot of work.
• Double re-scaling of images. You are exporting them at a certain quality/size and then google is redoing it. Bad for JPGs but not a show stopper.
• Changes in the archive aren’t automatically reflected in google photos since you aren’t syncing the archive.
• Underlying directory structure is lost.
• Google Drive is not a archive backup since it doesn’t contain your entire archive.

Syncing Software

Google’s cloud infrastructure is designed to be extensible. They’ve provided developers with ways to integrate with their products, including Google Photos. Google offers their own solution for syncing part or all of your hard drive: Google Backup and Sync, but there are also open source options available that run on a variety of operating systems.

Setting Up Google Backup and Sync

Most people, especially those that are non-technical, will want to use Google’s official Backup and Sync client. It is available for Mac and PC, but not Linux or other more fringe options so you can’t put it on a home-brewed Linux server. See the Alternatives section below.

Once you’ve downloaded and installed backup and sync directly from google and started it, there are a couple of options to check.

What to Sync

By default, google will include the option to sync your computers “normal” folders: Desktop, Pictures, Documents (may very on Windows). If your master archive is stored under one of these directories, enabling that directory (checking the box next to it) will be enough to include it in the sync.

In my case, my master archive is stored in a directory called “MasterPortfolio” under my home directory (/User/afritz/MasterPortfolio). To back this up, I need to add it. Click “Choose Folder” below the list of directories, and add it.

Resolution and Deletion

There are two other options to choose:

If you have unlimited storage, make sure to select “Original quality” under “Photo and video upload size”. If not, you probably want to choose “High Quality” so the space used doesn’t ding your quota.

Under “Removing items” there are a couple of possible options. If you chose option 1 above (sync your entire archive), any of the three options is acceptable. Decide if you want to keep google photos in sync if you remove local copies of the images, or if you want them to stay in the cloud.

On the other hand, if you chose option 2, exporting a google photos specific version, choose “Don’t remove items everywhere”. With that selected you can export some images, wait for them to sync, and then remove the local copy. They will stay in the cloud.

There are other options, but my goal isn’t a tutorial on Backup and Sync. Here is a good place to start to understand the options available.

Limitations

Google Backup and Sync is far from perfect. There are a few significant limitations.

You can’t sync network drivers…

If part or all of your master archive is on a network drive and you are using option 1 above, you will need to use an alternative syncing method for the network part of your archive. USB hard drives work just fine, but not network attached storage.

On that note, many commercial NAS solutions (such as the Synology) have built in google drive syncing which hand handle it for you. If you are running a home built setup, use RClone and CRON.

Problem images…

Backup & Sync chokes on some images, complaining that it can’t scale them and that they were stored at full resolution. This is strange since I’m NOT scaling and storing at full resolution. It appears to be a bug in the client. My long lived archive includes images saved from a wide variety of software that might be causing issues. It seems to happen on the more extreme files I have (like large panoramas), but, it is not 100% consistent.

The client will give you a notice that the image(s) were uploaded at full resolution and thus will count against your quota. You are given the option to remove the file from Drive. It gives this error even if you set it up to upload at full resolution, and even if you have unlimited storage like me.

The good news is that the images are still available in Google Photos. It’s mostly just an annoyance for me, but for those without unlimited storage, you might need to manually clean them up if you are low on space.

Lost or hidden meta-data…

Google Photos preserves the core meta-data in the file: camera settings, lens choice etc, but extended meta data, such as tags and ratings you might have added, are not accessible inside Google Photos. You can’t simply ask to see all 4 star images, or at least I’ve never found a way. You have to rely on Google’s (somewhat scarry) AI that identifies people, places and things.

The good news is, that AI does a pretty good job. For most use cases (e.g. I need to find pictures of uncle Sam), it does a pretty good job.

Your milage may vary. For me, it isn’t a show stopper because my real master archive allows me to search how I want. My Google Photos master archive is a convenience away from my main computer. If Google Photos is your only archive, and you also rely heavily on things like tags and ratings, be aware that they are no longer a way to find images.

One setup for all folders….

You can only choose one setup for Backup and Sync for all folders you sync. So, if you want to use “Remove items everywhere” on Documents, but “Don’t remove items everywhere” on your photo archive, you are out of luck. Most people won’t care, but some might.

Alternative Syncing Software

There are a number of google drive (and photos) clients out there. I’m only going to talk about one that I’ve used with great success:

Rclone.

Rclone is a open source project designed to synchronize directories between computers and cloud storage systems. It works with A LOT of different systems including google drive.It also runs on just about all operating systems including OSX (mac), Windows, Linux on multiple architectures, BSD and others.

Setup isn’t trivial, but, nor is it really that hard. Once you have granted it access (they provide detailed instructions) you have to use the command line to kick off the sync. Check out their documentation if you want to use it and aren’t scared of a command line.