Workflow: Taming the Beast by Splitting Your Photo Catalogs

In my previous post about high level workflow design and breaking vendor lock-in, I referenced different types of workflows: the monolithic, semi-monolithic and project based workflow. I didn’t go into detail then (that post was already getting long), but now it is time to talk about them in detail. This article is primarily about the Import/Edit/Deliver phase of a photo workflow and does not go into details of using any specific tool.

2 Phase Photography Workflow

I divide workflows into two distinct phases:

• Import/Edit/Deliver: Dealing with the initial import, edit and delivery to the “customer”.
• Archiving: Making sure your images are safe, easily searchable and usable long term.

I separate these 2 phases so that I can easily swap out the tool I use for the first phase (Import/Edit/Delivery) any time I want, even on a shoot by shoot basis. If Capture One is the right tool, I use it. If some other tool comes along I like more, I can use it for some or all of my photo work. For example, I normally don’t use Lightroom, but the photo-merge feature makes it convenient for some types of shoots (e.g. panoramas) when I’m more concerned about speed than ultimate control of the panoramas.

Types of Workflows

What happens when you get home from a trip or photo shoot? Which catalog do you import the photos into? Do you have just one catalog (e.g. common for Lightroom users) or a couple catalogs each for a different part of your photography (e.g. Professional, Personal), or do you create a new catalog for each and every shoot? There are a lot of possibilities. Different tools make some of them easier than others.

For example, in Lightroom you can only have one catalog open at a time, and switching catalogs is slow any annoying. In contrast, in Capture One Pro, you can have as many catalogs open as you want at once. Switching catalogs is quick and trivial. You just alt-tab to another Capture One window. Stepping back further, you could use a tool like Adobe Photo Bridge and a simple directory structure of photos as a “catalog” for a shoot.  Things are centered around a directory instead of your RAW tool.

3 Types of Workflows

There are 3 general ways you can divide up your raw files during the Import/Edit/Deliver phase:

Monolithic Catalog

One catalog to rule them all. All images, regardless of the project, end up inside the same catalog. The catalog grows forever. This is the “default” for many people as it was for me initially. Lightroom is designed so that you end up with this unless you intentionally break out of it. Even Capture One seems to expect this type of workflow but at least makes breaking out of it easy.

Semi-Monolithic Catalogs

All photos from a given subset end up in a catalog. The catalogs could be separated per shoot type (weddings, personal, sports) or by time (2018-q1, 2018-q2, 2018-q3). They could even be separated by a combination of the two (2018-weddings, 2018-personal, 2018-sports). Regardless, there are multiple catalogs, each containing some subset of your photography.

Project Based Catalogs

Each new photo shoot or project gets a new clean catalog (e.g. 2018-09-01 Surfer Photos, 2018-09-05 Joshua Tree Scouting, 2018-04-12 Frank and Ann Wedding). If you name them carefully, it is easy to keep track of what’s what. Each project can use it’s own tool too. Some shoots could use Capture One, others Lightroom and others less mainstream tools like Bridge, Affinity Photo, On1 Photo Raw, etc.

Monolithic Catalogs

Many people use this type of system with Lightroom, mostly because Lightroom makes it a PIA to switch catalogs. You have to shutdown and restart in a different catalog which isn’t fast. You can not have more than one open at a time. Therefore, people tend to use just one catalog until it gets huge and slow.

I don’t recommend Monolithic catalogs for anyone. They are a result of bad software design (looking at you Adobe) and lack of forethought. My favorite tool, Capture One (the Pro version anyway), makes starting and opening multiple catalogs trivial so it is less tempting to default to this terrible way of using the tool.

The Pros

• A monolithic catalog has the advantage (or disadvantage from my point of view) that it can also double as your Archiving tool combining the 2 phases of your workflow.
• All your photos live in one catalog so searching is easy.
• There is basically no work required to move photos “into the archive” since they are already there.
• You don’t have to spend time exporting archival JPGs for your own archive (although, you probably still need to for customers anyway).
• You don’t have to waste storage space on those archival JPGs.

The Cons

• Most people default to using their monolithic catalog as their archiving system. This leads to pretty strong vendor lock-in and pain when you find yourself wanting or forced to switch.
• Monolithic catalogs grow endlessly, and slow down the entire time. Eventually a monolithic catalog gets unusable.
• All your eggs are in one basket. A corrupt catalog DB file can destroy everything. You have Backups right?
• They don’t inherently lend themselves to knowing which shoots are finished and which still need work.
• They make it less obvious how to free up space. You shift things around inside the monolithic catalog after attaching drives to it so the catalog now spans multiple storage drives.
• It is hard to migrate away from when you realize they are terrible.

Semi-Monolithic Catalogs

Rather than one huge terrible never ending catalog of pain, you will set up a few catalogs divided along hopefully clear cut lines. I normally recommend people split things by date range. For light shooters, one catalog a year might be good enough. For more active shooters, one a quarter or a month might be warranted.

If you shoot heavy enough to do new catalogs monthly or if you have trouble deciding which catalog photos go in, a project based catalog system might be better for you. A semi-monolithic catalog will be the right choice for casual shooters and even some serious shooters. The choice between this and project based catalogs is mostly one of degrees. They are two versions of the same type of thing.

The Pros

• Splits things up so your catalogs don’t grow forever.
• Keeps the number of catalogs small so you shouldn’t have to switch very often (i.e. for Lightroom Users).
• You can still search your catalogs for photos from a relatively long timer period without going to a master archive. Maybe you can get by without one.
• You can move finished catalogs off your primary storage on a semi-regular basis.
• Catalogs should be small enough not to need to be split over multiple storage devices. They are bite sized.
• Catalogs get “finished” eventually and can be moved to cheaper and slower secondary storage (e.g. an external hard drive).
• Can easily be transitioned into a project based or finer grain semi-monolithic catalogs when you grow to that point. You can even mix and match some project catalogs with a few longer running catalogs.

The Cons

• There can be confusion about which catalog things should end up in. See “Clear Cut Lines” below.
• The catalogs can still get pretty large depending on your divisions. You might need to split the RAW files over multiple hard drives depending on your primary hard drive size.
• You can’t search everything in one place without a master archive. The smaller the chunks you split things into, the more places you have to look.
• Some tools (Lightroom) make switching catalogs a PIA.

Clear Cut Lines

The key to a happy existence with semi-monolithic catalogs is having clear cut lines between what goes in each catalog. Let me give you an example.

Lets say you setup a 2018-personal and a 2018-professional catalogs. Obviously, any photo taken in 2018 goes in one of these two catalogs, or is it? Lets say you go on a trip starting in late December 2017 and ending in January of 2018. It is personal (that is clear), but do you split it up between your 2017-personal and 2018-personal? Do you import it into the catalog when the trip started (2017-personal)? Either might be fine, but be consistent. Think through these type of situations before they occur and have a consistent rule.

A Murkier Scenario

There could be more difficult scenarios too. If you setup up 2018-landscape and 2018-wildlife catalogs for examples, where do photos from a day that you shot both landscapes and critters go? Do you need to waste time separating the intermixed photos? If you don’t, then one or the other catalog will have stuff that doesn’t belong in it, stored there muddying the lines between the catalogs.

These type of situations are why I suggest simple time based catalogs (by year, quarter year or month), or at most, time combined with very high level subject based (2018-professional, 2018-personal) where there is unlikely to be much mixing. Keeping the categories simple and high level avoid the mess. If you find yourself needing finer divisions than this, it’s time to switch to Project Based Catalogs anyway.

Project Base Catalogs

Project based catalogs take semi-monolithic catalogs to their natural conclusion. You separate every shoot into a separate catalogs. Catalogs become equivalent to jobs for a professions. For personal shooting (and non-professionals) knowing where to split things should be clear cut as well. If you take a trip then you create a new catalog. Each day trip somewhere becomes its own catalog. If you take your camera along to dinner and take some photos, that is a catalog.

A catalog might have 2000 photos in it (e.g. a wedding) or 20 photos (e.g. pictures from dinner out with friends). If it is easy to start new catalogs and jump between them, having lots of catalogs is no big deal. It’s important to remember that you don’t “reuse” catalogs either. If you return to Padre Island for more bird photography, you start a new catalog for that trip.

The key things required to make project based catalogs work well are:

• A RAW processor that lets you easily start and switch catalogs, open catalogs off network drives and even keep multiple catalogs open at once (i.e. Capture One Pro).
• Or, a raw processor that is not a catalog system, but that just opens raw files from your hard drive. Your directory structure becomes your catalogs.
• Consistency in starting and naming a new catalog for every project/trip/shoot.

Project based catalogs work well for those shooting heavily and who have a need to know what is finished and what isn’t. I use project based catalogs and love it. That said, I know a lot of people, especially casual shooters, that they are overwhelming for. In some ways Project Based catalogs are more complex, and in others, they are way simpler.

The Pros

• Small catalogs that contain only related images for one project/trip/shoot that are easy to move around and manage.
• It is easy to tell when a particular project is done or still needs work. This workflow encourages “finishing” shoots.
• Archiving completed work to free up space is trivial. Just move a given bite sized catalog to another hard drive.
• Catalogs can easily be kept self contained on a single storage device.
• Images are already organized. There is no need to spend time creating albums/collections/etc for each shoot to keep things straight.
• The entire catalog can be handed off sense it is a manageable size containing exactly one project. They work great in a team based editing environment.
• You can easily transition back to semi-monolithic catalogs if you find yourself shooting less or mix them as needed. They are compatible.
• You can use different tools for the Import/Edit/Deliver phases based on individual project needs. No need to pick just one.

The Cons

• Requires careful and thought out catalog naming to avoid a mess. Start right and it isn’t a problem.
• You can’t search any one catalog for more than a tiny fraction of your images.
• A master archive is required to search everything at once.
• You need to export and store archive JPGs for searching and quick use in the master archive.
• If your primary RAW tool isn’t designed to allow easily switching of catalogs (i.e. Lightroom) it can be a annoying to switch catalogs.

Picking a Workflow

Hopefully you have figured out that I do not like Monolithic catalogs. We all start there, and eventually we all have to deal with the problems they cause. You should never set out to create that type of system. It is just asking for trouble.

But, which of the other 2: semi-monolithic or project based, should you pick?

Jane: Project Based Workflow

Jane is a professional wedding photographer. She shoots 20 to 40 weddings a year plus numerous bridal and engagement sessions. Jane also makes photos for fun when she has time. She loves photographing birds and landscapes. Jane takes 10s of thousands of photos a year, sometimes 10s of thousands a month.

Jane is a classic example of someone that can use a project based workflow naturally. Her world is divided up into projects already. Each wedding, bridal shoot and engagement shoot is naturally a self contained project. She shoots it, imports the photos, edits them and then delivers them. Project done, and so is the catalog.

Her personal photography can easily fit into the same structure. Each day trip, workshop or vacation ends up in it’s own catalog.

Jane’s Catalogs

Her local catalog directory might look something like this:

• 2018-08-12 Banner Wedding
• 2018-08-15 Sam and Lacey Engagement
• 2018-08-30 Hiking at McKinney Falls State Park
• 2018-09-15 Julia Carter Bridal Shoot
• 2018-09-16 Bird Photography at Padre Island
• …

Even when she has an engagement, bridal and wedding photo sets from a customer, they each become their own catalog in this scheme, although, Jane could choose otherwise.

Jane Simplifies Her Personal Photography

Likewise, if Jane wants to simplify her life for her personal photography, she can change just that. Instead of the individual catalogs for personal shoots, she could use a year based semi-monolithic catalog for all of it, for example “2018 Personal”. Since Jane is a birder, that might be a bad choice though. Birders take lots of pictures and putting all of that in one catalog might lead to storage woes.

Regardless, if Jane did go to a yearly personal catalog, her catalog directory might look like this:

• 2018 Personal
• 2018-08-12 Banner Wedding
• 2018-08-15 Sam and Lacey Engagement
• 2018-09-15 Julia Carter Bridal Shoot
• …

Joe: Semi-Monolithic Workflow

Joe is a landscape photographer. He does not take money and is a amateur in the true sense of the word. He makes photos for the pure love of the craft. Joe loves to make landscape photos and doesn’t bother with much else. He shoots on trip he takes 3 or 4 times a year. In a big year, Joe might shoot 10,000 photos max.

Joe is a classic example of a low volume “casual” shooter who a semi-monolithic workflow will work well for. By casual, I mean there are not people he has an obligation to to finish the photos for. If he doesn’t feel like editing some photos, he can just ignore them.

Joe divides his catalogs up by year. Because he is a low volume shooter, he has no issues with storage space and can keep everything from each year together. Eventually he moves the catalogs for previous years to external hard drives to free up space, but only when he needs to.

Joe’s Catalogs

Joe’s local catalog directory might look something like this:

• 2016 Photography
• 2017 Photography
• 2018 Photography

That’s it. Each new years day Joe starts a new catalog.

Joe Expands to New Genres

If Joe suddenly finds himself shooting more, for example, he decides to start photographing local high school football for fun, he could easily switch to a project based catalog for just those shoots. He can easily mix his semi-monolithic catalogs with project based catalogs for specific things.

If he decided to do that, his project directory might look something like this:

• 2016 Photography
• 2017 Photography
• 2018 Photography
• 2018-10-12 Taylor vs Pflugerville Varsity
• 2018-10-15 Taylor vs Round Rock Junior Varsity
• 2018-10-18 Round Rock vs Thornedale Varsity

The yearly catalogs (e.g. 2016 Photography) contain all of his work except photos from football games. Sports photography is like bird photography. It involves lots of RAW files. By splitting out the sports photos, Joe can finish and move those to external storage to free up space, but still keep his annual catalogs for his much lower volume landscape and travel photography.

Hybrid Conclusions

Jane and Joe have different needs as photographers. They use different workflows. Both have made conscious decisions to handle their photos in specific ways.

Both Jane and Joe could easily use a hybrid. They each have a clear way to transition from semi-monolithic to project based and back. That is the power of thinking about how to split up your photo work and being systematic.  Because they both have clearly named catalogs and well thought out divisions in their work, they stay flexible. They aren’t locked-in to any one tool.  They know where to find any given photo they are looking for.

I use a hybrid approach when I need to as well. Note the “2018 – Teardrop v1” catalog that I use to store cell phone snap shots of my progress on my teardrop build. Make your workflow work for you!