FCPX SAN Workflow Dos and Donts

August 11, 2014 Tags: fcpx, Sam Mestman, SAN, workflow

Hey guys,

Sam here again… so, contrary to what a lot of people believe, SAN workflow in FCPX is actually very simple and straightforward, especially with the newly released version (10.1.2). While I’d still say that the Avid Unity workflow is still a little more robust with its bin locking features, working from a SAN/network in FCPX is still very practical. Also, in terms of price vs. performance, I’d say FCPX is the way to go, as Unity setups tend to be slow and extremely overpriced. The truth is that you can have multiple editors easily accessing the same media and passing projects to each other seamlessly. All you really need to know are a few things:

Do’s:

Keep all your media centralized on the SAN – For me, best practice is to put your original Camera negatives into a “Media” folder on your SAN. Within that, create a new folder based on your project, and then a “media” folder within that and place your camera/sound originals in there accordingly in their original directory structure.

Keep your media outside of the library – When you make a new library, from your preferences, first make sure that “leave files in place” is selected. Then, select your library, go to “modify settings” in the inspector, and set your media to be imported to a folder that is outside your library (this can be on the SAN). When you import media now, your media will added to this new folder, but it will be adding “Sym Links” (similar to aliases) that are pointing back to the original media that is also living on the SAN (see above). This will come in handy for Archiving and media management later (see below for why).

Keep your Libraries on an internal drive – If you’ve worked off a network in FCPX and experienced slow opening Libraries, and generally slow performance, it’s probably because you have your libraries on the SAN itself. The reason for this is because SAN’s are designed to handle large chunks of media, not the small database files that FCPX creates. If you’re on a network… run a test from a large library that has media outside the bundle (so it should be a lightweight file) and copy that file from the SAN to your desktop. You’ll notice that the copy time is probably far longer than it should have been. Now, copy a regular media file over. If you’re on a decently fast network, this copy should be MUCH faster than the library copy was, even though the library was much smaller in size. This illustrates this issue. For this reason, best practice is to keep your database on local storage (especially if you have a new Mac Pro which has an extremely fast internal SSD) or an external hard drive. You will see a significant increase in speed and application startup times doing things this way.

Make sure all Editors’ Libraries are pointing to the same place – The best way to do this is to make a master editorial Library for your primary editor using the import steps described above, and then duplicate that library and hand it off to each new editor. If you’ve kept your media outside your bundle, this will now be very similar to the standard FCP7 approach most of you are used to… and basically all you’re doing is passing each editor a duplicated “project file” that is making sure that your “capture scratch” is all being set to the same place for easy reconnecting later. This way, any time an editor imports media, you’re guaranteed that it’s going to the right place in the database, and that this database is the same as what your other editors are working from.

Use Xfer Libraries and keep those on the SAN/Network – Because two editors can’t work from the same library at the same time, you should have a centralized Xfer library that editors open and close when they want to pass new edits to each other (this can also live in a dropbox/google drive). If an editor needs to pass media or an edit to another editor, they should consolidate their library first to the network to make sure all media they’re referencing lives in the correct place (see below for the reason why), and THEN pass their project(s)/event(s) into the Xfer library to distribute to other editors. They should then close the Xfer library so others can access it.

Use consolidate commands and Hard links to seamlessly and non-destructively consolidate media in FCPX – Ok, so here’s something really cool not a lot of people are aware of. If you are using the FCPX media structure in the correct way, because of the way FCPX takes advantage of Hard Links (for the record, I have no idea what real definition of these are… just what they do), you can have multiple copies of the same file on a drive/SAN/network, and those files will only take up the space of a single copy of that file. To ilustrate, here’s a simple test you can run:

Create a new library set up the way I described above, and make sure the media folder in your library is set to the same drive your Camera Originals/files you want to import are stored on.
Check and see how much space is still left on the drive. Write this number down.
Take a relatively large file (minimum 5+ GB) and Import into the library and confirm that you have sym links show up in your original media folder.
Now, use the consolidate media command new in 10.1.2 to have that media you just imported copied over to the new library.
Ctrl click the file and select “reveal in finder” and then look in the original media folder and confirm that the file you imported no longer has an arrow next to it (meaning that this file is now an actual copy and no longer a sym link).
Check the total disk space on the drive/SAN/network. It should still be identical to the number that you wrote down.
This means that you have two “copies” of the file on the same drive, but you are only losing space for one of them because of “Hard Links.”

The reason for this is that because it’s using Hard Links for your media, FCPX is keeping track of the files you have on a drive, and if you use the FCPX commands to manage your media, you can have files living in more than one place on your drive, but not be penalized in terms of disk space.

What this means is that in terms of archiving and importing new media, if all your editors have their libraries pointing to the same media folder, and you are using the consolidate media commands correctly, you now have the best of both worlds; you can copy your media onto the network and have that directory structure remain untouched by FCPX… but you can also ensure that all of your editors have the same access to media that all your other editors have access to because everyone is consolidating to the same place when they import to new things, and you are never losing disk space because of this. Not only that, but when you’re done, everything is simple to archive because everything you need for a project will be living within a single directory which you can easily archive whenever you need to, and you’ll be sure that you’re not missing anything. Hard Links are great.

So, to recap… here are some don’ts:

Don’t keep your media inside the library on the network. This will make your libraries far less portable.
Don’t let your editors have their library Media folders pointing to different places.
Don’t keep your Library Media folders on different/networks/SAN’s drives than your original negatives if you can avoid it.
Don’t keep your libraries on the SAN/Network – except for a single Xfer Library for people to pass edits and events to and from (but even then, you should probably put this Xfer library in a dropbox).
Don’t try Group workflow on a large project without first running tests with your network (especially disk permissions which can do all kinds of weird things to you).
Don’t let your editors start cutting without explaining the workflow to them ahead of time and making sure they understand why they’re doing what they’re doing.

Also, for some more information about how media management works in FCPX, check out this awesome video by Dustin Hoye: