[This article was first published in the November, 2008, issue of
Larry's Final Cut Pro Newsletter. Click here to subscribe.
Updated: July, 2009 and May, 2010, to correct incorrect data rate entries.]

As we move from the world of standard-definition to high-def, understanding hard disk speeds and the data requirements of video formats can prevent a lot of problems.

Here's the key concept: hard disks are not infinite in speed, nor unlimited in storage.

The corollary to this rule is that dropped frame errors are most often caused by a hard disk that can't keep up with your video format.

During my recent seminars, I spent a lot of time explaining hard drive speeds and video format requirements. This is a summary of what I talked about.

HARD DRIVE SPEEDS

While hard drive speed, that is, the speed with which a hard drive transfers data between the disks and the computer, is impacted by how fast it rotates, a much greater controlling factor is how the hard drive attaches to your computer. We call this a drive's "data transfer rate."

Here are several things to keep in mind:

1. All FireWire drives are hubbed. This means that when both fast and slow devices are connected to the built-in ports of your computer, the slower devices (cameras and decks) slow down faster devices (hard drives).
   
   For instance, connecting both a FireWire 800 drive and a FireWire 400 drive to the same computer, slows the speed of the FireWire 800 device 35% - 40% when compared to the speed of the FireWire 800 device connected alone. This speed drop is true whether the drives loop through each other, or both are connected to two different ports on the same computer. This is NOT true if you have added a plug-in card to provide additional FireWire ports.
   
   
2. FireWire does not operate at its rated speed. While a FireWire 400 drive has the potential to transfer data at up to 50 MB/second, it doesn't. This is due to how the hard drive processes data internally using a FireWire bridge chip.
   
   
3. The more FireWire devices you add to your system, especially when you have more than five, the slower the data transfer rate is.
   
   
4. The more data you store on a hard drive the slower it goes. A drive is fastest when it is empty. When a drive is totally full, it neither plays back, nor records. I recommend keeping about 20% free space on your drive - though bar fights have broken out over what is the "best" proportion of free space to maintain.

While individual drives can vary, I've found that connection speeds generally group into the following speeds.

NOTE: By the way, if terms like RAID 0 or RAID 1 confuse you, read this short explanation of how RAIDS are defined.

With this as a background, let's see how we can apply that to video editing.

VIDEO FORMAT DATA TRANSFER REQUIREMENTS

Here's how you read the table below.

Format indicates the video format as indicated by Easy Setup.

Store One Hour indicates how much hard disk space it takes to store an hour of material in a particular format. This allows you to estimate what your total storage needs are based upon the amount of material shot. Remember, to add 20% for general storage overhead and free space.

Comp indicates how the codec compresses the video. GOP compression compresses pictures in groups, I-frame compression compresses individual images. GOP compression requires an additional conforming process prior to output, which increases the time it takes to output your sequence.

Dupe indicates those video formats that Final Cut duplicates the media from its source format (i.e. P2 or AVCHD) into its transcoded format (i.e. QuickTime or ProRes 422). As a general rule of thumb, double all storage requirements for these formats. While not totally accurate (for instance, if you don't transfer the footage into Final Cut Pro, no duplication occurs), its as reasonable a way to estimate file storage size as anything else.

Transfer Rate indicates how much data, on average per second, is required by that format. (The speed is measured in megabytes per second - MB/s.)

Keep in mind that a real-time dissolve doubles each of these transfer rates because during the dissolve two streams of video are playing. A four-camera multiclip increases each rate by four times because there are four streams of video playing.

Here's the key point: Assuming your hard drives are healthy, a dropped frame error means that you are trying to read or write more data than your hard drive will support. This table helps you to match the data rate of your video format with the speed of your hard drive, indicated in the table above.

Note: HD-CAM and HD-CAM SR create massively big files and the exact file sizes vary widely by frame rate and image size. In all cases, these formats should be edited using a RAID with a data transfer rate in excess of 200 MB / second. Faster is absolutely better.

NOTE: All AVC video formats are converted (transcoded) to ProRes422 during ingest into Final Cut Pro. So, while the AVC source video uses GOP compression, ProRes uses I-frame compression. Also, when the AVC footage is duplicated, the ProRes data rate and file size applies to the converted video.

ProRes is a variable bit-rate encoder, so file sizes will vary depending upon format, image size, and frame rate. The HD specs for ProRes are based on shooting 1080i/60 and taken from Apple's ProRes white paper. File sizes decrease for 720p files, or slower frame rates.

I found that understanding the relationship between hard disk speeds and video formats greatly simplified storage budgeting and solving dropped frame problems.

UPDATE - Nov. 17, 2008

Rich Roddman sent me the following note:

You have R3D files listed as not needing to create Pro Res files as they are imported into Final Cut. That is not quite true, while you can just drop the proxy file in the timeline to view or edit, if you use the Log & Transfer tool to import the R3D files they will transcode them into Pro Res (HQ). The 4K files are converted to 2K in in width, 3K import at 3K and 2K stays at 2K. Much like P2, it is not a fast process even with an 8 core tower.

Rob Naim adds:

There is a great widget from a UK company called Digital Heaven than can be helpful when working out video storage issues that you've discussed in your current newsletter.
Have a look at it here: http://www.digital-heaven.co.uk/videospace/

It doesn't do data rates but does do storage needed for length of video and its free.

Larry replies: Thanks for both your thoughts.

Larry Jordan is a post-production consultant and an Apple-Certified Trainer in Digital Media with over 25 years experience as producer, director and editor with network, local and corporate credits. Based in Los Angeles, he's a member of both the Directors Guild of America and the Producers Guild of America.

The information in this article is believed to be accurate at the time of publication. However, the author assumes no liability in case things go wrong. Please use your best judgment in applying these suggestions.

The opinions expressed in this article are solely those of the author. This newsletter has not been reviewed or sanctioned by Apple or any other third party. All trademarks are the property of their respective owners and are mentioned here for editorial purposes only.

Links to my website home page or this article are welcome and don't require prior permission.