It was not your normal day.

Before the doors to MacWorld opened, I had the chance to sit down with Gary Adcock, founder of Studio 37, who lives somewhere beyond the high-end of today’s market.

“So, Gary,” I asked, “is the world really going to 4K?”

“Absolutely,” he replied,

And we launched into a wonderful conversation about 4K images, quad-HD, monitors, codecs, camera formats, and an eclectic discussion of the challenges of working with $100,000 cameras and lenses.

I loved it. Gary is a gifted story-teller and I’ll write about our discussion once I understand it better. I left convinced that the march toward higher resolution was inevitable.

Then, the doors to MacWorld 2012 opened and the first booth I saw was i4Software extolling the virtues of editing video on an iPhone.

Talk about mental whiplash…!

Last night, I presented a demo on Final Cut Pro X at the San Francisco Apple Store to about 70 people. Over half the people there had not edited video before. And less than 10% (because I asked for a show of hands) had ever been paid to edit video.

This got me reflecting on who is a professional and the future of video editing.

If video is your hobby, you can view the tumult our industry is going through with a bemused smile, as you watch an industry reinvent itself from the inside out.

However, if your livelihood depends upon trying to figure out what the heck is going on, that bemused smile turns into a terrified grin because all this industry change makes you feel like you are riding in a little red wagon while traveling at breakneck speed down a foggy mountain road.

Terrifying is a good way to describe today’s professional video environment.

MACWORLD

MacWorld made an interesting shift this year. The last time I attended, three years ago, it was all iPod covers and iPhone cases. It was in danger of become Walmart.

This year, I felt like I was in Best Buy. Something intriguing lurked around each corner. There were the obligatory covers and cases – including, yes, an iPhone case that doubled as a bottle opener – but there were interesting vendors hidden in plain sight.

MacWorld showcased the world of mobile devices. I was reminded of the photographer’s credo: “The camera in your hand is better than the camera you left at home.”

As I walked the show floor I saw crowds in the i4Software booth learning how to shoot and edit video on an iPhone using their Video Camera app.

Cinefy showed its app for video editing on the iPhone.

Blue showcased their family of professional mics that plugged directly into an iPhone or iPad.

MicW Audio highlighted some amazing small mics – both lavaliere and condenser that plugged directly into an iPhone or iPad.

iPro Lens presented its wide angle and fish-eye lenses for the iPhone.

Audio Engine devoted their booth to wirelessly streaming 24-bit audio from a computer to a personal or professional sound system.

And, probably my favorite booth was IndiSystem. This was run by a long-time grip with an infectious smile, who, in addition to a wealth of ideas, has access to 3D modeling tools, aluminum extruders, precision lathes, and a plastic model shop. He runs a hardware-creation toy store! Out of this, he created iSupport: incredible camera accessories – like jibs, camera sliders, and grips – for the iPhone. Perfect tools for photographing miniatures!

After walking MacWorld, I was reminded of Wayne Gretzke’s famous quote: “A good hockey player plays where the puck is. A great hockey player plays where the puck is going to be.”

THE IMPLICATIONS

The high-end of the market is not going away; the world of Gary Adcock attests to that.

But the tools are changing. And I think MacWorld is showing us where the puck is going to be. Not for the high-end, perhaps, but for the rest of us.

Last night, at the SuperMeet, Alex Buono, the head of the Film Unit for Saturday Night Live, explained that he gets a script on Thursday, shoots on Friday, and airs on Saturday. Speed and workflow are everything to him. He shoots on a variety of cameras, with emphasis on the Canon 5D, 7D, and 1D. (And the new Canon C300.)

With budgets continually shrinking, camera technology morphing on a daily basis, and deadlines ridiculously short, I think we need to redefine “professional.”

There is a time and place for “heavy iron.” Some shows require all the equipment we can throw at them… (I was told recently that the SuperBowl will have 43 robotically controlled cameras this year.)

But all too often we define ourselves in terms of the tools we use. As soon as we do that, however, we limit our opportunities.

At the SuperMeet lounge last night, I was hanging around the Peachpit Press booth when an editor walked up and told me that when he first saw a demo of Adobe Premiere in 1997, he was busy editing linear tape and laughed when Adobe showed how to do a DVE move in software.

“Shoot,” he said, “with our equipment we pressed a few buttons on the switcher, recorded it live, and we were done.”

“Yes,” I said, “but it took you three-quarters of a million dollars of equipment to do it.”

“Ah,” he replied, laughing, “but it wasn’t my money!”

Today, it’s our money. And our time. And this affects our ability to feed our families. Technology this year is in a whirlwind of evolution, constrained only by the sluggishness of the economy.

Philip Hodgetts presented three ideas last night at the SuperMeet that can help us grow our business. He suggested we:

1. Mind our own business. That is, figure out what business we are actually in and why someone else would want to do business with you.
2. Give yourself an unfair advantage. That is, network with the people around you and stay informed and up-to-date.
3. Own a piece of the action. That is, create something that you own and can sell yourself.

Just as the DV revolution undermined film, the new mobile revolution threatens a lot of what we used to believe in. But it also provides us a lot of opportunities if we recognize that people coming new to video on mobile devices can benefit from what we know.

Forcing the story to fit the technology you know is a trap. It will work for the short-term, but longer term you get pigeon-holed in a backwater with no clients. Don’t let old habits prevent you from learning new techniques.

All of us are story-tellers at heart. Focus on the story and your creative approach to it. When you let the story guide you to the technology it needs to be told, you will always be in demand.

People pay money to hear good stories well told. They could care less about the gear you use to tell them.

Let me know what you think,

Larry

[Updated 12/13 with clarification on Promise and more info on the incremental cost.]

I needed to buy more storage for my editing system – as if needing more storage for video editing is a surprise or something…

Anyway, I decided that since one of my editing systems supports Thunderbolt, I would buy a Thunderbolt RAID. Except the only ones currently available seemed to be from Promise Technology. Promise has been making and selling RAIDS to the video editing market for more than a decade. As Andrew pointed out in the comments: “Promise succeeded Apple’s Xserve RAID with their vTrak RAID.” As you can see from the comments below, lots of people like them a lot.

However, I decided to see what else was on the market.

And, hmmm, there’s not a whole lot else on the market. So, I decided to do some research and had a number of off-the-record conversations with different hard drive vendors to figure out why – when we are almost a year after the launch – there are so few Thunderbolt units available. Here’s what I learned.

TOO EARLY

Basically, Thunderbolt was released to the market too early. Thunderbolt requires not just one chip from Intel, but a master chip and a variety of supporting chips to be able to handle all the signal processing.

The first problem was that there was a shortage of both the master chip and the supporting chips.

Next, all these chips needed to be integrated onto a new IC board. Anytime hardware engineering is involved – as opposed to making software changes – a minimum of six months of engineering and manufacturing time is involved.

HARDWARE TAKES SPACE

In fact, this hardware board with all these chips takes so much space that it won’t fit inside a standard enclosure – in general, it requires about another 1/2 in height inside the RAID unit.

This is space that, in smaller units, just doesn’t exist. This means that many manufacturers need to retool their enclosures to support the new protocol hardware.

APPLE AND INTEL NEED TO HELP

Once the hardware has been figured out, the SDK (software development kit) necessary to support Thunderbolt was several months late coming to market – most drive vendors didn’t get them until early summer. In fact, the specs were published several times – and kept changing with each iteration. This is good, in that it allowed the protocol to improve, but every change caused delays in creating software needed to support the protocol.

In addition, Thunderbolt requires significant support from Apple and Intel engineering to explain how the new system works so that RAID manufacturers can get their systems to interconnect properly with this new protocol.

And, because Apple and Intel do not have unlimited engineering support staff, not all RAID vendors were able to access the engineering help they needed in a timely fashion. So, some vendors are farther along in their development than others are.

There’s also, apparently, a political issue. I was told that some vendors were given market exclusivity for a year after the announcement – an exclusive period that expires just before NAB next year in April.

WHAT’S IN THE FUTURE?

There are many elements of potential good news here. First, every drive manufacturer that I spoke with is extremely interested in developing for Thunderbolt – so, while the devices may still take a while longer to get here, when they do arrive, they will arrive in quantity.

Second, Intel has already announced – at least to the drive vendors – a second generation bridge chip that includes both the Thunderbolt master chip and all the supporting chips on a single chip. This will vastly reduce both the amount of engineering that is necessary to support the new protocol and the amount of space it requires. From what I have been told, this new chip will support both FireWire and Thunderbolt on the same chip.

However, we probably won’t see these second-generation chips in quantity until late in the first quarter next year.

Intel is also working on a third-generation chip which supports data transfer over fiber-optic cable, but that chip probably won’t show up until 2013.

THERE’S ALSO THE ISSUE OF COST

As one of my contacts told me, all new technologies are more expensive than current technologies. This was true with USB, FireWire, even Blu-ray players; and is true with Thunderbolt. Costs don’t fall until enough people adopt the technology to allow increasing volume to drive down costs.

“The cost of Thunderbolt will remain higher than alternatives until native implementations are made into motherboards and the external devices and if and when this happens we could see the price drop very quickly.”

In this conversation, his feeling was not that the technology was announced too soon, but that the implementation of this new technology takes a while to figure out.

SUMMARY

So, what’s the take-away? Thunderbolt is coming – but the majority of units probably won’t arrive in quantity until the second quarter next year. By a year from now, we should have a wide variety of Thunderbolt devices to choose from.

Initially, most Thunderbolt devices will probably support RAID level 0 or 1 – which provide speed or redundancy, but not both. RAID level 5 units – which are both faster and more secure – will take longer to get to the market in quantity.

For now, though, we need to wait.

Let me know what you think.

Larry

‘TWAS THE NIGHT BEFORE SUPERMEET

I’m writing this at 9:50 PM from my hotel room at the 2011 NAB Show (National Association of Broadcasters) in Las Vegas, Nevada.

In less than 24 hours, we will know what the next version of Final Cut Pro will look like. If we are lucky, we may also learn the future of other software in the suite – Apple certainly has enough stage time to tell us.

What I am intrigued by is not what Final Cut Pro looks like – unlike most of the world, I’ve already seen it. What I am curious about is how much it has changed since Apple first revealed it.

When Apple presented the software to the group, they asked us to provide feedback. I provided a great deal, and I’m sure others did as well. I’m curious as to how Apple responded to what they heard.

I will be at the SuperMeet tomorrow, listening and taking notes. And, I’ll do a second NAB blog posting after the meeting about what I learned. But give me a few hours — initial reactions are not always accurate. I want to hear what Apple says, then allow myself some time to think about it.

Nonetheless, I’ve been reflecting about tomorrow night’s blog for almost two months. I’m looking forward to sharing my thoughts and – even more importantly – getting your reactions.

Tomorrow night is going to be … a jaw-dropper!

- – -

NABSHOWBUZZ.COM — LIVE FROM NAB!

My podcast – DigitalProductionBuzz.com – is at NAB, recording interviews and creating shows. Our first dozen interviews are online – we are posting interviews as fast as we get them recorded and edited. Then, we gather the highlights from our interviews and post a new BuZZ every night.

If you want to hear the newsmakers themselves, explaining what they are doing – in-depth interviews you can listen to quickly — take a listen to our NAB website: www.nabshowbuzz.com.

We haven’t done anything like this before and we are very interested in your reaction!

We hope to create about 20 interviews a day for the next three days, then create NAB Show Specials for the next week.

You’ll find everything posted here: www.nabshowbuzz.com.

- – -

LARRY IS A TWIT!

Well, ah, maybe that’s overstating things a bit. Let’s not get carried away.

Still, Leo Laporte invited me to be his lead guest at the start of his NAB coverage this morning on TWIT.TV. I was on for 15 minutes.

Plus, I’m invited back for an hour-long discussion on media this Wednesday morning on MacBreak Weekly. It will start sometime between 10:45 AM and 11:00 AM – Las Vegas time – this Wednesday.

See you then!

P.S. Thanks, John Stealey, for the screen-shot!

- – -

ARCHIVING UPDATE

I spoke with the president of The Tolis Group yesterday during our setup at NAB and discovered that they have an LTO-tape-based archiving solution, called BRU, for less than $4,000.

I told him that I could not find it easily on their website, at which point he and his webmaster and I are going to meet later this week to discuss this further.

However, I promised in my recent blog that if I discovered anyone with a reasonably-priced, LTO-based archiving solution that works on the Mac, I would mention it here.

BRU is highly-regarded and I’ve received many positive reviews of their gear from readers. Feel free to check it out for yourself.

The Tolis Group: www.tolisgroup.com

- – -

ADOBE’S BIG ANNOUNCEMENT

Adobe announced new upgrades today – they are releasing CS 5.5. But what really caught my attention was their announcement that they were now allowing you to rent their software.

Suspecting this was yet another ploy to get our data in “the cloud,” I spoke with Scott Morris, senior marketing director for Adobe.

What he told me was that when you rent software, your data stays on your system. “This is not software as a service,” he said. Instead, you are buying the full-featured software and installing it as usual. However, when you activate the program, it contacts an Adobe server to find out if this is a purchase or a rental.

If it is a rental, then each time the application starts, it pings the Adobe server to see if the rental is still valid. If it is, the program launches. If it isn’t, the program doesn’t launch.

In all cases your data is safe. If you send your files to someone who has the full program, the files can be opened, modified, and saved. If you stop renting for a few months, then start renting again, your files open exactly as they were last saved.

To me, this is a very elegant solution for people that need the power that Adobe software provides, but can’t justify the upfront cost of the purchase price. What I REALLY like is that our data remains “our data;” it is not forced to be stored somewhere “on the cloud.”

Weblink: Adobe Systems – www.adobe.com

For more details, listen to the Scott Morris interview.

- – -

AVID’S BIG ANNOUNCEMENT

Avid didn’t announce any new products – except for a technology demo of much tighter 3D integration within Media Composer which makes the process of handling 3D materials much more seamless.

However, what did catch my attention was that Avid announced a cross-grade for all Final Cut users to the latest version of Media Composer for only $999.

For more details, listen to the Maurice Patel interview.

- – -

HD IS THE NEW SD

It seems the video world is moving to higher resolutions.

Both Blackmagic Design and AJA announced support for 4K resolution playout. RED showed their Epic camera shooting 5K images. And Sony announced a new F65 camera that shoots 8K.

Clearly, the lesson is buy stock in companies that make or sell storage. These file sizes are going to be massive… And when you add stereo 3D, all your file sizes double!

Just as a rough guide, a 4K file is 4 TIMES bigger than a 1080p file. An 8K file is four TIMES bigger than a 4K file.

For more details, listen to the Nick Rashby interview.

- – -

THUNDERBOLT IS STARTING TO APPEAR

We are seeing previews of Thunderbolt support from a variety of vendors — AJA and Blackmagic Design for image capture and playback; and Promise, G-Technology, and LaCie for hard drives — there are probably others that I didn’t make note of.

Thunderbolt is in the very early stages of adoption – the real roll-out will occur this summer. But it promises to be everything we could want for speeding up data transfers and handling the massive file sizes we are about to experience.

Sigh… Just watch. Someday, not that far in the future, we’re gonna say: “Boy! I wish there was something seriously faster than Thunderbolt.” We are never satisfied…

- – -

More later, I’ll write again after Supermeet.

Larry

A few days ago, I wrote about my frustrations in finding archiving solutions that work on the Mac for protecting our video assets. (You can read that blog here — some of the comments are VERY interesting!)

The day after that posted, I had a long phone conversation with the executive team at Cache-A (which I can tell you about after NAB), plus a long email from HP.

Since I spent much of my post complaining about the HP gear, I wanted to share their response with you. This was written by Laura Loredo, World Wide Product Marketing Manager for HP-Nearline.

Larry,

I would like to apologize for your frustration trying to install the HP LTO-5 tape drive with LTFS.

My understanding is that most of your frustration came from trying to do a firmware upgrade. Unfortunately an old drive was sent to you that didn’t have the correct code. I can assure you that current shipping drives come with the correct firmware and customers do not have to spend 2 weeks of their time, and go through 30 steps, to get this done. I am really sorry that such a drive was sent to you and wasted your time.

By avoiding the firmware upgrade, the software install instructions would have been like this:

1) Install MacFuse (download from http://code.google.com/p/macfuse/)

2) Install two packages (download from ttp://hp.com/go/ltfs)

3) create a new directory in Finder. I’ll call it ‘mydirectory’.

4) Format and mount your tape (much like you would for a new disk). This requires two commands from a terminal window. The first command formats the drive:

mkltfs -d 0

The second mounts the drive

ltfs mydirectory [for "mydirectory" substitute the name of the folder you created in step 3]

Now you are ready to drop your asset directory (or whatever) onto the ltfs icon.

I realize that this still requires a certain level of IT expertise.

In your blog you made some very good recommendations on what you see as a great tape based archiving solution:

* Can be installed easily on the Mac OS

We are learning about mac users and we need some real improvements in our whole set up process. This is why your feedback is much appreciated.

* Is willing to list the product and its price on their website

Go to: www.hp.com/go/LTO you can find products and the prices for the LTO-5 tape drives.

* Has an announced shipping date

HP LTO-5 tape drives have been shipping since March 2010, LTFS since June 2010 - www.hp.com/go/ltfs

* Has an announced price of between $2,000 and $4,000

HP LTO-5 Ultrium 3000 tape drives can be bought at less than $3,000.

http://www.google.co.uk/products/catalog?q=hp+lto-5+ultrium+3000+external&hl=en&um=1&ie=UTF-8&cid=6433306862412258802&sa=X&ei=q1OHTeWsJdS3twf8j_TGBA&ved=0CFkQ8wIwAQ#

* Can connect to Macs via Thunderbolt or Firewire

This will be a major R&D project, but something we will be taking into account.

* Supports the LTO-5 standard, which makes file backup/restore VERY easy

* And includes archiving software that can be run by mere mortals and not IT gods…

With LTFS there is no need for special archiving software, you can drag and drop onto the tape device, just like you do when you use a disk for archiving.

This is the first time our products connect into the Mac environment, so much work is required in order to improve the overall customer experience.

If you would like to have a chat about any of these issues please let me know and we can arrange for a call.

Thank you very much for taking the time to look into this new technology.

Regards,

Laura Loredo

World Wide Product Marketing Manager

I want to thank Laura for taking the time to read and reply to this issue. Unless we solve this problem soon, archiving video assets is only going to get worse. I look forward to tape backup vendors providing products that meet the needs, and price-points, of video producers around the world.

The reason HP, and others, need to seriously think about how their tape drives connect to computers is that only the MacPro tower supports mini-SAS connections. And the majority of video editors are working with laptops and iMacs which don’t support mini-SAS. In other words, mini-SAS prevents HP from selling their products to potentially millions of editors that need to archive their assets.

Yes, it is an R&D effort. On the other hand, this effort opens up a vast, new market.

One last note, the LTFS File System allows users to simply drag files on top of the icon for their tape backup system to copy a file. However, as you would expect, tape drives are much slower than hard disks, so files take a while to transfer or restore. Also, if you know where a file is located (folder and path) you can easily find it using LTFS. However, at this point, LTFS is not yet integrated with Spotlight, so, for people not familiar with Unix command line search syntax, its search capabilities are less robust than we would expect.

Still, LTFS is a HUGE step in the right direction.

As always, let me know what you think.

Larry

It shouldn’t be this difficult.

NOTE: This blog grew out of my webinar last week on how to archive Final Cut Pro projects. If you are interested in archiving, click here to learn more.

The process of creating films and videos generates LOTS of data. From getting the footage shot, through the editing process, we are dealing with dozens, if not hundreds, of gigabytes of data.

And we solve this problem by buying ever more and ever larger hard disks and RAIDS. So far, a trifle expensive, but not really a problem.

However, once a project is done, we can’t just throw all these files away. The images and interviews are irreplaceable and have a value long after a project is complete.

We need to save these for the long-term, a process called “archiving.” And here, we are caught between a rock and a hard place.

In the camera industry’s head-long rush to convert to cameras that shoot tapeless video, they seem to have overlooked the fact that we need to save these video files after editing is complete.

I have spoken with executives at Sony, Canon, JVC, Panasonic, and the rest of the camera industry, not one offers any kind of long-term archiving solution.

NOTE: I define archiving as the ability to save all video project and source files for a period of 10 – 25 years. Backup is what we do to protect ourselves during a project from the risk of data loss.

One camera vendor told me, dismissively, that tapeless media should just be handled as part of an “IT workflow.” Which is great if you have an IT department and the budget to support it. But, I looked in my checkbook, and the ability to write a $100,000 check for an automated tape archive library is not possible this week.

For me, the long-term solution to archiving tapeless video is LTO-tape. This has the flexibility, storage, and longevity to meet all our archiving needs. But, while this seems obvious to me, manufacturers of LTO gear seem to be doing everything they can to avoid selling us the products we need.

HOW YOU CONNECT IS CRITICAL

Currently, most LTO-tape drives connect via FibreChannel, SAS, or mini-SAS. Which makes sense if you are selling to the corporate market, where every server supports plug-in cards, speed is everything, and budgets are always expressed in units of ten thousand dollars.

But that isn’t the reality of video editing.

More than 70% of all video editors work in companies of five employees or less. Generally, the video editor provides all their own tech support, without access to an IT department.

Additionally, I would guess that close to half of all video editing is done on a laptop with a FireWire-attached RAID or hard disk. Portability is more important to a producer shooting a documentary than sheer horsepower.

Or, take an independent filmmaker on a budget. They have a collection of multi-terabyte FireWire drives, containing all the elements of their film, scattered across their desktop while editing on an iMac to take advantage of the large screen. Saving money yet getting a large screen is more important than buying the fastest possible computer.

I don’t have any research here, but I would guess that two-thirds of video editors are working on a system that isn’t a MacPro. And every one of those editors is juggling terabytes of data and praying they can figure out a way to store it for the long-term.

NOTE: The newly announced Thunderbolt provides a very fast data bus that tape vendors can tap into that would give them access to every new Mac that Apple ships. However, it doesn’t solve the archiving problems of the millions of Mac users that already have gear.

SPEED ISN’T EVERYTHING

When backing up corporate servers, speed is important, because many of the files on a server change daily.

But this isn’t typical for video. Video and film shoot a lot of files in a very short period of time, then spend a while — sometimes a long while — editing them. In other words, once the initial backup is complete, the only files that change are relatively small project files, not the massive media files.

This means that a workflow that features easy-to-connect and easy-to-use is far superior to one that emphasizes blinding speed. We are making archives here to last for decades. Taking an extra few hours to back everything up is a non-issue.

My large media files don’t change that often. My small project files change daily. And I only need to archive my project once – when the entire project is complete.

A FireWire-attached LTO drive would not be a speed demon, but I’d bet you’d never be able to keep them in stock. Security of my assets is FAR more important than transfer speed.

THE SIMPLE TASK OF GETTING AN HP LTO-5 TAPE DRIVE TO WORK ON A MAC

Here’s a specific example.

HP is a leading manufacturer of LTO drives. About a month ago, I contacted them for a loaner unit to review so that I could use it to illustrate a webinar I just did on how to archive Final Cut Pro projects.

HP agreed and shipped me an HP StorageWorks Ultrium 3000 SAS LTO-5 tape drive. It uses an ATTO ExpressSAS H680 interface card, which, because it is a plug-in PCIe card, only works with a MacPro.

The drive arrived a month ago and I am STILL trying to figure out how to connect it to my MacPro!

NOTE: Installation on Windows and Linux is close to click-one-button easy. Not so on the Mac.

I teamed up with two software engineers to help me get this to work. The first took one look at the installation instructions and gave up.

The second spent many hours over the last four weeks trying to configure the system. Here’s what he learned:

1) The HP unit didn’t come with firmware compatible with the software
2) Updating the firmware is seriously non-trivial.
3) It’s loud. The drive is about two feet from his ear and measures 57dB in operation.

Here is a BRIEF summary of the steps necessary to install the HP StorageWorks Ultrium 3000 SAS unit on a MacPro. (Keep in mind this required a Unix-trained hardware engineer to accomplish — this is only a summary, the actual installation process took more than 30 steps and two weeks to figure out!)

1. Install the ATTO SAS card into PCIe slot #1 of the MacPro.

2. Go to Google and download MacFuse, which is special utility software that allows the drive to operate on a Mac)

3. Install MacFuse

4. Go to HP’s website — www.hp.com/go/ltfs — to download the latest drive firmware.
HINT: To get this to work, you need to know the version you need. To make this easier, HP doesn’t tell you which version that is.

5. Download and install ICUFramework.pkg

6. Download and install LinearTapeFileSystem.pkg

7. Format the tape cartridge – this is LOUD! 57 dB.
HINT: Formatting a tape can only be done from the Terminal window. You ARE familiar with entering Unix commands via Terminal, aren’t you?)

8. Update Firmware on your drive.
HINT: This involves a return to the Terminal.

9. Prior to installation move the downloaded firmware into the /VAR/ROOT/ folder.
HINT: You will be glad to know that this folder is invisible on the Mac.

10. Make yourself a SuperUser in order to install the software.
HINT: You should be familiar with how to become a root user.

And this is only the START! There are still about 20 steps left to go involving terminal commands, more downloads, accessing invisible folders, and, if I read the fine print correctly, sacrificing a small goat.

This is ridiculous!

THERE ARE OTHER SOLUTIONS

Yes, there are existing Mac solutions out there.

The Tolis Group — www.tolisgroup.com — has BRU Producer, a hardware/software system for archiving that runs on Mac, Windows, and Linux systems. I’ve spent the last 30 minutes on their website, I still have no idea what products I need or how they are priced. The purchase they product, they refer me to a dealer. When I go to a dealer’s site, for information or pricing, they refer me back to the Tolis Group site.

The Tolis Group has hardware bundles, software bundles, and add-ons, which, apparently, are neither hardware nor software, but assume you already have the hardware you need, whatever that is.

Cache-A — www.cache-a.com — is another company that makes archiving solutions for the Mac. They only have two products, which makes deciding a lot simpler. However, nowhere on their website do they mention a price.

OK, that means I need to buy it from a dealer. So, I went to four of the dealers listed on their website:

* AbleCineTech — doesn’t list Cache-A in their vendor list, and Cache-A can’t be found using a search
* Digital Video Group — doesn’t list the products they sell and their site is not searchable
* B & H Studios — Cache-A not listed, site not searchable
* TekServe — Cache-A can’t be found using a search

And these are not small dealers here. It seems I can only buy Cache-A through dealers and the dealers are too embarrassed to mention they carry the product!

Is selling archiving solutions REALLY that difficult??? Is this much secrecy about your pricing, your products, or your service necessary?

Apparently, archiving IS rocket science!

CONCLUSION

If you are an manufacturer that has an LTO-tape-based archiving solution that:

* Can be installed easily on the Mac OS
* Is willing to list the product and its price on their website
* Has an announced shipping date
* Has an announced price of between $2,000 and $4,000
* Can connect to Macs via Thunderbolt or Firewire
* Supports the LTO-5 standard, which makes file backup/restore VERY easy
* And includes archiving software that can be run by mere mortals and not IT gods…

Contact me. I will happily share the news of your product to the MILLIONS of video editors that are desperately searching for ways to safeguard their media.

Otherwise, we run the very real possibility that every movie we are watching today will slowly disappear over the next few years as our current archive solutions fail.

Larry

[ The following article was taken from my FREE Final Cut Studio newsletter for February.
Subscribe here. ]

The release of any new hardware is exciting; as we saw last week with the release of the new MacBook Pros.

However, the stunning news is the new Thunderbolt (née Light Peak) I/O technology. The ONLY reason this exists is because of the Pro market. The greatest novelist, or spreadsheet junky, in the world has absolutely no reason to get excited about this. FireWire meets their needs perfectly.

But for people pushing pixels (or waveforms) this is a birthday present come early.

WHAT IS THUNDERBOLT?

Thunderbolt is a very, VERY fast I/O (input/output) communications protocol (10 Gbps or 1.25 GB/second) that supports both hard drives and monitors. This is faster than anything else on the market.

Apple’s Thunderbolt white paper states:

For time sensitive data, such as video and audio during creation and playback, data transfer can be critical to the success of the work. Thunderbolt technology was specifically designed with video and audio applications in mind with inherently low latency and highly accurate time synchronization capabilities.

Currently, Thunderbolt is supported only on MacBook Pros. However, I have every expectation that this connection protocol will be added to all future Mac systems. (I don’t see it being added to IOS devices simply because they don’t hold enough data to require these kinds of transfer speeds. Well, at least not yet.)

Today, Thunderbolt connects two devices using copper wires, but the protocol has already been developed to support optical fiber as well, which provides longer cable runs and faster speeds; though without the on-board 10 watts of power.

The great news is that, unlike FireWire and USB, the protocol does not slow down when you have multiple devices attached to it.

Additional features include:

• Dual-channel, each 10 Gbps
• Bi-directional, equally fast in both directions
• Dual protocol, PCI Express for hard disks and DisplayPort for monitors
• Daisy-chain-able, supports up to six devices per channel
• Woks with both copper and fiber cabling, fiber allows longer cable runs
• Low latency, very, VERY short delays in transferring signals
• Delivers up to 10 watts of power over copper wiring, but not fiber, to support bus-powered devices. (Though this is not enough for most hard disks)

In short, this is some amazing technology!

For those that want an overview, here’s the relevant page from Apple on the technology: http://www.apple.com/thunderbolt/

For those that want more technical details, here’s Intel’s website on the new protocol: http://www.intel.com/technology/io/thunderbolt/index.htm

Here’s an additional analysis from Storage Newsletter: http://www.storagenewsletter.com/news/connection/analysis-intel-10gbs-thunderbolt

HOW FAST IS IT?

Remember the first time you saw Han Solo go into warp drive in Star Wars? It’s THAT fast!

It is:

• 10 gigaBITS per second (10 Gbps)
• 1.25 gigaBYTES per second (1.25 GB/s)
• More than twelve times faster than FireWire 800
• More than twenty times faster than USB 2
• Faster than USB 3, mini-SAS, FibreChannel, and all flavors of SCSI.

It is dazzlingly fast.

(This chart was published on Apple’s website.) PLUS, the spec calls for speeds ten times faster – up to 100 Gbps – in the near future. (No, not next week.)

Keep in mind that the only way you can take advantage of this new system is using a RAID. A single hard drive, no matter how large, does not begin to provide the speed offered by this new protocol. To take advantage of all the speed, you’ll need to invest in a RAID system.

UPDATE – Mar. 3, 2011

Adam Lloyd Connell sent me the following link to an EndGadget demo using Final Cut Pro.

http://www.youtube.com/watch?v=CCz_c_rDAXw

WHAT SUPPORTS THUNDERBOLT NOW?

Apple’s three latest MacBook Pros.

That’s it. No other hardware is currently shipping that supports the protocol.

On the other hand, until Apple ships a device containing this protocol, there is no incentive for any hardware manufacturer to create devices to support a protocol that doesn’t exist. First, Apple needs to ship something, then, the rest of the technology world can start to support it.

As always, this will take some time before we see actual products ship.

WHAT WILL SUPPORT THUNDERBOLT IN THE FUTURE?

Both Promise and LaCie have announced new hard drive systems that will support this protocol and both will be shipping in a few months.

I also sent emails to friends in the hard drive industry and every single one of them is frantically studying the new protocol so they can add it to their product line.

My expectation is that we will see Thunderbolt support on hard drives – more specifically RAIDs – sooner than on monitors. With nothing from any vendor before NAB in April. I expect most shipments to start in the early summer.

As one hardware vendor wrote:

Thunderbolt is THE hot topic of discussion everywhere. What happened is that when it was being developed, Apple went to Blackmagic Design, AJA, Promise, Western Digital, LaCie & even CalDigit to see if they’d be interested in being 1st onboard the tech, so it’ll be interesting to see what AJA, BMD & the rest will come out with. We already know Promise has a 4 bay and a 6 bay TB product line called "Pegasus" which doesn’t ship till April and LaCie has a new TB 2 bay, that I know of.

I don’t think it’ll kill FireWire anytime soon & our mini-SAS 8 bay product is still selling very solidly and will continue to do so at nearly 1000 MB/s with the ATTO R680 card.

CAN THUNDERBOLT BE RETROFIT ON EXISTING COMPUTERS OR STORAGE?

No.

Based on what I’ve learned so far, support for Thunderbolt can not be retrofit into current RAID technology. It requires a new chipset from Intel on both the computer and RAID to support the protocol. Also, the chipset does not look like it can be added via a PCIe card into a MacPro.

WHAT DOES THUNDERBOLT MEAN FOR THE FUTURE?

Thunderbolt is absolutely, and only, designed to meet the needs of the Pro market. No consumer needs speeds this fast – they can use it, they can appreciate it, but they don’t NEED it. Pros do.

To me, this is a strong indication that Apple has not forgotten the needs of Pro users. Which is a very good thing — for all of us.

Larry

The following is taken from my free, monthly Final Cut Studio Newsletter. Subscribe here.

I’ve had the pleasure of testing a variety of truly high-speed RAIDs over the years, and I’ve spent a lot of time worrying about whether I was getting the absolute maximum speed from each unit.

However, as I was investigating how Final Cut Pro handles multiclip editing, (see my earlier blog) it struck me that, after a certain point, the speed of your storage doesn’t really matter. Which means that we need to pay attention to more than just the raw speed of our storage systems.

Let me back up for minute, though, and explain where maximizing your speed is essential.

First, how you connect your storage to your computer determines transfer speed more than anything else. Take a look at the table below. These were measured about a year ago using the same computer, same hard disk and same test software (AJA System Test).

The key point isn’t the specific numbers, but the impact that your connection protocol has on transfer speed. You could have the world’s fastest hard disk, yet, if you connect it via USB, all that speed is wasted.

Second, single hard drives, while by far the most popular– and cheapest — method for storing video data for editing, suffer from two significant limitations. They can’t protect your data if the drive fails, and they have a maximum transfer rate of less than 100 MB/second.

This speed is fast enough for most single camera editing, but not fast enough for any significant multicam editing.

NOTE: Regardless of how you set the RT Menu in the Timeline, editing multiclips requires the full data stream from the hard drive for each angle in your multiclip. The RT Menu only reduces the strain on your CPU, not your storage.

For anyone using single drives as their main storage, speed is everything and you need to make sure you are connecting your hard drive using the fastest protocol it supports.

But the situation isn’t necessarily the same for RAIDs. Different factors come into play.

IMPORTANT NOTE: This article assumes you are directly connecting your storage to your computer. If you are connecting storage to a server, then speed is critical because the RAID needs to support more than one user.

If you connect your RAID via FireWire 800 (you should not even THINK about connecting a RAID via FireWire 400!) the maximum speed you can get from your RAID is limited by the FireWire bus to about 85 MB/sec. Even if you have a blazingly fast RAID, FireWire will limit the speed.

If you connect your RAID using iSCSI, your speed will be limited by the Gigabit Ethernet protocol to about 105 MB/second. This is fast enough for many single-stream video formats, but can be limiting for multiclip editing. Personally, I like iSCSI for its flexibility allowing me to use this same device on laptops and iMacs, as well as towers. iSCSI is faster than FireWire, but no speed demon.

It isn’t until we get into eSATA, Mini-SAS, or Fibre-channel that the connection protocol starts to match the speed the RAID is capable of sustaining.

At this point we need to be careful, lest we fall into a marketing trap; which is what got me started on this whole musing in the first place.

THINKING ABOUT RAIDS

If you are copying files, the speed of the copy is dependent upon your slowest device. So, if you have a blindingly fast RAID copying to a single drive, the speed of the copy is determined by the single drive, not the RAID.

Render files, even renders of multiple layers of video, play at the same rate as a single stream of the video in the sequence you are editing because the process of rendering creates a single video clip integrating all those different streams. That’s the magic of a render file — its the same size and speed as the video format of your sequence.

The speed of rendering is determined by the speed of your processor, not your storage. Render playback speed is the same as a single video clip.

In fact, once you have a hard disk with sufficient speed to play back two simultaneous streams of your video, your hard disk is "fast enough" for almost all editing, with one exception: multiclips.

A multiclip requires a storage system fast enough to play every video clip (also called a "stream") in the multiclip at full speed. So, depending upon how many streams and which format, you will quickly exceed the capability of a single drive, and start to tax a RAID.

However, there is an upper limit to the speeds you actually need; which is what I discovered when testing multi-clips.

Let’s do some simple math.

Final Cut Pro supports up to 16 simultaneous streams in a single multiclip. (You can link over 100 clips, but they don’t all play at the same time, they are simply tracked.)

The highest-quality, and largest, HD video files we can create are 1080p square pixel ProRes 422 HQ files.

NOTE: Yes, HDCAM SR files are larger, but transcoding them to ProRes 422 HQ is visually lossless and reduces files sizes close to 90%. You would use HDCAM SR files for final mastering, not for rough editing using multiclips.

For most transcoding I prefer ProRes 422. So, let’s do examples with both.

• A 16-stream ProRes 422 multiclip requires a total data rate of about 290 MB/sec.
  (16 * 18.1 MB/sec = 289.6 MB/sec.)
• A 16-stream ProRes 422 HQ multiclip requires a total data rate of about 425 MB/sec.
  (16 * 26.5 MB/sec – 424 MB/sec.)

In other words, the MAXIMUM speed you would use from any RAID for a fully-loaded multiclip is less than 450 MB/second. And, more practically, you would probably need less than 300 MB/second.

Granted, this is far faster than a single drive can support; or even a three drive internal RAID in a MacPro. However, there are many, many RAIDs on the market that far exceed these speeds.

Which means that you need to pay attention to more than maximum speed.

Here are other things to consider in deciding which RAID to buy:

1. How noisy is it?
2. How far can the RAID be placed from your computer?
3. How much total storage does it have?
4. How low is the seek time?
5. What’s the experience of the manufacturer?
6. What’s the duration of the warranty?
7. How long does it take to get a replacement drive?
8.
Does it require any special software to run?
9.
Does your computer provide an ability to connect to it?
10. What does it cost?

The older I get, the more noise matters to me. The old XRAID sounded like a jet taking off. It was awful. If companies expect me to connect a RAID to my computer in an edit suite, they need to make these units are REALLY quiet.

A while back, a RAID company sent me a unit to test (this was not AccuSys) and the interface cables were 18" long. I don’t have enough desk space as it is. Short cables are useless. I use RAIDs for their speed and storage, not because I like staring at them on my desktop.

Video devours storage. I will take a slower RAID with more storage than a faster RAID with less – assuming both are the same price.

Seek time is how long it takes the heads to get to a specific piece of data on the hard disk. With multiclips, low seek times are essential, since the RAID essentially needs to play all these files simultaneously. This means the drive heads need to move VERY quickly between all these files so that frames don’t drop.

Since there is no way I can benefit from a RAID that goes faster than 500 MB/sec – multiclips in Final Cut don’t need that much speed – I don’t see any reason to pay extra for speed I’m not using.

There’s an analogy that applies here. Car speedometers routinely display speeds in excess of 100 MP/H. Yet very few of us would EVER drive at that speed. A car that goes fast is good for the ego, but may not be practical.

Same thing with a RAID. It may go really, really fast. But at a certain point, you don’t need the speed. A fast engine, by itself, isn’t the answer. You also need a comfortable place to sit and a steering wheel that works.

You get the point.

The good folks at Accusys sent me an 8-drive RAID to test recently. (You can read my review of their ExaSAN A08S-PS unit in the next issue of my newsletter; coming out later this month.)

However, what I want to share with you today was something I discovered during my testing that totally surprised me.

Video editing is the most difficult thing we can do on a personal computer, because it requires really top-notch gear and editing software to all mesh perfectly in order to get any work done.

But what REALLY pushes your system to the limit is multicam editing. Working with more than two or three streams of simultaneous video truly separates the weak from the strong, from a hard disk point of view.

So, I decided to test this RAID by editing some 720p/60 P2 (DVCPROHD) footage with it.

NOTE: A quick technical note. DVCPROHD requires a data transfer rate from your hard disk to your computer of about 15 MB/second in order to display video in real-time. So, two streams of P2 video require about 30 MB/second. The key number to watch in the charts below is the READ number.

FireWire 800 drives can deliver up to about 80 MB/second of data; which means they are limited to about five streams of HD video. In practical terms, they may play less, depending upon the video format.

So, I created a multiclip containing 10 DVCPROHD clips, edited it into the Timeline, and started to switch between shots during playback by clicking the appropriate image in the Viewer.

NOTE: All the images match because, for simplicity, I took one image and duplicated it. These are all separate video files, not aliases pointing to the same file.

ONE CLICK and I got the dreaded “Dropped Frame” warning. (I hate this thing!)

Sigh…

So, I scaled back the number of clips in the multiclip to six, and everything worked perfectly.

But, what’s the sense of spending all the money to buy a fast RAID if it only does six streams of HD video.

NOTE: And if THAT isn’t a snobbish sentence, I don’t know what is. Imagine what you would have paid ten years ago to edit six real-time streams of HD video. You would have KILLED for the chance. Today, pffft…, can’t be bothered.

Then, as I was about to give it up as a bad job, a little voice in my head said: “Larry, what happens if you edit using keyboard shortcuts?”

So, I selected Tools > Keyboard Layout > Multi-camera Editing. This allows me to use the numeric keyboard to switch between cameras.

Six streams of 720p60 HD worked perfectly. (Which I would expect, since it worked in the Viewer.)

But TEN streams of 720p60 HD also worked perfectly, provided I use the keyboard shortcuts. And this I did NOT expect. I would have expected the same performance from the keyboard shortcuts as I would in the Viewer.

So then I tried 16 streams of 720p60 HD video. Worked PERFECTLY using keyboard shortcuts!

Even though running 16 streams requires Read speeds of 231 MB/second from the RAID; far more than I could get from a single drive.

Hmm… Time to get serious about this.

I transcoded 22 clips of 1920×1080 60 fps AVCHD into ProRes 422. ProRes 422 requires 17.4 MB/sec, at a minimum, for real-time playback of a single 1080p HD stream. The images are gorgeous, but the files are not small.

I linked all 22 clips together, turned off the display of six tracks (because a multiclip in FCP only displays 16 tracks at one time).

NOTE: There’s that snobbery again… ONLY 16 tracks of real-time full-res HD clips…! Sheesh…

There they are. (Thanks, Joe Centeno, for the video!)

WOW! 271 MB/sec. Playing smoothly and editing perfectly — and I was editing to a new shot about once a second, with ZERO dropped frames. Perfect playback!!

Holy Smoke! I did not expect this.

So, here’s the key thought. The next time you are editing multicam work, make it a point to use the keyboard shortcuts. You’ll be stunned at how much easier this is and how few dropped frames you experience.

And having a fast RAID is critical when you start editing more than four streams of video.

Larry

P.S. Here’s a short video tutorial, available in my store, that shows how to create and edit Multiclips using the Viewer, keyboard shortcuts, and buttons.

I’m a member of the IMUG mailing list, which has lots of interesting discussions on media, gear, and life. This morning, Carey Dissmore, Principal of Carey Dissmore Productions, Inc. and a co-founder of the IMUG, sent out a post that I found so compelling, I wanted to share it with you. So, with Carey’s permission, here it is.

There’s nothing brand new or earth-shattering about stating that technology moves really fast in this business. It’s been true of computers and software for awhile but it’s now also true of cameras and video formats.

I’ve often stated about any piece of gear: “It will always do what it always did”, implying another cliché “If it ain’t broke don’t fix it”. But that only works in a world that is standing still, and today we’re in a world that is moving faster (and becoming more fragmented) than ever, from a technology point of view. It also only holds until something DOES break, and then you’re really screwed trying to get parts (or the high, ROI-insane cost of said repairs) for old gear.

Nowadays, it’s getting very difficult to buy a camera, much less a cell phone that is NOT high-definition. It’s also hard to find one that shoots tape. The complete transition to a high definition, tapeless world is pretty much complete, and has brought radical changes to how we all do our work–not always adding speed or efficiency. This transition has also added a whole lot of media-management responsibilities at every step of the production workflow, particularly at the front and back ends of it.

So far I haven’t said anything most of us don’t already know. But sometimes it’s just good to talk about it.

But here’s the point: There are definite sweet spots (windows in time) to own and use gear. Implying all else is sour. Perhaps that’s a little strong, but let’s explore this a bit.

Bleeding edge/Early adopters: The pain of early adopters generally centers around infant hardware death, driver issues and software bugs. Being an early adopter can be a competitive advantage but tends to be most suited to those who are savvy enough to provide their own support–shops that are small and savvy enough to be flexible in dealing with hiccups and work around issues that arise. These are generally people who are already pushing the limits of their current technology pretty hard and need all the faster/better they can muster. Or they’re just tech geeks, or both, like me

Both the single system owner and the large “sausage factory” ends of the production business can be really hurt by bleeding edge issues. What is a minor inconvenience for the folks above can be devastating. The single system owner has no alternative systems to work around the problem and the “sausage factory” mega-shop basically shuts down because the large staff of “working class” employees probably don’t have the training or the analytical & technical skills to work around issues. Result in either case: Production grinds to a halt.

(Almost) Everyone else: Just behind them is a broad swath of general adopters. Where exactly these folks fall on the adoption curve is determined by project and budgetary cycles, but they’re all still kind of “in the zone” with reasonably current gear.

Then there are the technology Luddites (both witting and unwitting)…

I’ll directly address the Mac users. In the Mac-centric video world we’ve been through a lot of changes in the past 5 years. We’ve been through the PCI to PCI-X to PCI-Express transition, which made a lot of us have to buy new video I/O boards. We’ve been through the PowerPC to Intel transition (software upgrades), (Snow Leopard as well). Rapidly changing and fragmenting standards in storage connectivity, the move to multicore, the move to 64 bit, the move to GPU-centric software. It’s a lot. Fall too far behind on eBaying the aging gear to flip to the new gear and this big ballooning monster of total obsolescence will build up and crush you. (aka “Unwitting Luddites”)

Upgrade decisions are often driven by how closely one’s ability to get clients and work is tethered to your ability to support the latest production standards. The hobbyist who makes side income has a completely different threshold than someone who’s entire livelihood depends on continuing to secure more work. And therein lies the rub, because the upgrade cycles have slowed down for everyone in this economy, while the progress of technology, formats, cameras, etc. has surged. This makes for some hard choices. Sometimes it feels like trying to outrun an avalanche, but it’s better than being buried by it.

But there definitely comes a point, despite one’s budgetary challenges causing delay of investment in new gear, where you are expending so much time and effort to bootstrap old gear, trying to make it work with current standards, while being unaware of how much of the tangible (and intangible) benefits are to be had with upgraded kit. Somewhere in that mix also lies opportunity cost of jobs you could be getting if it weren’t for old gear. Tough to manage it all sometimes, but this is a terrible place to find oneself.

That doesn’t mean I’m not all about “bang for the buck” aka “value for money” and “the right tool for the job”. There IS such a thing as overdoing it. Everyone’s needs are different. A honest assessment of the work you do (and are likely to get) is required to match the right gear to it. For example, there are a lot of people in production who would do well to look at iMacs on “value for money” standpoints. iMacs don’t have the expansion options of the MacPro towers, but can do a lot for less depending on the kind of work you do. They can also be great support systems as long as you already have one MacPro configured with the necessary video I/O, which nowadays is more about driving displays than capturing anyway.

Now please excuse me, I’ve got an avalanche to outrun!
Carey

- – -

Thanks, Carey, for writing this and allowing me to share it. As always, I’m interested in your comments.

Larry

Just a very quick note – as it is now about 11:58 PM – that our podcast, the Digital Production Buzz is located on the trade show floor at the 2010 NAB Show as the official podcast. We are right next to the Post Pit – South Lower Hall, Booth 8826.

We’ve already created two shows – you can listen to them here: www.nabshowbuzz.com. (Well, OK, if you want to be technical, the second show is done as of ten minutes ago, and will be posted in the morning)

We have a team of almost 30 people – both staff and some GREAT volunteers – helping us put together a huge range of news and special reports. However, it’s too late at night for my brain to think clearly enough to blog about it. (By the way, while we are fully staffed for this year, we’d love to have your help next year!)

I’ll have all kinds of photos and gossip to post later in the week when I have time to breathe.

Starting tomorrow, Monday, at 9 AM, we are streaming and posting live five-minute News Briefs on the hour. Everything will be posted here — www.nabshowbuzz.com — as well as on iTunes and our website.

Our NAB coverage ends this Saturday – a WEEK of special shows and almost 40 News Briefs. Almost 14 hours of programming in five days, way more than 300 interviews. Then, I’m taking a nap. (Visit here to learn how you can put The Buzz on your website!)

By the way, thanks to the NAB for appointing the Digital Production Buzz the Official Podcast of the 2010 NAB Show. Thanks to our sponsors – Adobe Systems, Zeiss Lenses, Pond 5, Data Robotics, and Focal Press - for their financial support. And thanks to our incredible producer, Cirina Catania, and Associate Producer Debbie Price, for their tremendous efforts that make these shows possible.

More later, time to get back to work….

Larry