Data storage bedevils media companies

Preserving critical information for posterity is a universal headache. Overwhelming amounts of data are accumulating at staggering rates across all sectors. Although data storage costs are plummeting, demand is growing by about 50%, far outstripping the rate of cost reductions.And media companies have some unique headaches - huge unsearchable multi-media files to manage over time, plus associated intellectual property (IP) rights. So how can they tackle this fiendish problem?In the short term, tiered storage - moving less-frequently accessed data from servers to cheaper secondary storage media such as tape - can reduce costs. But this is only a stop-gap measure if companies don't do data triage, says John Webster, storage analyst at Nashua, N.H.-based Illuminata Inc. "All you're doing is creating a great big back-end that keeps growing."Preserving data for the long haul is even trickier. All forms of electronic storage media will eventually degrade over time. Popular formats such as CDs, DVDs and disks will only last a few decades as their magnetic fields deteriorate, and storage media is constantly evolving and being replaced with new formats.To tackle storage obsolescence, many organizations have a migration strategy to move critical files from older media formats to the next incarnation every few years. But no one has figured out how to deal with the software obsolescence problem associated with that, says Webster. The files may be available in the future, but will they be readable by whatever technology is in use then? Webster points out documents created in defunct software such as WordStar, which was a popular word processing package in the ‘80s but can't be read by other programs today.Media companies have bigger storage headaches than most. "They have more media to archive, and it's their bread and butter - unlike most enterprises, it's the product they're selling," says Mukul Krishna, global manager for digital media at Frost & Sullivan, a Palo Alto, CA-based consultancy. They also have a lot of unsearchable image, video and audio files to manage, says Joseph Martins, managing director at the Data Mobility Group, a Nashua, NH-based consultancy. Meta-tagging files with text-based descriptors to identify their content is critical, he says. While some software such as Adobe After Effects can capture primitive information in file headers, companies need an asset management tool to manage this area properly. "Most meta-tags have to be stored in an environment outside the assets themselves, so media companies have a dual problem: not only do they have to carry the files themselves through time, but also the meta-data for them - which means they have to preserve asset management software too," says Martin.There are also some dual problems on another front. Many companies - for example, animation houses - can't find software tools that meet their needs, so they often develop their own, he says. "So they will not only have the challenge of migrating their own proprietary formats over time and ensuring they're backwards compatible, but also commercial software. There is no conversion tool to go from Alias to Softimage, so whatever you build it in, chances are, you'll be locked in that environment."Many media companies are still relying on physical storage, as they've yet to make the move to digitize their assets, says Krishna. "They have to spend money on the transition, and they're dragging their feet," he says, pointing out the industry delayed the switchover from analogue to digital television in the US for many years to avoid expenditures. "Overall, they're conservative about IT investment."But the costs of maintaining physical archives are also huge, especially if a controlled environment to maintain fragile older footage is needed. And the savings in investing in digital asset management go well beyond storage - in one company's instance, Krishna says the payback period of implementing technology was two months in savings of time and resources spent managing assets manually.Once a digital archive with defined access rights is in place, media companies can look into ways to monetize their assets in the future. Newspapers started this trend by repackaging old photos and content in books, says Martin. "They were putting together books like old Kennedy photos not previously published." Now many companies such as the New York Times and Corbis offer users browsing access to their digital archives for a subscription fee, he says.If data continues to accumulate at current rates, media companies will need to make some hard decisions in the future. "They're going to have to face the fact, given the size of movies and clips, that they'll either have to put that stuff on cheap storage or nix it if isn't earning them money," says Martin. "Most of them could be doing a better job of monetizing their IP - it's costing them money to store their assets but they haven't figure out a way to tap into that."But it's hard to anticipate what kind of demand there might be for old content in the future in some sectors such as gaming. Martin points to a recent resurgence of interest in old Atari games from the ‘80s that could run on modern PCs with emulators, since the old hardware isn't around anymore."If Atari had figured out a way to monetize those games over time, they could have capitalized on this group of people who will pay for a software package that allows them to play their childhood games today, maybe just to show how goofy the old games dad played were," he says. "Who knows? Maybe there'll be a retro game trend in the future, just like ‘70s disco - 50 years of video games for $9.99." MEDIA STORAGE TIPS 1. File FormatsWhen migrating files, Martin says his first choice would be to go from the latest version of the same application to the next. "Don't go from Word to PDF, go to the newest Word," he says. Files can still be accessed with a tool that exists today, and there's still the option of converting the files to an open standard such as PDF or XML if necessary. Migrations should be planned every three to five years.2. Tiered storageMedia companies should prioritize their assets to determine which categories are accessed most frequently. Files that aren't frequently accessed should be moved off servers to cheaper tape storage media. Krishna notes storage technology is evolving, and costs continue to drop. Companies should look into migrating their files to new systems in the future. "The next big thing is holographic storage, which stores about four times the amount of traditional formats, as it uses multiple dimensions instead of just the surface of a disk."3. Meta-data is the foundation Storage isn't useful if you can't find information. The true value of an archive depends on deep indexing capabilities, which is based on meta-data, says Krishna. "You should define who created the file, what's in it, who had the original rights, and so on. If you want to protect the IP, you should define a policy around who can access it, what rights they have to manipulate it, and so on - an additional rights management package can be brought in for this."4. Version controlMedia companies need one good quality master copy of an asset with proper version control to ensure no one affects it. Changes should be saved to a different version, which then have to be reflected in the meta-data which will be migrated later along with the file. "Companies need to define policies around this and institutionalize the processes," says Krishna.5. Don't move too fastMany companies making the transition to digital try to roll out too much too soon, says Krishna. "This has to be a gradual process." Companies need to decide what sort of taxonomy they'll use, then implement the meta-data associated with that, and encode it in digital assets. It's best to run a pilot in one area to figure out what works, then roll out to the rest of the company.6. Don't move too slowPhysical storage is very expensive and affects the bottom line. The costs will only continue to rise for companies that drag their feet about making the switchover to digital. Asset management and archival systems have evolved significantly in the past few years, says Krishna. "For many companies, asset management is a four-letter word. They tried it in 2000 when the market was in its infancy and got burned. But vendors are now more attuned to customer needs, and there are some good systems available now."

Data storage bedevils media companies

Broadcast | August 17, 2007

Unlock all the Canadian telecom, broadcasting and digital media news you need.

FREE TRIAL

SUBSCRIBE

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us