Video Storage

  The most important considerations for video storage are performance (speed), volume and security. The goal is to keep enough performance headroom to avoid dropping frames and enough storage volume to accommodate your project requirements. Redundant data backup equates to security regardless of RAID type used. That said let's look at the performance requirements first since they are the most complex.

PERFORMANCE

  For every format of video there is a performance threshold or minimum performance that must be maintained to avoid problems when editing the footage. SD has very low performance requirements and can usually be accomplished with just a pair of drives in a simple RAID 0. HD, especially editing uncompressed has much higher performance requirements. For example, to edit 1080i uncompressed will require about 250 MB/s minimum and 2K starts at about 300 MB/s. Fortunately with today's high performance drives it requires less hardware to manage those speeds.

  To understand where the storage RAID array get it's speed there are a couple of realities that must be understood. The performance requirements dictate the number of drives that must be used. Each drive included in a RAID be it either a RAID 0 or a RAID 5 adds it's performance value to the RAID. Today's retail desktop drives are capable of 100 - 120 MB/s when empty. If you build a RAID out of multiple drives the number of drives included dictates the maximum performance available. Sometimes there are other limits to performance such as bandwidth or computer limitations.

  As data is added to a hard drive it's performance degrades, the more full a drive the slower it will be. At 80% of capacity a drive will provide approximately 70% of it's original speed when empty. The problem occurs when editing on a system that only provides enough performance for the required format when it's empty. As the drives fill up with data the performance degrades and the array starts dropping frames. The cure for this is to plan for the degraded performance of a loaded array. At 80% of capacity the performance is approximately 70% what the array started at. If you plan for this slow down and buy hardware accordingly you won't experience dropped frames.

Video format data rate chart is available at the bottom of this page

STORAGE VOLUME

  The volume of storage required is determined by the format of video multiplied by the hours of video that must be stored (MB/s X 3600). This includes the RAW footage as well as the post processed final product. With the huge 1500 GB drives available now this is easier than ever. When you choose what drives to RAID together to get the performance required for your project you will also choose the drive capacity. By considering the volume losses and your project's space requirements you should be able to decide what size drive is best for your needs. Keep in mind that you never want to fill a hard drive to it's capacity, always stop at about 80% when storing video. This provides room for the directory index which avoids data corruption. If the RAID you plan on using is a RAID 0 all but about 7% is available for storage. That 7% is lost to the formatting process. If you choose RAID 5 you will lose the volume of one of the drives as well as that 7%. For example if you build an 8 drive RAID 5 using 1TB drives you will lose 1TB for the parity and the 7% format loss leaving you with approximately 5.2 TB of usable space at 80% of capacity.

SECURITY

  Contrary to popular belief, a RAID 5 does not offer much more data security over a simple RAID 0. What it does offer is failure tolerance, a RAID 5 can fail a single drive and maintain it's data where a RAID 0 will lose all data if a member is lost. Parity RAID such as a RAID 5 or RAID 6 is a server technology designed primarily to maintain database uptime. It does this by storing parity data across all of the drives along with your video data, if a drive fails the RAID Controller can calculate what was on the missing drive. This makes for a very complicated algorithm which requires a high quality RAID Controller. The real quality comes into play when something goes wrong. These parity RAIDs are intolerant of data corruption. A little corruption introduced can make the RAID unreadable. Another issue is the complexity of the RAID itself. They are complicated to maintain especially when a problem arises. Should a drive failure occur and the operator accidentally remove the wrong drive the controller sees the missing drive as a failed drive , meaning that the array has lost two members and is now irreparably lost. We've seen this happen. Parity RAIDs have their own avenues of failure and for this reason can't be used without a backup copy of your data. Even a RAID 5 can fail and lose your valuable, irreplaceable data.

  The best plan that exists for protecting what you can't afford to lose is to have multiple copies. If you manage a backup copy of your data correctly it can protect you from events that the most robust RAID 5 can't. If a failure or corruption should occur, the backup copy can be used to recover or keep working. The backup should be automatically updated by a software application which keeps the backup version up to date with the primary. The one caveat to this method is should you experience a database problem or corruption you must stop the automated backup from occurring until the problem is rectified, otherwise that corruption will be carried over to the backup. Anytime an Operating System, Security Update or new software is installed you should disable your backup so that in the event of a problem it will be in pristine condition and available.

  For more information on RAID 5's check out our RAID 5 guide.

HARDWARE

  Most of our customers are processing their video on RAID 0 systems with a second array for backup. This requires a Port Multiplier capable Host Card installed inside the computer to provide external eSATA ports. Each of these ports is capable of up to 5 drives when using the Burly Port Multiplier Enclosures allowing enough drives to be mounted for today's video formats. These RAID 0 systems are the most cost effective and expandable storage available. One of our best sellers is the Burly Dual PM which uses a pair of Port Multiplier Boards and 6 drives in a small chassis with excellent cooling to provide 400 MB/s performance. That's fast enough to edit 2K or you can stripe 3 drives and back up to the other three in the same enclosure. Depends on your performance needs.

  For a true Hardware Based RAID 5 check out the Burly VR RAID Systems. This Enclosure comes with everything you will need already formatted into a RAID to your specifications. RAID 5 and RAID 6 is becoming popular for video and provides failure tolerance although a backup is always required. These systems have the dedicated CPU and RAM on a top level Areca card to manage the RAID leaving your computer's resources untaxed. The high performance CPU provides excellent performance in the 700 MB/s range.

  If you have specific questions don't hesitate to call or email us. We're always happy to help you configure a system that will best suite your needs.