New Developments in Nonlinear Editing, Part 2
Networking and Video Servers
[Last month's article discussed how to build nonlinear video editing systems. This article will discusses the basics of networking and video servers.]
By Chris Stakutis and Robert Lamm
Transferring files between personal computers was originally done with floppy disks. This was a clumsy way to exchange information, so it wasn't long before people figured out ways to network their computers together so that they could look at each others' files, share things like scanners and printers, and even run each other's applications. Most new computers now come network ready right out of the factory with inexpensive ethernet 10BaseT connections on them.
Network users often centralize all their storage on one of the stations. This computer, generally known as a server, is given most of the network's disk capacity and placed in a secure location. Aside from being a safer place to store important data, it's also more efficient since files are easier to find, redundant copies are avoided and free disk space is consolidated in a single location rather than scattered among lots of workstations.
Networking has proven very useful in the video production arena: text files, EDLs, and graphics are easily sent from room to room. But video files, which are much larger, take a long time to transmit. And it isn't possible to play video over a 10BaseT ethernet network in real-time, a neccessity if one is going to centralize video files on a server.
A 24-bit 720x486 image is about a Megabyte (MB), or 30 MB/second for moving video, not including the optional key channel or audio. CCIR-601, the uncompressed digital video that D1 machines work with, is 173 Megabits (Mb) (or about 22 MB)/second, Digital Betacam is 2:1 compressed down to 85 Mb (11 MB)/second, DV is 25 Mb (a little over 3MB)/second.
In contrast, the ethernet 10BaseT that most people have can only handle about 1MB/second. 100BaseT, a more advanced technology, can theoretically handle ten times as much, but in practice can rarely be tuned to exceed 3.5MB/second, barely enough for one DV channel. And protocol processing for this much data would eat up 80% of the clients resources and make it unavailable for other tasks.
There would also be a bottleneck at the server because all the video streams have to flow through it on their way from the disks to the network: The PCI bus that's in most personal computers can handle about 100MB/second, enough for a few high-quality video streams, but not for a dozen or more simultaneous users who may be accessing separate video and audio files and jumping from clip to clip during a nonlinear editing session.
As a result, most video servers aren't really servers at all: They're video devices that happen to be built out of computer components. An example would be the Tektronix Profile, which looks and acts like a videodisk recorder, complete with standard video input/output and RS-422 deck protocol. In fact, if one wants to incorporate Profile footage in a nonlinear edit, one has to redigitize the digital SMPTE-259 signal onto the nonlinear editor's drives! Most other video servers are similar: automated commercial and program playback devices which input and output only standard video/audio signals and whose storage is not accessible to other computers or applications.
If one builds a network with the computers' SCSI interfaces instead of the more usual ethernet connectors, the story changes. SCSI ports are the connectors that most people hook up their external drives to, including many nonlinear systems' media storage. Differential Ultra SCSI supports data rates of up to 40MB/second at distances up to 80 feet. SSA and fiberchannel, similar but more sophisticated technologies, offer 80-100 MB/s (SSA will soon offer 160MB/s); fiber-optic versions of these technologies can be networked across kilometers.
These data bus technologies also support a topology that eliminates the need for servers and the bottlenecks that can form there: Computers and drives are hooked up directly to a ring-shaped cable with all the data flowing from station to station in a circular direction. The drives place the data directly onto the bus as it's requested, computers take it off the bus as needed and vice versa. Unlike ethernet-based networks, where only one device can transmit over the bus at any one time, any device on a ring-shaped bus can transmit as long as the section between it and the next station is free. And since the data flows directly from disk to bus, there's no bottleneck at the server. All that's neccessary is some control software to manage the data flow and ethernet connections between the stations for control purposes.
Although there is no literal server, per se, the storage is centralized in a single, shared location that appears as a drive icon on all the users' screens and that all their applications can read and write to. And among the file types it can handle are multiple, real-time, broadcast-quality video streams going in and out to several users simultaneously.
Such a system is known as a 'multi-initiator shared-data-bus' system and is very economical to assemble: SCSI, SSA, fiberchannel and ethernet adapter boards are all standard, relatively-inexpensive computer accessories. Storage disks are the usual off-the-shelf AV-quality drives we all know and love. The basic control software that updates all the computers' directories of the common drives and keeps the users from writing on the same file simultaneously comes with the hardware or is available for a very small cost.
Putting a basic multi-initiator network together is surprisingly easy: The SCSI/SSA/fiberchannel and ethernet boards slip right in, the drivers are easily installed and the data rings and ethernet cables are easy to connect.
But there's a lot of room for optimization: One can support a lot more simultaneous users at higher data rates if one configures the network carefully so users are as topologically close to the data they use most and stripe the volume sets to take advantage of the bidirectionality of some data bus technologies.
Reliability is also an issue: If the data ring actually runs from computer to computer, any one of these machines can take down the network if it's shut off or disconnected. To prevent this, the large data ring can be replaced with a central 'hub'. This box contains the data ring inside it and feeds each station its part of the loop on a pair of dedicated lines. Hubs are intelligent, so they can sense if a station is off-line and route the data path around it. Hub-based systems are also usually easier to wire, since the physical arrangment of the various stations may not really accomodate a large loop of wire running from room to room.
The basic control software that comes with the hardware is also missing some critical functionality neccessary for most video applications: It really pays to get more sophisticated video-oriented control software. The most important function is the ability to prioritize stations' access to the data ring bandwith. Without this, the entire ring will slow down if enough people try to use it simultaneously. A lot of this demand will be noncritical: file copying, etc. Prioritizing stations ensures that critical activities like digitizing and printing-to-tape aren't disrupted by other tasks which don't need the uninterupped data flow.
The more sophisticated contol software also provides a lot of the functionality one finds on a normal server, such as storage analysis for defragmentation and failure detection, activity logging for billing and user management, and conveniences like e- mail and bulletin-boarding.
The Bottom Line
Open-system file servers like this are very attractive to the production community: The enormous amount of drive space neccessary for video recording can be centralized in a single location where its accessible to all systems. Workflow is a lot more efficient when all the various stations in the production process (graphics, animation, editing, color correction, editing, etc.) have access to the media. And traffic management at TV stations is a lot easier when all the media rests in a single location, yet can be accessed by anyone from reporter, to editor, to producer, to director, to switcher when needed.
Chris Stakutis is Director of Digital Products at Mercury Computers, a Chelmsford, MA manufacturer of open-system video servers. He can be reached at (508) 256-1300, firstname.lastname@example.org.
Bob Lamm works at CYNC Corp., which builds and sells nonlinear editing systems. He can be reached at (617) 277-4317, email@example.com.