Linux Movies Group

Linux Movies Group: Highlights of Meeting #8

Wednesday, August 14, 2002, 7pm
LinuxWorld
Moscone Center
San Francisco, California

Special meeting of LMG as a Birds-of-a-Feather session at LinuxWorld. Studios present include Disney, Dreamworks PDI, and Wild Brain.

Steve described how big pipes systems at large studios work. He mentioned that they were using a Cray. The backup archive was 8mm tape, now CD-R. Drew asked how they deal with the issue of CD-R and tape rot. Steve says that they keep evolving to new backup technologies, and migrating from old ones before they wear out. Steve mentioned using StudioPaint and DeepPaint. [Jason and Ramona may start working with those?]

Drew described Halo, the new compositor in Houdini that is being shown at LinuxWorld. He says it is hard to see where Halo begins and Houdini ends because they are closely integrated.

Steve and Skottie debated the value of having a highly customizable 3D pipeline. Nickelodeon kept the same design for years. Studios may take an incremental advancements or major revision approach to upgrading systems. If the latter, customization isn't important.

At Dreamworks Glendale a 20TB Network Associates box is used for network storage. A second unit is at Dreamworks PDI in Palo Alto. Skottie has seen XFS blow up on Irix, and therefore doesn't trust it on Linux either. He considers the assets too valuable to trust to Linux, although perhaps in the future a distributed Linux cluster solution may be attractive. Data services are special, requiring a much higher standard.

At Wild Brain, a much smaller studio in San Francisco, kernel 2.4.9 with RAID0 is performing at wire speed for critical storage. Jim says that they've had success relying upon Linux for data services, but that there are IP transport and NFS reliability issues.

It was mentioned that Clearcase, the version control system, built upon its own MVS file system in an effort to avoid weaknesses in available UNIX file systems.

Skottie says that file systems and databases based upon them always get out of sync. ReiserFS is trying to be both, however it isn't mature. Robin said that Hans Reiser said at the LUGOD meeting recently that the newest version of ReiserFS is a total rewrite. Coda, another file system, uses both kernel and user space code.

Jim pointed out that Pixar is keeping their movies online, buying new RAID systems for each new film. Skottie says that PDI is similar, with 2, 8, and 15TB for Shrek, Spirit, and Sinbad respectively.

Steve liked that ATM load balances automatically. Cards can share the same IP address. Trunking in TCP/IP Ethernet is a problem. However, ATM is too pricey, triples the kernel size, and driver updates are slow to be released.

There was some conjecture about multicast rendering and cachefs. That seems to handle 200-300 nodes ok, but how does Pixar handle 3,500? The typical studio network is like a freeway, with small on ramps to meter congestion on highways. Studios are limiting themselves to 100MB to the desktop deliberately. Using gigabit everywhere would choke the system. Performance is more about buying disk spindles, not mhz, to get speed. Balancing system is much more complex.

"Chunking" is the approach of batching ten 10-frame render jobs. [Notes say something about LSF here.] Scheduling for rendering is still tough, a non-linear challenge. Can choke queue with n-squared tasks! Skottie says 400 jobs per file server is the limit.

Thanks to everyone who came and helped make the meeting a success! Headcount was 10: Robin R., Gabrielle P., Drew P., Ramona H., Jason H., Skottie M., Steve G., Jim M., Jon K., and Daniel M. Beth M. and Greg joined us at dinner afterwards.

We might have had a bigger turnout had LinuxWorld been able to print our meeting in the conference program or had promoted us in any way. It wasn't confirmed until the last minute.