Daniel James
|
| Home | Wiki
| Perl | Meetings
| Press | Mailing
List | Software | Studios
| About | |
Daniel James
2006.12.16
Daniel James
The story behind this project goes back to early 2004, when I was talking to a musician from Los Angeles who had a sponsorship deal with AMD. The previous year, he'd been running Windows XP on a 32-bit dual Athlon processor workstation, which was one of the fastest PCs available at the time. Even so, he'd been running up against some hardware limitations with processor-intensive jobs, running real-time DSP effects on multiple audio tracks. Late in 2003, AMD had provided him with a workstation featuring a pair of their new 64-bit Opteron CPUs, and he told me that despite the fact that he was running exactly the same software, the performance problems he'd been having during heavy DSP loads went away.
I knew that Opteron chips were being used for servers and in high performance computing, but this was the first time I'd heard of them being used in a multimedia workstation. Microsoft was yet to release any native operating system for the platform, and while Linux distributions compiled for AMD64 were beginning to appear, discussion about native 64-bit computing often revolved around the Intel Itanium, or the PowerPC CPU used in the Apple G5.
The key differences were that the AMD64 chips were designed to be fully backward-compatible with 32-bit PC software, and that they cost about the same as existing 32-bit CPUs found in ordinary PCs. When Intel admitted in 2004 that they were adopting the technology for their own CPUs, and then Apple announced that they were abandoning the G5 in mid 2005, AMD64 was set to become the de-facto next generation CPU for all but the most high-end workstations.
If we take a look at the state of creative desktop computing tools, it's obvious that there has been a lot of consolidation among the proprietary software vendors in the last couple of years - Adobe's takeover of Macromedia is just one example. What this means is that regardless of the hardware or operating system in use, the mainstream creative desktop of the near future is likely to represent a highly integrated set of non-free applications from a very small number of vendors. We can expect these proprietary applications to be well tested for performance, reliability and usability.
We believe that it will be very difficult for GNU/Linux and other free software to compete for users on the multimedia desktop unless it can achieve a similar level of integration and polish. Without a significant user base, it becomes difficult for free software to maintain the hardware support that it needs. Reports indicate that it has become very difficult in recent years to obtain full specifications for video card driver development, and several of the most popular high-end audio interfaces, particularly the FireWire models not running BeBoB, remain without a free software driver for the time being. At 64 Studio, we aim to deliver a viable and sustainable creative platform based on free software, and partner with hardware manufacturers to ensure the availability of fully-supported components and peripherals.
When we launched our project in the spring of 2005, we decided to concentrate on the kind of desktop systems which we believe will be common among creative users in the future. This year we've seen four-core workstations and two-core laptops appear in the mainstream, and we can expect that eight-core and even sixteen-core x86 systems will be widely available next year. As well as these multi-way SMP systems becoming viable for the creative desktop, the amount of RAM available in a single system has increased exponentially since the Linux kernel was first released. In 1995 I had just one megabyte of RAM in my PC, and by 2005 I already had one gigabyte in my desktop machine. So it's not unreasonable to assume that within another five years we might have many, many gigabytes of RAM in an ordinary PC.
The relentless downward spiral in the cost of newer technologies looks set to make using any hardware older than a year or two quite counterproductive. For example, the HP laptop which I'll be demonstrating 64 Studio on later is an entry-level model from a department store in the UK. It has a 1.6GHz AMD Turion 64-bit processor and 1GB RAM as standard. It cost less than a far slower generic white-box PC of a couple of years ago, and it probably uses a great deal less energy too. Nevertheless, we recently started providing an 'alternative' 32-bit build, for users in mixed 32-bit and 64-bit hardware environments - it will be interesting to see how long demand remains for that version.
We've had native 64-bit Linux on the Alpha and the Itanium for years, but these architectures never reached the mainstream desktop. SGI had an Itanium2 based GNU/Linux desktop product aimed at the creative market, but it cost twenty thousand dollars per machine.
Compared to Windows or any other operating system, GNU/Linux clearly had a head start on AMD64, and you can choose from a range of natively compiled desktop distributions for the hardware. Unfortunately for the creative user, all of these are aimed at the general purpose computing audience. It's impossible to be all things to all people, and what's good for the so-called 'consumer' is rarely right for the content creator.
For example, typical distributions use Arts or ESD to share the sound card between applications, while most GNU/Linux musicians would want to use JACK - admittedly more complex, but far more powerful. (I was asked what was so difficult about JACK that means it isn't found as the primary sound server in any mainstream GNU/Linux distribution. I don't think it is difficult to use, but for the time being it still requires a patched kernel, and some knowledge of sample rates and buffers. Many non-musical users just want to be able to throw audio at any sample rate to the soundcard, and could care less about real-time priority.)
In addition, the creative user's default selection of applications would be very different to - for example - a sys-admin. Even gigantic distributions like Debian don't package all of the specialist tools needed for media creation, and the integration between packages is often less than perfect. So the goal of 64 Studio is to create a native AMD64 distribution with a carefully selected set of creative tools and as much integration between them as possible.
Today, we have free software applications covering many of the other creative disciplines, including 2D and 3D graphics, video and publishing for the web or print. Unfortunately media creation, when compared with media 'consumption', remains a niche activity, even on Linux. This niche status is reflected in the fact that none of the mainstream Linux distributions work particularly well 'out of the box' for media creation - but to be fair, Windows XP or OS X also require many additional packages to be installed before their users can realise the full creative potential of their chosen platform.
We know that specialist Linux audio distributions already exist, including AGNULA/DeMuDi, PlanetCCRMA, dyne:bolic and Studio to Go!, with a good level of integration for music-making. But all of these other audio distributions are x86 only so far, and there are few specialist distributions in the other creative fields. Ratatouille, a Knoppix-based distribution designed for animators, is one exception.
Switching to native 64-bit software doesn't always realise an instant and obvious improvement in performance on the same hardware, but we think that if we create a native platform, then application developers can begin to realise the benefits of multiple cores, 64-bit processor optimisation and an improved memory architecture.
But there's a problem with specialist distributions. Since they have relatively few users, they usually end up being maintained by a single person. External project funding, whether from a venture capitalist or a government agency, is often unreliable in the long term, and can steer the agenda of the distribution away from that of the users.
Since we believe maintaining a niche distribution is too much work for a volunteer, we set up a company to pay developers to create and maintain the system using the Custom Debian Distribution framework. You may know of lead developer Free Ekanayaka's work on CDD from later releases of the AGNULA/DeMuDi distribution. Most of the packages in 64 Studio come from the Pure 64 port of Debian testing, with some from Ubuntu, some from DeMuDi and some custom built.
A more obvious choice might be Red Hat, given that many of the high end (which is to say expensive) proprietary tools used in studios are sold as binary-only Red Hat packages. However, the split between Red Hat Enterprise and Fedora Core presents serious problems for any derived distribution. On the one hand, you could rebuild Red Hat Enterprise from source as long as you removed all Red Hat trademarks, but that's a lot of extra work - and you'd have to follow Red Hat's agenda for their distribution, which you couldn't have any input to. I doubt that you'd get much goodwill from Red Hat for 'improving' their distribution either - just ask Oracle!
On the other hand, you could build a distribution on top of Fedora Core. It's broadly Red Hat compatible, and there are the beginnings of a community process taking place - although it's still far more centrally controlled than genuine grass-roots distributions. The key problem with this approach is that Fedora Core is not designed or built to actually be used. I can say this with some confidence because I was able to ask Michael Tiemann, former Red Hat CTO and now vice president of open source, this question myself. Fedora Core remains just a technology preview for Red Hat Enterprise, and the Fedora Project has absolutely no commitment to stability or usability. If Red Hat wants to try a major update to see what breaks, they can.
Debian does have a commitment to stability, and a bona-fide community process. There are other reasons for favouring Debian over Red Hat. It's long been designed for seamless upgrades between stable versions. The work of the Debian Pure 64 port team is of a very high quality, not to mention that of all the many Debian package maintainers.
We recognise that whatever packages we put into 64 Studio, users will want some of the packages that we haven't included - so being able to use thousands of binaries straight from the Pure 64 port without modification is a major advantage. Because we're sticking very closely to Debian with the 64 Studio design, users can install any application from Debian testing, simply by enabling an additional apt source. This includes nearly all of the well-known applications with the exception of OpenOffice.org, which just won't build natively on AMD64 yet. After the stable release of Debian etch, expected next month, we will be working on a fully compatible 64 Studio 2.0 release, which will provide all the benefits of Debian's stable version, including security updates of course.
In fact, 64 Studio is not so much a distribution based on Debian as a Debian remix. Free Ekanayaka is an official Debian Developer, so we can contribute our improvements back directly - where they are Debian Free Software Guidelines compliant. However, we do benefit from the flexibility of not being an official part of Debian. For example, the Debian project has decided that it does not want to package binary firmware, which is required to be loaded by the driver for certain high-end audio interfaces to work. We understand the reasons for their decision, but it's a major usability problem if you own that kind of interface, because it won't work out of the box.
This kind of hardware - effectively reprogrammable on the fly with a new firmware blob - is only going to become more common, and not just for audio interfaces. So for the sake of our users, we have to support it. Otherwise, free software will become significantly harder to use than proprietary equivalents - and that's not a future we want to see. As users, we couldn't modify our soundcards when they were pure hardware, so we don't think it's any worse that they now require a binary upload. At least now there is the possibility of creating our own firmware and uploading that instead, which we didn't have before.
Our first alpha release was based on Fluxbox, because this window manager places minimal demands on system resources, and is very quick to learn, since there isn't much to it. However, we later switched to the Gnome desktop. Again, this is because we don't want to make free software too difficult for people who are used to proprietary tools. This doesn't mean that we will dumb down the interface or clone the look of other platforms, but - for example - we can expect around 95% of new users to assume that the program launching menu is in the lower left corner of the screen. There are also expectations about drag and drop file management, or GUI-based system configuration tools.
However, we have decided not to add 3D or animation effects to the desktop. For the time being, we can't see enough benefit in these gimmicks to justify the increase in complexity, and the dependence on proprietary video drivers. Our focus is on creating a dependable set of serious tools with the highest performance possible, not appealing to users on the basis of eye candy.
The maintainers of a distribution are fundamentally in an editorial role, selecting the most appropriate free software from the many thousands of packages available, and putting it into a convenient snapshot. Since the software is free software, it would be churlish of us to demand that people pay us to do this, but if we provide something of value then it should be worth a modest (and completely optional) subscription. We believe Red Hat's compulsory subscription model has cost its Enterprise distribution a lot of potential users. Apart from being ethically questionable in the context of free software, as a systems manager at a well-known studio with hundreds of Red Hat desktops put it, "Why should we have to pay for support every year whether we need it or not?"
Community support often meets or exceeds the quality that proprietary software vendors provide, but people tell us that it's reassuring to have some paid-for support available as an option. Sometimes our questions are just too ordinary to interest people on a mailing list or forum, or at the other end of the scale they can require patience and time-consuming research to answer. It can sometimes be difficult to get the help you need when you're up against a project deadline. We believe that by covering one kind of desktop user really well, we can provide detailed support for the people that need it at a reasonable cost. For the people that don't need support, or are planning large deployments where per-seat licences would be prohibitive, it's still free software - and we're not going to lock people into support contracts in order for them to access updates either.
We also offer the 64 Studio codebase as a development platform for OEMs building products on AMD64 hardware. We believe this enables these companies to reduce their development costs and time-to-market. We may also consider producing a server edition of the distribution in future that combines a fast and simple install with pre-configured services, so that a workgroup file server or a streaming media server could be set up in a few minutes - and these services would work right away with 64 Studio desktop machines of course. In the longer term, we hope that 64 Studio will go beyond packaging and integration work to contribute directly to application development, particularly where 'missing links' are identified.
There are a number of challenges we still have to face. The first is following the rapid pace of kernel development. We are currently shipping Linux 2.6.17 with Ingo Molnar's real-time preemption code and a few other patches. At one time these patches didn't build on AMD64 at all, and the stability of RT kernels is still variable. 2.6.18-RT wouldn't boot for a significant number of our testers, particularly those with multiple CPU cores in their machines. The first indications are that 2.6.19-RT is better.
Another challenge we have to deal with is the Debian community process. We are not in a position to demand anything from the Debian developers, we can only suggest and encourage. If there's a real roadblock within Debian, we have the option to create a custom package, but obviously that's something we'd rather not do.
A third challenge is the issue of support for proprietary formats within free software. At the level of encoding and decoding, we think the best solution we've seen is the GStreamer plugin collection, which as far as we can tell meets the requirements of free software licences regarding linking, and also the legal requirements of the patent holders. It's simply not sustainable to expect users to locate and download libraries of dubious legal status, and install these by themselves. Apart from any ethical problems, it's impossible to support users properly in that situation.
At the level of project interchange, for example moving a complex project from Ardour to ProTools, there does seem to be a move among proprietary audio applications towards support for AAF, the Advanced Authoring Format. Free software must support this kind of high-level project compatibility format, otherwise it doesn't stand a chance of gaining a significant user base in this area. When we talk to people in the music industry, it's almost a mantra that 'everyone mixes in ProTools'. We're not aware of any free software audio application that supports ProTools format import or export directly, but at least with AAF we have the chance of finding a middle way.
Some exciting possibilities are suggested by the work currently being done to add video frame support to the Jack audio server. This could potentially mean a large number of very different, processor hungry multimedia applications running concurrently on multiple cores, all locked together on the same project time-line with low-latency synchronisation, and a single transport control.
64 Studio 1.0 is available for download as an .iso image, and the distribution is seamlessly upgradeable with apt-get of course. We'd be more than pleased to hear your test reports and suggestions for the distribution - you can help us make free software the creative desktop of choice.
I'd like to thank Robin Rowe for organising this meeting, and the Tom Bradley Centre for hosting us this evening.
the end
Thursday, December 14, 2006, 7pm to 9pm
Tom Bradley
Youth & Family Center
5213 West Pico Blvd.
Los Angeles, CA 90019
Writer and technologist Daniel James on "64 Studio Linux Software". Daniel James, based in the Isle of Wight, UK, is the editor of LinuxUser & Developer Magazine. A must-see event for studio computer users interested in high performance Linux desktops and media apps! Linux is more popular than Windows or Macintosh for creating feature animation and visual effects at major studios including Disney, DreamWorks Animation, ILM, and Sony Imageworks.
The free 64 Studio Linux operating system distro is based on Debian and includes applications that cover audio and music, video, 2D and 3D graphics, publishing for the web or print, Internet and office tools. Available in both 64-bit and 32-bit versions. [Distrowatch]
You can Download the Installer for version 1.0. This will install Debian with X.org, the Gnome 2.14 desktop, Linux kernel 2.6.17 with realtime preemption patches (including a realtime SMP kernel for AMD64 dual core and multi-processor machines) and a selection of creative applications.
Press Release: "64 Studio Linux at Linux Movies Event in Hollywood"
Questions
to rower@movieeditor.com
Created Dec 17, 2006. Updated Dec
17, 2006