A few days after Independence Day, I’ll be flying to Mountain View for a week at the Googleplex, and from there to Seattle (or Kirkland), to start work as a software engineer on Google’s WebRTC team, within the larger Chromium development effort. The exact project I’ll be working on initially isn’t yet decided, but a few very exciting ideas have floated by since I was offered the position in March.

Last summer I told a friend that I had no idea where I would be in a year’s time, and when I listed places I might be — Boston, Madrid, San Francisco, Schenectady — Seattle wasn’t even on the list. It still wasn’t in March, when I was offered this position in the Cambridge (MA) office. It was an unfortunate coincidence that the team I’d planned to join was relocated to Seattle shortly after I’d accepted the offer.

My recruiters and managers were helpful and gracious in two key ways. First, they arranged for me to meet with ~5 different leaders in the Cambridge office whose teams I might be able to join instead of moving. Second, they flew me out to Seattle (I’d never been to the city, nor the state, nor any of the states or provinces that it borders) and arranged for meetings with various managers and developers in the Kirkland office, just so I could learn more about the office and the city. I spent the afternoon wandering the city and (with help from a friend of a friend), looking at as many districts as I could squeeze between lunch and sleep.

The visit made all the difference. It made the city real to me … and it seemed like a place that I could live. It also confirmed an impressive pattern: every single Google employee I met, at whichever office, seemed like someone I would be happy to work alongside.

When I returned there were yet more meetings scheduled, but I began to perceive that the move was essentially inevitable. The hiring committee had done their job well, and assigned me to the best fitting position. Everything else was second best at best.

It’s been an up and down experience, with the drudgery of packing and schlepping an unwelcome reminder of the feeling of loss that accompanies leaving history, family, and friends behind. I am learning in the process that, having never really moved, I have no idea how to move.

But there’s also sometimes a sense of joy in it. I am going to be an independent, free adult, in a way that cannot be achieved by even the happiest potted plant.

After signing the same lease on the same student apartment for the seventh time, I worried about getting stuck, in some metaphysical sense, about failure to launch from my too-comfortable cocoon. It was time for a grand adventure.

This is it.

So what can you do, when you find yourself compelled to do something that goes against your ethics? To make a choice that you feel is wrong on its own because it benefits you in other ways, a choice you would like to make only when really necessary and never otherwise? Any thinking person will have this problem, to greater and lesser degrees, throughout their lives. We are not martyrs, so we do what we have to do to survive and try to keep in mind our need to escape from the trap.

Organizations cannot simply keep something in mind, but they can adopt structures that remind their members of their values even when those values are compromised. A common structure of this type is the sin tax, a tax designed (in a democracy) by members of a state to help them break or prevent their own bad habits. Sin taxes work by countering the locally perceived benefit of some action that’s harmful in a larger way, by reminding us of less visible but still important negative considerations. Some of their effect is straightforwardly economic, but some is psychological, to help us remember the bigger picture.

Sin taxes are more or less involuntary, but when the government does not impose these reminders, we often choose to remind ourselves. One currently popular implementation of this concept is the Carbon offset, a payment typically made when burning fuel to counter the effect of global warming. Organizations that buy carbon offsets for their fuel consumption do so to send a message, both internally and externally, that they place real value on minimizing carbon emissions. They may send this message both explicitly (by publicizing the purchase) and implicitly (by its effect on internal and external economic incentives).

Carbon offsets may be in fashion this decade, but there are many older forms of this concept. Maybe the most quotidian is the Curse Jar*, traditionally a place in a home or small office where individuals may make a small payment when using discouraged vocabulary. The Curse Jar provides a disincentive to coarse language despite being strictly voluntary, and despite not purchasing any effect on the linguistic environment (although the coffee fund may help for some). The Curse Jar works simply by reminding group members which behaviors are accepted and which are not.

For Mozilla, the difficulty is not emissions, verbal or vaporous, but ethical behavior. How can Mozilla publicly commit to a standard of behavior while violating it? I humbly submit that the answer is to balance its karmic books, by introducing an Ethics Offset**. When Mozilla finds itself cornered, it may take the necessary unfortunate action … and introduce a proportionate positive action as a reminder about its real values.

In the case at hand, a reasonable Ethics Offset might look like an internal “tax” on all uses of patented codecs. For example, for every Boot2Gecko device that is sold, Mozilla could commit to an offset equal to double the amount spent on patent licenses for the device. The offset could be donated to relevant worthy causes, like organizations that oppose software patents or contribute to the development of patent-free multimedia … but the actual recipient matters much less than the commitment. By accumulating and periodically (and publicly) “losing” this money, Mozilla would remind us all about its commitment to freedom in the multimedia realm. A similar scheme may be appropriate for Firefox Mobile if it is also configured for H.264 support.

Without a reminder of this kind, Mozilla risks becoming dangerously complacent and complicit to the cartel-controlled multimedia monopolies. As long as H.264 support appears to serve Mozilla’s other goals, Mozilla’s commitment to multimedia freedom will remain uncomfortable, inconvenient, and tempting to forget. Greater organizations have slid down off their ethical peaks, on paths paved all along with good intentions.

Most companies would not even consider a public and persistent admission of compromise, but Mozilla is not most companies. Neither are the companies that produce free operating systems, and many other components of the free software ecosystem. None of them should be ashamed to admit when they are forced to compromise their values and support enterprises that, on ethical grounds, they despise … but they should make their position clear, by committing to an Ethics Offset until they can escape from the compromise entirely.

*: Why is there no Wikipedia entry for “Curse Jar”!?
**: Let’s not call it an indulgence.

I’ve spent the past few weekends digging into Broadway.js, stripping out the H.264 bits and replacing them with libvpx and libnestegg. Now it’s working, to a degree. You can see it for yourself at the demo page (so far tested only in Firefox 7…).

I’m not going to be able to take this much further … at least not right now. It’s been a fun exercise though. I invite all interested comers to read some more details and then fork the repo.

Take this thing, and make it your own.

Reactions: Hacker News, r/programming, and BadassJS, Twitter.

YouTube version
Sheet Music
Reference files at Archive.org

After about 6 years of covering pop songs in my a cappella groups, I really wanted to sing some original music. In part, I was motivated by the US’s aggressively restrictive copyright regime, which always prevented us from freely sharing recordings of our own performances.

I tried to write a song from scratch for a while, but it wasn’t working out, mostly because I don’t have anything interesting to say. Then I struck upon the idea of using the text of an old out-of-copyright poem (which, because of the US’s effectively perpetual copyright, has to be very old indeed). I started browsing through the poetry section of WikiSource, until I stumbled across this brilliant 1895 poem by Langdon Smith. The choice was clear.

I drew up a thoroughly derivative 4-part a cappella arrangement in MuseScore, and VoiceLab indulged me by adding it to the repertoire. We’ve sung it twice so far, but the first time we didn’t have a good recording, and then this time I had to solve this audio-video alignment problem… but now it’s here.

The recordings and sheet music are all CC0 dedicated to the public domain. I would appreciate attribution as the arranger, but I find threats of legal action to be just as distasteful as plagiarism. I wouldn’t want to do anything to discourage people from adopting and adapting the music as they see fit. Maybe someone will make a recording with a soloist who can really sing!

If you want to combine all these inputs into a video that anyone could watch, you’ll first have to line them up correctly in a video editor. This is a painful process of dragging clips around on the timeline with the mouse, trying to figure out if they’re in sync or not. The usual trick to making this achievable is to look at the audio waveform visualization, but even so, the process can be tedious and irritating.

This year, when I got three recordings from the VoiceLab spring concert, I resolved to solve the problem once and for all. I set about writing an automatic clip alignment algorithm as a patch to PiTiVi, a beautiful (if not mature) free software video editor written in Python.

Today, after about two months of nights and weekends, the result is ready for testing in PiTiVi mainline. Jean-François Fortin Tam has a great writeup explaining how it works from a user’s perspective.

I hadn’t looked into it until after the fact, but of course this is not the first auto-alignment function in a video editor. Final Cut Pro appears to have a similar function built in, and there are also plug-ins such as “Plural Eyes” for many editors. However, to the best of my knowledge, this is the first free implementation, and the first available on Linux. Comparing features in PiTiVi vs. the proprietary giants, I think of this as “one down, 20,000 to go”.

I guess this is as good a place as any to talk about the algorithm, which is almost The Simplest Thing that could Possibly Work. Alignment works by analyzing the audio tracks, relying on every video camera to have a microphone of its own. The most direct approach might be to compute the cross-correlation of these audio tracks and look for the peak … but this could require storing multi-gigabyte audio files in memory, and performing impossibly large FFTs. On computers of today, the direct approach is technologically infeasible.

The algorithm I settled on resembles the method a human uses when looking at the waveform view. First, it breaks each input audio stream into 40 ms blocks and computes the mean absolute value of each block. The resulting 25 Hz signal is the “volume envelope”. The code subtracts the mean volume from each track’s envelope, then performs a cross-correlation between tracks and looks for the peak, which identifies the relative shift. To avoid performing N^2 cross-correlations, one clip is selected as the fixed reference, and all others are compared to it. The peak position is quantized to the block duration (creating an error of +/- 20ms), so to improve accuracy a parabolic fit is used to interpolate the true maximum. I don’t know the exact residual error, but I expect it’s typically less than 5 ms, which should be plenty good enough, seeing as sound travels about 1 foot per ms.

My original intent was to compensate for clock skew as well, because all these recording devices are using independent sample clocks that are running at slightly different rates due to manufacturing variation. There’s even code in the commit for a far more complex algorithm that can measure this clock skew. At the moment, this code is disused, for two reasons: none of our test clips actually showed appreciable skew, and PiTiVi doesn’t actually support changing the speed of clips, especially audio.

If you want to help, just stop by the PiTiVi mailing list or IRC channel. We can use more test clips, a real testing framework, a cancel button, UI improvements, conversion to C for speed, and all sorts of general bug squashing. For this feature, and throughout PiTiVi, there’s always more to be done. I’ve found the developer community to be extremely welcoming of new contributions … come and join us.

gst-launch-0.10 multifilesrc location=images%05d.png caps="image/png,framerate=1/1,pixel-aspect-ratio=1/1" num-buffers=95 ! pngdec ! videorate ! alphacolor ! "video/x-raw-yuv,format=(fourcc)AYUV" ! matroskamux ! filesink location=images_raw.mkv

Boom. I just saved you my last two hours.

Having never attended FOMS before, I had little idea what to expect. I vaguely imagined a lot of sitting around at tables writing code together. Instead, we spent most of the two days in discussion groups. The talking covered a wide variety of subjects, but a majority of the time was spent on a single topic, one of which I was only barely aware before Sunday morning: adaptive bitrate streaming.

Adaptive bitrate streaming techniques go by many names, like “Microsoft Smooth Streaming”, “Apple HTTP Adaptive Streaming”, “Adobe Dynamic Streaming”, and “Free Software … uhh … hmm”. They work in a variety of ways, but all ultimately achieve the same goal: allowing clients to switch between different streams as their available download throughput changes. The trick, of course, is to do this without a pause during the transition. Because internet connections almost never actually guarantee the available throughput, this sort of glitch-free adaptation is crucial if you want to build a satisfactory replacement Television.

The existing systems for adaptive streaming are all tied to proprietary patented codecs, and so we spent half of Saturday and most of Sunday building up a sketch of what an unencumbered adaptive streaming system would look like. We didn’t manage to reach a complete consensus solution in 36 hours, but I think the discussions were productive nonetheless. We did, in my opinion, reach consensus on a few important points:

• HTTP is a terrible way to stream video.
• Especially for live streams
• Even more so when dealing with dumb servers that don’t know about video, and may even have incomplete, outdated implementations if HTTP
• We should do our adaptive streaming over HTTP.
• With support for live streams
• Even on dumb servers
• Ogg Theora|Vorbis and WebM are the only stream types we care about.
• We should build a solution that is basically universal across containers and codecs.

The unresolved issues are important, but in the broad perspective they are technical minutiae, and the atmosphere was clearly such that no one was going to get upset if their branch of the decision tree was not taken.

The one thing I wish I had heard at any point during the discussion was a mention of any other related work on this problem. With the imminent launch of Google TV, and I can easily imagine our design being nipped in the bud by an announcement from Mountain View.

Before:

In its HTML5 support, IE9 will support playback of H.264 video only.

After:

In its HTML5 support, IE9 will support playback of H.264 video as well as VP8 video when the user has installed a VP8 codec on Windows.

Time between quotes: three weeks. I feel sorry for the engineers.

It’s nicely designed, with a solid-feeling all-metal shell. I’m willing to pay the small weight penalty for rigidity. The keyboard seems great so far, and the screen is crisp and bright. The default subpixel fonts look beautiful. It’s a true 16:9 widescreen like most laptops these days, and it feels cinematic to look at. Dell has not tried very hard to minimize the bezel, which is 2-3 cm all around. I certainly don’t have an aesthetic problem with this, though, and it makes room for a nice full-size keyboard and wrist-rest.

The system came with Dell’s Ubuntu 9.10. There’s no Windows sticker or certificate of authenticity, but there is still an unfortunate Windows key. The software configuration, after using it for 10 minutes, has its ups and downs.

On the plus side, it’s a decent basic fresh install of Ubuntu. The only visible Dell-specific customization, and the only item on the desktop, is “Create Dell Recovery Media”, a program to create a USB stick (or CD if I had a CD drive) to restore the system software. There’s also an Ubuntu Install CD in the box, but this isn’t hugely helpful for a machine with no optical drives.

The Fn keys, like brightness and volume controls, seem to be well-integrated with the software. The wireless enable/disable seems to work as well, although the first time I tried it I got a kernel oops. I was notified of the oops by an oops-reporter tray icon, which offered to report the oops to kerneloops.org, then failed because the kernel image was “not a valid Ubuntu package”, or something to that effect.

The default application choices seem a little odd, but they’re probably Ubuntu’s, not Dell’s. The major list is pretty short: OO.o, GIMP, XSane, Empathy, Firefox, Evolution, Brasero, Rhythmbox, Tomboy, F-Spot, Bittorrent client, RDP client.

Intriguingly, the system seems to ship with no proprietary audio or video codecs, with the exception of the Adobe Flash 10 browser plugin, which is so far the only piece of non-free software I’ve identified on the system. Theora and Vorbis are of course installed, both in gstreamer and Firefox.

The system has compositing enabled, but the graphics don’t seem to be running fast enough to produce smooth whizzing animations. I’m surprised by this, given the Intel X4500 series GPU. I suspect a driver upgrade would make this much better. I can think of a few other minor issues too. For example, the default spell checking language in this Firefox window is set to Austrian German (de_AT), despite my specifically selecting en_US during the first-boot configuration wizard.

I ordered the machine with a webcam, but the default install doesn’t contain any software with which to check that the camera is working. “Ubuntu Software Center” shows Cheese but provides no option to install it until after first going to Synaptic and hitting “reload”. It seems that the system ships with only the info for packages that are already installed, and Ubuntu Software Center has no way to retrieve updates. This seems like a problematic design. After installing Cheese, though, the camera does seem to work well.

The touchpad defaults to edge-scrolling mode, with an option in System -> Mouse Preferences for two-finger scrolling. Unfortunately this option doesn’t actually work, even when configured via gpointing-device-settings.

Of course all of these niggles pale in comparison to the alternative, Windows. Booting into a working Linux distro out of the box is a welcome improvement over the usual rigmarole.

UPDATE: I clicked a big button that said something like “Upgrade to Ubuntu 10.04″, and when I woke up this morning these problems were improved. No more kernel oops. Windows swish smoothly. Mouse controls no longer offer multi-touch, which is at least better than offering an option with no effect. There’s also a video editor in the default install. In short, it seems like a nice upgrade. The inverse color theme, like the one I loved in Windows 3.1 at age 8, makes the laptop seems like a real infoweapon.