HTML5 Audio

Excellent news from Google, who have released a new beta of Chrome v28 for Android which includes a flag to enable Web Audio API support in the Android version. I don't currently have an Android device to test this on, so I'm anxious to hear about people's experience with this! Please send me feedback!

Stuart Memo has a new toy for us to play with, which is always good news! He has just posted a Web Audio API app called Beat Petite (also available as a Chrome app). Beat Petite is a simple drum machine that uses oscillators and noise instead of samples. Think of it as a way to make your chiptunes drums a whole lot easier!

Are you attending Google I/O this year? If so, you should definitely go check out Oskar Ericsson's talk with Xavier Barrade about how JAM With Chrome was created. The session will be Wednesday, from 12:45-1:45pm and I, sadly, can't be there. But as with last year's sessions, I expect it'll end up on YouTube, at which point I'll be sure to put it up here!

The 10th Nordic Game Conference is fast approaching and I will be there, talking about my favorite subject! If you are in Europe, you might want to come check out Nordic Game and learn about how the Web Audio API can help you put sound in your HTML5 game!

Along with multichannel audio functionality a couple days ago, the looping part of Web Audio API was revamped and shipped in Canary. Previously, one could only specify that a file looped and then entire file looped. In video game sound, however, we have need for the ability to set starting and ending points for a loop within a complete file, that way the sound can play smoothly from the start, into the looping section, and then out the tail once the loop has occurred enough times. A classic example of this would be a vehicle engine or an automatic weapon. I've put together an example that implements a machine gun, allowing the user to either hit the spacebar and keep it pressed in order to continue firing the weapon, or to click on the machine gun image and have the same functionality. As always, you can download the code.

Something worth noting is that I implemented this using an uncompressed WAVE file. It may be possible with a compressed file (MP3, AAC, Ogg), but you're likely to have to spend a lot of time tuning the loop and may never get it perfect. I didn't spend any time experimenting with this, other than to verify that an MP3 did in fact fail to loop smoothly on the first try.

The way the code works, you specify the starting and ending points of the loop as part of the context you're playing, like so:

// here are our starting and stopping points
// for this file's internal loop
var wpn_machinegun_fire_loop_start = 0;
var wpn_machinegun_fire_loop_end = 0.5782708333;

// let's play a sound
myContext = soundWpnMachineGunFire01.play();
// gotta set the loop points
myContext.loopStart = wpn_machinegun_fire_loop_start;
myContext.loopEnd = wpn_machinegun_fire_loop_end;

Then you can enable the loop:

myContext.loop = true;

Changing that back to false will cause playback to simply continue past the loopEnd point and play the remainder of the file.

You might be wondering how I came up with 0.5782708333 as the loopEnd point. Simple: I copied the loop of audio into a new audio file, found the length in samples, then divided by the sample rate. In this case, the loop was 27,757 samples long and the file is 48kHz sample rate. That equals 0.5782708333!

I saw a message come across the Web Audio Developers Community Group yesterday that got me all eager to run to my studio. Chris Rogers wrote:

For those interested in multi-channel output in the Web Audio API, here's
your chance to try out an early build. This could be of interest for
digital DJ type applications, or rendering to multi-channel speaker
installations...

Please note this is an early build and little testing has yet been made.
So far I've tested on several devices on OSX. I'm interested in your
feedback!

The .maxChannelCount attribute is now exposed to show the actual number of
hardware channels:
https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioDestinationNode

The .channelCount attribute should now be settable on the
AudioDestinationNode to values up to .maxChannelCount

Cheers,
Chris

To my knowledge, this is the first time discrete surround audio has ever been possible directly from a web browser. (It was previously possible to decode a Dolby Pro Logic II file once it left the computer's audio outputs, but that's matrixed surround.) Naturally, I had to try this and hear it for myself. It took a bit of effort — mainly because I wasn't aware that the PannerNode and ConvolverNode are both 2-channel only — but today I succeeded in playing both a 6-channel interleaved WAVE file and an Ogg Vorbis file created from the same source.

The functionality wasn't actually difficult to implement, once I removed my panner code, and I've got it here for you to see. All that needed adding to my recent interactive-audio example was the following code:

// test to see how many hardware channels we can output to
// if it's 6 or larger, we can play a 5.1 audio stream!
if (audioContext.destination.maxChannelCount >= 6) {
	audioContext.destination.channelCount = 6;
}
// otherwise, let's down-mix to 2.0
else {
	audioContext.destination.channelCount = 2;
}
audioContext.destination.channelCountMode = "explicit";
audioContext.destination.channelInterpretation = "discrete";

I have an example for you to try out (in the latest Google Chrome Canary), which uses a 6-channel Ogg Vorbis file. As before, I've base64-encoded the Ogg file, so give your browser a moment to load the 7.4 MB of data before pushing play! The code can be downloaded, too.

You might also be wondering how I got the audio into a form I could play it. I tried a lot of different tools, with a lot of failures. In the end, I bounced an interleaved 6-channel WAVE file directly from Pro Tools, then use VLC to convert the resulting WAVE file to Ogg Vorbis.

One last note: for some reason, I'm getting a DOM error when I try to play back the example on my laptop. But I assure you it definitely worked on my Pro Tools HD3 system at the studio.

Update: The DOM error appears to have been a result of my not testing for .maxChannelCount on the current playback hardware. I've added that in now, defaulting to a 2-channel down-mix when a 6-channel output is not possible.

One other thing I meant to mention before is that it would be wise to familiarize yourself with the ChannelMergerNode and the ChannelSplitterNode in relation to creating multichannel mixes on the fly.

Will you be attending the upcoming Game Developer Conference? If so, you might want to come get a peek at what's going on in Web Audio. I'll be presenting twice at the conference on Friday, briefly at the IASIG's Town Hall meeting, then as a more formal demonstration of the power of HTML5 Audio at 4:35pm, in Room 3000.

Additionally, there are some other HTML5-centered presentations, although I'm not sure how much they'll talk about audio. Mozilla's Director of Business Development will talk about "The Inexorable Rise of HTML5 Games". There will also be "Fast and Awesome HTML5 Games", presented by Mozilla's engineering director and a senior researcher. Ludei's CTO will discuss "HTML5 Cross-Platform Game Development". And Google will present "Native Apps? With HTML5? Yes you can!"

Hope to see you there!

I'm thrilled to tell you that Boris Smus has released a new book about the Web Audio API via O'Reilly. The title, available in electronic and hardcopy forms, provides everything you need to get started with audio programming for your HTML5 app and requires no prior knowledge of digital audio coding techniques.

The 76-page Creative Commons book will help you:

• Learn Web Audio API, including audio graphs and the audio nodes
• Provide quick feedback to user actions by scheduling sounds with the API’s precise timing model
• Control gain, volume, and loudness, and dive into clipping and crossfading
• Understand pitch and frequency: use tools to manipulate soundforms directly with JavaScript
• Generate synthetic sound effects and learn how to spatialize sound in 3D space
• Use Web Audio API with the Audio tag, getUserMedia, and the Page Visibility API

Be sure to check out the book's examples, which are also available on GitHub.

I was pretty excited when Jos Dirksen's article on visualizing sound appeared back in October. So you can imagine how I felt when I stumbled across Josh Marinacci's book HTML Canvas Deep Dive and discovered that Chapter 12 is dedicated to the Web Audio API! After quickly touching on the difference between the <audio> element and Web Audio API, the chapter provides some basic instruction on how to get an AudioContext working and then dives into visualizing audio on the Canvas using FFTs! A good set of fundamentals all the way around. And Chapter 13 continues with getUserMedia, so there's quite a lot worth taking in.

The source to the entire book can be found on GitHub and all the examples are available, too!

Last month the W3C's Audio Working Group created the new Web Audio Developers Community Group where developers and hackers interested in using the emerging Web Audio API can come together to discuss the issues they're having and ask questions of each other in an effort to build new applications and tools. The group, which is open to the public, will not be involved with the specifications aspect, but instead be a gathering place open to all comers.

So get yourself an account and join the discussion!