Monthly Archives: March 2013

Chuckles

Background

About 18 months ago, I had the idea of using Javascript to take live audio input and make it animate the mouth of a virtual ventriloquist dummy. The original seed of the idea was to play a prank on someone at SydJS. After a bit of research I was disappointed to find out there wasn’t any way to do it. The idea was pushed aside, but never completely forgotten.

Since then, there has been an explosion in new web technologies and standards. The ones that really piqued my interest were the Web Audio API and WebRTC.

The Web Audio API provides fine-grained control over sound processing and manipulation in JS.
WebRTC allows for peer-to-peer video communication between browsers, along with the associated access to webcams and microphones. Bingo.

Time to code

At the time I started playing with WebRTC, the few browsers that had implemented it only supported getting a webcam stream; microphone support was yet to come. I figured I could still work on getting the idea right using an HTML range input.

The first implementation was simple. I found a picture of a ventriloquist dummy online and hard-coded the image and drawing data, then set the position of the dummy’s mouth to be bound to the value of the input.

Then came the part I’d been waiting for: Google Chrome enabled getting microphone data in their Canary build.

All I had to do was get access to the microphone via the navigator.getUserMedia() API, pipe that input into a Web Audio API AnalyzerNode, then change the position of the dummy’s mouth based on the maximum audio level detected in the microphone. One final adjustment was made to lock the dummy’s mouth movement to regular “steps”, in order to give it a more old-fashioned wooden feel.

And thus was born the first demo of the library that has been christened “Chuckles”.

VIEW BASIC DEMO

More interaction

While the first version worked well enough, it still required hard-coding of all the data. So I built in some new display modes to make it more interactive:

  • Normal mode, the default state
  • Drawing mode, where you can draw on the image to define where the “mouth” segment is
  • Dragging mode, where you can drag the “mouth” around to set its position when fully open

A quick addition of drag-and-drop for adding your own image and you too can make your own ventriloquist dummy:

VIEW FULL DEMO

Next steps

The code is on GitHub at https://github.com/gilmoreorless/chuckles, and there are a few more things I’d like to do with it at some point.

  • Better definition for the mouth segment (possibly using a border).
  • Allow transforming the mouth segment using rotate, skew, etc.
  • Define eye regions that can move left/right independently of the mouth.

But adding those features all depends on whether I can be convinced the idea is actually useful, rather than just being a throw-away demo.

Presentation: WebRTC / Web Audio at SydJS

I gave a presentation about WebRTC and the Web Audio API (titled “WebRTC: Look Ma, no Flash!”) at the SydJS tech meetup on February 20, 2013.

There is no video available of the presentation (fortunately or unfortunately, depending on your perspective), but the slides are online at https://gilmoreorless.github.io/sydjs-preso-webrtc.

The presentation was really just me showing a heap of examples and demos of using WebRTC and Web Audio to do fun things. Most of the examples were written by other people, but a few of them were mine, showing things I’ve been doing in my experiments repository.

Instead of just being lazy and leaving my experiments hidden away, I’m going to force myself to write some blog posts explaining some of them and how they work.

The first one should be up tomorrow. Should.

The road to hell…

…is paved with good intentions.

One day I had an idea. It was the solution to two problems I had. “I’ll start a blog,” I thought.

It seemed simple. “There have been so many times when I’ve found solutions to obscure technical problems on random blogs, maybe I should start one myself and put down various tips and tricks I’ve learned, in order to help others.”

It neatly solved the other problem I had at the time (and still have today). “I’m currently only promoting the code projects I write by sending tweets on a stream that can’t be filtered and that easily gets lost in the noise. I need a better system.”

“And hey,” I added to myself, “people at work know I like to rant about things, now I can have a dedicated place to do it.”

The plan was obvious:

  1. Quickly throw together a WordPress install, because screw writing my own blogging engine.
  2. Grab the most basic template I could find, because my design skills are lacking (also, effort).
  3. Build a quick list of things I’ve learned that could be useful to other developers.
  4. Write lots of words ‘n’ shit.
  5. Bask in the inevitable glory.

It seemed to start well. One post done pretty quickly. So far, so good.

Two months later — at which point I hadn’t written any more posts — I came to realise that, thanks to writing blog posts on the company intranet, I didn’t actually enjoy blogging.
Sure, I liked the end result once it was written, but the actual process of taking words out of my brain and committing them to disk was not a fun one.

“Oh well, I can persevere. I’d really like to get these thoughts out into the wild.”

 

 

18 MONTHS LATER

Yeah, that didn’t work out so well.

So this, the second blog post, is an attempt to publicly commit to actually writing stuff here. Because, obviously, once it’s on the internet it MUST BE TRUE.

Let’s see how long this lasts.