Category Archives: Code samples

Reduce your spread

A quick prelude: I wrote the first draft of this post a couple of months ago, but it still needed editing and polishing. Then Rich Snapp published The reduce ({…spread}) anti-pattern and made all the main points I was going to make.

Feeling rather deflated, I initially abandoned the draft. However, to force myself to complete things, I’m publishing it anyway. Although our primary argument is the same, we approach the topic from slightly different angles. I recommend reading Rich’s post too as it’s well-written, and he explains some things better than I do. But mine has a bonus grumpy rant at the end. Contrast and compare.


Object spread — a primer

The object spread syntax has become one of the more readily-adopted[citation needed] pieces of the ever-evolving ECMAScript specification. Over the past few years I’ve worked on several application codebases that have made use of it, and seen many more open source libraries using it.

First I should define what I mean by the object spread syntax. It’s best to explicitly set the terms of reference here, because different people might have different interpretations of it. Other people might have used the syntax but not known what it was called.

Object spread, at its simplest, is used for copying the keys and values of one object into a new object. It’s important to note that it does a shallow copy rather than a deep copy, but in many cases that’s all that’s required anyway. (For more details on shallow vs deep copies, see this Stack Overflow post.)

let sourceObject = { element: 'input', type: 'radio' };

let sourceCloned = { ...sourceObject };
// A new copy of the original: { element: 'input', type: 'radio' }

let extendedObject = { ...sourceObject, name: 'newthing', type: 'checkbox' };
// A modified copy: { element: 'input', name: 'newthing', type: 'checkbox' }

The problem

Object spread syntax is not a problem in and of itself. Quickly and easily copying an object’s data to another object is a very common use case. Indeed, the plethora of solutions that were created in the years beforehand show how much it was needed. Long gone are the days where some projects would include the entirety of jQuery just so they could use $.extend().

The problem arises when it becomes the only way to assign properties to an object in a codebase. Over the past few years I’ve noticed a trend of using object spread wherever possible, even if it isn’t the best tool for the job.

In the worst cases, it has caused exponential performance bottlenecks. Helping someone fix a slow-running script is what prompted me to write this post. Object spread syntax should be used with caution.
Continue reading

SydCSS Wheel of Talks

Last December the SydCSS meetup had its first game show-themed “Wheel of Talks” evening. Three presenters each had to deliver a talk of 5-10 minutes on a topic related to CSS. The catch was that they didn’t know what their topics were until the start of the evening, at which point they had 20-ish minutes to prepare. The talks were randomly selected from the eponymous on-screen Wheel of Talks, built in CSS and JS.

This is the story of how the event was planned, and how I built the wheel.

I can’t hide from my mind

The best ideas arrive when you’re meant to be thinking about something else. The late, great Terry Pratchett imagined them as “inspiration particles” that, like muons and neutrinos, sleet through space until they strike the brain of an unsuspecting creature. I just call them brain farts.

This particular idea struck one morning as I was getting ready for work. I imagined a 3D, poker machine-esque, CSS-powered spinning wheel. Specifically, I imagined two of them — one to randomly choose a presenter, one to randomly choose their topic. I knew SydCSS was looking for new ideas to mix up their evenings, and this would be the perfect fit. I called it “presentation roulette”.

I whipped up a basic prototype on the train ride in to work that morning to make sure the concept worked. I showed it to David (one of the SydCSS organisers) and explained the idea. The next day I got a message from Fiona (the other organiser) saying that she loved the plan. A day later she said that someone else was already on board to do one of the impromptu talks, and that I should do one too.

And thus, a monster was born.

Continue reading

Redirecting GitHub Pages after renaming a repository

One of the golden rules I try to follow in web development is “don’t break URLs”. That is, even if pages get moved around, or a site is completely revamped, the old URLs should still redirect somewhere. It’s a terrible experience to bookmark a page or follow a link from another site, only to find that the page has completely vanished. Let’s try not to make the problem of link rot even worse.

The best way to redirect old pages is to get the web server to send a 301 Moved Permanently HTTP header. This has the benefit that search engines will see the 301 header and automatically update their caches to reference the new URL. Where this can fall down is when using static site hosting such as GitHub Pages or Surge.sh. By design, you’re not allowed to send any web server directives such as custom HTTP headers, so the 301 redirect option is out.

So what do you do when you want to move some URLs around when using GitHub Pages? Here are the options I found.

Continue reading

Using Make to generate Chrome extension icons

File this under “you don’t have to use JS for everything”.

Problem

I’ve built a few Chrome browser extensions, and there’s a part of making them that I generally find rather tedious: icons. To account for all the different places where icons are used, as well as multiple screen resolutions, 6 differently-sized icon files are required. Sometimes I’ve manually exported a few of these from Photoshop for one-off projects, but my most recent extension required a few updates to the icon. This was the tipping point for adding automation.

In this post I’ll detail how I used only a few lines of a Makefile to handle all icon size conversions. Yep, Make. No JavaScript-based build system with a multitude of npm modules and seconds of overhead for every run. Sometimes the old ways are the best.

Solution

Taking advantage of some special features of Make, this is what I came up with:

Continue reading

Strip analytics URL parameters before bookmarking

Have you ever found yourself bookmarking a website URL that contains a heap of tracking parameters in the URL? Generally they’re Google Analytics campaign referral data (e.g. http://example.com/?utm_source=SomeThing&utm_medium=email).

I use browser bookmarks a lot, mostly as a collection of references around particular topics. The primary source for most of my web development-related bookmarks is a variety of regular email newsletters. My current list of subscriptions is:

What many of these have in common is the addition of Google Analytics tracking parameters to the URL. This is great for the authors of the content, as they have the option to see where spikes in traffic come from. When I’m saving a link for later, though, I don’t want to keep any of the extra URL cruft. It would also be giving false tracking results if I opened a link with that URL data a year or two later.

So I wrote a quick snippet to strip any Google Analytics tracking parameters from a URL. I can run this just before bookmarking the link:

Continue reading

Responsive graphs and custom elements

A few weeks ago I published a statistical analysis of one-day cricket – specifically a verification of the old “double the 30-over score” trope.

This blog post is not about cricket – you can read the link above for that. This is a post about the technology behind that analysis. Ben Hosken from Flink Labs says around 60% of a data visualisation project is just collecting and cleaning the data – and so it was with this analysis. What was originally a “quick distraction” became my coding-on-the-train hobby project for 3 weeks. Here’s how I built it.

Data

All data came from Cricinfo, thanks to their ball-by-ball statistics. I won’t go into much detail about how I got the data because I have don’t know what their data usage guidelines are (I looked). Suffice it to say that I ended up with over 7000 JSON files on my machine.

From these I wrote a series of Python scripts to parse, verify, clean up, analyse and collate the data for each innings. The final output was a single JSON file containing only the relevant data I needed.

Custom element

In order to fit the graphs neatly into the context of a written article, I wanted to be able to embed them as part of the writing process. The article (source on GitHub) was written using Markdown with some custom additions (which are described later). Markdown was used despite my personal annoyances with some aspects of it, mainly because it’s widely supported.

The easiest way to put a custom graph into the article was to plonk some HTML in the Markdown source and let it be copied across to the rendered output – but what HTML? I needed containers for SVG graphs that would be rendered later via JavaScript. Given that there would be a few graphs, the containers needed to define customisations per graph, preferably as part of the article source (for better context when writing).

The usual choice for something like this would be a plain old <div> with lots of data- attributes, one for each option. Unfortunately this didn’t let me easily tweak the options after rendering, which I consider important for data explorations so that you can get the best possible view of the data.

Given the one-off nature of the article, I had the opportunity to use “new shiny” tech in the form of a custom element, part of the new Web Components specification. Obviously proper custom elements aren’t widely supported across browsers yet, so I used the excellent Skate library as a helper (disclaimer: I work on the same team as Skate’s author).

HTML API

Using a custom element meant that my in-context API was simple HTML attributes. I quickly wrote a potential API in a comment before I’d written any code for it.

/**
 * <odi-graph> custom element.
 * ___________________________
 *
 * Basic usage:
 *
 *   <odi-graph>Text description as a fallback</odi-graph>
 *
 * Attributes (all are optional):
 *
 *   <odi-graph graph-title="This is a graph"></odi-graph>
 *     Add a title to the graph
 *
 *   <odi-graph rolling-average="true"></odi-graph>
 *     Include a rolling average line (off by default)
 *
 * [... etc.]
 */

This allowed me to write the article with in-context HTML like so:

In order to find the answer, I graphed out a 100-innings rolling average, to give a better indication of trends over time.

<odi-graph graph-title="Overall vs rolling average"
    rolling-average="true" innings-points="false">
    IMAGE: A graph showing a rolling average halfway mark as described in the next paragraph.
</odi-graph>

Thanks to Skate, the initial definition of the custom element and its attributes was very simple.

skate('odi-graph', {
    attached: function (elem) {
        // Create internal DOM structure
        // [... SNIP: nothing but basic DOM element manipulation here ...]

        // Create the graph
        elem.graph = new ODIGraph();
        queue(elem, function () {
            elem.graph.init(cloneData(stats.data), configMapper(elem));
            onceVisible(elem, function () {
                elem.graph.render(elem);
            });
        });
    },
    attributes: {
        'graph-title': attrSetter,
        'rolling-average': attrSetter,
        'innings-points': attrSetter,
        'ybounds': attrSetter,
        'date-start': attrSetter,
        'date-end': attrSetter,
        'filter': attrSetter,
        'highlight': attrSetter,
        'reset-highlight-averages': attrSetter
    }
});

function attrSetter(elem, data) {
    if (data.newValue !== data.oldValue) {
        if (elem.graph && elem.graph.inited) {
            elem.graph.config(configMapper(elem));
        }
    }
}

The queue and onceVisible calls were just making sure that the graph wasn’t rendered too early, before the data finished loading. The key part is that it just calls out to a separate ODIGraph instance (described later on in this post) for separation of responsibilities. The attrSetter method made sure that the graph was re-rendered if any HTML attribute changed, after running the attributes through configMapper which just parsed the attribute strings into a JS config object.

This gave me the enormous benefit of being able to just tweak attributes for a graph in browser devtools and see the graph instantly update.

Editing a graph element's HTML attributes instantly updates the SVG internals

Accessibility

As shown above, all graphs were written with some text inside the custom element. During the initialisation of the custom element, a <figure> element is created inside it, then the custom element’s original text context is moved to a hidden <figcaption>. This provides a description that is read out by screenreaders for those users who can’t see the graph visuals.

An additional aspect of providing descriptive captions was making sure that the article text immediately after the graph described an interpretation of the graph data. Doing this ensured that no information was presented only via visuals.

For the finishing touches I added the aria-hidden="true" attribute to the axis and legend text in the graph. Since the graph was built with SVG, all the text elements would be read by screenreaders but without context. (After all, what does “15 20 25 30” mean without the visual positioning context of being on an axis?)

Responsive graph

For the actual graphs, I wanted them to be interactive, mobile-friendly, and (hopefully) not much work.

C3

I started off by using the C3 library for ease of development. It initially ticked a lot of boxes:

  • I could quickly get up and running with my existing data set.
  • It was responsive and auto-expanded to whatever size container it was put into.
  • Auto-scaling and highlighting of data points (with a vertical hover marker) came for free.

I was able to look at my data set in a few different ways with just some simple config tweaks. However, as I dug deeper into what I wanted the graphs to do, I found myself fighting against the library instead of working with it.

As an example, I wanted to combine a scatter plot for individual innings with one or more line plots for the averages. While this was possible with C3, it started to become an either/or situation. C3 gave me a vertical marker line when hovering anywhere on the graph (which is what I wanted), but only when I had just line plots – as soon as the scatter plot was added, the nice hover behaviour disappeared. The way the hover marker was implemented also started to fall down – it worked fine with around 100 data points, but as soon as the full data set of over 1800 points was added, the hovering stopped working.

Added to this was a performance problem of C3 with the size of my data set. In order to hide data point markers on the line plots, C3 adds an SVG DOM node per point then hides them via setting their opacity to 0. It does this so that they can be faded in with an animation (if they’re set to visible via an API call). In my case, this meant over 3500 DOM nodes were being created just to be hidden.

In the end I decided that while C3 is a nice charting library, it wasn’t set up for my use case. I wrote out a list of all the things I could think I would possibly want the graphs to do:

  • Combine scatter and line plots on the same graph.
  • Vertical marker to highlight data points when hovering, auto-snapping the line to the nearest data point to the mouse cursor.
  • Fully custom rendering of a tooltip when highlighting a data point.
  • Resize when the graph’s container size changes.
  • Plot data along the X axis by dates (C3 has an API for this, but it didn’t work with my data).
  • Define a title
  • Optionally add background highlight regions for grouping data points, with each region having its own title.
  • Show a horizontal background line marking 30 overs as a reference.
  • Have good performance – create only as many DOM nodes as necessary for displaying the data.

After quickly looking at some other charting libraries, I bit the bullet and accepted that I was going to have to roll my own.

Custom rendering

Note: All the code for the graphs can be found on GitHub. It changed structure many times as extra requirements turned up, so it’s far from perfect. I didn’t see the need for lots of refactoring, given its one-time-use purpose.

I won’t go into a super-detailed explanation of the code, as most of it is just calling D3 APIs. Instead I’ll detail the high-level concepts.

Most of the contents can be broken down into three main areas of responsibility: setup, sizing, and display.

First is setup. This is only called once per graph on initialisation. The only thing setup code is concerned with is setting up D3 axis components, creating placeholder DOM nodes and adding event listeners.

Second is sizing. This takes DOM nodes created in the setup phase and sets their positioning and dimensions based on the current size of the graph container. The ranges of D3’s axis and scale components are also defined here.

Finally there’s display. This is where the main rendering takes place. The data array that’s passed in is rendered according to a few constraints:

  1. The options already set – filtering, highlight regions, etc.
  2. The dimensions and positioning defined in the sizing phase.

This kind of structure means that when a graph is first created, it calls the setup, sizing and display phases in succession. From then on, resizing the browser window calls straight into sizing and display, meaning the graphs will always fit into the current window (after a short delay due to debouncing). Finally, if graph.data(someNewData) is called, only the display phase is called because the sizing hasn’t changed.

Tweaks for better responsiveness

Although the main data rendering was now responsive to any window size, I realised that some of the text elements didn’t work at small screen sizes. The different types of text required different solutions.

Graph titles could be highly variable in width, as the text is completely different per graph. I ended up with a solution that allowed me to specify different titles in the one attribute – a “primary” title and a shorter variant to be used only if the primary one didn’t fit.

I picked a couple of arbitrary unicode characters that were not going to appear anywhere else, and could be easily typed on my Mac keyboard. One character (· or Option+Shift+9) was used to define a cut-off point in a single title – if the full title didn’t fit in the graph, then only the text up to the cut-off point would be used. The other character (¬ or Option+l (lowercase L)) marked two completely different titles.

Examples from the code comments give a better idea of how they were used:

/**
 * Formatting for titles
 *
 *   `graph-title` attribute and highlight region names can indicate short vs long content.
 *   The short content will be used if the long version can't fit in the space available.
 *   « = short version; » = long version
 *
 *   "This is a title"
 *     « This is a title
 *     » This is a title
 *
 *   "Short title· with some extra"
 *     « Short title
 *     » Short title with some extra
 *
 *   "Short title¬Completely different title"
 *     « Short title
 *     » Completely different title
 */

The other text that didn’t work at small sizes was the legend at the bottom of the graphs. Intially I had added the legend as part of the same SVG element as the graph to keep everything together. I played with some different ways of trying to make the parts of the legend wrap nicely at smaller sizes, but this was one area where SVG worked against me. In the end I realised I was trying to replicate with SVG groups and transforms what CSS gave me for free. I ended up creating one SVG element per legend type, setting display: inline-block on them and letting CSS handle the rest.

Here is a comparison of how the title and legend worked at different graph sizes.

A wide graph with full title text

A narrow graph with reduced title text and wrapped legend

Final touches

With the core functionality done and the graphs finally doing what I wanted, it was time to add in some polish.

Only render when visible

There ended up being 7 graphs in the article. Rendering them all on page load would be a noticeable drag on performance, with over 5000 DOM nodes being inserted into the page.

Instead I used the Verge library to detect when a graph’s custom element became visible for the first time (using a debounced scroll event listener). Once the element becomes visible, the graph setup/sizing/render methods are called for the first time. This makes the initial page load snappy, and also stops the browser from having to render any graph that is never viewed.

Customised Markdown

The article was written using basic Markdown syntax, but with a couple of custom additions. One addition was for adding numeric callouts that appear on the side (using a syntax of ++1234++). The other addition was a helper for easily adding links to specific matches on Cricinfo. The Markdown source was read by a quick-and-dirty Python script, run through a Markdown renderer, then run through extra string replacements for the custom additions. Finally the HTML output was inserted into a basic page template to produce the final article.

So that’s how it was made. Enjoy.

Compute a DOM element’s effective background colour

While working on a new JS library today, I wanted to create a menu overlay element that automatically has the same background colour as its trigger element.

For some context, the basic structure looks like this:

Basic element structure

Over-the-top colours added to highlight element structure

My first thought was to just use <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window.getComputedStyle">getComputedStyle()</a> and read the backgroundColor property of the trigger. This works only if the trigger element itself has a background style.

Of course, if the background styles are set on a container element higher up the DOM tree, that method is a great big ball of uselessness. The problem is that the trigger element reports its background colour as being transparent (which is correct, but still annoying for my purpose).

My solution was to walk up the DOM tree until an element with a background colour is found:

function getBackgroundColour(elem, limitElem) {
  var bgColour = '';
  var styleElem = elem;
  var rTrans = /^transparent|rgba\(0, 0, 0, 0\)$/;
  var style;
  limitElem || (limitElem = document.body);
  while (!bgColour && styleElem !== limitElem.parentNode) {
    style = getComputedStyle(styleElem);
    if (!rTrans.test(style.backgroundColor)) {
      bgColour = style.backgroundColor;
    }
    styleElem = styleElem.parentNode;
  }
  return bgColour;
}

Notes

  • Different browsers return either “transparent” or “rgba(0, 0, 0, 0)” for a transparent background, so I had to make sure to account for both.
  • The limitElem parameter is there as a short-circuit, in case you only want to check within a certain container element rather than bubbling all the way up to document.body.
  • getComputedStyle() doesn’t work in IE8, but I wasn’t planning on making my library work in IE8 anyway. An equivalent API for that browser is element.currentStyle.
  • This behaviour assumes that there aren’t any funky absolute positioning tricks happening that make the child elements appear outside their parent container.

Why am I blogging about this seemly simple technique? Because I searched for a solution first and found bugger all, that’s why.

And before anyone asks: No, using jQuery will not help – it just defers to getComputedStyle() anyway.

Webcam tile sliding game

Another WebRTC experiment shown at SydJS

My first experiments with WebRTC were during the early phases of its development, at a time when there were very few demos and references to work from. One of those early demos was Exploding Pixels (part of Opera’s “Shiny Demos” to coincide with the release of Opera 12) which inspired my very first WebRTC experiment.

We all know the old tile-sliding games that are often given out as cheap toys to kids, where you’re presented with a jumbled picture or pattern and need to slide the pieces around one-by-one until the puzzle is complete.
I wanted to build one of those using JavaScript, but with a live webcam feed as the picture.

The first step was to build a simple tile game library that took some basic parameters and handled the interaction and game state.

var tiler = new Tiler('game', {
  width: 640,
  height: 480,
  rows: 3,
  cols: 3
});
tiler.setSrc('../common/img/catimov.jpg');

VIEW BASIC DEMO

Hooking up a webcam

Next came the fun part — integrating with WebRTC. To do this I build the library in such a way that the game could be redrawn at any stage with a different background image. It also accepted a parameter for the background that could be a URL path to an image, an image element or a canvas element.

Then it was a simple matter of hooking up the webcam using navigator.getUserMedia, updating a hidden canvas with the video stream (I didn’t use a video element directly as I wanted to flip the image so it acted like a mirror), then setting it as the game’s background image repeatedly.

var tiler = new Tiler('game', {
  width: 640,
  height: 480,
  rows: 4,
  cols: 4
});

var video = document.getElementById('webcam');
var source = document.getElementById('source');
var ctx = source.getContext('2d');
navigator.getUserMedia({video: true}, function (stream) {
  // A custom helper method to set the webcam stream
  // as the video source - different for each browser
  setStreamSrc(video, stream);
  // Flip the video image so it’s mirrored
  ctx.translate(640, 0);
  ctx.scale(-1, 1);
  (function draw() {
    ctx.drawImage(video, 0, 0);
    tiler.setSrc(source);
    requestAnimationFrame(draw);
  })();
}, function () {
  console.error('Oh noes!');
});

VIEW WEBCAM DEMO

Extra difficulty

Taking advantage of the fact that these aren’t real tiles, I added some extra difficulty by randomly flipping or rotating some tiles during setup. It obviously means that the game needs a “selection mode”, where clicking a tile selects it (at which point it can be flipped or rotated) and a double-click moves it.

var tiler = new Tiler('game', {
  width: 640,
  height: 480,
  rows: 4,
  cols: 4,
  move: 'dblclick',
  flip: true,
  rotate: true
});

This works for both static images and live webcam feeds.

VIEW STATIC IMAGE DEMO

VIEW WEBCAM DEMO

All the code for these demos is on Github at https://github.com/gilmoreorless/experiments/tree/gh-pages/tiles

NOTE: The usual caveats apply for these demos (i.e. They were built as quick experiments for a presentation and are not guaranteed to work properly in all browsers). I’ve tested in Chrome, Opera and Firefox as they were the first browsers to enable WebRTC.