New project: A Visual Studio Code extension for time zone data files

(A boring title, I know, but it’s stuffed full of juicy SEO keywords. Mmmmm.)

As mentioned in my previous project-related post

From now on, I’m going to attempt to write a proper explanation here for each side project I complete. Even if no-one else reads it, at least I’ve created some historical documentation for my future self.

As many people know by now, I’m a time zone nerd. I’ve given talks at conferences and meetups about problems and solutions when dealing with time zones. I sit on the time zone database mailing list and read about last-minute changes to a country’s time zone with a mixture of fascination, horror, and (mostly) amusement.

The IANA time zone database (also known as the “Olson database”) source files are incredibly rich with information. About two thirds of the lines in each file are comments about the data and its history. The project is part database, part history lesson, part social commentary.

A couple of years ago I wrote a Sublime Text package that added syntax highlighting specifically for the formatting of the time zone database files (available via Package Control). While it’s great to read the comments surrounding the data, sometimes you want the comments to be styled like comments and fade away a little, to let the lines of data stand out.
It seems to have been well-received, and Matt Johnson of Microsoft (a frequent contributor to the mailing list) suggested an improvement to the text formatting.

A few months ago, Matt contacted me asking if I was interested in porting the package over to an extension for Visual Studio Code (hereafter referred to as “VS Code”, because I’m lazy and it’s easier to type). I regularly use VS Code for coding, so I figured it was a good way for me to explore the extension ecosystem. I’ll describe how I converted it, and some of the mistakes and improvements I made along the way, in case it’s helpful for someone else.

If you want to cut to the chase and use the finished extension, it’s available as zoneinfo in the VS Code marketplace.

Version 1 — Conversion and syntax highlighting

The initial conversion was remarkably easy. As highlighted in the documentation, the VS Code team provides an extension generator. The generator has a feature that will take a Text Mate language definition file (used by Sublime Text) and convert it into a VS Code extension. Using the generator in this way meant that most of the hard work was done in one go.

npm install -g yo generator-code
yo code

A screenshot of using the "yo code" command line generator tool

I provided the name vscode-zoneinfo and the location of the .tmLanguage file from the Sublime Text package, and used the default values for the rest of the options (which were mostly derived from the contents of the .tmLanguage file). At the end of it I had a zoneinfo directory containing most of what I needed.

I had to tweak some of the generated data to properly match the Sublime Text package. For example, the zoneinfo.tmLanguage file defined specific file names (due to the naming convention of the tzdb source files), but the generator converted those to file extensions, which I had to change back.

The Sublime Text package also contains a default settings file to use the correct indentation, which was not detected by the VS Code extension generator.

{
    "detect_indentation": false,
    "translate_tabs_to_spaces": false,
    "tab_size": 8
}

Adding this into the VS Code extension was as easy as adding the same JSON properties to a specific section of the package.json file:

{
  // Other package.json stuff goes here

  "contributes": {
    "configurationDefaults": {
      "[zoneinfo]": {
        "editor.detectIndentation": false,
        "editor.insertSpaces": false,
        "editor.tabSize": 8
      }
    }
  }
}

Testing that the changes worked was easy thanks to the extension generator’s default configuration. It provides a “Launch Extension” command that starts up a new window of VS Code with your extension installed.

After that, it was a matter of the usual tuning and polishing of a new project before publishing it: a proper description and keywords in package.json, taking screenshots, writing a README and changelog, adding a licence file, and going through the rigmarole of signing up to yet another publishing platform.

Version 2 — Taking advantage of a new editor

With the initial request taken care of (thanks for the review, Matt), I decided to take advantage of the extra features VS Code provided. Sublime Text is a text editor, while VS Code is an IDE, so there are many more things that can be done.

This also provided a good excuse for me to try using TypeScript. VS Code is written using TypeScript and has first-class support for it, while the extension generator can also handily create all the files and configuration needed to get started.

The first step was to turn my simple config-based extension into something that could hook into the VS Code APIs and lifecycle. I re-ran the extension generator in a separate directory to create a default TypeScript extension, then copied over the files I didn’t already have in my project.

Rather than trying to do everything at once, I broke down the tasks into manageable chunks. I knew that I’d probably have to rewrite a fair amount of code by the end of it, but it helped me stay on track (and not get overwhelmed to the point of giving up). I’ll just highlight the main points here, rather than going into full super-technical details of each step (because that’s what a commit history is for).

Get something working

“Make it work, make it right, make it fast.”

The first step when venturing into new territory like this (new APIs and new syntax in TypeScript) is to follow the documentation and get the smallest, simplest thing working. Quickly getting to a state of “yep, that works” is the best motivating factor. If it takes a long time to get anything working, that doesn’t bode well for the rest of the project.

With that in mind, I whipped up the quickest working implementation I could. A command to “Go to Symbol in File” would parse the currently-focused file, extract the names of any Link, Rule, or Zone declarations, and return them as a list of symbol objects.

But first, the documentation says there are two different models for creating a language extension:

  • In-process — everything is handled within the extension process.
  • Client/server — the extension process just sends commands to a different process, and receives data back.

Uh oh, an architectural decision to make already! Luckily, in this case, the decision turned out to be an easy one. The client/server model is best for programming languages that have their own runtime. The JS-run extension can call out to a server written in another language (Go, Ruby, Python, etc.). Whereas my extension is for a “language” that is just specially-structured data with a whole lot of comments. In this way it’s similar to writing an extension for Markdown or SQL files.

So I followed the documentation examples, hooked up the simple text parser and… huzzah it worked!

A screenshot of the "Go to Symbol in File" feature

Restructure

“Now that I can get symbols for one file, it should be easy enough to get them for all the tz source files in the workspace,” thought Past Gil. Of course, with the benefit of Current Gil’s hindsight, Past Gil was naïve and should have foreseen that there would be problems. Silly Past Gil.

Overall, the documentation for writing VS Code extensions is very good, and tells me everything I need to know. However, trying to complete this particular task was the one area where I felt lost. The API docs didn’t quite explain the difference between a few seemingly-similar APIs, and one method didn’t work the way I expected it to based on its name. I tried looking for an example of what I was trying to do in the vscode-extension-samples repository, but all the language extension examples were for the client/server model, not the single-process model I was using.

I saved some notes about “things that could be improved in the docs” and persevered. Between confusion of which APIs to use and confusion about how I wanted this feature to work, I ended up changing the file structure a few times.

Some basic diagrams describing multiple attempts at structuring file includes. It took 4 attempts to get the right structure.

Eventually I got it working, and I was now able to quickly search for a zone definition across the whole tz directory. Double huzzah!

A screenshot of the "Go to Symbol in Workspace" feature

Shiny and chrome

Having got the big chunk of confusion out of the way, the rest was pretty smooth. I had to make the text file parsing smarter to find all references from one symbol to another, which then allowed for more features:

  • Click-through navigation for going to the definition of a Zone or Rule. This also brought in an inline preview when hovering, absolutely free of charge (i.e. I didn’t have to do anything, it Just Worked™).
  • Find all references to a Zone or Rule, which is really just the reverse operation of “go to definition”.
  • Smarter text matching when searching for symbols in a workspace
  • Hooking into document events to re-parse a changed document. This ensures that symbol references are correct even for unsaved changes.

Async I can

While looking for examples of other extensions, I realised that TypeScript would allow me to use the async/await syntax without adding any extra build processes. Since most of the VS Code APIs already use promises, it was fairly straightforward to convert to use async/await. The only real trouble I had was working out exactly where to put the async and await keywords when using Promise.all() and Array.prototype.map(). Once that was sorted, the code certainly did look neater and easier to follow. I’ll definitely be trying to use async/await more in future where I can.

After those improvements, it was time for another round of cleanup: remove all the debug logging, update the README file, and add more screenshots (often the most time-consuming part of publishing a project). And lo, version 2 was done.

Campsite rule

One thing I try to abide by when working with any external system (browser, IDE, someone else’s open source code, etc.) is the Campsite Rule: “Leave the campsite cleaner than you found it.”

As mentioned earlier, while I was building the extension I made a note of anything I thought could be improved about the experience. Once the extension was finished, I started raising issues and pull requests in VS Code-related repositories. I doubt that they’ll all get fixed, because there are priorities to be maintained on a large project, but at least I can say I did my bit to try to make it better for the next person.

Here’s a list of all the issues and pull requests I created for various VS Code projects:

In the end, I’m not too fussed about the future popularity of this extension. It was designed for a niche audience (i.e. people who read and navigate the source files of the IANA time zone database), and will get limited use. But it was a great learning opportunity for me, and I had fun making it. When my free time for coding is restricted to my morning train trip only, that’s mostly what counts.