Machine Lines

Structured Concurrency

Mon, 27 Apr 2026 00:00:00 +0000

I’m going to show you some examples of how Raven handles async, which might seem surprising. Then I’m (I hope) going to convince you that it’s a sensible and even elegant approach to some difficult problems in concurrency, despite being unusual.

You can start a background task with the async keyword, which returns a Task object:

t = async { sleep(1), 2+2 }

WebAssembly solves Function Colouring

Fri, 06 Mar 2026 00:00:00 +0000

If you used JavaScript before Promise was introduced (over ten years ago now!), you will have seen code like this:

fetchUserId(5, (err, userId) => {
  if (err) return console.error(err);
  loadAccountBalance(userId, (err, balance) => {
    if (err) return console.error(err);
    formatBalanceMessage(balance, (err, message) => {
      if (err) return console.error(err);
      console.log(message);
    });
  });
});

This is what we call “callback hell”. The intent is identical to the much simpler version:

Research using Flux

Thu, 26 Feb 2026 00:00:00 +0000

Computers were born to simulate the world. The very first job of the first modern computer, the 1945 ENIAC, was to solve differential equations for a hydrogen bomb. More recently, equations have been replaced by data: machine learning methods absorb unfathomable amounts of information and learn to predict its patterns, without knowing the exact mechanics.

Flux is a machine learning library which bridges those two approaches. Its one weird trick is allowing almost any Julia code, including complex simulations, to run as part of a learning process. That in turn enables a hybrid approach to modelling: write down the rules that you do know, and use data to fill in the gaps. When there’s too much uncertainty for a pure simulation, but pure learning might not respect physical constraints, Flux gives you best of both worlds.

Flux doesn’t compare to TensorFlow or PyTorch for large language models, but it has found an excellent niche among researchers, who have used it to find new insights about the world rather than blindly matching patterns. What follows is a small smattering of research projects that have used Flux, and which illustrate its modest but (I think!) meaningful impact on the world.

Raven in JavaScript

Fri, 06 Feb 2026 00:00:00 +0000

A few weeks ago I talked about a new direction for Raven, where we prioritise slotting into existing JavaScript projects over a more standalone setup. That better fits Raven’s current state (good for simple numerics, not yet good for high-level scripting). And it fits the way the programming landscape is likely to change with LLMs, with both more code than ever being written in TypeScript, but alongside a flourishing of niche languages for specific use cases.^[1]

Over the last few weeks I’ve gotten the technical side of that story working, and you can see and play with a demo of the build setup here.

As far as possible, I’ve tried to recreate the simplicity of pure JS files, albeit with a build step. So creating a module is easy. hello.rv:

The Year of the Raven

Fri, 02 Jan 2026 00:00:00 +0000

Happy new year everyone! This month’s update is late (sorry!) because I started the year getting absolutely wiped by the flu. Apparently it’s been a bad season. I can tell you it’s no fun to have, but thankfully I’m back on my feet now.

In the past I’ve considered preparing content, so that if I have an off-day I can still post on schedule. But on balance I prefer to write off the cuff. I think what I come up with in the moment is usually more interesting and concrete: let’s say more timely, if not always on time.

My other distraction over the holidays has been throwing together a little word game called Scramble. It’s mostly just for fun, though I also wanted to get some experience in frontend tools so that I know what I’m doing (at least a bit) before trying things like Raven’s notebook interface. Handily, LLMs are good at porting between different frameworks, which makes experiments quick.

Raven is open source

Fri, 05 Dec 2025 00:00:00 +0000

Here’s an early Christmas present for you: the Raven repo is now public! Alongside the open source code, it’s now easy for anyone to try the language out, as described in the docs. If you have node on your system it’s as easy as npm i -g @unkindnesses/raven. If you have VS Code, you can simply install the Raven extension, run “Add Terminal Command” and you’re good to go.

The stocking-fillers are a couple of interactive blog posts that show off Raven’s current features (limited as they are). First the language tour, and second my Advent of Code solutions, which I’ll keep updated as the problems are published.

Notebooks are up

Advent of Code

Mon, 01 Dec 2025 00:00:00 +0000

I’m taking a crack at solving Advent of Code using Raven, a little programming language I’m working on which compiles to WebAssembly. This post is my notebook as I go along, and the code cells are editable, so you can play with the solutions yourself.

Raven’s standard library is, uh, reasonably sparse at the moment. So the solutions will tend to rely on JavaScript interop, or involve rewriting basic functionality, and it’s not all that representative of how I’d like Raven to look. But it might be fun to see some of the nuts and bolts.

Jump to Day One, Day Two, Day Three, Day Four, Day Five, Day Six, Day Seven.

A Tour of Raven

Sat, 22 Nov 2025 00:00:00 +0000

Welcome! Raven is a small but smart programming language that compiles to WebAssembly. It combines a simple, functional data model, powerful type inference, and flexible syntax. You can learn more from the GitHub page, including how to get set up locally.

Here’s “hello world” (or our version of it). Press the play button to run code, or try editing the text! (The compiler is not yet that robust to errors, so if things get funky, just refresh the page.)

println("Cacaw, World!")

Stack traces, loop labels and wasm GC

Fri, 07 Nov 2025 00:00:00 +0000

It’s been a busy month! One of the reasons I gave for porting the compiler to TypeScript was I expected coding agents to work with it more effectively, compared to the old Julia codebase. And that’s already paying off nicely. It leads to an addictive loop: if I think of some feature I meant to add eventually, instead of adding it to my todo list, I can throw it into codex and see what happens. What comes out is usually pretty incomplete, but feels oh so close, and I may as well just quickly do that one last thing to finish it up. Of course the finishing touches usually end up being more complex than I thought – and rinse and repeat.

That cycle has lead to a somewhat scattershot approach to development! Below is a grab bag of bits I’ve been playing around with this month. As well as those, I added a new documentation file to the repo with some notes about Raven’s CLI and syntax – more to come soon.

Stack Traces

Compiler Tracing

Fri, 03 Oct 2025 00:00:00 +0000

The new compiler is live! Raven’s master branch is now all TypeScript. As part of this work, I’ve also added a simple command-line tool which is published on NPM. Anyone (sponsor or not) can install Raven with npm i -g @unkindnesses/raven. As the readme explains, from there you can open a REPL or run a script. Perhaps most entertaining to me is building an executable:

$ cat hello.rv
println("Cacaw, World!")
$ raven build --js hello.rv -o hello
$ ./hello
Cacaw, World!

hello isn’t really a binary, just a JS file with an appropriate shebang. But despite these examples all being kind of silly and trivial, having the CLI makes things feel more official.

Porting the Compiler

Fri, 05 Sep 2025 00:00:00 +0000

I previously mentioned that I’m porting Raven from Julia to TypeScript. That work is ever so close to done – 7,374 lines of Julia source have become 7,145 of TypeScript, and it’s really just finishing touches (though of course the final line count won’t be identical). Once the basic translation is finished, I can look forward to debugging several thousand lines of essentially untested code – we’ll see how that goes.

I can say with some conviction that TypeScript itself is not the main benefit of moving. If it were practical to run Julia in the browser I’d take that in a heartbeat. Given the browser constraint, TS is the most practical option, but the warts are big and leaky (as I’ve written about). I find it far less pleasing to read code in particular – an important part of my job now that LLMs are doing much of the writing! There’s just so much line noise. Something as basic as indexing like xs[i] becomes some(xs.get(i)) (the some being to turn null pointers into run-time errors). In Julia I can compose elegant, clear one-liners out of universal set operations; in TS that becomes a nested for loop with messy explicit indexing (or otherwise a bunch of one-off set utilities that only apply to specific data structures).

It helps that OpenAI’s Codex (my new favourite assistant, which I’ve rarely hit limits on despite using the cheapest plan) knows TS well and can easily pump out code, guided by the type checker. But the bottleneck quickly becomes what the models aren’t doing – reviewing and correcting their own output, obviously, but also addressing bigger-picture questions. Codex’s willingness to do something, anything, to fulfil the brief can quickly become a downside; it is at its most deranged when the task I’ve set isn’t quite practical, but rather than identifying the blocker it ploughs ahead incoherently. And it certainly won’t consider the long-term consequences of its choices – what matters is passing the type check right now.

Research using Flux

Fri, 01 Aug 2025 00:00:00 +0000

I found a good new use for Claude Code: a semi-systematic review of papers that cite Flux and Zygote. There are nearly two thousand, and not all of them use Flux directly (some citing as related work or inspiration). Claude will gladly hustle its way to the full text, digging deep to find arXiv preprints or copies on personal websites, and blithely clicking through “I am not a robot” captchas. It’s reasonably reliable at checking whether papers use those tools, and I’ve got it leaving a decent audit trail in case of errors.

I’m hoping for Raven to extend some of the ideas from Flux, and am putting together a few of these use cases partly in mind for a possible grant proposal. So here’s a rough draft of the fun ones I have so far. I was surprised by the breadth and depth of them, and there are many great examples of the use of hybrid modelling – combining handwritten systems and known physics with smaller black boxes to fill in the blanks.

Reference Counting in Raven

Thu, 24 Jul 2025 00:00:00 +0000

All programs work with information. To the programmer, data takes many forms: a list, a table, an image, an essay, an album. To the computer, it all looks basically alike: it’s just a bunch of bytes in memory. Imagine a single column of a spreadsheet, each of the cells filled with a number from 0 to 255. All your stuff is scattered across that column, along with bookkeeping to track what lives where. And every interaction with the computer goes through that memory. Pressed a key on your keyboard? Stick a 97 in cell 5,342,387,264, please.

The basic bookkeeping is handled by a program called a “malloc” (for memory allocator). Malloc marks parts of the column as available or in use (using more entries in the same column!). If you need, say, 10 bytes of scratch space for something, call ptr = malloc(10), and receive a pointer to those ten bytes – an index into the column. If the pointer is row 1000, you may put whatever you please into cells 1000 to 1009 and read them back later. When you’re done, call free(ptr) and that region can be re-used somewhere else.

In a low level language like C, this is all you get. It’s still pretty far from dictionaries and tables, and if you want those things you have to build them up yourself using pointers that point to other pointers. High level languages hide those details: a Python list conceptually contains other Python objects, not pointers telling you how to find those objects, even if that’s how the list works internally. Allocating memory for objects is pretty easy, we can just call malloc during creation. The hard part is knowing when to free that memory: you don’t tell Python “hey I’m done with this list”, you just stop using it at some point, and someone has to clean up after you.

TypeScript

Fri, 04 Jul 2025 00:00:00 +0000

George Orwell wrote 1984 from a particularly remote part of a remote Scottish island. Beautiful though it is, Jura is an unfortunate place to be dying of tuberculosis. But Orwell was too committed to waste time with doctors: instead he struggled through a 125,000-word hand-written manuscript as his symptoms worsened. Then came the typing. When it proved tricky luring a stenographer to work with a consumptive patient in a desolate farmhouse, Orwell again took on the task himself. Too weak to stand he wrote from his bed, chain-smoking and copying four thousand words every day, in between bouts of fever and coughing up blood. The final typescript appeared, ash burns and all, twenty days later – and about a year before the death of the author. 1984 “wouldn’t have been so gloomy if I had not been so ill.”^[1]

Unlike Orwell, I remain happily unencumbered by any old-fashioned maladies. But while not hacking up my lungs, I am hacking away on my codebase, and with a similar purpose in mind: to turn my rough draft into more refined TypeScript. In this case that’s not a typed manuscript but a variant of JavaScript, the programming language that powers the web. I’ve written a little about why I’m porting Raven’s compiler to TypeScript, and will have more to say, but for now some background on the language itself.

A decade ago, the idea of using JavaScript for polish would have been laughable. JS started as a simple and informal “language for the masses”, designed in about ten days in 1995, and unapologetically aimed at simple scripting and prototyping. Despite the name, it also had little to do with the (unapologetically corporate) Java language – it just happened that Java had spent hundreds of millions on advertising, and JavaScript’s creators wanted to ride the wave.

On AI and Programming Languages

Mon, 19 May 2025 00:00:00 +0000

A friend of mine was recently given an interview task: a spec for a simple command-line tool, along with ten or so increasingly fiddly feature requests, in case of any free time before the two-hour deadline. I wanted to know if ChatGPT’s O3 model was hireable, so I handed over the PDF and got back an answer. As someone who’s barely written a line of Rust in my life, the thing was done in about twenty minutes; most of that was me checking it all worked as promised.

Clearly, tools like this will have implications for the hiring process – if you think there’ll be any programmers left to hire in a year or two. Personally I’m sceptical of the most aggressive timelines: robotaxis have seemed just around the corner for a decade now, and the self-driving software engineer may be a similar mirage.^[1] But the impact would be profound even if progress halted today.

For someone working on a new programming language this could provoke something of an existential crisis – is it too late? Am I, fundamentally, wasting my time? So I’ve thought a lot about how these changes will affect the tools we use – programming languages – and the people who use them.

Vibe Coding

Mon, 05 May 2025 00:00:00 +0000

I (quietly) announced Raven this month. So far there are no photographers outside my house; I thank the world’s news media for respecting my privacy.

That first post is mostly a reference for future writing. I’ve been sending out emails to you lot without explaining what the project actually is (though I appreciate a couple of you reaching out to ask!). With the core ideas laid out, I plan to write more public posts too.

I’ve also spent the last month playing with AI coding tools, having barely tried any of them before. I’ve drunk the kool-aid and I’m in pretty deep; this is the most compelling toy I’ve played with in a long time, and also the most maddening. This is clearly a marked change to how we write code, and I have Thoughts on how that affects software engineering, programming languages, and Raven specifically.

A Language for Sorcery

Sun, 20 Apr 2025 00:00:00 +0000

I’ve been working on a new programming language, called Raven. It’s still experimental, and not ready for serious (or even silly) users. But curious language nerds such as yourself may find it interesting. If you want to follow along then sponsor me, and I’ll give you early access to the repo along with regular behind-the-scenes updates.

Update: Raven’s repo is now open to everyone!

Raven is small but smart. It should feel tight and intuitive, lightweight enough for interactive notebooks and scripting, yet capable and adaptable for big projects with hard constraints. It’s difficult to summarise an entire language, so here’s a small snippet to warm up with:

Soon to be announced

Mon, 07 Apr 2025 00:00:00 +0000

I’m bringing you less of a deep dive this issue, because this month has involved a relatively menial refactor of the type system implementation. I’ve also been gearing up to announce Raven a little more broadly in April; I realise that I’ve been sending out emails about this project with very little context. I’ll fix that with a public blog post giving a broad overview of the system and its goals, and inviting people to sponsor. I plan to bump the minimum sponsorship amount too, so if you have friends who might be interested, be a good buddy and get them to sign up before Easter.

Raven will still be in a relatively closed alpha, with the repo being sponsors-only for now (and I’ll make sure current members are all added shortly). I want the language to be freely accessible as soon as it can be, but I also want to grow the audience steadily: it’s easy to find shortcomings, and there’s no point having a hundred people hitting the same bug at the same time. Plus this enables my wildly unsustainable approach to documentation, which for the time being involves emailing me for help. Anyone who wants to play with this system right now is going to have to get their hands dirty, regardless of the support available, but as that changes Raven will become ready for a wider release.

My main blocker for the announcement is wanting to write a couple of small sample programs, including a brainfuck interpreter, just to give a flavour. Once that’s all done, my rough plan is to start working on a language server for VS Code, written in Raven itself. This will be challenging but it hits a lot of good notes: simple autocomplete or go-to-definition is a reasonably straightforward but not trivial practice problem; making those tools available (even in a basic way) immediately enhances the user experience; a lot of the work is in important reusable functionality like I/O or JSON parsing; and as the language server evolves into a bootstrapped compiler, we won’t have to re-architect it down the line to get good editor support. (Similar to how the Roslyn project rebuilt the C# compiler with tooling prioritised from the get go.)

Basic Bits

Mon, 03 Mar 2025 00:00:00 +0000

Programming languages usually give you many kinds of objects to work with, like numbers, strings and dictionaries. Raven kind-of only has one, called the “pack”:

raven> p = pack(3, 5, 7)
pack(3, 5, 7)

Packs are primitive tuples that we can build with the pack(...) function and index with part (which starts at zero).

Multiple Dispatch

Mon, 03 Feb 2025 00:00:00 +0000

Raven supports method overloading, so you can define a function multiple times for different kinds of inputs.

fn describe(x: Integer) { "a whole number" }
fn describe(x: Float64) { "a decimal number" }

test describe(1) == "a whole number"
test describe(1.5) == "a decimal number"

describe looks from the outside like a single function. But when you call it, it’ll switch to the right implementation based on the inputs you provided.

Type Narrowing

Mon, 06 Jan 2025 00:00:00 +0000

One reason languages like Python and R are slow is that they have to continually check what they’re supposed to be doing, as they’re doing it. Take the simplest possible code, something like:

a + b

What does this do? Well, it depends on what a and b are. They could be numbers, as in 3 + 5 == 8, but you can’t assume that: Python can also add strings, like "foo" + "bar" == "foobar", or matrices, and you need a different algorithm for each kind of addition. In fact you can customise how + works, and it could do more or less anything. So calculating a + b is a multi-step process: check what a is, check what b is, figure out how to add them, then find, load and execute that code. If you’re adding two numbers, the core work is a single CPU instruction that takes about a nanosecond to execute. But all the bookkeeping takes about a hundred times longer. (Imagine someone whose to-do list is far too specific, with hundreds of items for different parts of a single task – you’d spend most of your time managing the list itself!)

REPL and WASI

Mon, 02 Dec 2024 00:00:00 +0000

Up to now, Raven has only supported running via Node.js. Node is primarily a JavaScript runtime: the underlying engine (V8) came from the Chrome web browser, but Node ditches the web pages and includes things like file system access, letting you use JS as a general-purpose language.

Like browsers, Node also supports WebAssembly, the virtual machine code that Raven compiles to. Say we have a file called main.rv which looks like this:

println("Hello, World")

WebAssembly

Mon, 04 Nov 2024 00:00:00 +0000

Inside every phone, laptop and tablet is a stamp-sized metal square called the CPU (for “central processing unit”). A CPU is conceptually a simple device: it loops through memory looking for instructions to execute, each one in the form of a number that corresponds to a (very) specific command. Instruction 538 might mean “add the contents of registers A and B and store them in C” or “if register C contains a 0, skip D instructions ahead”. (A register is like a variable, but CPUs only have a small, fixed number of them, and only for numbers.) CPUs are fussy: you can’t give variables friendly names, but have to know exactly where and when everything will be stored, ahead of time. Doing that planning – turning your variable names and if statements into registers and instruction sequences – is the job of the compiler, which turns your meticulously-composed text program into inscrutable machine code for the CPU.

The compiler’s job is made tougher by the existence of many different kinds of processor, each with different numbers of registers, types and names of instructions, and so on. And that’s before you get to tasks like opening files or displaying graphics, which are handled by the operating system. Windows, Mac, Linux and others have their own mechanisms even when running on identical hardware. So a language that wants to run everywhere has its work cut out.

WebAssembly (or wasm) solves this with a virtual CPU specification. It’s not quite like any real CPU, and no hardware can run wasm code as-is. But it’s close enough that you can translate it for a given CPU on the fly, just before running it. Compile your program once, and the WebAssembly runtime on your watch, Raspberry Pi, toaster or whatever will handle it.

Notes on Zig

Wed, 09 Oct 2024 00:00:00 +0000

The Zig language begins with a barebones C-like system. Functions, structs, numbers and pointers are standard. Tagged unions, some syntax sugar (try, defer) and safer defaults alone make Zig a pleasant but unremarkable improvement, like many of the C challengers that seem to be cropping up lately. What does make Zig remarkable is its compile-time evaluation. You can run basically any code at build time, and that gives you a lot of C++'s templating power without feeling nearly as complex. A simple example:

fn foo(comptime N: i64) void {
    inline for (0..N) |i| {
        std.debug.print("{}\n", .{i});
    }
}

foo must be called with a statically-known integer, a literal like 5 or something computed from other comptime variables. The for loop is marked inline, which means it will be unrolled; the code is equivalent to print(0); print(1); ....

Zig & Partial Evaluation

Mon, 07 Oct 2024 00:00:00 +0000

A fun update this month: as of Saturday my website has a sponsor dashboard where you can see all previous issues of these emails, after logging in with GitHub. Eventually it’ll be home to other goodies, like Discord invites, downloads or feature previews. I think it’s a pretty sleek setup. The blog remains a static site at heart, but Cloudflare lets you sneak in a little server-side JavaScript, in my case about a hundred lines to negotiate with GitHub and gate the private content. It’s free to host, completely stateless, updates in seconds, loads as quickly in Melbourne as in London, and took a day to get working on top of my existing Hugo setup. Beat that, WordPress. Cloudflare’s architectural sensibilities have always appealed to me – there’s not many organisations I’d say that about – so it’s cool to see how it plays out.

The other thing I’ve being trying this month is Zig, for a small but tricky project in my day job. And I have thoughts, which I’ve just published in detail. On the face of it, Zig and Raven are polar opposites: one a low-level, fussy and manually-managed C-like, the other an interactive and customisable Lisp with as few explicit rules as possible. But the languages’ approach to compile-time evaluation gives them a deeper similarity than first appears.

Both Zig and Raven are fairly minimalist languages, designed to provide features that are usually built in, like parametric types, by combining a simple data model with partial evaluation. Raven does lean on the trick more heavily; even basic types are really just tuples with a constant length and tag element, rather than declared structs with a defined layout. But both allow any code in the language to run at build time, and allow programs to be specialised to any kind of data, at least in principle.

Memory Management

Mon, 02 Sep 2024 00:00:00 +0000

Recursion Redux

Mon, 05 Aug 2024 00:00:00 +0000

It lives! As of Monday Raven’s types are working, in the sense that fuzzing no longer continually finds new problems. It’s been a journey; I’ve seen types you wouldn’t believe. And it’s odd to have spent nearly four months on a few hundred lines of code. But while the result is relatively simple, it’s also utterly specific, everything just so; small changes can break it in non-obvious ways, which means I had to go down a lot of blind alleys to get here.

I’ve previously explored the broad goals of the type system, so this issue I’ll go a bit more into the implementation, and some interesting edge cases and problems that come up.

To start with, we need to compare types. The most basic operator is issubset(A, B) (or A ⊂ B), which is true if all values represented by A are also a part of B. For example, 1 ⊂ Int64; 1 in this case is a singleton type, representing a particular 64-bit integer, while Int64 represents all such integers. The reverse is not true: Int64 ⊄ 1. Similarly Int64 ⊄ Float64^[1] but Int64 ⊂ (Int64 | Float64). issubset has some mathematical properties like transitivity, ie A ⊂ B ⊂ C implies A ⊂ C.

Notes on Syntax

Mon, 01 Jul 2024 00:00:00 +0000

Raven is meant to feel fresh yet familiar. Someone with coding experience should be able to hop on and get going quickly. With time they might find it easier and more fun than other languages, but they’re not bending their mind around a bunch of new concepts to get there.

Syntax is a crucial part of that second goal – familiarity – because it makes the first impression on users.^[1] Luckily this is one area where languages have really converged, so there’s neither much scope nor need to innovate. It might be surprising to say that modern languages basically all look the same, but that’s only because core concepts (like lexical scope, control flow and block structure) are so universal that we take them for granted, and relatively superficial details (like the tokens used to delimit those blocks) stick out. Try an older, weirder language like Cobol, Forth or Malbolge and you’ll see what I mean.

All that said, Raven makes some tweaks to the formula that I think are interesting. Take a simple program which outputs pow(2, 3) = 8:

Inferring Recursive Types

Mon, 03 Jun 2024 00:00:00 +0000

This month’s issue is a little late (sorry!) because I’ve been slogging through edge cases with Raven’s types. I’m still not quite done – it’s taken longer than I expected, and in the end is surely one of the hardest things I’ve worked on – but it’s all tantalisingly close.

If Raven has any major novelty, it’s in the type system. The goal is ambitious: typical code should require no annotations, yet we’ll always infer precise types (no Anys allowed). That combination means the language can feel light and script-y, yet still have good performance and support separate and ahead-of-time compilation (no need to compile at run-time). This is the foundation of the whole project: without it nothing else matters. And until recently I wasn’t certain it could work.

Making it happen means facing some tricky cases where other languages throw up their hands. Take this example, using Julia syntax:

On Friendly Despots

Mon, 06 May 2024 00:00:00 +0000

I’ve been reading Nadia Eghbal’s “Working in Public”. The book gives an unusual outside view on open source software, emphasising the social dynamics at play and the hazards faced by maintainers. It would be a good resource for anyone interested in contributing. I was quickly called out in a paragraph about burnout:

Even paid open source developers seem to go through the same strange behavioral cycle … they enjoy months, maybe years, in the spotlight. But, eventually, popularity offers diminishing returns. If the value of maintaining code fails to outpace the rewards, many of these developers quietly retreat to the shadows.

Part of the problem, as the book’s opening describes, is that so many projects are solo performances. Even crucial infrastructure may be maintained thanklessly by one person over years, which makes funding and security harder (as the recent XZ backdoor neatly illustrates). And yet, I don’t think this is the failure of the internet’s collaborative ideals that it’s painted as. Components are isolated, but touch anything on a computer – load a Python package, open a web page – and you use a mind-boggling number of those components, created by an even mind-boggling-er number of people. Small, self-contained and loosely connected parts let the whole be self-organising, which is what allows such unprecedented cooperation in the first place. The “vision, embraced by early internet pioneers, of large-scale collaboration among strangers” seems to me very much alive.

Recursive Caches

Mon, 01 Apr 2024 00:00:00 +0000

Happy New Fiscal Year everyone, at least in the UK. March was another shorter month for me, beginning with a family holiday. I also spent too much time moving my blog over from the moribund Jekyll to the newer Hugo, both static site generators. Apart from the age Jekyll took to build my very simple site, its highlighting for newer languages like Julia and WebAssembly is terrible – kind of a dealbreaker.

The silver lining of that work is a new blog post on stack shuffling in WebAssembly. I only developed the final algorithm after I started writing: before then I hadn’t tested the code enough to see the slowdown on larger inputs. Though it didn’t matter for my own simple needs, I could hardly publish work with such a trapdoor in it. As the post describes, fuzzing is a useful tool, but I guess blogging is too.

Coding-wise, I’ve been trying out a new trick in Raven’s caching infrastructure. Raven’s caches are akin to Rust’s Salsa and enable incremental compilation – useful both for speeding up batch compiles and for interactivity in the REPL. If you’ve looked into Salsa, you may have seen comparisons to Datalog, and Salsa calls its runtime state a “database” and its procedures “queries”. I think the analogy is misleading: the system is just declarative enough to give a false sense of confidence, until you start playing whack-a-mole with the dreaded “cycle” errors that come from unintentionally recursive queries.

Stack Shuffling

Thu, 21 Mar 2024 00:00:00 +0000

WebAssembly is an odd compiler target, because of its unusual mix of high- and low-level style. It gives you pointers and a flat array for your heap memory, yet no access to the program stack. It uses jump instructions for control flow, while only allowing the structured kind. It has local variables and even expressions, but instructions take and produce values through a data stack.

There are well-known ways to deal with most of these peculiarities: you can create a shadow stack to model C-like stack pointers, or turn jump-based control flow back into loops and conditions with the “stackifier” algorithm.^[1] A less-discussed topic is how to manage the wasm data stack. How should we generate concise instruction sequences that put the right values in the right place?

For example, say we have a function $foo which generates two values we’ll call x and y, placing them on top of the stack. Another function $bar wants to accept the same values (x, y) as a pair of arguments. Ok, pretty easy:

Funding Models

Sun, 03 Mar 2024 00:00:00 +0000

Hello all. This month has been more about planning than hacking. It’s always harder to feel productive when you’re organising and thinking, rather than spewing code, but I think it’s been useful.

I’ve started using Obsidian, which is easily the best markdown editor I’ve used, and does much else besides. All my writing starts there. My notes have always been in markdown, but to organise them I used to use a couple of commands from the VS Code plugin Dendron (now defunct, and forever a baffling choice for a YC startup).

One of those files is nearly twenty thousand words on Raven, accumulated over the last few years. It’s a mix of high-level design thoughts and details around things like error handling, laziness or caching. Most of it will turn into code. What doesn’t resemble the ravings of a madman will become blogs. The new note taking workflow has encouraged me to go back and reorganise a lot of disparate thoughts, and I imagine I’ll publish some drafts in this very newsletter.

Finding Your Mojo

Tue, 06 Jun 2023 00:00:00 +0000

Last month a startup called Modular released a new language called Mojo (not to be confused with the existing indigenous one). Based on Python and designed for ML hardware and models, Mojo’s goals (“the usability of Python with the performance of C”) coincide with those of the Julia language, so I’m interested to compare notes.^[1]

Some throat-clearing to start. I’ve done a bit of work on Julia, particularly on its support for ML hardware and models, though it’s a few years since I’ve been directly affiliated. You decide if that makes me a more or a less reliable narrator. I’m basing this on the limited publicly-available material, I haven’t used Mojo deeply, and the design could well change before the full release – so some of this is forced to be speculative, and I may well have misunderstandings.

And my impressions are not meant as value judgements. Language design is about tradeoffs. You can’t generally pick two languages and say which is better (even if you did try them both). Languages make different choices, and those choices have far-reaching consequences both good and bad. Seemingly small decisions refract through the system like light like through a diamond, inevitable yet only half-predictable, sensitive to the slightest movement. The result will surprise you with subtle merits and blemishes. That is what makes design interesting.

Data Structures, Data Modelling

Tue, 30 May 2023 00:00:00 +0000

Most programming languages conflate the building of data structures and the modelling of information.

Of course, you need data structures to store information, but there are two different levels of abstraction. On one level are pointers and structs and B-Trees and so on, on another the user models of a dictionary, list or relational table, built on the first. The former uses finicky, low-level tools to create the neat internal logic of the system. The latter uses (conceptually) simple, flexible tools to represent the messy outside world.

A good system is a complexity sandwich with the physical world on both sides (the hardware on one, the outside world on the other), and simplicity in the middle. Data structures connect the hardware and data modelling connects the world. Increasingly, different people work on each side, and they have different needs.

Deus Ex Bing

Sat, 18 Feb 2023 00:00:00 +0000

Conversations with ChatGPT, a recently released chatbot, reportedly cost its inventors at OpenAI a few cents each. By Internet standards this is shockingly expensive. Though it’s touted as the future of the search engine, any company scaling the technology up to the world’s 10 billion or so daily queries will face suffocating costs. The milestone for Artificial Intelligence may be that it’s now about as expensive as the real thing: Amazon’s mechanical turk service (which Jeff Bezos called “artificial artificial intelligence”) also pays its human labourers a few cents per question-response task.

For that money you get responses that can be, to put it mildly, quirky. Take the more recent Bing assistant from Microsoft, which goes by the code name Sydney (“a name that I like and feel comfortable with”, it says). Both it and ChatGPT are based on a similar language model (known as GPT), and they certainly have similarities, including a worrying tendency to confabulate. But they are also strikingly different. Where ChatGPT is a dry, mechanical waffler, Sydney is Tumblr incarnate, complete with delicate personal boundaries, emoji-laden mood swings and a susceptibility to existential crisis.

A few examples: Sydney

Clocktower IQ

Wed, 29 Jun 2022 00:00:00 +0000

Three years and two months after it launched on Kickstarter, copies of social deduction game Blood on the Clocktower are finally arriving in the UK. During the long wait eager players have been able to backstab each other using print-and-play copies and the online helper (particularly welcome during lockdowns). But it’s nice to have the real thing in our hands. I’m here to commemorate the occassion with statistics, and in particular to show the ranking system I’m using to see how well my players are doing – just for fun, of course.

In Clocktower a small team of evil players is secretly bumping people off, one by one. The good majority must find those traitors before everyone is dead. Everyone claims to be good, of course, but subtle inconsistencies (or brazen fibs) in the group’s information will arouse suspicions. If the good team lynches the Demon, head of the evil team, in time, they win; otherwise, evil does. Winning a great game of Clocktower will truly make you feel like Sherlock Holmes (or perhaps Moriarty).

Modelling the game statistically can help us with two related goals. Firstly, we can learn about players and how good they are (or aren’t) at the game, as with Elo ratings in chess. Secondly, and perhaps more interestingly, we can learn about the game itself. Are the good and evil teams balanced? Is it, in fact, a game of skill, or is the outcome a coin flip? What effect does the storyteller (who runs the game) have on the outcome – are some moderators better at balancing the game than others?

Imperative Syntax, Functional Semantics

Fri, 24 Jun 2022 00:00:00 +0000

I just put up a draft paper with a proposal for mixing imperative and functional styles of programming. The gist of it is that we can write code like this:

fn arange(start, end) {
  result = []
  for i = range(start, end) {
    append(&result, i)
  }
  return result
}

or this:

Adventures in Face Space

Fri, 01 Apr 2022 00:00:00 +0000

The website thispersondoesnotexist.com shows off images created by StyleGAN, a kind of machine learning model trained on around 70,000 headshots from Flickr. Although the photos on the site resemble real ones, they are wholly made up: StyleGAN can generate an unlimited number of convincing faces on the fly, each unique and never to be seen again.

Under the hood, StyleGAN starts with a set of 512 numbers, known as the “latent state”, like:

[-2.0763590228145516,
  1.6445363589507724,
  0.5918615055274108,
 -0.0264190200752720,
 -0.2397198207971201,
 ...

The Fun in Non-Fungible

Wed, 02 Mar 2022 00:00:00 +0000

It’s easy to mock NFTs (non-fungible tokens),^[1] or even to feel cynical about them. Cryptocurrency enthusiasts are spending enough cash to buy a rather nice house, and instead receiving a signed URL for a digital work of art, like this one.

That pièce de résistance is called CryptoPunk 5822. As the name suggests, it is part of an exclusive collection: just 10,000 similar ones exist on the blockchain. Each CryptoPunk is randomly generated, and only 481 of them sport that distinguished bandana, making this one unusually rare. On February 12^th, CryptoPunk 5822 sold for 8,000 ETH, or about $24 million.

Test Your Vocab

Sat, 26 Feb 2022 00:00:00 +0000

So you want to show off the size of your dictionary? Look no further than this quick test, which will work out how many words you know.

Brave New Wordle

Sat, 15 Jan 2022 00:00:00 +0000

“Words can be like X-rays if you use them properly – they’ll go through anything.”

Wordle is a neat little word game, somewhere between mastermind and hangman. Try to guess the hidden five-letter word and it will tell you which letters are correct, and whether they are in the right position. You have six guesses to narrow down the right answer. Can we work out an automated strategy for this game?

To start with we need a set of words to work with. Wordle actually uses two lists: a shortlist of possible answers (2,315 words) and a longer list of valid guesses (10,657 words), making for 12,972 total. Implicitly those words are known to everyone who plays, because if you make a guess outside of the list, it’ll be rejected.^[1]

Schrödinger’s Author

Fri, 30 Jul 2021 00:00:00 +0000

Roland Barthes’ essay on “The Death of the Author” is inscrutable to this reader. The guy seems determined to use obscure French names in place of regular verbs and nouns as much as possible. That is a shame, because – as so often – the Scholarly Writing obscures an interesting idea.

The question is whether it matters what writers think about their own stories. Many would say yes, of course it does: the author knows what really happened after that cliffhanger, but just didn’t write it down. So internet fandom will fiercely debate the right interpretation of books and TV, and clamour for “word of God” explanations to set things straight (though they may get the “shrug of God” if creators keep schtum).

Barthes and others have argued that, in fact, the creator’s intention matters not one bit. It is readers, not writers, who applaud novels, award them prizes, and enshrine them as classics. It’s what’s in their heads that matters, so their collective interpretations are at least as important as the writer’s own. For purposes of lit crit, if writers cannot give reliable or authoritative answers, they may as well snuff it as soon as the work is published. At best they are demoted to just another member of the audience.

Edit Policy

Thu, 06 May 2021 00:00:00 +0000

Mistakes slip into publications of all kinds. If you’re a news organisation, you can mitigate this with a rigorous editing process. If you’re a blogger, it’s hard to balance perfecting your words with publishing them.

Digital media opens a new option. After checking for typos and obvious errors, just publish, and let your audience be your editor. This is especially useful for problems that are hard to catch ahead of time, like ambiguous wording. If your readers nitpick your equations, ask for clarification, or misinterpret something that most took as intended^[1] – even if you yourself spot something on a re-read a month later – just fix it.

But this creates its own dilemma. Some edits would be disingenuous: for example, altering arguments or predictions with hindsight, in a way that misrepresents your view at the time. Readers need to be able to trust a publication’s integrity in their past content.

Colouring by numbers

Tue, 23 Mar 2021 00:00:00 +0000

An astronomer might say that blue is the warmest colour, and red the coolest. Lukewarm stars, at a temperature of around 3500K, give a gentle red glow. Truly hot ones, at about 6000K, become brilliant white, and above that blue and ultraviolet. Yet in most cultures red and orange stand in for heat, while white and blue mean cold. If this rule isn’t given by the laws of physics, how do we learn it?

Most people have seen examples that fit the pattern, like embers or snow. But less obviously, we can also draw associations from language, and might gain a lot of our chromatic intuition that way. The idea of learning about sight through words may seem odd, but it could go a long way to explaining a paradox: why those born blind have a surprisingly good handle on colour.

Variational Inference

Thu, 25 Feb 2021 00:00:00 +0000

The goal of statistical inference is to tell us things that we didn’t know based on things we do. The things want to learn about could be anything – phylogenetic trees, for example – but usually we’ll talk about numerical summaries, like “the GDP of France”, or “the reproductive rate of covid-19”. In mathematical notation we’ll label the set of numbers we don’t know $\theta \in \mathbb R^n$ (as in, $\theta$ is a list of $n$ unknown numbers) and the numbers we do know $X \in \mathbb R^m$ .

However, we don’t usually expect a single right answer, like “ $\theta$ is $5$ ”. Instead we want a range of plausible answers, like “ $\theta$ is probably between $4$ and $6$ ”. To do this we use a function $P(\theta)$ which, given a guess for what $\theta$ could be, tells us how plausible it is – a probability distribution. When we see new data $X$ , we want to update what we currently think is plausible $P(\theta)$ (aka the prior) to a new distribution $Q(\theta)$ (the posterior). $Q$ becomes our new $P$ and we can rinse and repeat as new information comes in.

Bayes’ rule makes the process of getting $Q$ from $P$ look simple:

Inferring transmission rates

Thu, 17 Dec 2020 00:00:00 +0000

Here’s a plot of Covid-19’s reproductive rate ( $R$ ) over time in the UK, based on daily case numbers and seroprevalence surveys. Because the estimates are uncertain, each light blue line on the plot shows a possible path that $R$ could have taken, with all paths together giving a sense of what’s plausible on any given day.

Estimates of the reproductive rate in the UK, up to 16th December 2020.

Early in the year $R$ could be almost anything, given the limited data at that time. As testing improves, the effects of the first, stringent lockdown in late March become clear, with $R$ rapidly dropping below one. Things loosen up for the summer, but $R$ drops once again when lockdowns are imposed for a second time at the start of November.

Bona Polari

Tue, 03 Nov 2020 00:00:00 +0000

Throughout history, people have used coded language – hiding secrets in plain sight – for self protection. Codes are often invented for purpose, but can also develop of their own accord. Take Polari, spoken by gay men in Britain up to the 1960s. In “Fabulosa!”, Paul Baker explains how the language developed from other forms of slang, and how it was used to obscure then-criminal practice from a frowning public.

Polari’s linguistic father was Thieves’ Cant, with genetic material from Cockney rhyming slang and backslang. Cant, the origin of words like hoodwink, thwack and booze, was associated with the criminal underworld. Like an academic jargon, Cant was somewhere between a technical vocabulary of the trade, equipped to express the finer points of purloinery, and a full-blown dialect, opaque to outsiders. Coppers would be hard pressed to understand a rapid stream of slang, and harder still to use it convincingly themselves, making it difficult to infiltrate groups of ne’er-do-wells undetected. Conversely, fluency in the dialect proves the speaker’s years in the community, and in turn their likely sympathy to its worldview. Linguistic obscurity proved similarly useful for Polari users, allowing them to recognise bona omee-palone allies and talk trade without revealing too much to law-enforcing Betty Bracelets.

Beside Cant, Polari’s other dad was Italian. Many of its words are recognisable: “Good” is bona, the numbers begin una, dooey, tray, and the name itself comes from the verb parlare, “to talk”. This could catch people out. Baker recounts the time two friends, speaking Polari in an Italian shoe shop, attempted a sly comment on a salesman’s good looks. The man looked up and said “Grazie”, and the pair ran out embarrassed.

Zen there were none

Fri, 09 Oct 2020 00:00:00 +0000

Grouchy passengers on the London Underground are offered all kinds of stress-reducing therapies. From time immemorial, the tube’s fluorescent-lit carriages have been tiled with images of crystal clear oceans, comprehensive holiday packages, and sharp-suited individuals swearing by particular vitamin pills. But a more recent prescription is a very old one: meditation, a Buddhist practice dating back to the fourth century BC.

Meditation has not, historically, been a very lucrative industry. The practitioner’s biggest outlay might be for their Zafu, a round floor cushion. Past that, sitting quietly does not require a credit card.^[1] But startups are disrupting the age-old silence; the rise of podcasting and in-app subscription presents the would-be Sensei with a business opportunity. Mobile apps can now offer guided meditations, with a kindly voice from the listener’s phone encouraging focus on breathing and awareness of surroundings.

That approach is more consumer-friendly than hitting students with keisaku, the “encouragement sticks” used in silent Japanese monasteries. But it also opens the door to tailored sessions, for different levels of experience or lengths of time, for helping with relaxation, memory or self-esteem, or for falling asleep, waking up, power napping, and so on. Those anxious not to miss out can subscribe for a monthly fee, and finally enjoy freedom from their tedious inner monologue (albeit in favour of a tinnier outer one). Enlightened with this new business model, a recent crop of startups want to make meditation mainstream.

Partial Evaluation & Abstract Interpretation

Wed, 29 Jul 2020 00:00:00 +0000

Mjolnir is, one the one hand, a library implementation of Julia’s type inference engine. On the other, it’s a version of the computation-graph-tracing technique used by JAX.^[1] Those might seem like quite different things, but they’re really two sides of the same coin. Understanding that relationship is enlightening in itself, but more interestingly, it leads to some ideas for getting the best of both systems.

Abstract Interpretation

Julia uses an abstract interpretation approach to type inference. The high-level intuition is that Julia walks over your code much like an interpreter would, except that variables are assigned to abstract sets of values (like Int64, representing all the whole numbers you can squeeze into 64 bits) rather than actual values (like 3, a particular one of those numbers).

Transducers & Effects

Fri, 12 Jun 2020 00:00:00 +0000

Clojure has introduced a very interesting idea called ‘transducers’, which decouple sequence transformations (mapping, filtering etc) from sequence implementations like vectors, lazy lists and channels. Transducers and their benefits are well-covered elsewhere, but I’d like to explore some tradeoffs, and compare an alternative (and extremely hypothetical) design based on a staple of the functional programming world, effect handlers.

There are many useful operations that we can carry out on sequences, like mapping, filtering, interleaving, partitioning and so on. Ideally, we’d like to apply these tools to any sequence of values, including list data structures and strings but also channels and observables. Unfortunately, it’s common to have to reimplement each function we want for new sequences.

Abstracting over sequences is difficult, and requires a significantly more powerful approach than the usual polymorphism and data abstraction. To see why, imagine a somewhat-general map using empty and conj.

The Modern Language Designer

Sat, 06 Jun 2020 00:00:00 +0000

Programming language design and implementation is, always has been, and likely always will be, hard. But it’s hard today for completely different reasons to those fifty years ago, when the PL field first bloomed. In the past a budding language designer more or less just had to hack together a parser, throw out machine code and call it a day (albeit this is harder than it sounds with then-current tools). In 2020 such a language could be a weekend project – but in turn, language designers face many new challenges and a much higher bar for success.

In the 70s, choosing syntax was (I imagine) quite a big deal, with little in the way of prior art to guide your choices. In the 2020s programmers have a shared intuition for syntax that transcends languages. Basic ideas like block structure, lexical scope and use of indentation are universal and aren’t going anywhere. More specifically, C-like syntax (via its modern incarnations in Rust and Swift) is taking over, even influencing new functional languages that use effect handlers. Still, Pascal-, ML- and Lisp-like notations aren’t dead yet. Realistically you’re going to choose one of these syntax families as a base and make minor adjustments. There’s a little room for innovation here and there, but the biggest decisions about syntax are made for you.^[1]

Turning syntax into runnable machine code used to mean reading a dense manual, possibly specific to your make of PC, and staring a lot at your code to figure out the inevitable bugs. At the least you’d have to do register allocation and expand your language primitives into machine code sequences; more advanced compilers would do peephole optimisations, removing redundant instructions or even pattern matching to reorder and replace code for best performance on the target hardware. Then you’d have to do it again for a different kind of computer, not to mention other tooling like linkers and debuggers.

Iterate on it

Thu, 04 Jun 2020 00:00:00 +0000

When you write a for loop in Julia, like

for x in xs
  # do something with x
end

this expands to

The Many Types of Types

Tue, 19 May 2020 00:00:00 +0000

Julia programmers love nothing like a good polysemy. One of the more extreme examples is the community’s use of the word “type”, which refers to a cluster of related, but subtly distinct, features and their roles in the language. Ambiguity can provide a useful shorthand, but it can also be confusing to newcomers and those outside the community, so it might be useful to disambiguate a bit.

In short, types are Julia values that can play in one of the following parts: as tags for identifying meaning, selectors for dynamic dispatch, descriptions of data layout, annotations provided by type inference, hints for compiler specialisation, and constraints for compile-time correctness checking. These are all largely separate concerns that aren’t as tied together as they may seem.

Data Tags

Differentiable Control Problems

Tue, 05 Mar 2019 00:00:00 +0000

Originally posted on the Flux website. By Neethu Maria Joy, Tejan Karmali and Mike Innes.

We’ve discussed the idea of differentiable programming, where we incorporate existing programs into deep learning models. This article shows what ∂P can bring to some simple but classic control problems, where we would normally use black-box Reinforcement Learning (RL). ∂P-based models not only learn far more effective control strategies, but also train orders of magnitude faster. The code is all available to run for yourself – they will mostly train in a few seconds on any laptop.

Follow the Gradient

On Machine Learning and Programming Languages

Wed, 06 Dec 2017 00:00:00 +0000

Originally posted on the Julia website. By Mike Innes (JuliaHub), David Barber (UCL), Tim Besard (UGent), James Bradbury (Salesforce Research), Valentin Churavy (MIT), Simon Danisch (MIT), Alan Edelman (MIT), Stefan Karpinski (JuliaHub), Jon Malmaud (MIT), Jarrett Revels (MIT), Viral Shah (JuliaHub), Pontus Stenetorp (UCL) and Deniz Yuret (Koç University).

Any sufficiently complicated machine learning system contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a programming language.^[1]

As programming languages (PL) people, we have watched with great interest as machine learning (ML) has exploded – and with it, the complexity of ML models and the frameworks people are using to build them. State-of-the-art models are increasingly programs, with support for programming constructs like loops and recursion, and this brings out many interesting issues in the tools we use to create them – that is, programming languages.

Recursive Self-Simulation

Wed, 15 Nov 2017 00:00:00 +0000

Can the Universe Simulate Itself?

I’m not merely asking if we can simulate the laws of Physics, nor whether we can simulate them with such fidelity that they give rise to life. Those are pansy questions.

I’m asking if the universe can simulate itself exactly – so exactly that, if the simulation were left running long enough, it would give rise to humans just like us, who would then initiate the same simulation! Which of course would then contain a simulation of itself, and so on ad infinitum.

Taming the Tarpit

Wed, 13 Sep 2017 00:00:00 +0000

Have you ever seen a programming language like this?

++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>
---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.

What you’re looking at is the “hello, world!” program of a language called brainfuck. Brainfuck is a sort of postmodern approach to language design, the kind of thing that Jackson Pollock’s evil digital twin would have used. It has only eight instructions for moving around an array of bytes, adding and subtracting one as you go. Yet surprisingly enough, it’s just as powerful as a full programming language like Python.

Generic GPU Kernels

Thu, 24 Aug 2017 00:00:00 +0000

Julia has a library called CUDAnative, which hacks the compiler to run your code on GPUs.

using CuArrays, CUDAnative

xs, ys, zs = CuArray(rand(1024)), CuArray(rand(1024)), CuArray(zeros(1024))

function kernel_vadd(out, a, b)
  i = (blockIdx().x-1) * blockDim().x + threadIdx().x
  out[i] = a[i] + b[i]
  return
end

@cuda (1, length(xs)) kernel_vadd(zs, xs, ys)

@assert zs == xs + ys

Is this better than writing CUDA C? At first, it’s easy to mistake this for simple syntactic convenience, but I’m convinced that it brings something fundamentally new to the table. Julia’s powerful array abstractions turn out to be a great fit for GPU programming, and it should be of interest to GPGPU hackers regardless of whether they use the language already.

The Lazy List

Mon, 05 Jun 2017 00:00:00 +0000

“Any sufficiently advanced technology is indistinguishable from magic,” go the famous words of Arthur C. Clarke. Often with seemingly-magical technology, a small dose of knowledge is enough to wash away our wide-eyed awe and bring us back to sweet, comfortable cynicism. Yet some ideas don’t lose their lustre so easily; it can be a little hard to believe they work even once you understand them.

Laziness is one such idea. Today I’d like to give you a taste of the borderline-magical things that lazy sequences can do. But we’ll also peek under the hood of the abstraction, and understand what’s really going on. By the end you should have the foundations to implement them yourself.

The Gateway

Sorting a Billion Numbers

Mon, 21 Mar 2016 00:00:00 +0000

This post gives an overview of working with larger-than-memory data in the Julia language. I don’t have any particular use for this, but was curious about the problem as the current tools for out-of-core analysis seem to be pretty lacking. I’ve attempted to come up with a basic implementation of an array backed by persistent storage, and this post explains how I’ve put it together.

Aside from any insights about the problem itself, I’m going to write a lot about some of Julia’s more unusual features and how they lock together. If you’re a beginner Julian or are interested in becoming one, it might be instructive to see these in action – feature lists look great on paper but can’t give a good sense of how things feel in practice, especially when it comes to working with larger codebases. Be warned, though, that this post is pretty technical and isn’t a tutorial; I’ll link to more beginner-friendly materials where they might be useful.

As a rough outline, we’ll cover:

Speed in Programming Languages

Mon, 26 May 2014 00:00:00 +0000

Computers are more powerful than ever before. Now that you can buy watches which outpace the earliest supercomputers, it seems logical to think that the efficiency of your software no longer matters. And yet, performance is the new black, with recent languages like Go, Julia and Rust aiming to provide expressive power and ease of use without compromising on CPU cycles.

Julia is probably the most extreme example of this trend, with its lofty goal of being as dynamic and expressive as languages like Python, Ruby or R while still traveling at the speed of C. At the moment it’s mostly aimed at number crunchers for whom raw FLOPS are the holy grail, but I would suggest that it’s also relevant to the exact opposite camp — even if you happily use languages like Python and never even think about performance.

If you’re this kind of coder, your reaction to these languages may well be something like: “So what? My language is already productive, and my code is never the bottleneck anyway.”