unifyDB Dev Diary 1: the query system

Posted on October 3, 2020

This is the first development diary for the database I'm writing, unifyDB. I wrote a brief introduction to the project here. In this post I'm going to talk about unifyDB's query system: what it does and how it works.

I want to start with an example of a unifyDB query, but to understand that we need to understand a bit about how unifyDB represents data. All data in unifyDB is stored as a collection of facts. A fact is a tuple with three pieces of information: an entity ID, an attribute name, and a value (actually, a fact has two additional fields, a transaction ID and an added? flag, but we won't worry about those until we talk about time-traveling queries, which deserves its own blog post). For example, we might represent some user records with the following set of facts:

(1, "username", "harry")
(1, "role", "user")
(1, "preferred-theme", "light")
(2, "username", "dumbledore")
(2, "role", "user")
(2, "role", "admin")
(2, "preferred-theme", "light")
(3, "username", "you-know-who")
(3, "role", "user")
(3, "role", "user")
(3, "role", "admin")
(3, "preferred-theme", "dark")


unifyDB Dev Diary 0: I’m building a database!

Posted on August 9, 2020

Phew, it’s been a while! Over a year, in fact. And what a wild year! Lots of good things happened: I got married, got a new job that I love, moved to a nice new apartment. Also some not-so-nice things, but since you are all living through 2020 just like me I don’t think I need to go into those. But I have still found some side-project time, and I’d like to start talking about what I’m building.

So – I’m excited to announce that I’m building a database! I’m calling it unifyDB. It’s going to be a general-purpose database with some pretty interesting properties:

  • It maintains a complete history of changes to all entities stored in the database
  • You can make queries for historical data, e.g. “what did this user record look like last Tuesday?”
  • Arbitrary metadata can be attached to transactions – for example, you can add an application user ID to every transaction your app makes
  • Fine-grained access control is built into the database, allowing developers to limit access to particular attributes across all entities


More than JSON: ActivityPub and JSON-LD

Posted on April 23, 2019

In which our hero discovers the power of normalization and JSON-LD

The problem with JSON

I’ve been doing a lot of research for my current side project, Pterotype. It’s a new kind of social network built as a WordPress plugin that respects your freedom, encourages choice, and interoperates with existing social networks through the power of ActivityPub. It’s undergone several iterations already – the beta has been out for a while now, and I’ve been working hard on a version 2 for the last several months.


ActivityPub: Good enough for jazz

Posted on January 7, 2019

Kaniini, one of the lead developers of Pleroma, recently published a blog post called ActivityPub: The “Worse is Better” Approach to Federated Social Networking. It’s a critique of the security and safety of the ActivityPub protocol. They make some good points:

  • ActivityPub doesn’t support fine-grained access control checks, e.g. I want someone to be able to see my posts but not respond to them
  • Instances you’ve banned can still see threads from your instance in some ActivityPub implementations, because someone from a third instance replies to the thread and that reply reaches the banned instance


Announcing Pterotype

Posted on November 15, 2018

In my last post, I wrote about an emerging web standard called ActivityPub that lets web services interoperate and form a federated, open social network. I made an argument about how important this new standard is – how it tears down walled gardens, discourages monopolies and centralization, and encourages user freedom.

I genuinely believe what I wrote, too. And so, to put my money where my mouth is, I’m excited to announce Pterotype! It’s a WordPress plugin that gives your blog an ActivityPub feed so that it can take advantage of all the benefits ActivityPub has to offer.


What is ActivityPub, and how will it change the internet?

Posted on September 15, 2018

A new kind of social network

There’s a new social network in town. It’s called Mastodon. You might have even heard of it. On the surface, Mastodon feels a lot like Twitter: you post “toots” up to 500 characters; you follow other users who say interesting things; you can favorite a toot or re-post it to your own followers. But Mastodon is different from Twitter in some fundamental ways. It offers many more ways for users to control the posts they see. It fosters awareness of the effect your posts have on others through a content warning system and encourages accessibility with captioned images. At its core, though, there’s a more fundamental difference from existing social networks: Mastodon isn’t controlled by a single corporation. Anyone can operate a Mastodon server, and users on any server can interact with users on any other Mastodon server.


A DSL for Music

Posted on August 5, 2018

Haskell School of Music

I recently discovered Haskell School of Music. It’s a book about algorithmic music, which is awesome because: a) I’ve been obsessed with procedural generation for years and b) I like music as much as I like programming. So you can imagine my excitement when I discovered that someone had written a textbook combining my favorite areas of study.

Haskell School of Music is aimed at intermediate-level CS students, so it covers a lot of the basics of functional programming. It aims to be an introduction to the Haskell programming language while also thoroughly examining computer music. It starts simply by defining the data structures that represent music, and progresses to functional programming concepts, procedurally generating music, and doing signal processing and MIDI interfacing to actually play the songs.