Using Windows Job Objects for Process Tree Management

Using child processes to perform various tasks is a standard construct in larger programs. The simplest reason is this gets you memory isolation and resource management for free, with the OS managing scheduling and file descriptors and other resources. A common requirement when using multiple processes is the ability to wait on or kill one or more of these children. It is not always possible to record process IDs at fork(), since the fork may happen in a library that does not give you such access.

Debugging MacOS file locking with DTrace

A few days ago, I was stymied at work by a set of tests that had intermittent failures on OSX but not Windows. There was a process which would try to obtain an exclusive lock on a file, using the lock-on-open provided by the BSD/MacOS O_EXLOCK flag to open(2). It also used O_NONBLOCK; if the file was locked by another process, it could be skipped. The process would hold the lock and remove (unlink(2)) the file, before close(2)-ing the descriptor.

Canceling socket operations using I/O multiplexing.

I spent the last couple of months diving into networking code at work. This led to some interesting discoveries, including how to allow an in-progress network operation to be canceled on demand. This can be an attempt to establish a connection, or a socket read or write. The standard use case is to allow the user to cancel an operation or to allow a clean shutdown when multiple threads are operating on sockets.

Systems We ❤

I had a really great time at Systems We Love 2 weeks ago. It was refreshing to attend a conference that had genuine talks about complicated systems that keep or kept the “world” running. No sponsor-driven drivel and no “here is how to do X” talks that could’ve been summarized by documentation. Ozan Onay has already summarized all the talks, including links to videos, so I don’t have to. A few talks stood out in particular.

Why Golang is great for network services

I’ve been using Go professionally for over a year now. While I’m not a fan of every decision made by the language, I will admit that Go gets a lot of things right for one segment of the industry - network daemons and similar systems software. Google’s ethos are engineering solid and reliable services at scale; various design decisions make Go well suited for this. Go favors composition at multiple levels.

Introduction to Programming: What I wish I knew

Every Introduction to Programming class starts with a programming language, introduces various concepts like variables and functions and leaves with an understanding of some simple algorithms. Mine was no different. We used C and spent a lot of time trying to make students comfortable using Linux. This leaves a lot to be desired. Software engineering is hard and laying out certain principles at the beginning can help tremendously. This is a short list of supplemental resources I would have liked to have when I started programming.

Understanding Ansible and Jinja2's default() filter

I have been using Ansible at work for the past month to set up some infrastructure. I wanted to express the following in the setup script: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') }}" but wanted to set the user to ubuntu if SSH_USER was not set. For some reason, this would not work: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') | default('ubuntu') }}" Ansible provides several functions that can extract various string values from the environment.

Linux Application Container Fundamentals

At my current job, I spend a lot of time coaxing Docker to run containers while trying to avoid network failures, Docker bugs and kernel reference count issues. Recently, I’ve gotten into reading about how Docker and other containerization software is implemented under the hood. This is a write-up of my exploration and experiments looking at how container runtimes are implemented. Nothing in this essay is original, but I hope it helps some people.

Writing a simple database in rust, part 1

A Simple and Efficient Implementation for Small Databases is an old paper that I've wanted to implement for a long time. It describes a simple application that embodies a lot of "systems" knowledge at the same time: Using the filesystem as a persistence and synchronization mechanism.Using a write-ahead log as a source of truth.Creating crash tolerant (or crash-only) systems.While ideally you'd want to write this in C or C++, mucking around with strings and files isn't a very pleasant experience unless you are willing to pull in some third party libraries.