Using Windows Job Objects for Process Tree Management


Using child processes to perform various tasks is a standard construct in larger programs. The simplest reason is this gets you memory isolation and resource management for free, with the OS managing scheduling and file descriptors and other resources. A common requirement when using multiple processes is the ability to wait on or kill one or more of these children. It is not always possible to record process IDs at fork(), since the fork may happen in a library that does not give you such access.…
Read more ⟶

Debugging MacOS file locking with DTrace


A few days ago, I was stymied at work by a set of tests that had intermittent failures on OSX but not Windows. There was a process which would try to obtain an exclusive lock on a file, using the lock-on-open provided by the BSD/MacOS O_EXLOCK flag to open(2). It also used O_NONBLOCK; if the file was locked by another process, it could be skipped. The process would hold the lock and remove (unlink(2)) the file, before close(2)-ing the descriptor.…
Read more ⟶

Canceling socket operations using I/O multiplexing.


I spent the last couple of months diving into networking code at work. This led to some interesting discoveries, including how to allow an in-progress network operation to be canceled on demand. This can be an attempt to establish a connection, or a socket read or write. The standard use case is to allow the user to cancel an operation or to allow a clean shutdown when multiple threads are operating on sockets.…
Read more ⟶

Systems We ❤


I had a really great time at Systems We Love 2 weeks ago. It was refreshing to attend a conference that had genuine talks about complicated systems that keep or kept the “world” running. No sponsor-driven drivel and no “here is how to do X” talks that could’ve been summarized by documentation. Ozan Onay has already summarized all the talks, including links to videos, so I don’t have to. A few talks stood out in particular.…
Read more ⟶

Why Golang is great for network services


I’ve been using Go professionally for over a year now. While I’m not a fan of every decision made by the language, I will admit that Go gets a lot of things right for one segment of the industry - network daemons and similar systems software. Google’s ethos are engineering solid and reliable services at scale; various design decisions make Go well suited for this. Go favors composition at multiple levels.…
Read more ⟶

Introduction to Programming: What I wish I knew


Every Introduction to Programming class starts with a programming language, introduces various concepts like variables and functions and leaves with an understanding of some simple algorithms. Mine was no different. We used C and spent a lot of time trying to make students comfortable using Linux. This leaves a lot to be desired. Software engineering is hard and laying out certain principles at the beginning can help tremendously. This is a short list of supplemental resources I would have liked to have when I started programming.…
Read more ⟶

Understanding Ansible and Jinja2's default() filter


I have been using Ansible at work for the past month to set up some infrastructure. I wanted to express the following in the setup script: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') }}" but wanted to set the user to ubuntu if SSH_USER was not set. For some reason, this would not work: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') | default('ubuntu') }}" Ansible provides several functions that can extract various string values from the environment.…
Read more ⟶

Linux Application Container Fundamentals


At my current job, I spend a lot of time coaxing Docker to run containers while trying to avoid network failures, Docker bugs and kernel reference count issues. Recently, I’ve gotten into reading about how Docker and other containerization software is implemented under the hood. This is a write-up of my exploration and experiments looking at how container runtimes are implemented. Nothing in this essay is original, but I hope it helps some people.…
Read more ⟶

Writing a simple database in rust, part 1


A Simple and Efficient Implementation for Small Databases is an old paper that I've wanted to implement for a long time. It describes a simple application that embodies a lot of "systems" knowledge at the same time: Using the filesystem as a persistence and synchronization mechanism.Using a write-ahead log as a source of truth.Creating crash tolerant (or crash-only) systems.While ideally you'd want to write this in C or C++, mucking around with strings and files isn't a very pleasant experience unless you are willing to pull in some third party libraries.…
Read more ⟶

Banner Peak


Note: Image titles can be viewed by hovering over the image. My interest in Banner Peak and Mt. Ritter began when I encountered them while hiking the John Muir Trail in August 2014. Immortalized by Ansel Adams, the 2 mountains framed by Garnet Lake and Thousand Island Lake are well known and frequently climbed. I snagged permits for Ediza Lake several months ago, with an intention to climb both Ritter and Banner via their scrambling routes from the Ritter/Banner saddle.…
Read more ⟶