Debugging MacOS file locking with DTrace

A few days ago, I was stymied at work by a set of tests that had intermittent failures on OSX but not Windows. There was a process which would try to obtain an exclusive lock on a file, using the lock-on-open provided by the BSD/MacOS O_EXLOCK flag to open(2). It also used O_NONBLOCK; if the file was locked by another process, it could be skipped. The process would hold the lock and remove (unlink(2)) the file, before close(2)-ing the descriptor.

Canceling socket operations using I/O multiplexing.

I spent the last couple of months diving into networking code at work. This led to some interesting discoveries, including how to allow an in-progress network operation to be canceled on demand. This can be an attempt to establish a connection, or a socket read or write. The standard use case is to allow the user to cancel an operation or to allow a clean shutdown when multiple threads are operating on sockets.

Systems We ❤

I had a really great time at Systems We Love 2 weeks ago. It was refreshing to attend a conference that had genuine talks about complicated systems that keep or kept the “world” running. No sponsor-driven drivel and no “here is how to do X” talks that could’ve been summarized by documentation. Ozan Onay has already summarized all the talks, including links to videos, so I don’t have to. A few talks stood out in particular.

Why Golang is great for network services

I’ve been using Go professionally for over a year now. While I’m not a fan of every decision made by the language, I will admit that Go gets a lot of things right for one segment of the industry - network daemons and similar systems software. Google’s ethos are engineering solid and reliable services at scale; various design decisions make Go well suited for this. Go favors composition at multiple levels.

Introduction to Programming: What I wish I knew

Every Introduction to Programming class starts with a programming language, introduces various concepts like variables and functions and leaves with an understanding of some simple algorithms. Mine was no different. We used C and spent a lot of time trying to make students comfortable using Linux. This leaves a lot to be desired. Software engineering is hard and laying out certain principles at the beginning can help tremendously. This is a short list of supplemental resources I would have liked to have when I started programming.

Understanding Ansible and Jinja2's default() filter

I have been using Ansible at work for the past month to set up some infrastructure. I wanted to express the following in the setup script: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') }}" but wanted to set the user to ubuntu if SSH_USER was not set. For some reason, this would not work: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') | default('ubuntu') }}" Ansible provides several functions that can extract various string values from the environment.

Linux Application Container Fundamentals

At my current job, I spend a lot of time coaxing Docker to run containers while trying to avoid network failures, Docker bugs and kernel reference count issues. Recently, I’ve gotten into reading about how Docker and other containerization software is implemented under the hood. This is a write-up of my exploration and experiments looking at how container runtimes are implemented. Nothing in this essay is original, but I hope it helps some people.

Writing a simple database in rust, part 1

A Simple and Efficient Implementation for Small Databases is an old paper that I've wanted to implement for a long time. It describes a simple application that embodies a lot of "systems" knowledge at the same time: Using the filesystem as a persistence and synchronization mechanism.Using a write-ahead log as a source of truth.Creating crash tolerant (or crash-only) systems.While ideally you'd want to write this in C or C++, mucking around with strings and files isn't a very pleasant experience unless you are willing to pull in some third party libraries.

Banner Peak

Note: Image titles can be viewed by hovering over the image. My interest in Banner Peak and Mt. Ritter began when I encountered them while hiking the John Muir Trail in August 2014. Immortalized by Ansel Adams, the 2 mountains framed by Garnet Lake and Thousand Island Lake are well known and frequently climbed. I snagged permits for Ediza Lake several months ago, with an intention to climb both Ritter and Banner via their scrambling routes from the Ritter/Banner saddle.