The main thread name and process name are the same thing on Linux!

On typical days, we software engineers are usually stuck due to head scratching bugs, instead of actually writing interesting software. I had made some small changes and suddenly our end-to-end tests were timing out, with no useful clues. The harness emits a crazy number of log lines, so digging through them was difficult. Fortunately, the problem was easily reproduceable on a local VM and with some piecemeal commenting I was able to isolate it to changing the thread name of the main thread, that I had done for some debugging information.

Why does my stack have an extra 4 bytes? Digging into Clang's return value implementation

I was futzing around with some C code a few days ago and noticed that executables generated by Clang would sometimes have an extra 4 bytes on the stack. This was just for the main function. We can verify this is Compiler Explorer. Try switching to GCC and this doesn’t happen. This was interesting, so I spent a few hours over the holiday digging into why and how this happens.

CppCon 2017 Talks I enjoyed

I spent a recent holiday listening to several CppCon talks. I’m hooked! I was impressed by the generally high quality of the talks. The lack of “use my/my company’s framework which is so awesome!” talks was refreshing. In addition, the use cases where C++ trumps most competition are often either performance sensitive, or correctness sensitive under strong constraints. Of course, this means contending with the syntactic and semantic complexities C++ throws at you.

Using Windows Job Objects for Process Tree Management

Using child processes to perform various tasks is a standard construct in larger programs. The simplest reason is this gets you memory isolation and resource management for free, with the OS managing scheduling and file descriptors and other resources. A common requirement when using multiple processes is the ability to wait on or kill one or more of these children. It is not always possible to record process IDs at fork(), since the fork may happen in a library that does not give you such access.

Debugging MacOS file locking with DTrace

A few days ago, I was stymied at work by a set of tests that had intermittent failures on OSX but not Windows. There was a process which would try to obtain an exclusive lock on a file, using the lock-on-open provided by the BSD/MacOS O_EXLOCK flag to open(2). It also used O_NONBLOCK; if the file was locked by another process, it could be skipped. The process would hold the lock and remove (unlink(2)) the file, before close(2)-ing the descriptor.

Canceling socket operations using I/O multiplexing.

I spent the last couple of months diving into networking code at work. This led to some interesting discoveries, including how to allow an in-progress network operation to be canceled on demand. This can be an attempt to establish a connection, or a socket read or write. The standard use case is to allow the user to cancel an operation or to allow a clean shutdown when multiple threads are operating on sockets.

Systems We ❤

I had a really great time at Systems We Love 2 weeks ago. It was refreshing to attend a conference that had genuine talks about complicated systems that keep or kept the “world” running. No sponsor-driven drivel and no “here is how to do X” talks that could’ve been summarized by documentation. Ozan Onay has already summarized all the talks, including links to videos, so I don’t have to. A few talks stood out in particular.

Why Golang is great for network services

I’ve been using Go professionally for over a year now. While I’m not a fan of every decision made by the language, I will admit that Go gets a lot of things right for one segment of the industry - network daemons and similar systems software. Google’s ethos are engineering solid and reliable services at scale; various design decisions make Go well suited for this. Go favors composition at multiple levels.

Introduction to Programming: What I wish I knew

Every Introduction to Programming class starts with a programming language, introduces various concepts like variables and functions and leaves with an understanding of some simple algorithms. Mine was no different. We used C and spent a lot of time trying to make students comfortable using Linux. This leaves a lot to be desired. Software engineering is hard and laying out certain principles at the beginning can help tremendously. This is a short list of supplemental resources I would have liked to have when I started programming.

Understanding Ansible and Jinja2's default() filter

I have been using Ansible at work for the past month to set up some infrastructure. I wanted to express the following in the setup script: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') }}" but wanted to set the user to ubuntu if SSH_USER was not set. For some reason, this would not work: ansible_ssh_user: "{{ lookup('env', 'SSH_USER') | default('ubuntu') }}" Ansible provides several functions that can extract various string values from the environment.