Ruby on Rails is a great website framework, and I really like it for developing web applications. It has a lot of opinionated defaults that make things that are hard in other frameworks easy in Rails.

Maybe it’s that, or you shouldn’t use Flask for large web applications unless you want to write everything yourself. What I’m talking about is my emotes project which uses Flask and Peewee. Basically, the Flask/Peewee codebase was becoming very un-fun to maintain because every time I wanted to make a change, it ended up taking many, many hours because of the ORM layer (Peewee doesn’t have a migrator, so there are plenty of database issues every time I do an update — I wrote my own terrible migrator). I figured that if I was going to replace the ORM layer (for example with SQLAlchemy), I could as well just rewrite the entire emotes in Rails. So that’s exactly what I did.

It took me less than a day (although writing Emotes for the first time took quite a while[!!]: at least a few days). Obviously, the final product doesn’t have feature-parity with the Python version of Emotes, but it will get there eventually. But before I went any further, I wanted to deploy my testing version of Emotes to my server, which runs NixOS.

This is where the fun started. I’m 100% going to do another blog post of how I completely rebuilt my broken homelab infrastructure and how I set all that up on NixOS, but that’s another post. Effectively, if I wanted Emotes on Rails to be running on my servers, I’d have to deploy it with the NixOS packaging tools (which can be painful since the NixOS documentation is awful). Or did I? There were other options for a declarative Emotes on Rails, and I explored that as well (I’ll go into more detail later :P).

Also, if you don’t want to read this long post, here are the "tl;dr"s:

tl;dr how to do it

It’s actually a bit complex so I’d recommend reading the entire article, but just look at the rails branch of Emotes here and look at the files default.nix and build.nix.

Here’s a shortened summary, though:

  1. You have to create the tmp/sockets directory when making your derivation.

  2. You have to change the Bootsnap cache directory to something else, or just disable Bootsnap entirely.

  3. You have to set the puma PID file location to somewhere outside of the derivation directory (which you can do programatically or with the PIDFILE environment variable).

  4. When starting and doing anything with Rails, you have to use the bundle binary provided in with bin/bundle in your derivation directory (so ${derivation}/bin/bundle).

  5. You need to generate a Rails master key with bundle exec rake secret (yes, this is with the bin/bundle from #4), save that somewhere if it doesn’t exist, and load it back when you start your Rails application (or when you’re doing operations with Rails like migrating, I think). In order to load the Rails mater key, you can set the environment variable RAILS_MASTER_KEY.

  6. If I’m missing anything, it’s more than probable I set some environment variable for it in the default.nix file in the aforementioned rails branch.

tl;dr what not to do

  1. Use nix-shell to generate your Gemfile and Gemfile.lock if you don’t use NixOS already (I don’t — I use Arch™). Using nix-shell will magically solve lots of annoying and vague problems you encounter.

  2. When writing your SystemD service, use the bundle binary that’s in the bin directory of your Rails application. That’s what I saw the redmine packaging for NixOS do, and that’s what I did, and it solved many problems.

  3. You have to reconfigure or hack some things (the puma PID file, the tmp directory creation, and the bootsnap cache directory). I’ll go into more detail on this later in the post.

Tip
 I’ll be referring to my Emotes on Rails source code many times here, which is located on the rails branch of emotes (direct link on GitHub here)

Attempt 1: bundlerEnv and mkDerivation

This was my starting point for building a Ruby derivation for Emotes on Rails. I’m still not 100% on how the Nix package manager works, but I’ll try to explain from what I understand. You need a program called bundix that will turn your Gemfile 's gems into Nix packages. Now, why would you need this? See, in a normal environment, you’d be able to call bundle install and have everything work just fine. In a Nix environment, however, when making a derivation (which I would just say is a fancy term for 'package'), the derivation doesn’t have access to the internet (and you can’t make it have access to the internet either), so when you call bundle install, you’ll just get some sort of network error and nothing will happen. In order to get things from the network, you have to use a special function to fetch the sources from the internet before the build happens, and then link all of it together at the end, which is why Nix needs special packaging tools like bundix. For Rust, there’s cargo2nix, crate2nix, and naersk, which serve the same purpose of bundix but for a different programming language. I wouldn’t realize until later the implications of a derivation not having network access (which you’ll see in Attempt 2). So I went ahead and ran bundix on the Rails project, and bundix generates a file called gemset.nix which you’d then import when you call nix-build and add bundlerEnv/mkDerivation that are documented in the Nix wiki as I linked above.

Now, if you go ahead and try buliding a Rails application with this method, it won’t work. You’ll get dependency errors (because, for example, the nokogiri gem requires a native library [I think it was zlib in this case and a few others]). The problem is that the Nixpkgs wiki doesn’t really document how you can add the zlib dependency when nokogiri is compiling. Luckily, I (and) you don’t have to figure out how to do this; someone’s already documented how to compile these Gems for a Rails project (the difference between what I’m linking and what I’m doing is that the repo I’m linking only built a Rails project while I aim to integrate my Rails project as a service). That Rails-Nix template is here (specifically the file nix/rubyenv.nix file). In case something happens to that repository, I’ll put the relevant part of that file here:

bundlerEnv {
    # Other things before the gemConfigs

    gemConfig.openssl = attrs: {
        buildInputs = [ openssl ];
    };

    gemConfig.pg = attrs: {
        buildInputs = [ postgresql ];
    };

    gemConfig.sqlite3 = attrs: {
        buildInputs = [ sqlite ];
    };

    gemConfig.nokogiri = attrs: {
        buildInputs = [ libiconv zlib ];
    };

    gemConfig.sassc = attrs: {
        buildInputs = [ libsass ];
        shellHook = ''
        export SASS_LIBSASS_PATH=${libsass}
        '';
    };
}

I went ahead and copied that and things started to build (actually, I had a very strange issue with a Nokogiri hash mismatch, but that was because you can’t use the Arch Ruby and expect things to work right with Nixpkgs — again, see my tl;dir).

Once you get your project to build, you can move on to turning your Rails project into a service (which is the… much harder part). I started by copying what I’d written for mymysqlbackup (which is another piece of my homelab infrastructure that’s basically a specialized script to backup MySQL databases). Once I had this, I got the configuration file to a state where it had most of the options I needed to configure emotes on my server (you can see what I had by browsing the commit history of the rails branch of Emotes).

Then, I tried making a SystemD service using the Bundler from Nixpkgs. I think you can see what I did in this file and this file. Basically, I was trying to execute Rails with bundlers that weren’t correct (eg. the nixpkgs.bundler, or something like that). That doesn’t work and you shouldn’t use that or the SystemD service will literally tell you to bundler install and fail, which is absolutely not supposed to happen. At this point, I just got confused, frustrated, and gave up with the direct NixOS approach. I still wanted a 100% declarative configuration, and, surprisngly, I had a different idea that would allow me to keep the 100% declarativeness (on NixOS!) while letting the process be easier. This idea isn’t 100% invalid, and while it’s not what I chose in the end, I think it could still work in a difference scenario.

Attempt 2: Docker

Yes, this was my second idea. Docker. There’s a very cool option in NixOS called virtualisation.oci-containers that lets you declaratively make Docker containers. So, figuring this would be easier (since there is thankfully plenty of documentation about how to Dockerize Rails on the internet), I went headfirst into making a Dockerfile for Emotes on Rails. It wasn’t too hard, and I got a building image quickly (whether that image actually worked was a different story—I didn’t actually test it until much later on [and yes, I did get it to run just fine]).

If I wanted to make this simple for myself, I figured that I’d have the Docker container connect to the host MySQL, which would also get rid of the opportunity for UNIX socket authetnication (basically where the UNIX user connects based on their user authentication to the system over having a separate user/password to MySQL), and that would require more configuration on my end when deploying Emotes, but I figured it would be much less work to set this up than to deal with packaging Rails on NixOS. There is actually a way to do this if you look through the NixOS options — with virtualisation.oci-containers.containers.<name>.extraOptions, which are basically extra command line options to add to Docker when starting up the container. The example is is exactly the option I want, ["--network=host"], so I knew exactly what to do (this option basically attaches the Docker container to the host network).

Declaratively Building the Dockerfile on NixOS?

This was the first hurdle (and I never cleared it, although I have some ideas on how to now). How do you take that aforementioned Dockerfile and convert it into a Docker image that you can run from NixOS declaratively? NixOS had its own tools for building Docker images, which is documented here in the Nixpkgs manual (what I’m referring to is buildImage and I think buildLayeredImage).

The problem was that I was pretty sure these two functions were to build NixOS-based Docker images, which would’ve defeated the purpose because of the struggles I had earlier—I didn’t want to touch NixOS for the Docker-component of the Nix build. I still wanted to keep my Alpine-based Docker image and use it with NixOS. I looked around a little, but there was no way to build a Dockerfile programmatically with the Nixpkgs tools. So… I set out to build that Dockerfile myself into an OCI image, which is how I realized a few things about mkDerivation. I’ll briefly outline the experience of my failed attempt to build the Dockerfile.

  1. The virtualisation.oci-containers option can use a .tar.gz file as your image, so I had to figure out how to export my built Dockerfile. You can do it with docker save, but…

  2. The command docker build won’t work in general, since it requires a running Docker daemon, which is not going to exist when building a derivation.

  3. I turned to tools such as img and buildah which could do the job without a Docker registry. The buildah package was already in Nixpkgs, but I couldn’t quite figure out how to use it with a Dockerfile—because it kept telling me it couldn’t find any registries. I found out later that you have to either write a config file with the registries you want or you have to specify the registry you’re pulling your base image from.

  4. Because I couldn’t figure out buildah, I tried img, which I couldn’t get to compile at all with the buildGoModule helper in Nixpkgs. So I gave up and went back to buildah, which I figured out how to use.

  5. The problem with any of the Docker helpers is that they wanted some kind of root access from the derivation, and on top of that, it wanted to access a specific timezone file, and if it couldn’t find that, it failed. I couldn’t figure out how to solve any of these problems (I think I found a Gist on GitHub that outlined some fixes for buildah), but even then…

  6. Remember how I said derivations have no internet access? Well… that defeated the purpose of using any builder like buildah or img since they wouldn’t be able to access the internet to download the image or install the Alpine packages I needed in my container. Building a Docker image from a Dockerfile at this point seemed… not simple. However, there was another solution: building the Dockerfile manually and pushing it to the global registry. It wasn’t optimal, but it would do, so that’s what I did. But before I actually went through and finished updating everything to work with the global Docker registry, I decided to revisit doing Rails with a standard derivation because pushing to the Docker registry didn’t feel like an optimal solution to me (as in the entire build wouldn’t be happening locally). Also, knowing that using a Dockerfile to build locally (without lots of work, at least) would be a dead end also swayed my opinion. However, I also later learned that the buildImage function supposedly is supposed to emulate the functionality of a Dockerfile and doesn’t necessarily use Nix as a base image (but rather copies a derivation), which I suppose could be used if you did want to go the contianerization route for deploying your Rails application on NixOS.

Attempt 3: Fixing Attempt 1

So I decided to back to the old, non-Docker setup. Now, there’s already been a Rails application turned into NixOS configuration, so I took a lot of inspiration from it. The application I’m talking about is Redmine, whose service configuration is here. After learning that they used the bundler from bin/bundle in their Redmine derivation directory, and knowing I found that my previous attempts had always used the wrong bundler, I found fresh hope for this approach. Sure enough, I got a little bit further. Instead of my SystemD service telling me that I’d need to rune bundle install, I got some… other errors. These other errors, thankfully, were hard to fix but not major — caused because my Ruby/Bundler versions were different than their Nix versions. By regenerating the Gemfile.lock with an older Ruby (which I used a nix-shell to acquire) and updating my Gemfile to require only a Ruby version greater than 2.6, I was able to get rid of these errors.

(The following is more or less documented in the tl;dr how to do it section, but I’ll try to go into a bit more detail here)

Now, I was on to the Nix quirks. Since derivation directories are read-only, but Rails likes to write to things in the derivation directory, this poses some problems. The first thing is that for Rails to work, you have to make sure all the groups are built (eg. the default, development, and test) groups. Next, you’ll encounter issues with the bootsnap gem trying (and failling) to cache things since it wants to write to the read-only derivation directory. To fix this, you need to manually override the Bootsnap cache directory. You’ll see that for Emotes, I used the following code in config/boot.rb:

if ENV["NIXOS"] == "1" && ENV["RAILS_TMPDIR"]
  require 'bootsnap'
  Bootsnap.setup(
    cache_dir: ENV["RAILS_TMPDIR"]
  )
else
  require "bootsnap/setup"
end

Basically, I created an environment variable for NixOS. If both that is set and a directory to cache bootsnap exists, then it’ll load Bootsnap with a custom directory. Otherwise, it does the default bootsnap setup.

Note
 Yes, I see a problem with this. It’ll fail if NIXOS=1 but RAILS_TMPDIR is unset. Also yes, my Ruby isn’t that great and I could probably work on making my code shorter, but that’s a different issue…

Next, you’ll find that the Rails master key isn’t set. There’s probably a number of ways around this problem, but I just added a preStart to my SystemD service that created a master key file if it didn’t exist and wrote to it with rake secret, then loaded this key from the file every time something with rails was done (eg. migrating the database or starting the server). It’s also worth noting that I couldn’t just set the environment variable for the master key in the environment section of the systemd.services.<name>.environment option for NixOS, since environment read from a file (but maybe there’s a way to do this). Instead, I set this variable separately to read from the key file (see default.nix at this line number for more).

Once that’s done, there are basically two more hurdles that I’ll explain. The first is create_tmp_directories, which is located here in the Rails source code. Basically, it tries to make these three directories inside the tmp directory, which will not work. The first is the pids directory, which I’ll talk about next. The second is the cache directory, which, while I’m not certain of the full extent of how it’s used, although I think you can get away with its not being writable — the only thing that needs it seems to be bootsnap, which we took care of earlier. The final directory is the sockets directory, which, at least from my experience and my lazy two-second search of the Rails repo, doesn’t really seem to be used. Maybe I’m wrong about that last one, but my application is working just fine without tmp/sockets. As for the pids directory, you need to override the PIDFILE environment variable as I did here in default.nix to some writable file, so Rails won’t actually use the tmp/pids directory.

But what about actually making those directories so Rails doesn’t fail at create_tmp_directories? Well, the way I found was to just make those three directories during the mkDerivation process.

Adding this to my installPhase in mkDerivation seemed to suffice:

mkdir -p $out/tmp/{pids,cache,sockets}

This is, of course, assuming that $out is the directory where your Rails project was copied. I suppose you could adjust $out for where your Rails project lives.

With that, assuming I didn’t forget some small roadblock I ran into along the way, your Rails application should work (this is, if you do your SystemD service right, configure your database right, et cetera, et cetera [you can see how I did it in default.nix on the rails branch of Emotes]).

And… that’s it (for getting it to work).

What next (and some closing thoughts)?

Well, you can get Rails to run on NixOS with some twiddling and work. You can also probably use buildImage to make this happen more easily although I haven’t tested it. Personally, I still want a Docker solution for running my web application. Since NixOS can run Emotes just fine now, I wonder if I have Docker build me a NixOS image (although I haven’t really explored that yet) with my derivation and configuration (although… I’m not sure if a Systemd service is the right approach for Docker).

As for the future, I’ll be adding yew—which is a Rust frontend web framework that uses WASM—to Emotes, so I guess I’m gearing up for some multi-language Nix deployment fun!