Directory-specific environment variables and program versions

Direnv is a tool for loading custom environment variables per directory in your shell. With it, you can use environment variables to configure your applications and system environment. And it combines especially well with asdf.

Asdf is a shell extension for managing multiple versions on a single system. Think Rbenv, but for any tool, not just Ruby. Asdf allows us to specify that for a given project, we want to use Ruby 2.7.2, PostgreSQL 10.3 and Node.js 14.10. Once set up, asdf will intercept calls to, say, ruby and route it to the right version. Because it also comes with nice tooling to install said versions, I try to use it as much as I can for my system dependencies.

Setting up

Getting started with direnv and asdf takes a few steps. We are going to activate asdf in our shell through direnv, and we’re going to install and activate direnv through asdf. Don’t worry, it’ll make sense soon!

Installing asdf

First, install asdf using your method of choice. I used Homebrew, so I ran:

brew install asdf

This will give you the asdf command. Unless you hook it into your shell, as per the installation instructions, asdf will not do much. But we’ll skip the manual shell setup for now, and instead go through direnv.

Installing direnv

Second, install the asdf-direnv plugin:

asdf plugin add direnv

This will allow you to use asdf to install different versions of direnv. Use it to install the latest version:

asdf install direnv latest

Then, mark whatever version was just installed as the global or “default” version:

asdf global direnv 2.21.3

Note that we still haven’t hooked asdf into our shell yet, so won’t do anything yet. We’ll get to that in a bit — but first, let’s activate direnv.

Hooking direnv into our shell

We’ll hook direnv into our shell using its hook function, explicitly ran through asdf:

# in ~/.bashrc
eval "$(asdf exec direnv hook bash)"

Running direnv hook bash will output a short shell snippet for bash to that will call direnv when appropriate. Prefixing this with asdf exec ensures we use whatever version of direnv asdf thinks is appropriate — i.e. the version we set as “global” in the previous step.

We’re almost there! Now we have set up our shell to use direnv, as installed by asdf; all that is left is to allow direnv to activate asdf on a per-directory basis for us. Let’s edit the global direnvrc file to set up the proper asdf hooks:

# ~/.config/direnv/direnvrc
source "$(asdf direnv hook asdf)"

This invokes the asdf-direnv plugin to load the shell functions necessary for us to be able to “enable” asdf where we want it.

Recap

What the hell did we just do?

1. We installed asdf, which allows us to install run specific versions of programs using asdf exec.
2. Using asdf, we installed direnv, which allows us to customise our shell environment on a per-directory basis.
3. We used asdf exec to run direnv to include its setup code in our shell, so direnv is now active in new shells.
4. We finally added some code to the direnv “library” in ~/.config/direnv/direnvrc that allows us to activate asdf’s shell integration on a per-directory basis.
5. Now we can run asdf-managed programs without using asdf exec.

But why?

The reason we go through this whole routine is that we want to use asdf to manage our system versions. In the traditional approach without direnv, asdf would maintain shims of the programs it knows about. For example, running ruby would be routed by asdf’s shell integration to ~/.asdf/shims/ruby, which would be a wrapper program to figure out the right version of Ruby to use, and then run that. That’s a little overhead on every invocation we don’t need to pay. Moreover, it might sometimes lead to compatibility issues. With direnv though, instead of at invocation time, we figure out when changing directories what the right versions to use are. Then we add those to our $PATH, so we can directly invoke the right executables without having to go through a shim or incurring any overhead.

Let’s use our shiny new tools in a new project!

Playing around

Let’s start a new project to try things out:

mkdir blog
cd blog

To activate asdf and direnv in it, we create a .envrc file in our project directory. The direnv initialisation code we inserted into our shell profile will look for this file when we change directories and run it.

We can use it to activate asdf for our project. Create the .envrc file like so:

#!/bin/bash
use asdf

The shebang line (#!/bin/bash) is not strictly necessary but I do usually add it, as it helps code editors understand the syntax of the file. use is a function provided by direnv, and the asdf argument triggers the setup functions we loaded in ~/.config/direnv/direnvrc. A shell, when changing into this directory, will now use asdf!

The threat of untrusted code

…and immediately, we see an error in our shell if we save and close the .envrc file. It will read:

direnv: error /path/to/blog/.envrc is blocked. Run `direnv allow` to approve its content

This is a security feature of direnv. Since running a shell script when you cd into a directory has the potential to run untrusted and potentially malicious code, direnv will not load the .envrc file by default. It will tell you that you will need to explicitly allow it. It will record a fingerprint of the file, so whenever a trusted file changes, you will have to approve it again. Go ahead and approve it by running direnv allow.

Picking a version of Ruby to use

To demonstrate how this works, let’s say we’ll create Ruby project. We haven’t configured anything with asdf yet, so let’s check what Ruby we will be using:

$ which ruby
/usr/bin/ruby

Running ruby will invoke the system Ruby installation. Let’s change that. Create a file called .tool-versions in your project directory with this content:

ruby 2.7.2

Asdf will look for this file to determine what version to use. You can now ask it what versions it would use of the programs it knows about:

$ asdf current
direnv          2.21.3          /Users/avdgaag/.tool-versions
ruby            2.7.2           Not installed. Run "asdf install ruby 2.7.2"

Asdf tells us running direnv will use version 2.21.3 as specified in our global .tool-versions file, which we created using asdf global earlier. For ruby it wants to use version 2.7.2, but that is not installed. Helpfully, it tells us how to install it.

Installing requested versions

We can install just the requested version of Ruby, or anything listed in our .tool-versions file that we don’t have yet. asdf install ruby 2.7.2 will install only Ruby 2.7.2, but asdf install will install everything listed in .tool-versions. Run that and go get a coffee while asdf downloads and installs Ruby for you.

Note: under the hood, asdf will use ruby-build to install Ruby, just like Rbenv does.

When it’s done, check what Ruby is used now:

$ which ruby
/Users/avdgaag/.asdf/installs/ruby/2.7.2/bin/ruby

Ah, now we’re not using the same system-wide Ruby anymore, but a custom version managed by asdf.

Managing environment variables

Let’s also understand what direnv does with an example. When working with Ruby projects it is common to use environment variables such as RAILS_ENV or RACK_ENV. We can check their values and see they are empty:

$ echo $RAILS_ENV

But we can edit our .envrc file and add a new line to it to set that environment variable:

#!/bin/bash
use asdf
export RAILS_ENV=development

Now, when you have saved the file and ran direnv allow to trust the new .envrc file, we can see the environment variable was now correctly set:

$ echo $RAILS_ENV
development

Diving deeper

Asdf has installed direnv for us and hooked it into our shell, so that direnv can hook asdf into our shell on a per-directory basis. We can now modify our environment in our .envrc file and specify tool versions to use in our .tool-versions file. That’s a lot! But there’s more.

Direnv ships with some useful functions in its stdlib, that we can use in our .envrc files. Two functions I like, are:

• dotenv .env loads the contents of .env into the shell. .env is a conventional place to put key/value pairs, which are to be loaded into an application’s environment on start-up by tools such as Foreman or language-specific tools such as Dotenv. Direnv can handle it for us!
• PATH_add ./bin will add the bin directory in our current directory to our $PATH, which means executables in that directory can be run without specifying their full path, and take precedence over system-wide programs of the same name.

But there are also a few layout functions that wrap up commonly used functionality per project type.

Layout functions: Ruby

First, there’s layout ruby. As per the documentation:

Sets the GEMHOME environment variable to $PWD/.direnv/ruby/RUBYVERSION. This forces the installation of any gems into the project’s sub-folder. If you’re using bundler it will create wrapper programs that can be invoked directly instead of using the bundle exec prefix.

Using layout ruby in your .envrc file will set $GEMHOME to a project-specific subdirectory. This effectively creates a gemset like RVM once provided. You still use Bundler to manage your Ruby gems, but there is no longer a need for running executables using bundle exec because your project gems are isolated. I also quite like the idea that removing a project directory also removes all the gem dependencies it was using.

One more thing I like to do is to let Bundler install binstubs in my project directory. In a Rails project, I then let Spring modify those binstubs to have them go through Spring, giving me a performance boost of not booting Rails on every command. Finally, I add my project-specific ./bin directory to my $PATH using direnv:

#!/bin/bash
use asdf
layout ruby
PATH_add ./bin

Now, I can run rails s or rspec in my shell and know it will do the right thing.

Layout functions: Node.js

Second, there’s layout node. It does something similar for Node.js:

Adds “$PWD/nodemodules/.bin” to the PATH environment variable.

No more installing NPM packages globally; everything can be installed in your project directory. By adding the node_modules/.bin directory to your path there is no more need to type out the path to an executable, or use npx. Of course, if you want to install and manage Node.js versions via asdf, you will need to install the appropriate asdf plugin.

Custom layout functions

Finally, since direnv is “just” shell code, it allows us to write our own functions. For example, we can use direnv to write a layout_postgres function that will configure PostgreSQL to be completely project-specific, including its version, data and configuration. See the Direnv wiki on GitHub for reference.

Integration with other tools

Running the right versions of programs in the right environment from your shell is one thing. But oftentimes, other programs need to use your project configuration, from outside your project directory. The primary example is your code editor needing to use your particular version of eslint or needing to know the right environment variables. How can we make sure these programs are aware of the right configuration?

For starters, we can look for integration packages. My editor of choice is Emacs and there is a neat direnv package for Emacs that will look for and load the .envrc contents into Emacs’ concept of the environment.

For asdf versions, we can invoke asdf directly to run a program and let asdf pick the right version for us:

asdf exec ruby

We can even hard-code the right version number to use.

ASDF_RUBY_VERSION=2.7.2 asdf exec ruby

That should give you everything you need to use direnv and asdf in your development setup.

Conclusion

Asdf and direnv can be a bit dizzying to set up. But it pays off by simplifying your daily workflow, removing some friction we have grown accustomed to in modern development, such as prefixing all your commands with bundle exec. I have found using these tools for the most part “just works”. I encourage you to try them out and read their documentation to discover what else they have to offer.