Managing your environment

Trying to avoid ‘it works on my machine’

YiWen Hon

Matt Dray

12 March 2026

Storytime #1

• Ronald worked on a project last year. He did some quick analysis and saved his scripts.
• He’s revisiting the project now but the code isn’t running because one of the functions has now been deprecated.

Storytime #2

• Freud’s finished working on a data pipeline! It runs on his machine.
• However his colleague Pavlova can’t run it on her laptop because she doesn’t know what package versions she needs to install.

Steps

There’s plenty to discuss, but we’ll focus on:

• Basic portability
• Environment managers
• Containers

Roughly: ‘the least you can do right now’ to ‘probably the best choice long-term’.

Step 1: portability

• Can you ‘lift and shift’ your whole, discrete project?
• Does the recipient have everything they need?
• Consider IDE-specific helpers (VS Code Workspaces, R Projects)

• You should be able to hand-over a discrete folder that contains everything needed to recreate the work.
• Some integrated development environments have features top encourage this.

Step 1: portability

Helpers for Python:

• os.path.join for OS-agnostic filepaths
• .env files and the dotenv package for environment variables

For R:

• .Rproj and {here}
• an .Renviron file

• Live demonstration goes here showing what these functions are doing.
• File paths in scripts should be ‘relative’ (to where the request is coming from), not ‘absolute’ (from the root of the computer it’s being run on).
• Store personal keys and other variables in gitignored files, which the recipient must have their own copy of.

Step 2: environment managers

• Record language and package versions at a specific time.
• Like a time capsule, “freeze” your programming environment.

Step 2: environment managers

For Python:

• there’s many, many different options (see table here)
• we use uv in the DS team and recommend it

For R:

• {renv} with caveats (see Shannon’s talk)
• version-switching (try Positron, rig)
• rv?

• Live demonstration goes here
• Tooling in R is developing, but isn’t as strong or diverse as for Python.
• People seem to use {renv} a lot, it lets you install and snapshot packages so others can later restore.
• But it isn’t a perfect solution: it doesn’t handle external system dependencies including RTools for Windows; binaries may not be available from package repositories (e.g. CRAN), or need compilation.
• Switching R versions is also not handled. It’s possible manually, but ugh.
• rv is a tool that sits outside of R and works differently (see uv later).

Step 3: containers

• Docker 🐳 / nix ({rix} for R)
• Recreates the WHOLE MACHINE, not just the environment!
• Docker creates a temporary virtual computer with the sole purpose of running your script

What are you trying to achieve?

Do you want to make your project:

• work right now on someone else’s machine?
• work reliably again in future?
• easier to work with collaboratively?

Bottom line:

• you have to make conscious decisions
• different projects need different solutions

• What mode? Are you sharing, recreating, reproducing?
• What timescale? Now, the short-, medium- or long-term?
• Do you want to freeze the project in place forever, or occasionally make use of newer package versions?