Build processes should only generate new files

Last edited April 2021

Steps in a build process should not modify files — neither files that are checked-in into the version control system, nor files that were generated previously in the build process.

This is essentially side-effect-free programming. Writing to disk is not a side effect; it is the main effect. Build step inputs and outputs are immutable.

There is one exception to this guideline: a build step can generate files and mutate them in the same build step. This works, because the mutability is localized: it manifests itself only inside the build step, and so the inputs and outputs of this build step are still immutable.

Advantages

(“Target” here means an output directory or output file.)

The set of build rules will be easier to understand, because there is only one rule for each target.

Incremental rebuilds will be faster. Previously generated output that is still up to date will not need to be regenerated. If mutation is allowed, this is not necessarily true.

State does not need to be tracked explicitly. Questions like “was operation X already performed on this target” are not trivial to answer when allowing mutation: this information needs to be tracked explicitly. When mutation is disallowed, the existence of the output file/directory is enough.

Note, however, that to avoid the need to track state explicitly, targets must be created atomically: if the target exists, it must be up to date. Targets need to be created elsewhere and then moved into place. This works as long as the move operation is be atomic (i.e., within the same filesystem).

Other thoughts

To do: Don’t dynamically calculate filenames. This makes it a pain to track what depends on what. Maybe not quite a related thought.