Architecture¶
Terminology¶
- Package identifier: a package name and version, e.g. text-1.2.1.0
- GhcPkgId: a package identifier plus the unique hash for the generated binary, e.g. text-1.2.1.0-bb83023b42179dd898ebe815ada112c2
- Package index: a collection of packages available for download. This is a
combination of an index containing all of the .cabal files (either a tarball
downloaded via HTTP(S) or a Git repository) and some way to download package
tarballs.
- By default, stack uses a single package index (the Github/S3 mirrors of Hackage), but supports customization and adding more than one index
- Package database: a collection of metadata about built libraries
- Install root: a destination for installing packages into. Contains a bin path (for generated executables), lib (for the compiled libraries), pkgdb (for the package database), and a few other things
- Snapshot: an LTS Haskell or Stackage Nightly, which gives information on a complete set of packages. This contains a lot of metadata, but importantly it can be converted into a mini build plan...
- Mini build plan: a collection of package identifiers and their build flags that are known to build together
- Resolver: the means by which stack resolves dependencies for your packages. The two currently supported options are snapshot (using LTS or Nightly), and GHC (which installs no extra dependencies). Others may be added in the future (such as a SAT-based dependency solver). These packages are always taken from a package index
- extra-deps: additional packages to be taken from the package index for dependencies. This list will shadow packages provided by the resolver
- Local packages: source code actually present on your file system, and
referred to by the
packages
field in your stack.yaml file. Each local package has exactly one .cabal file - Project: a stack.yaml config file and all of the local packages it refers to.
Databases¶
Every build uses three distinct install roots, which means three separate package databases and bin paths. These are:
- Global: the packages that ship with GHC. We never install anything into this database
- Snapshot: a database shared by all projects using the same snapshot. Packages
installed in this database must use the exact same dependencies and build
flags as specified in the snapshot, and cannot be affected by user flags,
ensuring that one project cannot corrupt another. There are two caveats to
this:
- If different projects use different package indices, then their definitions of what package foo-1.2.3 are may be different, in which case they can corrupt each other's shared databases. This is warned about in the FAQ
- Turning on profiling may cause a package to be recompiled, which will result in a different GhcPkgId
- Local: extra-deps, local packages, and snapshot packages which depend on them (more on that in shadowing)
Building¶
Shadowing¶
Every project must have precisely one version of a package. If one of your local packages or extra dependencies conflicts with a package in the snapshot, the local/extradep shadows the snapshot version. The way this works is:
- The package is removed from the list of packages in the snapshot
- Any package that depends on that package (directly or indirectly) is moved
from the snapshot to extra-deps, so that it is available to your packages as
dependencies.
- Note that there is no longer any guarantee that this package will build, since you're using an untested dependency
After shadowing, you end up with what is called internally a SourceMap
, which
is Map PackageName PackageSource
, where a PackageSource
can be either a
local package, or a package taken from a package index (specified as a version
number and the build flags).
Installed packages¶
Once you have a SourceMap
, you can inspect your three available databases and
decide which of the installed packages you wish to use from them. We move from
the global, to snapshot, and finally local, with the following rules:
- If we require profiling, and the library does not provide profiling, do not use it
- If the package is in the
SourceMap
, but belongs to a difference database, or has a different version, do not use it - If after the above two steps, any of the dependencies are unavailable, do not use it
- Otherwise: include the package in the list of installed packages
We do something similar for executables, but maintain our own database of installed executables, since GHC does not track them for us.
Plan construction¶
When running a build, we know which packages we want installed (inventively called "wanteds"), which packages are available to install, and which are already installed. In plan construction, we put them information together to decide which packages must be built. The code in Stack.Build.ConstructPlan is authoritative on this and should be consulted. The basic idea though is:
- If any of the dependencies have changed, reconfigure and rebuild
- If a local package has any files changed, rebuild (but don't bother reconfiguring)
- If a local package is wanted and we're running tests or benchmarks, run the test or benchmark even if the code and dependencies haven't changed
Plan execution¶
Once we have the plan, execution is a relatively simple process of calling
runghc Setup.hs
in the correct order with the correct parameters. See
Stack.Build.Execute for more information.
Configuration¶
stack has two layers of configuration: project and non-project. All of these
are stored in stack.yaml files, but the former has extra fields (resolver,
packages, extra-deps, and flags). The latter can be monoidally combined so that
a system config file provides defaults, which a user can override with
~/.stack/config.yaml
, and a project can further customize. In addition,
environment variables STACK_ROOT and STACK_YAML can be used to tweak where
stack gets its configuration from.
stack follows a simple algorithm for finding your project configuration file:
start in the current directory, and keep going to the parent until it finds a
stack.yaml
. When using stack ghc
or stack exec
as mentioned above, you'll
sometimes want to override that behavior and point to a specific project in
order to use its databases and bin directories. To do so, simply set the
STACK_YAML
environment variable to point to the relevant stack.yaml
file.
Snapshot auto-detection¶
When you run stack build
with no stack.yaml, it will create a basic
configuration with a single package (the current directory) and an
auto-detected snapshot. The algorithm it uses for selecting this snapshot is:
- Try the latest two LTS major versions at their most recent minor version release, and the most recent Stackage Nightly. For example, at the time of writing, this would be lts-2.10, lts-1.15, and nightly-2015-05-26
- For each of these, test the version bounds in the package's .cabal file to see if they are compatible with the snapshot, choosing the first one that matches
- If no snapshot matches, uses the most recent LTS snapshot, even though it will not compile
If you end up in the no compatible snapshot case, you typically have three options to fix things:
- Manually specify a different snapshot that you know to be compatible. If you can do that, great, but typically if the auto-detection fails, it means that there's no compatible snapshot
- Modify version bounds in your .cabal file to be compatible with the selected snapshot
- Add
extra-deps
to your stack.yaml file to fix compatibility problems
Remember that running stack build
will give you information on why your build
cannot occur, which should help guide you through the steps necessary for the
second and third option above. Also, note that those options can be
mixed-and-matched, e.g. you may decide to relax some version bounds in your
.cabal file, while also adding some extra-deps.
Explicit breakage¶
As mentioned above, updating your package indices will not cause stack to invalidate any existing package databases. That's because stack is always explicit about build plans, via:
- the selected snapshot
- the extra-deps
- local packages
The only way to change a plan for packages to be installed is by modifying one of the above. This means that breakage of a set of installed packages is an explicit and contained activity. Specifically, you get the following guarantees:
- Since snapshots are immutable, the snapshot package database will not be invalidated by any action. If you change the snapshot you're using, however, you may need to build those packages from scratch.
- If you modify your extra-deps, stack may need to unregister and reinstall them.
- Any changes to your local packages trigger a rebuild of that package and its dependencies.