Overview

Most Haskell distributions provide a single version of the Glasgow Haskell Compiler and a single set of Hackage libraries — generally the Haskell Platform de jour plus some other libraries. If a user needs to set up a serious project then they will need to use cabal-dev or similar. The cabal-dev README explains the problem thus:

Performing consistent builds is critical in software development, but the current system of per-user and per-system GHC package databases interferes with this need for consistency. It is difficult to precisely identify the dependencies of a given project, and changes necessary to enable one project to build may render another project inoperable. If each project had a separate package database, each project could be built in a sandbox.

The JustHub Haskell distribution builds this functionality into the distribution, providing a quite complete back-catalogue of pre-built GHC and Haskell Platform packages, all of which can be deployed simultaneously in different projects.

All of the GHC tools and Cabal-install have been integrated into the system so that they can be used in the normal way with each work tree, each potentially connected to its own private sandbox (work trees can also share a sandbox).

Sandboxes are called hubs in this system and they get managed with the hub commands.

(The examples in this overview consist mainly of ghc/cabal toolkit commands with a few hub commands; the reader is assumed to be mostly familiar with the standard toolkit commands.)

Getting Going

The first step is to install the 'justhub-release' package (see the Download page). This will make the rest of the JustHub packages available for installation with yum.

You could install haskell:

    sudo yum install haskell

(This depends upon there not being any other GHC installations using /usr/bin — see the Download page) for an explanation of how to deploy JustHub along side other Haskell distributions.)

This will get you the current Haskell Platform and associated GHC and Cabal-install distributions (2012.2.0.0/7.4.1/0.14.0 at the time of writing).

Working with Sandboxes and a Single Toolchain

You are in business. Suppose you wanted to work with the SHA package.

    cabal update
    cabal unpack SHA
    cd SHA-*
    hub init
    cabal install

This is what happens.

  1. First of all we update Cabal's local copy of the Hackage package summaries.

  2. Then unpack the latest version of the SHA package and change directories into the root of it's source tree.

  3. Create a private hub for the work tree.

  4. Build and install the SHA package into your private Hub.

The SHA package only gets installed in our private hub. We can add other packages.

    cabal install utf8-string
    cat >sha.hs <<EOF
    import Data.ByteString.Lazy.UTF8
    import Data.Digest.Pure.SHA

    sha1String :: String -> String
    sha1String = showDigest . sha1 .  fromString

    main :: IO ()
    main = interact sha1String
    EOF

Now you can run the program in the usual ways.

    ghci sha.hs
    echo "foo" | runghc sha.hs
    ghc --make sha.hs && { echo "wibble" | ./sha; }

Listing the packages,

    ghc-pkg list

should give something like this:

    /usr/hs/db/2012.2.0.0.d
       Cabal-1.14.0
       GLUT-2.1.2.1
       HTTP-4000.2.3
       HUnit-1.2.4.2
       OpenGL-2.2.3.1
       QuickCheck-2.4.2
       array-0.4.0.0
       base-4.5.0.0
       bin-package-db-0.0.0.0
       binary-0.5.1.0
       bytestring-0.9.2.1
       cgi-3001.1.7.4
       containers-0.4.2.1
       deepseq-1.3.0.0
       directory-1.1.0.2
       extensible-exceptions-0.1.1.4
       fgl-5.4.2.4
       filepath-1.3.0.0
       ghc-7.4.1
       ghc-prim-0.2.0.0
       haskell-platform-2012.2.0.0
       haskell-src-1.0.1.5
       haskell2010-1.1.0.1
       haskell98-2.0.0.1
       hoopl-3.8.7.3
       hpc-0.5.1.1
       html-1.0.1.2
       integer-gmp-0.4.0.0
       mtl-2.1.1
       network-2.3.0.13
       old-locale-1.0.0.4
       old-time-1.1.0.0
       parallel-3.2.0.2
       parsec-3.1.2
       pretty-1.1.1.0
       process-1.1.0.1
       random-1.0.1.1
       regex-base-0.93.2
       regex-compat-0.95.1
       regex-posix-0.95.1
       rts-1.0
       stm-2.3
       syb-0.3.6.1
       template-haskell-2.7.0.0
       text-0.11.2.0
       time-1.4
       transformers-0.3.0.0
       unix-2.5.1.0
       xhtml-3000.2.1
       zlib-0.5.3.3
    /home/chris/.hubrc/lib/__h025/package.config
       SHA-1.5.0.1
       utf8-string-0.3.7

If you don't normally get involved with package databases then don't worry — you don't need to. The aim of this setup is to provide sandboxes while abstracting away from us details. Those used to caring about such details will note that the user package database managed by the hub has been set up under .hubrc/lib/__hxxx/package.config from you home directory (for some value of 'xxx').

You can request summary of your hub.

    hub info

which will respond with something like this.

    __h025 [DIR]           -- /home/chris/SHA-1.5.0.1
          GHC              : 7.4.1
          Haskell Platform : 2012.2.0.0
          Tools            : /usr/hs/ghc/7.4.1/bin
          Cabal Install    : 0.14.0
          Package DBs
              global       : /usr/hs/db/2012.2.0.0.d                            (2012.2.0.0)
              user         : /home/chris/.hubrc/lib/__h025/package.config       (__h025)

The first line tells you the name (which will vary here as we didn't specify a name on creation) along with and indication of how it has determined which hub to use ([DIR] indicating a configuration file in the current directory or one of its parents) and an annotation or comment — here just listing the path to the root of the work tree when the hub was created.

The subsequent lines tell you which edition of GHC the sandbox is using along with the Haskell Platform version (if applicable), the directory containing the compiler tools, the version of Cabal Install the hub is configured to use (if it isn't using the distribution's default version) and the locations of the package databases used by hub.

We could have specified a name at the start (see below) but we can rename the hub at any time. (If you are trying these examples out, you will have to substitute __h025 with whichever name was chosen for your hub — maybe `__h000'.)

    hub mv __h025 test-SHA

Now if we do hub info it says:

    hub mv: __h025: no such hub

The problem is that by renaming the hub we have broken the link between the work tree and the hub, which is maintained in a configuration file in the root of the work tree set up by the above hub init. This is easily corrected.

    hub set test-SHA

And order is restored, hub info giving:

    test-sha [DIR]         -- /home/chris/SHA-1.5.0.1
          GHC              : 7.4.1
          Haskell Platform : 2012.2.0.0
          Tools            : /usr/hs/ghc/7.4.1/bin
          Cabal Install    : 0.14.0
          Package DBs
              global       : /usr/hs/db/2012.2.0.0.d                            (2012.2.0.0)
              user         : /home/chris/.hubrc/lib/test-sha/package.config     (test-sha)

This makes it trivially easy to share a hub between different work trees: just change to the root directory of the other work tree and issue another hub set command

    cd ~/other-project
    hub set test-SHA

You can also replicate (hub cp), annotate (hub comment), remove (hub rm) and lock a hub against having any packages added or removed, or the hub being renamed or removed (hub lock) — see the hub(1) man page or hub usage or hub help for details.

Traditional GHC Behavoiur is Retained Outside of the Hubs

The above worked out example shows how it is possible to create a sandboxed environment, install packages into it and then compile and run programs in the sandbox.

Note that if you are outside of any sandbox the tools work just as if they had been installed in the conventional way.

    cd ~
    cabal install LDAP
    ghc-pkg list

Assuming no hub has been associated with the home directory, this would produce something like this:

    /usr/hs/db/2012.2.0.0.d
       Cabal-1.14.0
       GLUT-2.1.2.1
       HTTP-4000.2.3
       HUnit-1.2.4.2
       OpenGL-2.2.3.1
       QuickCheck-2.4.2
       array-0.4.0.0
       base-4.5.0.0
       bin-package-db-0.0.0.0
       binary-0.5.1.0
       bytestring-0.9.2.1
       cgi-3001.1.7.4
       containers-0.4.2.1
       deepseq-1.3.0.0
       directory-1.1.0.2
       extensible-exceptions-0.1.1.4
       fgl-5.4.2.4
       filepath-1.3.0.0
       ghc-7.4.1
       ghc-prim-0.2.0.0
       haskell-platform-2012.2.0.0
       haskell-src-1.0.1.5
       haskell2010-1.1.0.1
       haskell98-2.0.0.1
       hoopl-3.8.7.3
       hpc-0.5.1.1
       html-1.0.1.2
       integer-gmp-0.4.0.0
       mtl-2.1.1
       network-2.3.0.13
       old-locale-1.0.0.4
       old-time-1.1.0.0
       parallel-3.2.0.2
       parsec-3.1.2
       pretty-1.1.1.0
       process-1.1.0.1
       random-1.0.1.1
       regex-base-0.93.2
       regex-compat-0.95.1
       regex-posix-0.95.1
       rts-1.0
       stm-2.3
       syb-0.3.6.1
       template-haskell-2.7.0.0
       text-0.11.2.0
       time-1.4
       transformers-0.3.0.0
       unix-2.5.1.0
       xhtml-3000.2.1
       zlib-0.5.3.3
    /home/chris/.ghc/x86_64-linux-7.4.1/package.conf.d
       LDAP-0.6.6

LDAP has been installed in the usual default package database for GHC-7.4.1.

Working with Multiple ToolChains and Platforms

By installing some packages we can see how to work with multiple GHC tool chains.

    sudo yum install haskell-platform-2011.4.0.0-hub ghc-7.4.2-hub  

This gets us the 2011.4.0.0 platform and the latest edition of GHC (7.4.2).

We can see what new hubs that has got us with

    hub ls

which will list all of the named user hubs belonging to the logged in user, and all of the global hubs, produce something like this:

    2011.4.0.0             -- Haskell Platform 2011.4.0.0
    2012.2.0.0             -- Haskell Platform 2012.2.0.0
    7.4.1                  -- GHC 7.4.1
    7.4.2                  -- GHC 7.4.2
    7.0.4                  -- GHC 7.0.4
    test-sha               -- /home/chris/SHA-1.5.0.1

The name of the hub is given in the left hand column and a textual annotation sdescribing the hub is given in the right column.

The system packages (installed above with yum) install global hubs whose names always start with a digit (2011.4.0.0, 7.4.2, etc). Hubs the user creates (test-sha above) are the user hubs; they are private to the account of the logged-in user and their names start with a non-digit (usually a letter or a (double) underscore in the case of "anonymous" hubs, which could have been included in the listing by adding a -a flag:

    hub ls -a

).

One thing we can immediately do is list the packages that belong to the global hubs we have just installed.

    HUB=2011.4.0.0 ghc-pkg list

(This sets the HUB environment variable to 2011.4.0.0 for the duration of the ghc-pkg command and then forgets the binding.)

By setting the HUB environment variable to some hub for the duration of a toolkit command, the command will run in the context of that hub/sandbox. In this case ghc-pkg will list all of the constituents packages of the 2011.4.0.0 global hub.

Alternatively we could just use the hub list command:

    hub list 2011.4.0.0

which is just a shorthand for the above ghc-pkg command.

Code can be tested on the different platforms thus,

    cabal-unpack GLUT-2.1.2.1
    cd GLUT-2.1.2.1/examples/BOGLGP/Chapter01
    HUB=2012.2.0.0 runghc Simple.hs
    HUB=2011.4.0.0 runghc Simple.hs

which will run one of the GLUT example programs, first on the 2012.2.0.0 platform, and then on 2011.4.0.0.

Projects can be set up based on any of the installed platforms or (bare) GHC tool chains. The next example is taken from the current hub.cabal description. It uses the 2011.2.0.1 platform so it will need to be installed first.

    sudo yum install haskell-platform-2011.2.0.1-hub

The example sets up a hub in a work tree containing the latest edition of hexpat, installs hexpat in the hub and then switches into the test suite and installs that too. Finally the test suite is run.

    cabal unpack hexpat
    cd hexpat-*
    hub init -s 2011.2.0.1 hexpat-test
    cabal install
    cd test
    cabal install --bindir=.
    ./testsuite

The above command to create the hub is a new variant of hub init.

    hub init -s 2011.2.0.1 hexpat-test

It creates a new hub based on the global 2011.2.0.1 hub (a.k.a., Haskell Platform 2011.2.0.1), names it hexpat-test and associates the current work tree with the new hub (the -s flag is necessary to force the configuration of the directory when creating hubs that we have named).

Erasing Packages

For a variety of reasons, the Haskell tools have been a little weak in supporting the removal of packages. (No single tool in the standard set has enough definitive information to collect the garbage.) With the new powers to replicate and manage package databases this shortcomming could become really noticeable but fortunately the hub system is in a position support the full deletion of packages from user hubs.

    hub list hexpat-test

The package list for the above example should produce this list of packages:

    /usr/hs/db/2011.2.0.1.d
       Cabal-1.10.1.0
       GLUT-2.1.2.1
       HTTP-4000.1.1
       HUnit-1.2.2.3
       OpenGL-2.2.3.0
       QuickCheck-2.4.0.1
       array-0.3.0.2
       base-4.3.1.0
       bin-package-db-0.0.0.0
       bytestring-0.9.1.10
       cgi-3001.1.7.4
       containers-0.4.0.0
       deepseq-1.1.0.2
       directory-1.1.0.0
       extensible-exceptions-0.1.1.2
       ffi-1.0
       fgl-5.4.2.3
       filepath-1.2.0.0
       ghc-7.0.3
       ghc-binary-0.5.0.2
       ghc-prim-0.2.0.0
       haskell-platform-2011.2.0.1
       haskell-src-1.0.1.4
       haskell2010-1.0.0.0
       haskell98-1.1.0.1
       hpc-0.5.0.6
       html-1.0.1.2
       integer-gmp-0.2.0.3
       mtl-2.0.1.0
       network-2.3.0.2
       old-locale-1.0.0.2
       old-time-1.0.0.6
       parallel-3.1.0.1
       parsec-3.1.1
       pretty-1.0.1.2
       process-1.0.1.5
       random-1.0.0.3
       regex-base-0.93.2
       regex-compat-0.93.1
       regex-posix-0.94.4
       rts-1.0
       stm-2.2.0.1
       syb-0.3
       template-haskell-2.5.0.0
       text-0.11.0.6
       time-1.2.0.3
       transformers-0.2.2.0
       unix-2.4.2.0
       xhtml-3000.2.0.1
       zlib-0.5.3.1
    /home/chris/.hubrc/lib/hexpat-test/package.config
       List-0.4.2
       ansi-terminal-0.5.5
       ansi-wl-pprint-0.6.4
       hexpat-0.20.1
       hostname-1.0
       test-framework-0.6
       test-framework-hunit-0.2.7
       test-framework-quickcheck2-0.2.12.2
       utf8-string-0.3.7
       xml-1.3.12

Supposing you were considering using a different package from xml-1.3.12 in your project. You could try erasing it:

    hub erase xml-1.3.12

Hub will report that a number of packages are dependent on xml-1.3.12 and that they will be deleted too — once the user gives the nod.

    Packages requested to be deleted:
      xml-1.3.12
    Dependent packages also to be deleted:
      test-framework-0.6
      test-framework-hunit-0.2.7
      test-framework-quickcheck2-0.2.12.2
    Would you like to delete these packages? [n]

Currently this won't remove any code — that requires a garbage collection to make sure that no other hubs are referencing the code and documentation we have just unlinked from. To reclaim orphaned code and documentation we will have to run the garbage collector manually.

    hub-gc

and that will place the collected nodes in ~/.hubrc/garbage for the user to remove. (In fact these examples — by themselves — won't have freed up any nodes for the garbage collector to collect. This somewhat conservative approach to garbage collection is being retained for beta testing. In future erase commands will likely run the garbage collector automatically, and perhaps remove any collected nodes.)

Archiving and Restoring Hubs

The exact configuration of packages used to build a library or executable is an important part of the project's context that needs to be represented in and managed with the project's source management system along with the other scripts and source code. The

hub save build-hub

command can be used to provide a dump of this information on the standard output; in this example a dump of hub build-hub will be generated.

The saved archive used by the hub make file looks like this:

    ^=7.4.1
    List-0.4.2
    fgl-5.4.2.4
    hexpat-0.20.1
    mtl-2.1.1
    regex-base-0.93.2
    regex-compat-0.95.1
    regex-posix-0.95.2
    text-0.11.2.1
    transformers-0.3.0.0
    utf8-string-0.3.7

The first line starts with a ^= and lists the name of the global hub that it is based on — in this case GHC-7.4.1.

The subsequent lines just list the packages that need to be loaded into the user hub to make up the set.

The command to re-load a hub from a saved configuration is (surprise):

    hub load build-hub

It will check to see if the hub exists and create it if it isn't. If the hub exists it check that is is based on the correct global hub, otherwise it will remove it and start over. Next, the list of packages in the hub are checked for any that are not mentioned in the archive; any such packages and their dependents will be removed. Finally any missing packages are installed into the hub with cabal.

The hub makefile uses this fragment to prepare build environment.

    prep:
        hub load    build-hub <build-hub.har || printf $(OP)
        hub comment build-hub $(CM)          || true
        hub set     build-hub                || true
        runhaskell prep

(Running the prep script has nothing to do with setting up the sandbox — it is only included for completeness; the OP variable contains a long message set up in the header of the makefile.)

The prep target is a fake target that gets run each time make is invoked to build the executable. On checkout of the source tree in a user account that hasn't built this source tree the hub load command will most likely observe that there is no such build-hub hub and duly create one based on the 7.4.1 global hub and install the specified packages. This will of course take some time, but on subsequent builds this will just act as a guard, instantly checking that the build environment hasn't been compromised and proceeding with the next stage of the build.