Build Chain Setup

Build Chain Setup
	Chapter 2. Setting up for packaging

The basic tool to create a package is dpkg-buildpackage. But PBuilder creates packages in a minimal chroot, which ensures a clean build and enables us to build for multiple versions of multiple distributions, and even multiple architectures, all on a single build host. COWBuilder adds copy-on-write tactics to PBuilder, which saves significant amounts of time. And PDeBuild is a wrapper around either COWBuilder or PBuilder directly.

Into a package goes upstream source. But the packager may have to patch the source. And he certainly will have to adjust the package to its contents. For lack of a better word, I'll call the efforts of the packager the container here.

It is useful to keep the container under version control. Git being my VC-tool of choice, I use git-buildpackage to do that. So the list of tools, a simplified call graph if you will, becomes:

git-buildpackage -> pdebuild -> cowbuilder -> pbuilder -> dpkg-buildpackage

Procedure 2.2. Setting up PBuilder/COWBuilder

A ~/.pbuilderrc for multiple distributions

Put the following in ~/.pbuilderrc. (It also works to put it in /etc/pbuilderrc.)

# This is to enable building packages for multiple distributions

# Change 'cowbuilder' to 'pbuilder' below if you prefer that
PDEBUILD_PBUILDER=cowbuilder

if [ -n "${DIST}" ]; then
    export DISTRIBUTION=$DIST

    AUTO_DEBSIGN=yes

    BASETGZ="$(dirname $BASETGZ)/$DIST-base.tgz"

    export BASEPATH=/var/cache/pbuilder/${DIST}-base.cow
    export BUILDPLACE=/var/cache/pbuilder/${DIST}.cow

    install -d ../lastbuild/${DIST}
    export BUILDRESULT=../lastbuild/${DIST}

    if echo $@|grep -qe --create ; then
        BUILDPLACE=/var/cache/pbuilder/${DIST}-base.cow
        install -d ${BUILDPLACE}
    fi  

    APTCACHE="/var/cache/pbuilder/$DIST/aptcache/"

    case ${DIST} in
        'lenny'|'squeeze'|'wheezy'|'sid' )
            MIRRORSITE=http://ftp.nl.debian.org/debian
            ;;
        'lucid'|'maverick'|'natty'|'oneiric'|'precise' )
            MIRRORSITE=http://osmirror.rug.nl/ubuntu
            ;;
    esac

fi

Creating COWBuilder chroots

COWBuilder builds packages in chroot jails, which we need to create. Here's a script /usr/local/sbin/cowbuilder-create-base that will do it:


#!/bin/bash

declare DISTROS BP_ASIDE BUILDPLACE DIST DISTRO DEBOOTSRAPOPTS MIRROR PBUILDEROPTS
declare CREATE=create

PROGRAM=${0}
OPTARGS=$(getopt -n $PROGRAM -o 'uh' -- "$@")
eval set -- "$OPTARGS"
while (( "$#" > "0" )) ; do
    case "$1" in
        '-h' )
            cat <<EOF

            usage: cowbuilder-create-base [-u] [dists]

            with -u meaning 'update instead of create' and
            [dists] a space separated list of common Debian or Ubuntu
            distribution codenames like 'precise' or 'squeeze'

EOF
            exit 0
            ;;
        '-u' )
            shift 1
            CREATE=update
            ;;
        '--' )
            shift 1
            break
            ;;
        * )
            echo "unknown option: $1"
            exit 3
            ;;
    esac
done   

DISTROS=${@:-squeeze wheezy sid precise}

restore_old_buildplace()
{
    [ "${CREATE}" = "create" ] \
        && [ -n \"${BP_ASIDE}\" ] \
        && [ -d ${BP_ASIDE} ] \
        && rm -rf ${BUILDPLACE} \
        && mv ${BP_ASIDE} ${BUILDPLACE} \
        && return 0
    return 1
}

for DIST in ${DISTROS} ; do
    DEBOOTSTRAPOPTS=
    PBUILDEROPTS=
    case ${DIST} in
        hamm|slink|potato|woody|sarge|etch|lenny|squeeze|wheezy|sid )
            DISTRO=debian
            DEBOOTSTRAPOPTS="--debootstrapopts --keyring=/usr/share/keyrings/debian-archive-keyring.gpg"
            MIRROR=http://ftp.nl.debian.org/debian
            ;;
        hardy|lucid|maverick|natty|oneiric|precise )
            DISTRO=ubuntu
            MIRROR=http://osmirror.rug.nl/ubuntu
            PBUILDEROPTS="--components \"main universe\""
            ;;
        * )
            echo "unknown distro ${DIST}, please add it to the script before re-running"
            exit 2
    esac

    echo "Creating CowBuilder base for ${DISTRO} ${DIST}"
    BUILDPLACE=/var/cache/pbuilder/${DIST}-base.cow

    unset BP_ASIDE
    if [ "${CREATE}" = "create" ] ; then
        if [ -d "${BUILDPLACE}" ] ; then
            echo "Moving existing ${BUILDPLACE} aside to ${BP_ASIDE}"
            # If pbuilder fails, we want to restore this old original
            BP_ASIDE=$(mktemp -d --tmpdir=$(dirname ${BUILDPLACE}))
            mv ${BUILDPLACE} ${BP_ASIDE}
        fi
        install -d ${BUILDPLACE}
    fi

    trap restore_old_buildplace INT

    COMMAND="pbuilder --${CREATE} \
--logfile pbuild-${DIST}-create.log \
--buildplace ${BUILDPLACE} \
--distribution ${DIST} \
--mirror ${MIRROR} \
--no-targz \
--extrapackages cowdancer ${PBUILDEROPTS} \
--debootstrap debootstrap ${DEBOOTSTRAPOPTS}"

    if eval ${COMMAND} ; then
        trap - INT
        [ "${CREATE}" = "create" ] && [ -n "${BP_ASIDE}" ] && [ -d ${BP_ASIDE} ] && rm -rf ${BP_ASIDE}
    else
        FAILLIST="${FAILLIST} ${DIST}"
        trap - INT

        if restore_old_buildplace ; then
            FAILLIST="${FAILLIST}(but existing version was saved and restored)"
            echo "The command \"${COMMAND}\" failed."
            echo "Old chroot restored from ${BP_ASIDE}"
        fi

    fi

done


if [ -n "${FAILLIST}" ] ; then
    echo "The following distributions failed to be (re)created: ${FAILLIST}"
    exit 1
fi

exit 0

It can be called as either of these:

apprentice@packaging-host:~$ sudo cowbuilder-create-base

apprentice@packaging-host:~$ sudo cowbuilder-create-base squeeze

apprentice@packaging-host:~$ sudo cowbuilder-create-base -u squeeze

(The last one merely updates an existing chroot.)


GPG keys		Setting up gpg-agent