Packaging Software (for your reference)

This used to be an objective for LPI. Since it no longer is, this section serves as a reference. This section includes building (or rebuilding) both RPM and DEB packaged software.

Key files, terms and utilities include:

rpm
SPEC file format
/debian/rules

Resources: Jackson01

While there is a vast quantity of software available in the DEB or RPM format, sometimes it is necessary to install software that isn't. It is often possible to install the software from source, but you may want to create a package from the source and install the package. This is useful if you want to be able to do updates without worrying if all obsolete files are removed or if the software needs to be installed on more than one machine.

Both DEB and RPM use a similar structure to build a package. A script, in one way or another, is used to build the package to run in /usr and install it in a temporary directory (DEB: <sourcedir>/debian/<packagename> ; RPM: /var/tmp/<packagename>-buildroot). The appropriate files are gathered in an archive and some control data is added. The resulting DEB or RPM can be installed on the target platform.

Both package systems start from the original source and apply a patch where necessary to correct any problems that might arise in packaging.

The major difference between the packaging formats is the location and structure of the scripts. In the DEB format, a directory (debian) which contains a number of files is added to the source. RPM uses a spec-file, which is separate from the source.

DEB Packages

As mentioned in the Debian Policy Guide (Jackson01), the debian directory structure can be generated with dh_make. This utility will try to guess a suitable debian/rules file based on information in the source-directory.

The generated debian directory contains a number of files. Some files will need to be edited, e.g., the Description: needs a proper description in the control file and documentation you want to have installed in /usr/share/doc/packagename will have to be added to the docs file. See Jackson01 for more information.

Only the required files will be mentioned here. These are control, changelog, copyright and rules.

control file

This file contains various values which dpkg and dselect will use to manage the package. Here is an example:

Source: bao
Section: games
Priority: optional
Maintainer: Ben Mesman <ben@mesman.nu>
Build-Depends: debhelper (>> 3.0.0)
Standards-Version: 3.5.2

Package: bao
Architecture: any
Depends: ${shlibs:Depends}
Description: A mancala game from Tanzania and Zanzibar
 Bao is what we call a mancala game. Mancala is the term to denominate
 games with one shared characteristic: moves are not executed as in
 chess or checkers, instead moves are executed by sowing seeds (or
 other playing pieces) into holes.

The first six lines are the control information for the source package.

The Section: field is the Debian section this package goes to. This package will be installed in the section games. Other sections include for instance x11 for X11 specific programs or devel for programmer tools.

The Priority: field describes how important it is for the user to install this package. Possible priorities are required, important, standard, optional and extra. This should probably be optional or extra for self-created packages.

The Maintainer: field is the maintainer of the Debian package, not the upstream maintainer. The Standards-Version is the version of the Debian Policy Manual the package conforms to.

If you want to resolve the build dependencies (i.e., the packages needed to build this package) automatically, you need to specify the packages in the Build-Depends field.

The second part of the file contains information on the binary package that will be generated. If more binary packages should be created from the same source, there will be more of these sections.

Note that the Architecture: field should be set to all for most packages. Se the Debian Policy Manual for more information on architecture dependent packages.

Apart from the package description, the most important information here is the relationship with other packages. There are seven possible relations:

Depends:

Used if the program absolutely will not run (or will cause severe breakage) unless a particular package is present.

Recommends:

Used for packages that are not strictly necessary but are typically used with the program.

Suggests:

Used for packages which will work nicely with the program but are not at all necessary

Pre-Depends:

This is stronger than Depends:. The package will not be installed unless the packages it pre-depends on are installed and correctly configured. Use this very sparingly and only after discussing it on the debian-devel mailing list.

Conflicts:

Used if the program absolutely will not run (or will cause severe breakage) if a particular package is present.

Provides:

For some types of packages where there are multiple alternatives, virtual names have been defined. You can get the full list in /usr/share/doc/debian-policy/virtual-package-names-list.text.gz file. Used if the program provides a function of an existing virtual package.

Replaces:

Used when the program replaces files from another package or completely replaces another package (used in conjunction with Conflicts:). Files from the named packages will be removed before installing.

Note that in this example the Depends: field contains $(shlibs:Depends). This will be automatically generated by dh_shlibdeps, and filled in by dh_gencontrol, with the names of any shared libraries the program uses, such as libc6 or xlib6g, so they don't have to be explicitly specified.

copyright file

This file contains the information about package upstream resources, copyright and license information. Its format is not dictated by the Debian PolicyJackson01, but the contents is (section 6.5). dh_make creates a default one. This is what it looks like:

This package was debianized by Ben Mesman <ben@snow.nl>
on Fri, 23 Nov 2001 11:42:37 +0100.

It was downloaded from <fill in ftp site>

Upstream Author(s): <put author(s) name and email here>

Copyright:

<Must follow here>

If the program is licensed under one of the common free software licenses, such as the GNU GPL or LGPL, BSD or the Artistic license, you can refer to the appropriate file in the /usr/share/common-licenses/ directory. If the program has its own license, the complete license must be included in this file.

Changelog file

This is a required file and uses a special format. This format is used by dpkg and other programs to obtain the version number, revision, distribution and urgency of the package.

For the maintainer of the program, it is also important, since it is good to document all changes. It will help people downloading the package to see whether there are any unresolved issues that they should know about before installing. It will be saved as /usr/share/doc/bao/changelog.Debian.gz in the binary package.

This is an example Changelog:

bao (0.1.0) unstable; urgency=low

  * Initial Release.

 -- Ben Mesman <ben@snow.nl>  Fri, 23 Nov 2001 11:42:37 +0100

Local variables:
mode: debian-changelog
End:

The first line is the package name, version, distribution and urgency. The name must match the source package name, distribution should be either unstable or experimental (for now), and urgency probably shouldn't be changed to anything higher than low.

The third line is a log entry describing the changes in this release. The fifth line closes the comments and specifies who made the release and when. The name <email> must be followed by two spaces and the date in RFC822 format (e.g., as generated with date -R).

New releases should be added at the top.

rules file

Now we need to take a look at the exact rules which dpkg-buildpackage will use to actually create the package. This file is actually another Makefile, since it is executed with make -f, but different from the one in the upstream source.

The important part to know about the rules file created by dh_make is that it is just a suggestion. It will work for simple packages, but, for more complicated ones, don't be afraid to add and subtract from it to fit your needs. The only thing that you must not change are the names of the rules, because all the tools use these names, as mandated by the Packaging Manual.

#!/usr/bin/make -f
# Sample debian/rules that uses debhelper.
# GNU copyright 1997 to 1999 by Joey Hess.

# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1

# This is the debhelper compatibility version to use.
export DH_COMPAT=3

configure: configure-stamp
configure-stamp:
        dh_testdir
        # Add here commands to configure the package.
        ./configure --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info

        touch configure-stamp

build: configure-stamp build-stamp
build-stamp:
        dh_testdir

        # Add here commands to compile the package.
        $(MAKE)
        #/usr/bin/docbook-to-man debian/bao.sgml > bao.1

        touch build-stamp

clean:
        dh_testdir
        # dh_testroot
        rm -f build-stamp configure-stamp

        # Add here commands to clean up after the build process.
        -$(MAKE) distclean

        dh_clean

install: build
        dh_testdir
        dh_testroot
        dh_clean -k
        dh_installdirs

        # Add here commands to install the package into debian/bao.
        $(MAKE) install DESTDIR=$(CURDIR)/debian/bao

# Build architecture-independent files here.
binary-indep: build install
# We have nothing to do by default.

# Build architecture-dependent files here.
binary-arch: build install
        dh_testdir
        dh_testroot
#       dh_installdebconf
        dh_installdocs
        dh_installexamples
        dh_installmenu
#       dh_installlogrotate
#       dh_installemacsen
#       dh_installpam
#       dh_installmime
#       dh_installinit
        dh_installcron
        dh_installman
        dh_installinfo
#       dh_undocumented
        dh_installchangelogs 
        dh_link
        dh_strip
        dh_compress
        dh_fixperms
#       dh_makeshlibs
        dh_installdeb
#       dh_perl
        dh_shlibdeps
        dh_gencontrol
        dh_md5sums
        dh_builddeb

binary: binary-indep binary-arch
.PHONY: build clean binary-indep binary-arch binary install configure

As you can see from the first line, this script will be processed by make.

The configure rule (and its child, config-stamp) specify that the program will be configured to run in /usr (instead of the default /usr/local). Also the manual pages and the shared files will be installed in the Debian default location.

The build rule (and its child build-stamp) specify how the program is to be built. In most cases, the program's own Makefile will be used.

The clean rule cleans up any unneeded binary or auto-generated stuff left over from building the package. This rule must be working at all times (even when the source tree is cleaned up!), so please use the forcing options (e.g., for rm, that is -f), or ignore return values (with a - in front of a command name).

The installation process, the install rule, basically runs the install rule from the program's own Makefile, but installs in <pwd>/debian/bao directory.

As the comments suggest, the binary-indep rule is used to build architecture independent packages. As none are specified in the control file, nothing will be done. If the package was an Architecture: all one, all the commands for building the package would be under this rule, and the next rule (binary-arch) would be empty instead.

The binary-arch rule uses several small utilities from the debhelper package to perform various operations to make the package Policy conforming.

The names start with dh_, and the rest is the description of what the particular utility really does. It is all quite self-explanatory, but here are some additional explanations:

  • dh_testdir checks that you are in the right directory (i.e. the top-level source directory),

  • dh_testroot checks that you have root permissions which is needed for binary* targets, and clean,

  • dh_installmanpages copies all the manpages it can find in the source tree to package,

  • dh_strip strips debugging headers from executable files and libraries, to make them smaller,

  • dh_compress gzips manual pages and documentation larger than 4 Kb,

  • dh_installdeb copies package related files (e.g. the maintainer scripts) under debian/tmp/DEBIAN directory,

  • dh_shlibdeps calculates shared libraries dependencies of the libraries and executables,

  • dh_gencontrol adds stuff to, and installs, the control file,

  • dh_md5sums generates MD5 checksums for all the files in the package.

For more information on these and other debhelper commands, read the debhelper documentation.

Building The Package

When the rules file is adapted to the program, a package can be build by executing

dpkg-buildpackage -rfakeroot

This will do everything for you, including the signing of the files. You will just have to enter your PGP (or GPG) secret key twice.

In our example, the following files will be generated:

bao_0.1.0-1_i386.deb

This is the completed binary package. You can use dpkg or apt to install or remove this just like any other package;

bao_0.1.0.orig.tar.gz

This is the original source code gathered up so that if someone else wants to recreate the package from scratch they can. Or if they aren't using the Debian packaging system, but need to manually download the source and compile.

bao_0.1.0-1.dsc

This is a summary of the contents of the source code. The file is generated from the bao-0.1.0/debian/control file and is used when unpacking the source with dpkg-source. This file is PGP signed, so that people can be sure of its origin.

bao_0.1.0-1.diff.gz

This compressed file contains each and every addition made to the original source code, in the form known as unified diff. It is made and used by dpkg-source.

bao_0.1.0-1_i386.changes

This file describes all the changes made in the current package revision, and it is used by the Debian FTP archive maintenance programs to install the binary and source packages in it. It is partly generated from the bao-0.1.0/debian/changelog file and the .dsc file.

People downloading the package can look at this file and quickly see what has changed. The long strings of numbers are MD5 checksums for the files mentioned. Those downloading the files can test them with md5sum and if the numbers don't match, they'll know the file is corrupt or has been hacked. This file is PGP signed, so that people can be even more sure that it is authentic.

RPM Packages

The SPEC file is a description of how a (source-)RPM can be built from source. The SPEC file should be named according to a standard convention.

<program name>-<version number>-<release number>.spec

This will ensure that if you install multiple source RPMs for different versions of the same package that the spec files remain intact.

Before you can actually build an RPM from a spec file, you should make sure you have a proper build tree. It should contain the following directories:

  • BUILD is the directory where all building occurs by RPM. You don't have to do your test building anywhere in particular, but this is where RPM will do its building,

  • SOURCES is the directory where you should put your original source tar files and your patches. This is where RPM will look by default,

  • SPECS is the directory where all spec files should go,

  • RPMS is where RPM will put all binary RPMs when built,

  • SRPMS is where all source RPMs will be put.

The default location is of the build tree is /usr/src.

Here is a sample of a spec file (bao-0.1.0-1.spec):

Summary: A mancala game from Tanzania and Zanzibar
Name: bao
Version: 0.1
Release: 1
Copyright: GPL
Group: Amusements/Games
Source: ftp://ftp.somesite.com/pub/games/bao-0.1.0.tar.gz
# Patch: none
BuildRoot: /var/tmp/%{name}-buildroot

%description
Bao is what we call a mancala game. Mancala is the term to denominate
games with one shared characteristic: moves are not executed as in
chess or checkers, instead moves are executed by sowing seeds (or
other other playing pieces) into holes.

%prep
%setup

%build
./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info
make

%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$(RPM_BUILD_ROOT)

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root)
%doc README TODO

/usr/bin/bao

%changelog
* Tue Dec 04 2001 Ben Mesman <ben@snow.nl>
- initial release

The Header

The header consists of a series of fields that provide general information about the package. These fields are:

  • Summary: This is a one line description of the package,

  • Name: This is the <name> string from the rpm file,

  • Version: This is the <version> string from the rpm file,

  • Release: This is the <release> string from the rpm file,

  • Copyright: This is the copyright of the package. It should be a short description, like GPL, BSD, MIT, public domain, distributable or commercial,

  • Group: This is used by higher-level installation programs, such as gnorpm, to place this package in its proper place in the hierarchical structure. A description of the hierarchy can be found in /usr/doc/rpm*/GROUPS,

  • Source: This is the location of the original source file. The filename on the local system must match the filename in this line (i.e., do not rename the file after downloading). Several source files can be specified along these lines:

    Source0: http://site.name/downloads/hopla-01.tar.gz
    Source1: http://site.name/downloads/hopla-02.tar.gz
    Source2: ftp://other.site/pub/patches/addition-1.tar.gz
    

  • Patch: This works basically, the same as the Source: field. It designates the location of the patch file. Multiple patches can be specified in the same manner as in Source:,

  • BuildRoot: This is the root for building and installing the new package. This can help test the package before it is installed,

  • %description This is not really a header item, but should be described with the header anyway. It is a multi-line description of the package. There should be one %description field per package and/or subpackage.

Prep

This is the second section of the spec file. It should contain all commands to set-up the source to do a make. This is where you unpack the source(s) and apply the patch(es).

One thing to note is that each of these sections is really just a place to execute shell scripts. So, it is possible to create a sh script and put it after the %prep tag to unpack and patch the sources. However, there are macros to aid in this.

The first macro, shown in the example, is the %setup macro. It unpacks the sources mentioned in the Source: line in the header and cd's into the source directory.

The second macro is called %patch. It will apply all patches mentioned in the header. Since there are no patches for this package, the %patch macro is not included in the example.

If something needs to be done that can't be handled by these macros, you can include another setup. Everything up to the %build section will be interpreted as an sh script.

Build

There aren't really any macros for this section. Just use any commands here that you would need to build the software once you have untarred the source, patched it and cd'ed into the directory. This is just another set of commands passed to sh, so any legal sh commands can go here (including comments).

Important

It is important to keep in mind that your current working directory is reset in each of these sections to the top level of the source directory. You can always cd into subdirectories if necessary.

Install

There aren't really any macros here, either. Basically, use any commands that are necessary for the install. If you have make install available to you in the package you are building, put that here. If not, you can either patch the Makefile for a make install and just do a make install here, or you can hand install them here with sh commands. You can consider your current directory to be the top-level of the source directory.

The variable RPM_BUILD_ROOT is available to tell you the path set as the Buildroot: in the header. Using build roots are optional, but are highly recommended because they keep you from cluttering your system with software that isn't in your RPM database (building an RPM doesn't touch your database...you must install the binary RPM you just built to do that).

Clean

It's a good idea to always make sure there is a clean build root before building a package a second time on a system. The %clean macro will help with that. Simply put the proper commands there to blow away a former build root.

In the example, the RPM_BUILD_ROOT is removed. To be prudent, it is a good idea to check that RPM_BUILD_ROOT was not set to / before doing something this volatile.

Files

This is the section where you must list the files for the binary package. RPM has no way of knowing what binaries get installed as a result of make install. To circumvent this, some have suggested doing a find before and after the package install. With a multiuser system, this is unacceptable as other files may be created during a package building process that have nothing to do with the package itself.

There are some macros available to do some special things as well. They are listed and described here:

  • %doc is used to mark documentation in the source package that you want installed in a binary install. The documents will be installed in /usr/doc/$NAME-$VERSION-$RELEASE. You can list multiple documents on the command line with this macro, or you can list them all separately using a macro for each of them.

  • %config is used to mark configuration files in a package. This includes files like sendmail.cf, passwd, etc. If you later uninstall a package containing config files, any unchanged files will be removed and any changed files will get moved to their old name with a .rpmsave appended to the filename. You can list multiple files with this macro as well.

  • %dir marks a single directory in a file list to be included as being owned by a package. By default, if you list a directory name without a %dir macro, everything in that directory is included in the file list and later installed as part of that package.

  • %defattr allows you to set default attributes for files listed after the %defattr declaration. The attributes are listed in the form (mode, owner, group) where the mode is the octal number representing the bit pattern for the new permissions (such as chmod would use), owner is the username of the owner, and group is the group you would like assigned. (You may leave any field to the installed default) by simply placing a - in its place, as was done in the mode field for the example package.

  • %files -f <filename> will allow you to list your files in some arbitrary file within the build directory of the sources. This is nice in cases where you have a package that can build its own file list. You then include that file list here and you won't have to specifically list the files.

A potential problem in the file list is listing directories. If you list /usr/bin by accident, your binary package will contain every file in /usr/bin on your system.

Changelog

This is a log of the changes that were made when the package was updated. If you are modifying an existing RPM it is a good idea to list the changes here.

The format is simple. Start each new entry with a line with a * followed by the date, your name and your email address. The date should appear in the same format that is output by:

date +"%a %b %d %Y"

The rest of the section is a free text field, but should be organised in some coherent manner.

Building RPMs

When the SPEC file is finished, you can build an RPM with:

rpm -bb <spec-file>

The -bb indicate that rpm should build a binary package. -bs builds a source rpm and -ba builds both (all). The RPM will be placed in /usr/src/RPMS/ and the source RPM will be placed in /usr/src/SRPMS (or wherever your RPM-build tree should be).

Copyright Snow B.V. The Netherlands