Creating a BINHOST for can be a real time saver when managing multiple Gentoo servers. By using a BINHOST other systems can simply download and use those packages, without waiting for building. This makes an install of those packages almost as fast as an apt-get might be.

Preparing for Binary Hosting

At Edoceo we have one machine that handles all of our Binary Host needs. It updates portage nightly, builds the packages and exposes the packages via FTP and HTTP.

Update /etc/make.conf

To have this machine always build packages merge the following entries into /etc/make.conf. 

FEATURES="buildpkg"

Create cron task

Update Portage every night for this machine and send a report about available updates Please note that this does not build the packages. This will only update the local portage tree and fetch the packages that need to be built. Building the packages is done manually. 

02 2 * * * /opt/edoceo/sbin/portage-update

The command above is a simple shell script that does the following. 

#!/bin/bash

export EMERGE_DEFAULT_OPTS="--alphabetical --color=n --nospinner  --quiet"
export FEATURES="nocolor notitles"

# Update my Tree
emerge --sync >/dev/null

# Fetch the stuff that this system (and presumeably all nodes) need
emerge --deep --newuse --update --fetchonly world >/dev/null

# Report these updates
# emerge --deep --newuse --pretend --update --verbose world

Expose FTP and/or HTTP

We use Pure-ftpd and Apache for these procedures. These settings will need modified based on the environment.

To expose the Portage binaries via FTP simply point to the package root ($PKGDIR) In Portage 2.1 the packages are /usr/portage/packages/All in Portage 2.2 it's just /usr/portage/packages. 

useradd -c'Portage Binary Host Users' -d /usr/portage/packages -g ftp -s /bin/false pbhu

For the HTTP exposure Apache is given an Alias and some options for directory indexing. 

Alias /portage-i686 /usr/portage/packages
<Directory /usr/portage/packages>
	Options +Indexes
	IndexOptions +FoldersFirst +IgnoreClient +VersionSort
</Directory>

Building Binaries

The binary system uses the above command to sync portage and generate a report about the updates. This will of course only report about packages installed on the binhost system. The binary host system will likely not have all the packages that all of the other nodes would need.

Above a command was given that updates Portage nightly and fetches the packages and the output would be mailed to root. The reason that binary packages are not automatically built is to ensure the quality of the built packages. An extensive list of per packages USE flags is maintained to ensure each packages functions exatcly as desired. This is list is not complete so the choice is made to audit each of the builds. It is very undesireable to create binary packages that will not operate as expected on all of the nodes using this binhost.

Determine what needs to be updated, this info would also be in the email from the cron job. The emerge command must be run with -g (--getbinpkg), or set FEATURES="getbinpkg" in /etc/make.conf. 

emerge -gpvuDN world
[ebuild     U ] net-dns/dnsmasq-2.34 [2.31] USE="-resolvconf%"
[ebuild   R   ] dev-libs/glib-1.2.10-r5  USE="(-debug%) -hardened%"
[ebuild     U ] sys-fs/fuse-2.6.1 [2.6.0_rc1]

Verify the above is what is desired and then do the update of the binhost which will also create the packages. 

emerge -guDN world

Build Specific Packages

Our binhost server does not run some software that other systems do. Specific pacakges can be built and added to the exposed binhost directories using the ebuild command. 

ebuild /usr/portage/net-dns/dnsmasq/dnsmasq-2.34.ebuild package
stat /usr/portage/packages/All/dnsmasq-2.34.tbz2
  File: `/usr/portage/packages/All/dnsmasq-2.34.tbz2'
  Size: 156935          Blocks: 320        IO Block: 4096   regular file
Device: 901h/2305d      Inode: 18497628    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2007-02-21 12:01:07.000000000 -0800
Modify: 2007-02-21 12:01:07.000000000 -0800
Change: 2007-02-21 12:01:07.000000000 -0800

Another option is to manually add the desired packages to /var/lib/portage/world and they will crawl in as updates occur. This will also install those packages on the binhost, which may block some items e.g. ssmtp blocks postfix.

Subscribing to the Binary Host

Once the host is operational and exposed other nodes may begin using this system. Merge the following into /etc/make.conf of these nodes. For clarity many examples are shown for the PORTAGE_BINHOST url, pick only one. 

PORTAGE_BINHOST="http://cdn.edoceo.com/praxis/x64/"
PORTAGE_BINHOST="ftp://pbhu:pbhu@pbh.edoceo.com/"
PORTAGE_BINHOST="http://pbhu:pbhu@pbh.edoceo.com/portage-i686/"
PORTAGE_BINHOST="https://pbhu:pbhu@pbh.edoceo.com/portage-i686/"

# Maybe the binhost can also be the rsync mirror too
SYNC="rsync://pbh.edoceo.com/edoceo-portage"

When performing an update on the subscriber node the output from emerge will be slightly different. This output has been truncated. 

emerge -gpvuDN world
[binary     U ] net-dns/dnsmasq-2.34 [2.31] USE="-resolvconf%" 250 kB
[binary  N    ] net-analyzer/traceroute-1.4_p12-r5  USE="-static"
[ebuild   R   ] sys-kernel/gentoo-sources-2.6.17-r8  USE="-build -symlink* (-ultra1)" 40,553 kB
[binary     U ] app-misc/ca-certificates-20061027.2 [20050804]

The packages listed as binary will be updated from the binhost, ebuild packages will be downloaded and compiled locally.

It may be handy to have each of the nodes on the network send a report about what packages they need built. Then binhost would process that list and build the packages, nodes would need to be notified of the updates.

See Also

ChangeLog