Remote poudriere build

What I had forgot to include in a previous blog post was how I managed to successfully setup poudriere on atlas for use with remote package serving to ichigo.  I initially followed a Vermaden blog post Simple FreeBSD poudriere harvester guide which helped me get the majority of the setup configured, adjusted.  I attempted to leave a comment on the blog post but for whatever reason it failed, so I am going to provide that here as well.  The process is not difficult but there is at least one detail left out which could be discernable to some but it took a little for me to discover.

Vermaden does a very good job for the community to provide insights and guides and is part why I chose to write my own blog as well, beginning years ago.  It became obvious to me that writing about my experiences on FreeBSD would help me to remember and help others to find their way more easily.  I would recommend to any FreeBSD user to do the same for at least the same or similar reasons, it may seem a no-brainer.

The essentials of Vermaden's blog which I needed were as follows:

Adjust the rc.conf (or rc.conf.local) file to include:

  sshd_enable=YES
  nginx_enable=YES

Those lines will permit a web browser such as firefox to view the progress and results of the current poudriere bulk build, and permit ssh including access to X windows applications via ssh -X and sftp for any LAN file transfer needs.

Add needed packages:

  poudriere
  nginx
  git-lite

Although when I read the blog post I believe I already had those installed and some other things he mentions.

The most important step that I needed was for the pkg signing which is a series of steps, each command as root:

# mkdir -p   /usr/local/etc/ssl/keys /usr/local/etc/ssl/certs /usr/ports/distfiles
# chmod 0600 /usr/local/etc/ssl/keys
# openssl genrsa -out /usr/local/etc/ssl/keys/poudriere.key 4096
# openssl rsa -in /usr/local/etc/ssl/keys/poudriere.key -pubout -out /usr/local/etc/ssl/certs/poudriere.cert

I wasn't sure that mkdir would create multiple paths in one command, and I already had /usr/ports/distfiles, so I did them individually but testing a moment ago I now know I could have just copied that line entirely.  Those steps above are perfect for the poudriere build machine but a detail that is essential here which can be done now or anytime before trying to use the signed pkgs, is to copy the /usr/local/etc/ssl/certs/poudriere.cert to the client box(es).  This is one thing that is missing and tripped me up, surely it is assumed and obvious but not to me right then.

Since I already had poudriere running previously for use on the same machine, a poudriere.conf already existed.  I edited the file to be sure to add and revise as needed the lines he provided:

  ZPOOL=zroot
  BASEFS=/usr/local/poudriere
  ZROOTFS=/dev/null
 #FREEBSD_HOST=ftp://ftp.freebsd.org
  POUDRIERE_DATA=/usr/local/poudriere/data
  CHECK_CHANGED_OPTIONS=verbose
  CHECK_CHANGED_DEPS=yes
  PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key
  URL_BASE=http://172.16.0.12
  USE_TMPFS=all
  TMPFS_LIMIT=5
  MAX_MEMORY=4
  MAX_FILES=2048
  DISTFILES_CACHE=/usr/ports/distfiles
  KEEP_OLD_PACKAGES=yes
  KEEP_OLD_PACKAGES_COUNT=2
  CHECK_CHANGED_OPTIONS=verbose
  CHECK_CHANGED_DEPS=yes
  CCACHE_DIR=/var/cache/ccache
  RESTRICT_NETWORKING=yes
  

Since I use the same ports tree for the poudriere machine and also for poudriere, I did not create a seperate ports tree for it.  I also have the /etc/makeconf symbolic linked to /usr/local/etc/poudriere.d/make.conf because I want local things built to match what poudriere builds.  This way hopefully there is no difference but automation.

I setup nginx as described in Vermaden's blog post, possibly from a previous nginx poudriere progress view attempt, only needed to check it was accurate.  I have also had ccache setup for ages as well, so did not need to look at that much either.

One of the last steps in this whole process is to define the repos.  I had already setup poudriere.conf in /usr/local/etc/pkg/repos for the build machine to use the pkgs it makes, but I revised it to use the signature, the two lines were added.

poudriere: {
        url      : file:///usr/local/poudriere/data/packages/14amd64-default,
        signature_type: "pubkey",
        pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
        enabled  : yes,
        priority : 10
}

On the client machine, I needed a new pkg repo configuration file.

atlas: {
  url: "http://172.16.0.12/packages/14amd64-default",
  signature_type: "pubkey",
  pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
  enabled: yes,
  priority: 100
}

Since when I originally followed the instructions I missed placing a cert file on my client machine, this setup failed inexplicably.  I used pkg -v -v update -f to try to figure it out.  The results did not clearly say that I had no cert or that that was the issue, because something else was interfering.  I had pf setup on the poudriere build machine, and this halted the client from requesting anything, it literally silently failed, no explanation.  I probably had it in the back of my mind that pf could be the reason, so I eventually tried service pf stop and tried again, this time it was able to update the meta file and others.  Now was when it took a bit to figure out that I needed to copy the cert file to the client.  Once everything was in place, all I needed to remember was to periodically disable pf as needed.  I knew I could permanently cut a hole in pf for the pkg update or install requests but that was something for another time.

Remote LAN work

A new PC makes the old one a file server and sometimes build server.  Previously I had setup poudriere so I could periodically update the ports via pkg on my box with my own desired options.  I also have been keeping a number of unofficial ports updated as well, most notably, luanti (formerly minetest) which I described in an old blog post and revisit periodically it seems.  My plan is to continue to build on the same machine and use it to update ports on that box as well as on my new pc.  I also decided I wanted to figure out how to build my custom unofficial ports for pkgs to install on the new pc.  It turns out that there was a thread of documentation to find which eventually got me to the destination.

Firstly, I put both boxes onto the same LAN which is an obvious step, except that my new PC has an indirectly supported ethernet device on the motherboard.  I sidestepped this issue after temporarily solving it during a test install on an SSD.  The realtek 2.5GB ethernet card requires a driver from ports and an rc.conf adjustment to override the normally included re driver.  There was a little too long of a moment when everything should have worked but did not, an ethernet cable with a broken clip had fallen out of the switch.  With both machines on the same network and functioning, I could setup some things to make my transfer to the new box easier.

Since I have both machines accessible to each other it is a simple matter of setting up ssh so that I can sftp or ssh -X and use X server applications from the old box.  ssh is easy to setup, and with ssh you get the ability to use a secure ftp (sftp) the exact same way as the original deprecated ftp.  At this point the new box was barely configured and had a lot of what I needed for my GUI but my customizations for FVWM were not yet present.  There are other configuration files I transferred over using sftp, such as rc.conf.local and loader.conf.local which I believe I may have named rc.conf and loader.conf on the new box after I moved them.  With things mostly usable and shortly after a number of applications installed on the new box, some days later I finally got motivated to get poudriere setup.

The setup for poudriere to handle building and for pkg to access the files it builds took a little too long to figure out, even though its not particularly complicated.  The later step to handle the default ports tree and my custom local ports tree all in one step took a very close read of documentation to solve.  The basic idea is to use an overlay which is a relatively new addition to ports functionality.  Poudriere supports overlays but I was unable to figure it out with the anecdotal documentation I found.

First, after reading some forum posts and obviously missing some detail I went to the manpages and fell upon a couple details:

FreeBSD	13.2			August 9, 2024		     POUDRIERE-BULK(8)
(excerpt)
       -n	Dry run.  Show what would be done, but do not  actually	 build
		or delete any packages.

       -O overlay
		Specify	an extra poudriere-ports(8) tree to use	as an overlay.
		Multiple -O overlay arguments may be specified to stack	them.

       -p tree	Specify	 on  which  ports  tree	 the  bulk build will be done.
		(Default: "default")

FreeBSD	13.2			April 26, 2021		    POUDRIERE-PORTS(8)
(excerpt)
EXAMPLES
       Creates a new checkout from Git called default from FreeBSD's  official
       ports tree branch main.

	 # poudriere ports -c
       Creates	a  new checkout	from Git called	quarterly from FreeBSD's offi-
       cial ports tree branch 2021Q1.

	 # poudriere ports -c -p quarterly -B 2021Q1
       Updates ports tree named	quarterly.

	 # poudriere ports -u -p quarterly
       Import a	local manually-managed ports tree named	local from /usr/ports.

	 # poudriere ports -c -p local -m null -M /usr/ports

After reading this I now better understood what the command that was in the forum post was about:

With overlays:
Code:

# poudriere bulk accessibility/sct www/firefox tools/tool1 -p default -O privateportstree

The part -p default is essentially automatic as a default itself, so I can ignore it, the other part, -O privateportstree is important, but there needs to be additional setup to use this.  Since my alternative ports tree is incomplete but already existing, I used the command from the examples with adjustments:

  poudriere ports -c -p local -m null -M /usr/local/ports

I did that after I copied all of the contents of my /home/tigersharke/Ported_software directory into the newly created ports directory in the path /usr/local and to help me if I forget, I also created a symbolic link in my user directory to this named Local_ports.

Now I can use a similar command as in the past, to build all ports I wish with my unofficial ones included:

  poudriere bulk -O local -j 14amd64 -f `p-keg-deblack installed-pkgs-gross`

I had to make some adjustments to my blacklist because I was assuming I would not be able to do this combined build and maybe only might use poudriere on specific unofficial ports.  I have been working on getting pkgs updated and installed onto my new ichigo box which get built on the former ichigo now called atlas.  A number of adjustments to my make.conf and blacklist files were necessary, as well as avoiding some non-vital options in some ports which presently cause them to fail building.  While all this is going on, my repeated builds and adjustments, I am editing this blog post using bluefish which I am running via ssh -X from atlas to appear on my monitor which shows me the new ichigo desktop.  A lot of remote lan work including poudriere, ssh edits and an X app appearing from a different machine.  It is amazing what can be done and much of this technology is not brand new, poudriere is the most recent of the bunch actually.