Remote poudriere build

What I had forgot to include in a previous blog post was how I managed to successfully setup poudriere on atlas for use with remote package serving to ichigo.  I initially followed a Vermaden blog post Simple FreeBSD poudriere harvester guide which helped me get the majority of the setup configured, adjusted.  I attempted to leave a comment on the blog post but for whatever reason it failed, so I am going to provide that here as well.  The process is not difficult but there is at least one detail left out which could be discernable to some but it took a little for me to discover.

Vermaden does a very good job for the community to provide insights and guides and is part why I chose to write my own blog as well, beginning years ago.  It became obvious to me that writing about my experiences on FreeBSD would help me to remember and help others to find their way more easily.  I would recommend to any FreeBSD user to do the same for at least the same or similar reasons, it may seem a no-brainer.

The essentials of Vermaden's blog which I needed were as follows:

Adjust the rc.conf (or rc.conf.local) file to include:

  sshd_enable=YES
  nginx_enable=YES

Those lines will permit a web browser such as firefox to view the progress and results of the current poudriere bulk build, and permit ssh including access to X windows applications via ssh -X and sftp for any LAN file transfer needs.

Add needed packages:

  poudriere
  nginx
  git-lite

Although when I read the blog post I believe I already had those installed and some other things he mentions.

The most important step that I needed was for the pkg signing which is a series of steps, each command as root:

# mkdir -p   /usr/local/etc/ssl/keys /usr/local/etc/ssl/certs /usr/ports/distfiles
# chmod 0600 /usr/local/etc/ssl/keys
# openssl genrsa -out /usr/local/etc/ssl/keys/poudriere.key 4096
# openssl rsa -in /usr/local/etc/ssl/keys/poudriere.key -pubout -out /usr/local/etc/ssl/certs/poudriere.cert

I wasn't sure that mkdir would create multiple paths in one command, and I already had /usr/ports/distfiles, so I did them individually but testing a moment ago I now know I could have just copied that line entirely.  Those steps above are perfect for the poudriere build machine but a detail that is essential here which can be done now or anytime before trying to use the signed pkgs, is to copy the /usr/local/etc/ssl/certs/poudriere.cert to the client box(es).  This is one thing that is missing and tripped me up, surely it is assumed and obvious but not to me right then.

Since I already had poudriere running previously for use on the same machine, a poudriere.conf already existed.  I edited the file to be sure to add and revise as needed the lines he provided:

  ZPOOL=zroot
  BASEFS=/usr/local/poudriere
  ZROOTFS=/dev/null
 #FREEBSD_HOST=ftp://ftp.freebsd.org
  POUDRIERE_DATA=/usr/local/poudriere/data
  CHECK_CHANGED_OPTIONS=verbose
  CHECK_CHANGED_DEPS=yes
  PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/poudriere.key
  URL_BASE=http://172.16.0.12
  USE_TMPFS=all
  TMPFS_LIMIT=5
  MAX_MEMORY=4
  MAX_FILES=2048
  DISTFILES_CACHE=/usr/ports/distfiles
  KEEP_OLD_PACKAGES=yes
  KEEP_OLD_PACKAGES_COUNT=2
  CHECK_CHANGED_OPTIONS=verbose
  CHECK_CHANGED_DEPS=yes
  CCACHE_DIR=/var/cache/ccache
  RESTRICT_NETWORKING=yes
  

Since I use the same ports tree for the poudriere machine and also for poudriere, I did not create a seperate ports tree for it.  I also have the /etc/makeconf symbolic linked to /usr/local/etc/poudriere.d/make.conf because I want local things built to match what poudriere builds.  This way hopefully there is no difference but automation.

I setup nginx as described in Vermaden's blog post, possibly from a previous nginx poudriere progress view attempt, only needed to check it was accurate.  I have also had ccache setup for ages as well, so did not need to look at that much either.

One of the last steps in this whole process is to define the repos.  I had already setup poudriere.conf in /usr/local/etc/pkg/repos for the build machine to use the pkgs it makes, but I revised it to use the signature, the two lines were added.

poudriere: {
        url      : file:///usr/local/poudriere/data/packages/14amd64-default,
        signature_type: "pubkey",
        pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
        enabled  : yes,
        priority : 10
}

On the client machine, I needed a new pkg repo configuration file.

atlas: {
  url: "http://172.16.0.12/packages/14amd64-default",
  signature_type: "pubkey",
  pubkey: "/usr/local/etc/ssl/certs/poudriere.cert",
  enabled: yes,
  priority: 100
}

Since when I originally followed the instructions I missed placing a cert file on my client machine, this setup failed inexplicably.  I used pkg -v -v update -f to try to figure it out.  The results did not clearly say that I had no cert or that that was the issue, because something else was interfering.  I had pf setup on the poudriere build machine, and this halted the client from requesting anything, it literally silently failed, no explanation.  I probably had it in the back of my mind that pf could be the reason, so I eventually tried service pf stop and tried again, this time it was able to update the meta file and others.  Now was when it took a bit to figure out that I needed to copy the cert file to the client.  Once everything was in place, all I needed to remember was to periodically disable pf as needed.  I knew I could permanently cut a hole in pf for the pkg update or install requests but that was something for another time.

Remote LAN work

A new PC makes the old one a file server and sometimes build server.  Previously I had setup poudriere so I could periodically update the ports via pkg on my box with my own desired options.  I also have been keeping a number of unofficial ports updated as well, most notably, luanti (formerly minetest) which I described in an old blog post and revisit periodically it seems.  My plan is to continue to build on the same machine and use it to update ports on that box as well as on my new pc.  I also decided I wanted to figure out how to build my custom unofficial ports for pkgs to install on the new pc.  It turns out that there was a thread of documentation to find which eventually got me to the destination.

Firstly, I put both boxes onto the same LAN which is an obvious step, except that my new PC has an indirectly supported ethernet device on the motherboard.  I sidestepped this issue after temporarily solving it during a test install on an SSD.  The realtek 2.5GB ethernet card requires a driver from ports and an rc.conf adjustment to override the normally included re driver.  There was a little too long of a moment when everything should have worked but did not, an ethernet cable with a broken clip had fallen out of the switch.  With both machines on the same network and functioning, I could setup some things to make my transfer to the new box easier.

Since I have both machines accessible to each other it is a simple matter of setting up ssh so that I can sftp or ssh -X and use X server applications from the old box.  ssh is easy to setup, and with ssh you get the ability to use a secure ftp (sftp) the exact same way as the original deprecated ftp.  At this point the new box was barely configured and had a lot of what I needed for my GUI but my customizations for FVWM were not yet present.  There are other configuration files I transferred over using sftp, such as rc.conf.local and loader.conf.local which I believe I may have named rc.conf and loader.conf on the new box after I moved them.  With things mostly usable and shortly after a number of applications installed on the new box, some days later I finally got motivated to get poudriere setup.

The setup for poudriere to handle building and for pkg to access the files it builds took a little too long to figure out, even though its not particularly complicated.  The later step to handle the default ports tree and my custom local ports tree all in one step took a very close read of documentation to solve.  The basic idea is to use an overlay which is a relatively new addition to ports functionality.  Poudriere supports overlays but I was unable to figure it out with the anecdotal documentation I found.

First, after reading some forum posts and obviously missing some detail I went to the manpages and fell upon a couple details:

FreeBSD	13.2			August 9, 2024		     POUDRIERE-BULK(8)
(excerpt)
       -n	Dry run.  Show what would be done, but do not  actually	 build
		or delete any packages.

       -O overlay
		Specify	an extra poudriere-ports(8) tree to use	as an overlay.
		Multiple -O overlay arguments may be specified to stack	them.

       -p tree	Specify	 on  which  ports  tree	 the  bulk build will be done.
		(Default: "default")

FreeBSD	13.2			April 26, 2021		    POUDRIERE-PORTS(8)
(excerpt)
EXAMPLES
       Creates a new checkout from Git called default from FreeBSD's  official
       ports tree branch main.

	 # poudriere ports -c
       Creates	a  new checkout	from Git called	quarterly from FreeBSD's offi-
       cial ports tree branch 2021Q1.

	 # poudriere ports -c -p quarterly -B 2021Q1
       Updates ports tree named	quarterly.

	 # poudriere ports -u -p quarterly
       Import a	local manually-managed ports tree named	local from /usr/ports.

	 # poudriere ports -c -p local -m null -M /usr/ports

After reading this I now better understood what the command that was in the forum post was about:

With overlays:
Code:

# poudriere bulk accessibility/sct www/firefox tools/tool1 -p default -O privateportstree

The part -p default is essentially automatic as a default itself, so I can ignore it, the other part, -O privateportstree is important, but there needs to be additional setup to use this.  Since my alternative ports tree is incomplete but already existing, I used the command from the examples with adjustments:

  poudriere ports -c -p local -m null -M /usr/local/ports

I did that after I copied all of the contents of my /home/tigersharke/Ported_software directory into the newly created ports directory in the path /usr/local and to help me if I forget, I also created a symbolic link in my user directory to this named Local_ports.

Now I can use a similar command as in the past, to build all ports I wish with my unofficial ones included:

  poudriere bulk -O local -j 14amd64 -f `p-keg-deblack installed-pkgs-gross`

I had to make some adjustments to my blacklist because I was assuming I would not be able to do this combined build and maybe only might use poudriere on specific unofficial ports.  I have been working on getting pkgs updated and installed onto my new ichigo box which get built on the former ichigo now called atlas.  A number of adjustments to my make.conf and blacklist files were necessary, as well as avoiding some non-vital options in some ports which presently cause them to fail building.  While all this is going on, my repeated builds and adjustments, I am editing this blog post using bluefish which I am running via ssh -X from atlas to appear on my monitor which shows me the new ichigo desktop.  A lot of remote lan work including poudriere, ssh edits and an X app appearing from a different machine.  It is amazing what can be done and much of this technology is not brand new, poudriere is the most recent of the bunch actually.

Gentoo does its way

Since I have been trying Gentoo, I have learned that it is definitely a bit more tedious and detailed to accomplish many things as compared to FreeBSD.  There isn't much automation in a default install and at the moment I am not sure what automation could be added.  Gentoo just does things differently, and you may need a crib sheet to know what to use or how to do something, until it becomes second nature.  It can be learned but there may be many bumps or hiccups along the way, correcting misunderstood methods and finally getting success.

For FreeBSD there is a bootable image which guides you through the entire process from preparing the storage medium (hard drives) all the way to installing and configuring the base system, and adding users.  A reboot into a fresh install drops you to a shell login prompt and then you need to setup your GUI.  Some flavors of FreeBSD such as long defunct PC-BSD and present Ghost BSD will finish by installing your desired GUI or a default GUI.  There is also sysutils/desktop-installer in ports which can be very helpful to automate a number of GUI installations.

If you have to setup a FreeBSD desktop, you can use a desktop installer and choose one of those it supports, or you can work from the basic level.  When you install a GUI for FreeBSD, you first need to get X windows with its assorted support mechanisms (including xterm), and then you choose a GUI like FVWM and maybe a login session manager such as SLiM.  After that is accomplished, you can continue to use pkg or the FreeBSD ports system to install further software.  The hierarchy of the ports system allows for easy discovery of software one may wish to add to their system.  There is plenty of associated documentation that is available online or installed, even as part of the base system, so that many questions can be answered easily.  One can fairly quickly and simply get a FreeBSD system up and running as desired with all the software of interest assuming it is already in ports or a pkg exists.

Gentoo does have documentation, it does have a handbook and wiki and forums.  What it does not seem to have, at least from a default basic install is automation or scripting all set-up to be used.  There is no poudriere, there is no ports tree with its hierarchy and predefined make files that easily facilitate the process.  Gentoo does have a build system, it does have packages.  Installing a fresh system is a long process, every step and detail the user must type and carry out.  There is great flexibility in this but some things may take using or trying to know if those are the best choices to make, the best configuration to use.  There are things that act like helpers, which complete tasks in a similar fashion as sysrc on FreeBSD can set variables for rc.conf, but aside from those it is a long journey with small step after small step process.  Gentoo has its niche, it has its advocates and fans, and it may aid me with porting things to FreeBSD, but if compared to Ubuntu, or as FreeBSD -release compared to -current, it is just a bit more difficult to handle and possibly not really for the novice.

This has though been the most time I have spent trying to use and understand any specific Linux distro and I believe the experience has been useful.  I am certainly not going to erase the long fought work of getting a usable Gentoo in Virtualbox, but instead move it to a faster machine to learn more.  My preference is to use FreeBSD, I have been using it sporadically since before 2000 as v2.2.6 and almost constantly since about 2008 beginning with PC-BSD 8.x which I eventually abandoned (due mostly to my tinkering) for FreeBSD 9.x and FreeBSD stable-9.  I eventually decided to work on porting things to FreeBSD in an unofficial capacity, which is where understanding Linux will be useful.  Gentoo in a virtual box might help me with that.

Involved with Gentoo

Since I have been stymied in my attempts to get gaming going on FreeBSD for myself, I decided to attempt another tactic.  After some effort I was able to get VirtualBox installed and setup to function properly, and it took a number of attempts to get the Gentoo iso to startup within it.  I chose Gentoo due to a remembered mention of it being somewhat like FreeBSD, maybe I was remembering something else but I went with Gentoo.  My plan was to see if I could get some stuff to work in the virtual environment which had failed me on FreeBSD.

The Gentoo install is involved.  We may hear complaints from those who try to install FreeBSD or any other BSD as it not being easy or is complicated, or that updating things later is difficult.  I would doubt most of those people have used Gentoo or have gone through all the details necessary to install it.  Luckily for me there is a guy in Kentucky who regularly (annually or so) records a step by step install video for Gentoo. He says that things change from time to time and he likes to keep an updated walkthrough for those who would need it.

I followed his video, pausing and rewatching in order to be sure I got the steps correct as I worked to setup Gentoo in virtualbox.  He tells of some settings that he might not use himself but that others may, and I chose to use some of those.  When it came to setting up the EFI partition and installing the needed files, it did not work.  It took me three iterations through the whole process to this point, restarting from a fresh install each time, to discover what I did wrong.  There was a path or file or directory name that I mistyped each of those previous times.  How I noticed that I made this mistake I am unsure, except that perhaps the emphasis on a third attempt made me look closer.  I made it past this and it felt like I would finally finish.

I was able to reach nearly the last steps when grub is setup for the last reboot.  This is the moment I had difficulty again.  What went wrong I may need to revisit, watching the video again to be sure.  I corrected whatever it was, and succeeded to get grub working, I thought.  Yes it would reboot and grub did function, but the trouble now was that when I went through the second video to install a GUI, I had made some mistakes with SLiM and FVWM3.  How I was trying to execute slim and direct it to start fvwm3 was inaccurate.  I solved this mistake but then discovered that due to the entire Gentoo install being rather bare bones, I was again stopped because although fvwm did load up, I had no way that I knew how to continue.

I had to install xterm, and I tried to solve a script issue that uses python.  Gentoo has a lot of things compile on your system with the emerge commands.  Getting most of what I needed built (and installed) took nearly a day to complete.  I still did it, using FreeBSD as the host, virtualbox provided with as much ram (22G) as I could and as many processor cores (5) as I was able.  The host was still mostly responsive but Gentoo inside virtualbox worked at a snails pace in comparison.

I hoped that if I could install Lutris, and get that to function, I would learn what I am missing with my unofficial port of it, the same for any other port like the ryzom client, or wine functionality, or steam.  Each install took forever which somewhat dampened my expectations, and then when I believed I had done everything for Lutris and the ryzom client to function, there were still unsatisfied dependencies or misconfigured things.  My hopeful panacea for Gentoo in virtualbox to help me solve problems became barely different than attempting directly from FreeBSD.

I still have a lot to learn about how to install a linux thing for Gentoo and how to find and discover dependencies, and where to look for documentation, and which documentation might be most accurate.  I have not yet abandoned it as both a challenge to learn more about and as a possible route to discovering how to make things function properly on FreeBSD.  The amount of non-automated detail that goes into doing things on Gentoo is mostly okay, I do tedious tasks on FreeBSD also, but a little more automation and speed would definitely not hurt, especially speed.  Waiting a day to verify success, to try something is a bit depressing.

I recently bought and assembled a new pc, about a month after I had my virtualbox experience above.  One massively beefier than my present Phenom IIx6, it is a Ryzen 3 and ram jumps from 32 gigabytes up to 128.  When I transfer the virtualbox machine to the new hardware (I have very little intent to go through THAT process again), perhaps the build and install process with emerge will not be so onerous.  Maybe at some future date I can write about how my understanding of Gentoo helped me succeed with something like dbeaver or lutris.