Warning: there may be occasional oddness due to css and blog edits. **KNOWN ISSUE: possible hidden text**

Wednesday, June 23, 2021

To history no more mergemaster

Just what I love to hear, and you should already guess that is sarcasm, that yet another tool is being retired.  Our old faithful mergemaster is being replaced by etcupdate which I will now have to become a bit more familiar with using.  The former tool is mentioned in my Kernel and world update post and page here, so I will revise those as well.  Also, once more, I run across this news accidentally and rather fortuitously, though my next Kernel and world update is on the horizon.  I only suggest it is good to know about this now primarily because the retirement is imminent, arguably not particularly sudden as the PR for it is from January this year.  Even so, that PR is intended to track the change, so a more precise end of usefulness is not perfectly known (to me).  There is also discussion about this change in a post on the FreeBSD forums which manages not to get heated but does echo a little displeasure at the relative silence until now regarding the change.

According to pieces of the forum posts and the PR, it appears that my decision to update my /usr/src in order to check for a more current /usr/src/UPDATING file, has caused me to lose the easier ways to switch to etcupdate.  I think I can certainly double my builds so that after the first one it will be in sync as it must be for the switch, or I could struggle to cause git to backrack the /usr/src directory to the prior version or whatever version (which I do not know) I used for my system as it is presently built.  It may be that there is another method which is less easy but a halfway between those yet only involves etcupdate.

I am now reading that the etcupdate method will usually mean one extra, it seems, manually initiated step each time, since I know that nearly every Kernel and World build and install process for me has differences between what it wants to do or include in various config files compared to what I have already.  I am not sure yet, not until I have been through the process at least once, or better, twice, quite what to expect.  I am going to assume at least one extra step compared to what I have been doing though for these past many years.

Here is the part of the etcupdate(8) manpage which tells me that I doubled my work:

   Bootstrapping
     The etcupdate utility may need to be bootstrapped before it can be used.
     The diff command will fail with an error about a missing reference tree
     if bootstrapping is needed.

     Bootstrapping etcupdate requires a source tree that matches the currently
     installed world.  The easiest way to ensure this is to bootstrap
     etcupdate before updating the source tree to start the next world upgrade
     cycle.  First, generate a reference tree:

           etcupdate extract

     Second, use the diff command to compare the reference tree to your
     current files in /etc.  Undesired differences should be removed using an
     editor, patch(1), or by copying files from the reference tree (located at
     /var/db/etcupdate/current by default)

     If the tree at /usr/src is already newer than the currently installed
     world, a new tree matching the currently installed world can be checked
     out to a temporary location.  The reference tree for etcupdate can then
     be generated via:

           etcupdate extract -s /path/to/tree

     The diff command can be used as above to remove undesired differences.
     Afterwards, the changes in the tree at /usr/src can be merged via a
     regular merge.

The second part suggests extracting a reference tree but I am not going to fall down into that pit.  Instead I will take my medicine, update and install kernel and world to the version of /usr/src I have now, then bootstrap as described above. After all that, I can do another update when /usr/src is revised with the next git pull (yeah, I don't worry about using gitup for this, but one could).

Somewhat distressingly the etcupdate extract does not itself provide any output or clue that it is doing the job, so shortly after noticing this, I ran htop in another xterm to see  The process was fairly quick so it shouldn't be too much concern but I would guess that futher steps will certainly show whether it did its job to setup for bootstrap or not  The second part, mentioned above is not as clear as I think it should be, not only to say "use the diff command" which I will have to check the manpage to see if it is a sub-command of etcupdate, but that the manpage ought to also show "etcupdate diff" if this is the case.

     etcupdate diff [-d workdir] [-D destdir] [-I patterns] [-L logfile]
   Diff Mode
     The diff mode compares the versions of files in the destination directory
     to the “current” tree and generates a unified format diff of the changes.
     This can be used to determine which files have been locally modified and
     how.  Note that etcupdate does not manage files that are not maintained
     in the source tree such as /etc/fstab and /etc/rc.conf.
So it is as I suspected and would have known if I had read through the whole manpage five or six times, a function of etcupdate but the problem is that without the example, there is no mention of where the default locations are for the extract and what to use for workdir and destdir.  Maybe this information is also "buried" in the manpage in a seperate place, and it turns out that suggesting it was buried is a bit too harsh, BUT, an example which uses the defaults would be much more helpful than "use the diff command" since this bootstrapping must be done right the first time.  These options provide a way to set alternate locations so maybe an example using those defaults would be excessive, but should still show the command as otherwise the mention is a little vague.
     -D destdir         Specify an alternate destination directory as the
                        target of a merge.  This is analogous to the DESTDIR
                        variable used with ‘make installworld’.  The default
                        destination directory is an empty string which results
                        in merges updating /etc on the local machine.

     -d workdir         Specify an alternate directory to use as the work
                        directory.  The work directory is used to store the
                        “current” and “previous” trees as well as unresolved
                        conflicts.  The default work directory is
                        /var/db/etcupdate.

I am going to try simply etcupdate diff and see what it does, maybe it will give feedback on any additional information it might need before it begins.  It just runs and goes through most of the usual files that mergemaster would, to see what has changed from 'stock' according to the output below.

root@ichigo:/usr/src # etcupdate extract
root@ichigo:/usr/src # etcupdate diff
Index: /.cshrc
===================================================================
--- /.cshrc (stock)
+++ /.cshrc (local)
@@ -44,3 +44,4 @@
endif

 endif
+source /root/.cshrc-personal
Index: /root/.cshrc
===================================================================
--- /root/.cshrc (stock)
+++ /root/.cshrc (local)
@@ -44,3 +44,4 @@
endif

 endif
+source /root/.cshrc-personal
Index: /root/.login
===================================================================
--- /root/.login (stock)
+++ /root/.login (local)
@@ -7,6 +7,15 @@

 # Query terminal size; useful for serial lines.
 if ( -x /usr/bin/resizewin ) /usr/bin/resizewin -z
+# -z     Do nothing unless the current kernel terminal size is zero.
+# This is useful when run from a user's profile (shell startup) scripts:
+# querying the window size is required for serial lines, but not when
+# logging in over the network, as protocols like TELNET or SSH already
+# handle the terminal size by themselves.
+setenv MANWIDTH 240
+setenv MANCOLOR yes
+#setenv LC_CTYPE POSIX

 # Uncomment to display a random cookie on each login.
 # if ( -x /usr/bin/fortune ) /usr/bin/fortune -s
+ if ( -x /usr/bin/fortune ) /usr/bin/fortune -s
Index: /etc/master.passwd
===================================================================
--- /etc/master.passwd (stock)
+++ /etc/master.passwd (local)
@@ -1,6 +1,6 @@
 # $FreeBSD$
 #
-root::0:0::0:0:Charlie &:/root:/bin/csh
+root::0:0::0:0:Charlie &:/root:/bin/csh
 toor:*:0:0::0:0:Bourne-again Superuser:/root:
 daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
 operator:*:2:5::0:0:System &:/:/usr/sbin/nologin
@@ -27,3 +27,35 @@
 hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin
 tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin
 nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin
+tigersharke::1001:1001::0:0:tigersharke:/usr/home/tigersharke:/bin/tcsh
+eviltwin::1002:1002::0:0:My Evil Twin:/usr/home/eviltwin:/bin/tcsh
+messagebus:*:556:556::0:0:D-BUS Daemon User:/nonexistent:/usr/sbin/nologin
+polkitd:*:565565::0:0:Polkit Daemon User:/var/empty:/usr/sbin/nologin
+saned:*:194:194::0:0:SANE Scanner Daemon:/nonexistent:/bin/sh
+squid:*:100:0::0:0:squid caching-proxy pseudo user:/var/squid:/usr/sbin/nologin
+vboxusers:*0:920::0:0:Virtualbox user:/nonexistent:/usr/sbin/nologin
+_ntp:*:2::0:0:NTP Daemon:/var/empty:/usr/sbin/nologin
+cups:*:193:19::0:0:Cups Owner:/nonexistent:/usr/sbin/nologin
+tests:*:977:6534::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin
+git_daemo64:964::0:0:git daemon:/nonexistent:/usr/sbin/nologin
+polkit:*:5662::0:0:PolicyKit User:/nonexistent:/usr/sbin/nologin
+_tss:*:601::daemon:0:0:TrouSerS user:/var/empty:/usr/sbin/nologin
+avahi:*:558:8::0:0:Avahi Daemon User:/nonexistent:/usr/sbin/nologin
+cyrus:*:60:60:0:0:the cyrus mail server:/nonexistent:/usr/sbin/nologin
+colord:*:970:970::0:0:colord color management daemon:/nonexistent:/usr/sbin/nologin
+clamav:*:1006::0:0:Clamav Antivirus:/nonexistent:/usr/sbin/nologin
+spamd:*:58:58:0:0:SpamAssassin user:/var/spool/spamd:/usr/sbin/nologin
+minetest:*:976:976::0:0:& server:/nonexistent:/usr/sbin/nologin
+_sndio:*:702:702::0:0:sndio privsep:/var/empty:/usr/sbin/nologin
+slurm:*:8::0:0:SLURM Daemon:/nonexistent:/usr/sbin/nologin
+pulse:*:563:563::0:0:PulseAudio System User:/nonexistent:/usr/sbin/nologin
+sddm:*:219:219::0:0:SDDM Display Manager user:/var/lib/sddm:/usr/sbin/nologin
+mysql:*:88:88::0:0:MySQL Daemon:/var/db/mysql:/usr/sbin/nologin
+webcamd:*:145:145::0:0:Webcamd user:/var/empty:/usr/sbin/nologin
+fahclient:*:246:246::0:0:Folding@home User:/nonexistent:/usr/sbin/nologin
+qbittorrent:*:850:850::0:0:qBittorrent Daemon User:/var/db/qbittorrent/conf:/usr/sbin/nologin
+mud:*:97:97::0:0:MUD Owner:/nonexistent:/bin/sh
+bnetd:*:700:700::0:0:Bnetd user:/nonexistent:/usr/sbin/nologin
+wolfpack:*:138:138::0:0:Wolfpack Empire Server:/usr/local/share/wolfpack:/usr/sbin/nologin
+mcserver:*:199:199::0:0:Minecraft Server:/nonexistent:/bin/sh

Removed: /etc/regdomain.xml

Index: /etc/hosts
===================================================================
--- /etc/hosts (stock)
+++ /etc/hosts (local)
@@ -10,8 +10,8 @@
 # not be consulted at all; see /etc/nsswitch.conf for the resolution order.
 #
 #
-::1    localhost localhost.my.domain
-127.0.0.1      localhost localhost.my.domain
+::1    localhost ichigo.underworld
+127.0.0.1      localhost ichigo.underworld
 #
 # Imaginary network.
 #10.0.0.2      myname.my.domain myname
@@ -29,3 +29,8 @@
 # numbers but instead get one from your network provider (if any) or
 # from your regional registry (ARIN, APNIC, LACNIC, RIPE NCC, or AfriNIC.)
 #
+#209.94.190.55 distcache.FreeBSD.org
+#38.109.202.29 www.ixsystems.com
Index: /etc/loin.conf
===================================================================
--- /etc/loconf (stock)
+++ /etc/loconf (local)
@@ -51,6 +5@@
:charset=\
:lang=C.UTF-8:

+
 #
 lle of common class names - forward them all to 'default'
 # (login wou normally do this anyway, but having a class name
@@ -86,6 +87,11 @@
:ignorenol\
:memorylocked=unlimited:\
:tc=defau
+:charset=UT:\
+:lang=en_US.UTF-8:\
+:setenv=LC_LATE=C:
+:setenv=PATHOCALE=/usr/local/lib/X11/locale:\
+:path_locale=/usr/local/lib/X11/locale:\

 #
 # Russians Accounts. Setup proper environment variables.
Index: /etc/nt.conf
=================================================================
--- /etc/ntnf (stock)
+++ /etc/nf (local)
@@ -37,7  @@
 # the nex, and replace CC with the country's abbreviation.
 # Make sure that the hostname resolves to a proper IP address!
 #
-# pool 0.ol.ntp.org iburst
+pool 0.U.ntp.org iburst
+pool 0.freebs.pool.ntp.org iburst
+pool 0.netpool.ntp.org iburst
+pool 0.ope.pool.ntp.org iburst
+pool 0.ope.pool.ntp.org iburst

 #
 # Tonfi a specific server, such as an organization-wide local
Index: /etc/gup
===============================================================
--- /etc/gro(stock)
+++ /etc/group (local)
@@ -1,12 +1,12@@
 # $FreeBS #
-whe:*:
+wheel:*:0:root,tigersharke
 daemon:*:1:  kmem:*:2:
 sys:*:3:
*:4:
-operatoroot
-mail:*:6:
+operator:oot,tigersharke
+mail:*:6:clamav
 bin:*:7:
 news:*:8 man:*:9:
@7,9 +17,12 @@
 smmsp:*:2
 mailnull:*:
 guest:*:31:
-video:*:44:
+audio:*:43:ersharke,minetest
+video:*:44:tigersharke,sddm,root
 bind:*:5
+spamd:*:58
 unbound:*:5
+cyrus:*:60:
 proxy:*:62:
 authpf:*:63
 _pflogd:*:64
@@ -29,9 +32,4 @@
 network:*
 audit:*:77:
 www:*:80:
+mysql:*:8ud:*:97:
+squid:*:100:
+clamav:*:106:
 ntpd:*:123:
+wolfpack:*:138:
+webcamd:*:145:
 _ypldap:*:160:
+cups:*:193:
+saned:*:194:
+mcserver:*:199:
+sddm:*:219:
+_ntp:*:233:
+fahclient:*:246:
+slurm:*:468:
+messagebus:*:556:
+pulse-rt:*:557:
+avahi:*:558:
+polkit:*:562:
+pulse:*:563:
+pulse-access:*:564:
+polkitd:*:565:
+_tss:*:601:
+bnetd:*:700:
+_sndio:*:702:
 hast:*:845:
+qbittorrent:*:850:
+vboxusers:*:920:tigersharke
+git_daemon:*:964:
+colord:*:970:
+minetest:*:976:
 tests:*:977:
+tharald:*:1003:
+plugdev:*:1011:tigersharke,root
+tigersharke:*:1012:tigersharke
 nogroup:*:65533:
 nobody:*:65534:
Index: /etc/devfs.conf
===================================================================
--- /etc/devfs.conf (stock)
+++ /etc/devfs.conf (local)
@@ -41,3 +4 @@
 # Allow ms of group operator to cat things to the speaker
 #own   speaker root:operator
 #perm  speak 0660
+
+# ad A21 2020
+link    acd0   cdrom
+link    acd0   dvd
+perm    acd  0660
+perm    cd0   0660
+perm    pass  0660
+perm    pass  0660
+perm    pass  0660
+perm    pass  0660
+perm    xpt0  0666
root@ichigo:/r/src #

The diff above looks fine to me, although there is one explanatory line in one file I would prefer to keep.  Something with the copy-paste or other manipulation of the xterm to get a complete copy-paste has caused some text damage, but you can get the idea of how the diff result looks anyway.  The manpage paragraph below should provide any sort of example on how one would make an adjustment with patch as suggested, I cannot guess how this wouldbe done, so for bootstrapping I will lose the ntp.conf line "# pool 0.CC.pool.ntp.org iburst" which makes the rest of the comment above it a bit less clear.  Much of the differences between the files which etcupdate scrutinized were not long go cleaned up a bit, to organize and group my changes together and make the default and my revised version match more closely.  Perhaps it has now been enough time that the /usr/src directory can be updated and I can rebuild (world takes 1.5 hours, kernel in 45 minutes I think) and reinstall and try etcupdate after.  I decided to look at /etc/ntp.conf to see what it was like, the line I wanted was gone, so I restored it, then looked at some other settings in the file and made some minor revisions.  I am guessing it should come up in the next etcupdate after the rebuilds and reinstalls, and might expect me to resolve, but I won't know until I do so.

In the place of mergemaster -p I used etcupdate -p which I was reasonably sure about.  After that, I used etcupdate -n and tried etcupdate status but both gave no output, all I could do is bite the bullet and try just etcupdate.  None of it gave any output this time around, so perhaps in a future update to my world and kernel there will be a bit more noise from etcupdate.  So for now, I will stick to the same procedure for that next time, to go ahead with the prepare step and then after installing world, the dry run and finally run of etcupdate.  Until experience proves differently and I must make adjustments this seems reasonable and safe.  This command/tool has been around a while already and except for some tweaks to the manpage, it seems to be a good tool to replace mergemaster.  I look forward to using it in the future, especially the next time or two, to see if it prompts me for anything or provides more output.

How many of you read the title as I humorously intended?  "To history no more" meaning the record of commands, 'history' and also "no more mergemaster" as most obvious it is being retired, there is also the other blatant meaning, just that mergemaster would be relegated to history.  I wasn't sure what title to use until that struck me as perfect.

Frequently viewed this week