Custom kernel 11 stays

I recently looked over my custom kernel configuration file prior to updating kernel and world on Dec 5th 2024.  I compared the kernconf I revised to GENERIC to see whether anything was new or changed.  I noticed a bunch of extra space that I had included which I forget why, except that it was to match an older GENERIC or an prior kernconf.  There were a few things missing which were present in GENERIC and I added those, using the same nomenclature as I had before.

I noticed that I was including a pile of compatibility shims for older versions of FreeBSD.  I decided to revise this to exclude all but the last version, just in case.  I had a comment in the file at the end which mentions an issue with building certain ports.  Since the newer 14-stable was what I was building, I pared them down to just the 13, and I still hope to use 32bit wine, so I made sure to keep the i386 compatibility.  My method (nomenclature) is to copy the GENERIC kernconf as a custom kernconf, keep used options in my custom kernconf but commented, and unused options prefixed with no.  I described creating a customized kernel in another blog post.

#options    COMPAT_FREEBSD32    # Compatible with i386 binaries
nooptions   COMPAT_FREEBSD4     # Compatible with FreeBSD4
nooptions   COMPAT_FREEBSD5     # Compatible with FreeBSD5
nooptions   COMPAT_FREEBSD6     # Compatible with FreeBSD6
nooptions   COMPAT_FREEBSD7     # Compatible with FreeBSD7
nooptions   COMPAT_FREEBSD9     # Compatible with FreeBSD9
nooptions   COMPAT_FREEBSD10    # Compatible with FreeBSD10
nooptions   COMPAT_FREEBSD11    # Compatible with FreeBSD11
nooptions   COMPAT_FREEBSD12    # Compatible with FreeBSD12
#options   COMPAT_FREEBSD13    # Compatible with FreeBSD13

Building world and kernel went fine.  No issues, the etcupdate was silent, and I rebooted into the updated kernel and world without issue.  I started up FVWM and noticed it felt faster already.  I cannot say exactly why it felt faster, I believe it was mostly my imagination.  Once I had my GUI of choice up, next is to return to my usual programs, firefox in particular.  I used my configured gui option to startup firefox but in never appeared after multiple attempts.  At this moment I believe firefox was present but would not start for an unknown reason.  Without keeping any notes on the output I am sure I saw when I tried to start firefox from an xterm, I chose to reinstall firefox.  It wasn't present in my local poudriere repository, so I guess it failed to build the last time, was removed due to some other update and the firefox rebuild failed as well.  When I got set to rebuild firefox outside of poudriere, just in the ports tree, I discovered my mistake.

root@ichigo:/usr/home/tigersharke # cd /usr/ports/www/firefox
root@ichigo:/usr/ports/www/firefox # make reinstall
===>  Deinstalling for firefox
===>   firefox not installed, skipping
===>   firefox-133.0_2,2 depends on package: nspr>=4.32 - found
===>   firefox-133.0_2,2 depends on package: nss>=3.106 - found
===>   firefox-133.0_2,2 depends on package: icu>=73.1 - found
===>   firefox-133.0_2,2 depends on package: libevent>=2.1.8 - found
===>   firefox-133.0_2,2 depends on package: harfbuzz>=10.0.1 - found
===>   firefox-133.0_2,2 depends on package: graphite2>=1.3.14 - found
===>   firefox-133.0_2,2 depends on package: png>=1.6.44 - found
===>   firefox-133.0_2,2 depends on package: dav1d>=1.0.0 - found
===>   firefox-133.0_2,2 depends on package: libvpx>=1.14.1 - found
===>   firefox-133.0_2,2 depends on package: py311-sqlite3>0 - found
===>   firefox-133.0_2,2 depends on package: v4l_compat>0 - found
===>   firefox-133.0_2,2 depends on executable: autoconf2.13 - found
===>   firefox-133.0_2,2 depends on executable: nasm - found
===>   firefox-133.0_2,2 depends on executable: yasm - found
===>   firefox-133.0_2,2 depends on executable: zip - found
===>   firefox-133.0_2,2 depends on file: /usr/local/share/wasi-sysroot/lib/wasm32-wasi/libc++abi.a - found
===>   firefox-133.0_2,2 depends on file: /usr/local/share/wasi-sysroot/lib/wasm32-wasi/libc.a - found
===>   firefox-133.0_2,2 depends on package: wasi-compiler-rt17>0 - found
===>   firefox-133.0_2,2 depends on package: rust-cbindgen>=0.26.0 - found
===>   firefox-133.0_2,2 depends on package: rust>=1.83.0 - not found
===>  Deinstalling for rust
===>   rust not installed, skipping
===>   rust-1.83.0 depends on file: /usr/local/bin/cmake - found
===>   rust-1.83.0 depends on executable: ninja - found
===>   rust-1.83.0 depends on package: pkgconf>=1.3.0_1 - found
===>   rust-1.83.0 depends on file: /usr/local/bin/python3.11 - found
===>   rust-1.83.0 depends on file: /usr/local/lib/libcrypto.so.12 - found
===>   rust-1.83.0 depends on file: /usr/local/bin/ccache - found
===>   rust-1.83.0 depends on shared library: libcurl.so - found (/usr/local/lib/libcurl.so)
===>  Configuring for rust-1.83.0
Bad system call
=> Sanity check failed: kernel is missing COMPAT_FREEBSD11
=> Aborting build
*** Error code 1

Stop.
make[5]: stopped in /usr/ports/lang/rust
*** Error code 1

Stop.
make[4]: stopped in /usr/ports/lang/rust
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/lang/rust
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/www/firefox
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/www/firefox
*** Error code 1

Stop.
make: stopped in /usr/ports/www/firefox
root@ichigo:/usr/ports/www/firefox #

Glorious, right? My kernel is missing something I should not have removed.  The nooptions COMPAT_FREEBSD11 # Compatible with FreeBSD11 line needs to be restored by switching nooptions to #options which would then not override the GENERIC kernel configuration file.  Since I have not updated /usr/src after building (the faulty) kernel and world, I can simply fix the kernconf file and remake the kernel and reboot.  Another thing I have avoided due to discovering the firefox issue is that normally right after I rebuild my kernel I also rebuild my poudriere to match.  I did not get to that step so I don't double my repair work and waste three times as much time.  Along with fixing my error, I decided that since I have made the transition from FreeBSD 13-stable to 14-stable, I actually no longer need the compatibility to FreeBSD 13.  It is a good idea to keep compatibility with the prior major release initially with your first rebuild after the transition, it may take some time to get all ports rebuilt.

It is somewhat amazing that FreeBSD 14.x still needs to be compatible with FreeBSD 11.  What is preventing the applicable changes to eliminate this issue?  Maybe there needs to be work done on cargo/rust so that we can build them without needing to rely upon a bootstrap from the days of FreeBSD 11.  Luckily this is a hidden pitfall which most vanilla users would never encounter, because the GENERIC kernel providers and the port maintainers both keep firefox and rust easy to install without needing to think about any issues at all.  Even though many users would not be hit by this issue with compatibility with an old FreeBSD version, it remains a less than ideal solution.  How far into the future will we need to keep the compatibility for this solitary purpose?  This is now an additional firefox caveat and a rust caveat that I have added to the distilled notes.

Port breaks kernel breaks port

So many of us chug happily along without completely realizing or recognizing how some of the present FreeBSD build mechanisms have become a bit more complex.  Those who never have any need of graphics and remain in a text mode commandline interface for the duration of their use of FreeBSD would not know that there is indeed at least one situation, now, which ties a port and the kernel together.  When everything is working perfectly, this would likely never come up, but a relatively small problem inflated itself to cause my kernel build to fail.

During this troubleshooting quest, first I tried the obvious things, re-rebuild world just to be sure it was ok, then rebuild a GENERIC kernel instead of my custom kernconf and that after having re-enabled some possibly related things in that kernconf to no avail.  After beating my figurative head against the wall for quite a while, I went to twitter to see if @FreeBSDHelp had any ideas.

The details and comments on that discussion thread didn't solve my issue as I could not comprehend how our kernel build was now in any way tied to the build of a port, though the comment that was made may not have explicitly indicated this.  So my next thought was, if I could get a different, earlier version of the /usr/src from git in some way, then rebuild kernel from before the error seemed to appear in /usr/src.  I didn't know this was not the path to take to solve this, but I still wasted far too much time trying to go backward with git to an earlier commit.  I am definitely not a particularly big fan of git, and this exertion didn't help me love it any more.

The reason, besides that it had been in excess of 10 days since the last time I rebuilt my kernel and world, was to get virtualbox working which needed bits from the kernel build which I didn't have, and those need to be the same version as the running OS.  This meant my long journey to rebuild my kernel and world (multiple times each) so that I could use Virtualbox to try the game Veloren which due to whatever is different than expected (FVWM3 and Radeon graphics probably, or similar) does build and install but does NOT run.  I am sure that if I could startup Virtualbox, put tinycore Linux in there, and install Veloren for Linux, I would probably succeed where I was prevented otherwise.  I could not install another OS in Virtualbox because I could not boot the iso, and this due to the virtualbox kernel object not having been loaded.  I couldn't load the needed kernel object since it needed to be built, and now you know why I got stuck down this rabbit hole.

It has been some time since setting that whole "Play Veloren in Virtualbox via tinycore Linux" idea on a back burner or in a box on a shelf somewhere.  One of my incomplete projects is to get my port attempt of Reshade rebuilt, which I was attempting and it ran into some conflicts with python items.  The various python things it needed were installed as version 310 while I already had version 39 of those same ports.  The only way forward was to remove each of the python 3.9 ports to let the reshade build then install what it needed as version 3.10.  Among all of the things that were removed as a consequence of this, was vlc and firefox, both I use daily.  So I gave poudriere a gross list of everything I had installed on my system, let it build, and then discovered a number of things that failed.  The graphics/drm-fbsd13-kmod port was among the fairly long list of things that didn't get built, and it was in the smallish group of "lynchpin" ports, meaning that others failed (were skipped) because it failed.

And so, I thought that's not a problem, I'll go investigate what happened with graphics/drm-fbsd13-kmod to make it fail.  I remembered that poudriere keeps logs of all (or most everything) it does, and I just had to find it.  Since I often end up trying to remember where any certain important thing is and its path, I have been keeping a directory of symbolic links with sometimes more descriptive names.  The appropriate one was,

root@ichigo:~ # ls -l Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg
lrwxr-xr-x  1 root  wheel  66 May  5  2021 Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg -> /usr/local/poudriere/data/logs/bulk/13amd64-default/latest-per-pkg

and from there I could do

root@ichigo:~ # tail -n 15 Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg/drm-fbsd13-kmod-5.4.144.g20220223.log
===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: %%KMODSRC%%/linuxkpi/dummy/include/linux/random.h
Error: Orphaned: %%KMODSRC%%/linuxkpi/dummy/include/linux/suspend.h
===> Checking for items in pkg-plist which are not in STAGEDIR
===> Error: Plist issues found.
*** Error code 1

Stop.
make: stopped in /usr/ports/graphics/drm-fbsd13-kmod
=>> Error: check-plist failures detected
=>> Cleaning up wrkdir
===>  Cleaning for drm-fbsd13-kmod-5.4.144.g20220223
build of graphics/drm-fbsd13-kmod | drm-fbsd13-kmod-5.4.144.g20220223 ended at Sat Feb 26 00:30:21 CST 2022
build time: 00:04:43
!!! build failure encountered !!!

Firstly, the build failure is due to my choice to be a bit more stringent on builds, to test for various things, so it is possible that this might not appear to most users, although it truly should be visible to all the port maintainers and various FreeBSD developers.  It tells me, as did the small highlighted concise reason in the failed build list output from after I ran poudriere, that it is an issue with the pkg-plist.  This I correctly believed was a simple issue, and easy to fix since this process is something I have repeated many times with my own repos for FreeBSD Port Tree Leaf items such as for Minetest-dev which I wrote about in another blog post.

What I needed to do was go to /usr/ports/graphics/drm-fbsd13-kmod and rename the pkg-plist to pkg-plist-old, and then do a fresh build of it.  Once the build completes, I create a fresh pkg-plist by
make makeplist > pkg-plist
in order to do a comparison between this new fresh list and the old original list.  This is accomplished by
diff -y pkg-plist pkg-plist-old | more
to step through the output, looking for something that is present or absent in the newly generated pkg-plist as compared to the old one.  Since a pkg-plist that is in the ports tree may have %%text%% type tags which are often not generated by the make makeplist script, I usually modify the pkg-plist-old to match the one freshly generated.  Once the edits are made, I rename the pkg-plist-old to pkg-plist and then rebuild once more to prove no errors related to the file remain.

Now that graphics/drm-fbsd13-kmod successfully builds and installs, I thought from the back of my mind, that I would try to update kernel and world, due to that vague mention of these two things being related.  World builds as expected, so I go on to the kernel, and then it fails.  It complains that kconfig.mk was missing.  I remember that that was one of the things that I had removed from the graphics/drm-fbsd13-kmod pkg-plist for a reason I am already uncertain about now-- and this is being written within hours of having done it.  I go back to that port tree directory and either the pkg-plist-old was still present or I went through the steps to generate fresh and make the needed edits to fix it.  Whatever actually happened seems to have fallen out of my mind but the result was "Hey! graphics/drm-fbsd13-kmod needs to be built in order for the kernel to build, gee that is weird."

I have been writing about all of this within a relatively short period after succeeding to build kernel and world when it had been broken some week(s) ago.  My new kernel has not yet been installed and I have to build the virtualbox thing(s) that are dependent upon the source.  It is nice now to have this mess cleared up and better understood.  I'll be adding this nit to the lists of build issues for kernel or world, and add emphasis on the relationship between this port and the kernel which likely many of us had not known.  The kernel failure meant the virtualbox port for a kernel object couldn't be built, but the kmod graphics port is what broke the kernel.

FreeBSD source via git

Some time ago I went through the process of switching to git from svn for updating /usr/src.  I believe the newly git-created directories sat unused after completing that step those 20+ days ago, until I finally set aside the time to work on rebuilding my kernel and world again.  I could have rebuilt and reinstalled my kernel and world right then but as it had been a while, I figured I should update my source again.  This is where I needed to research a bit more, the conversion method is readily available but mention of how to update was not and the commands are different than svn so I couldn't just use update as before.

First the conversion process which I did, the first step simply as a precaution.

1. Copy your last /usr/src updated with svn to a backup directory:
   cp -pRP /usr/src /usr/src-last_svn_update
   
2. Remove the directory and all contents then create a fresh directory.  As /usr/src contains hidden files or directories, this may be more appropriate and an easier sure step.  After, we still need the directory to use with git, so re-create it.
   rm -rf /usr/src
   mkdir /usr/src
   Or alternatively just delete the /usr/src directory contents:
   rm -rf /usr/src/*
   
3. Now that the way is prepared we can use git to get the source tree:
   git clone -b master --single-branch --depth 1  git://github.com/freebsd/freebsd.git /usr/src
    Github changed security in September 2021 
   git clone -b master --single-branch --depth 1  https://github.com/freebsd/freebsd.git /usr/src
   
4. Since I use the stable branch of version 12, below is how I obtain a specific branch, the manpage also indicates that --depth implies --single-branch, so we can omit that:
   git clone -b stable/12 --depth 1 git://github.com/freebsd/freebsd.git /usr/src
   git clone -b stable/12 --single-branch --depth 1  https://github.com/freebsd/freebsd.git /usr/src
   

The output is much less verbose than svn.  We can follow the initial clone command with a few more which provide added detail and verification.

Cloning into '/usr/src'...
remote: Enumerating objects: 84805, done.
remote: Counting objects: 100% (84805/84805), done.
remote: Compressing objects: 100% (71708/71708), done.
remote: Total 84805 (delta 17987), reused 37808 (delta 10001), pack-reused 0
Receiving objects: 100% (84805/84805), 276.31 MiB | 10.02 MiB/s, done.
Resolving deltas: 100% (17987/17987), done.
Updating files: 100% (81378/81378), done.

Looking further into the insanely extensive git manpages, I find git reflog which appears to display in reverse order, the most recent action at the top.
git reflog

94a5e942b (grafted, HEAD -> stable/12, origin/stable/12) HEAD@{0}: clone: from git://github.com/freebsd/freebsd.git

We can discover the config settings by typing the command below from /usr/src.  Many of those settings are defaults and I must have setup my email address previously for git.  I have used git sporadically for various things including my own repos of random projects so this is not a big surprise.  The manpage for git-config lists a LOT more options.
git config --list

user.email=username@mailsvc.com
user.name=username
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=git://github.com/freebsd/freebsd.git
remote.origin.fetch=+refs/heads/stable/12:refs/remotes/origin/stable/12
branch.stable/12.remote=origin
branch.stable/12.merge=refs/heads/stable/12

The last thing I need to do is get the most recent updates, to refresh my /usr/src, which if you read the initial paragraph is really all I needed to do when I began this post, and all I will need to do from now on.  The first time updating /usr/src it will be a two step process, the first line is only needed once.
git config pull.rebase true
git pull

If you had not used that first line for your first update, git will give you a reminder as below.

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint:
hint: git config pull.rebase false # merge (the default strategy)
hint: git config pull.rebase true # rebase
hint: git config pull.ff only # fast-forward only
hint:
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.

Lets set the config and show the resulting config list change.
git config pull.rebase true
git config --list

user.email=username@mailsvc.com
user.name=username
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=git://github.com/freebsd/freebsd.git
remote.origin.fetch=+refs/heads/stable/12:refs/remotes/origin/stable/12
branch.stable/12.remote=origin
branch.stable/12.merge=refs/heads/stable/12
pull.rebase=true

Now that the git method for obtaining FreeBSD source is reasonably described, the post about updating kernel and world can be updated to match as well as the specifically related distilled page.  Everything is setup, any further need to obtain source can be accomplished with one simple command:
git pull 

Kernel and world rebuild

I prefer to use the -STABLE branch which means there are many more updates compared to RELEASE.  I try to do an update once a week or at least a couple times a month.  This is not a particularly difficult task but I still refer to a printout of a tutorial page from an old BSD Now podcast.  I have a number of customizations which I have used in the past along with a custom kernel, but I have not used them lately, or at least not the customized KERNCONF.  This entire process will need to be done as root due to standard default permissions.

** Please note that FreeBSD switched to git and so svn is not the method since Dec 18th 2020 or so. **

If you have not done so already, you will need to obtain the source for the build you intend to use.
git clone -b master --single-branch --depth 1 git://github.com/freebsd/freebsd.git /usr/src

Github changed security in September 2021
git clone -b master --single-branch --depth 1  https://github.com/freebsd/freebsd.git /usr/src
and then to prepare now for future updates do:
git config pull.rebase true

Since I use the stable branch of version 12, below is how I obtain a specific branch:
git clone -b stable/12 --single-branch --depth 1 git://github.com/freebsd/freebsd.git /usr/src
git clone -b stable/12 --single-branch --depth 1 https://github.com/freebsd/freebsd.git /usr/src

Assuming you hadn't just completed the above, or sometime next week you choose to rebuild, one of the first steps is to update /usr/src.  This can be as easy as changing to the directory /usr/src then type git pull.  Either before or directly after this step, it is very important not to have any leftover remnants from the previous builds, use the command make cleanworld and then to be sure, the command make clean, prior to any building.

Now that the preparations have been made, we can begin.  You can take advantage of multiple cores in your system by setting one job per core or cpu.  If you are uncertain you can check by sysctl -n hw.ncpu and place the resulting number after -j for the number of jobs.  Wow, I just realized that I hadn't used all my six cores the last time I rebuilt my kernel and world.  I would have had I used the command below, because it will always provide the correct number, supposing I have 8 cores sometime in the future I wouldn't be using only six for the build.

make -j `sysctl -n hw.ncpu` buildworld

Now that that is complete, I can do the next step. This one I discovered can be combined with the step after it, as below.  This explicitly defines the kernel configuration file, which must reside in /usr/src/sys/amd64/conf as a real fle, or a symlink to it from your home directory which is a good way to keep it against accidental erasure. By using kernel rather than buildkernel the scripts involved will build and install the kernel in the default location, so this is particularly helpful if the process is unattended such as a slow task handled while the admin is sleeping or handling other matters.
make -j `sysctl -n hw.ncpu` kernel KERNCONF=GENERIC

Although I have had a custom kernel in the past, and I still have the modified kernconf file, I am using GENERIC because shockingly, I had some difficulties. My troubles with booting and with kernel builds have been primarily related to graphics. I think I have a decent handle on them now, but rather than use the monolithic configuration I made, I want to take a closer look at how it compares to GENERIC and also discover which items can be built as kernel objects. Once I know that, I would limit kernel object builds to those I need or might use, and reduce rather than increase the kernel by removing things I never use.

The next step is to go to single user.  This I do with a reboot which also verifies that the kernel I just installed functions as intended to that point.
reboot

Watch for the FreeBSD startup menu, press space as needed to pause it -- it may be set to be a very short countdown here, then choose 'singleuser' by hitting '2'.  It is NOT recommended to use another shell for handling the buildworld or buildkernel processes, so when it finishes and shows the prompt below, tap the return or enter key.

Enter full pathname of shell or RETURN for /bin/sh

As I mentioned in another blog post, now I can't boot the way to access /usr/src again requires some mounting actions, below mounts all items in /etc/fstab as read-write.
mount -u /
and then if your system is ZFS rather than UFS, this mounts all datasets:
zfs mount -a

Once in /usr/src there are a few more things to do and then we can reboot into the new system.  First, what the manpage calls a pre-buildworld mode mergemaster which compares the installed /etc/group and /etc/master.passwd with the new defaults so that you can choose what to adjust as needed, and then the install command.

Mergemaster removed Nov 20, 2023 replaced by etcupdate.

Function of etcupdate is essentially the same, pre-buildworld option remains -p for this step.
mergemaster -p
etcupdate -p
make installworld

Should you see a message like below, which can appear,

At installworld Make: "/usr/src/Makefile" line 360: Check your date/time

it is a simple fix, type the following command to continue:
adjkerntz -i

Now that the error is corrected, finish the command that was interrupted:
make installworld

The next steps will help update configuration files, allow you to merge your customized lines with the new file as needed, while automatically handling those files which are essentially identical or have not been user-modified.
mergemaster -iUF
etcupdate -n
etcupdate

man 8 etcupdate

     -n             Enable “dry-run” mode.  Do not merge any changes to the
                    destination directory.  Instead, report what actions would
                    be taken during a merge.  Note that the existing “current”
                    and “previous” trees will not be changed.  If the -r
                    option is not specified, then a temporary “current” tree
                    will be extracted to perform the comparison.

     -N             Perform a NO_ROOT build when building a “current” tree.
                    The resulting tree will include a corresponding METALOG
                    file at its root.

     -p             Enable “pre-world” mode.  Only merge changes to files that
                    are necessary to successfully run ‘make installworld’ or
                    ‘make installkernel’.  When this flag is enabled, the
                    existing “current” and “previous” trees are left alone.
                    Instead, a temporary tree is populated with the necessary
                    files.  This temporary tree is compared against the
                    “current” tree.  This allows a normal update to be run
                    after ‘make installworld’ has completed.  Any conflicts
                    generated during a “pre-world” update should be resolved
                    by a “pre-world” resolve.

It is good practice to be certain old libraries and other assorted files are not left around to cause problems, especially true between updates of major versions, so the next two commands will help remove all of them automatically. The command below, yes repeats 'y' to fulfill prompts, thus agreeing to each deletion which automates the script.  Yes as described by its manpage can be used to repeat any text.
yes|make delete-old
yes|make delete-old-libs

Now that everything is finished, you can reboot one last time to load up and use your newly updated system.  If for any reason there is a failure with booting, refer to now I can't boot for some steps to help solve it.