Warning: there may be occasional oddness due to css and blog edits. **KNOWN ISSUE: possible hidden text**

Tuesday, September 5, 2023

14-stable leads to a winding path

When I set out to update my system to use FreeBSD 14-stable in lieu of 13-stable, I knew there was a chance things could go awry.  Usually a switch between major versions will involve substantial changes to how things are done and thus indirectly invalidate a large swath of (or everything) that is installed.  I was ready for this task but first I had to get a running system to begin my other (userland) updates.

As usual, essentially the same way I've done many times, I updated my /usr/src to the most current 14-stable, and began the usual buildworld and kernel processes as described in another blog post,Kernel and world rebuild.  Since this was going to be a new ball of wax instead of some minor tweaks of 13-stable, I wisely chose to install a GENERIC kernel.

What I was not so aware of, was how issues with etcupdate could thwart a complete update.  After a couple attempts to rebuild and reinstall, I looked closer at etcupdate and what might possibly be the wrench in the works.  This discussion of my kernel and world challenge is being written from memory as I didn't remember to take any notes to be more precise.  What I recall is that I eventually clarified six or eight files so that they would match what was expected in the new system, etcupdate could then be satisfied (no more conflicts) and my usual process could be completed.

Aside from the etcupdate hiccup, the transformation from 13-stable to the new shiny 14-stable went rather smoothly and easily.  Since I now had a fresh install, I could work on getting my userland (ports) updated and back to a functioning desktop.  I now have a list of ports I need to re-install when I update my kernel and this along with a helpful script to iterate through each of them, would update anything needing the kernel and all the rest.

misc/pciids
graphics/libdrm
graphics/linux-c7-libdrm
graphics/libglvnd
graphics/libGLU
graphics/gpu-firmware-amd-kmod
x11/xorg
graphics/mesa-libs
graphics/mesa-dri
graphics/drm-kmod
devel/libudev-devd
x11-drivers/xf86-input-mouse
x11-drivers/xf86-input-keyboard

I had those ports rebuilt and reinstalled, and made sure that x11-wm/fvwm was installed as well.  When I tried to startx it told me there were no screens detected, which is an obscure way to say that there is no graphics driver, or that is just one cause of the message.  I tried to reinstall the few items I knew would affect this, and made sure to install graphics/gpu-firmware-amd-kmod because that is the hardware I have.  I was still unable to issue startx and get my fvwm desktop back up.  I figured that since I had trouble in the past and desktop-installer cured things before, I could use it now and get back a gui.

I recall there being a moment (but I am very uncertain where in the order of events) when I was unable to connect to the freebsd pkg repo, and I hadn't changed anything myself (directly, intentionally) so I looked online for some documentation about it.  At that time I may have noticed that ports were no longer updating.  Strangely, what had always worked was no longer functioning, I had to change my freebsd.conf to get that back.

This is my /usr/local/etc/pkg/repos/FreeBSD.conf with the former text all commented out with # characters, the working portion below.

# $FreeBSD$
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#
#
#FreeBSD: {
#  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
#  mirror_type: "srv",
#  signature_type: "fingerprints",
#  fingerprints: "/usr/share/keys/pkg",
#  enabled: yes
#  priority: 0
#}

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
}

Since desktop-installer succeeded to set me up with SDDM and fluxbox, I had a GUI to work on the rest of my updates.  I needed to figure out or disable SDDM and get fvwm going.  At some point I noticed that graphics/drm-kmod had a way to auto choose the right port to install based upon the version of FreeBSD.  The Makefile did not permit the newer version for 14-stable to be drm-515-kmod as it at that moment could only install drm-505-kmod.

I tried to get things updated using the file list mentioned earlier but kept running into problems until I happened to use graphics/gpu-firmware-kmod to install things instead of the more direct amdgpu in the list.  Once I did this, both drm-505-kmod or drm-515-kmod worked fine.  When I reinstalled that list, I kept thinking that maybe one thing in the list affected others, and maybe they do but repeated installs did not solve it.

So finally again I had a gui and everything seemed to be going the right direction.  All it would take to bring things back to a broken state yet again is to update those ports.  I liked the idea of sysutils/auto-admin and it mostly worked fine, aside from how it would insist to reinstall (from pkg) versions with one-size-fits-all default port configs.  The more ridicules was ftp/curl which by removal caused auto-admin scripts themselves to fail.

Eventually I realized that if auto-admin wanted to deinstall ftp/curl and use bad defaults for it (I must have static for some other port) I could build it the way I need then lock it.  This became the solution for a number of issues.  Pango also needed the libthai option because firefox insisted upon it, so that was built and locked.  Some time later, vlc was also affected by changed options, MAD being the only mp3 decoder available, so this too rebuilt and locked.  I know ffmpeg and firefox were both eventually locked as well.

It seems that within the week or so that I rebuilt and reinstalled FreeBSD from 13-stable to now 14-stable, more things mysteriously broke and had to be fixed than usual.  I do not know what to point to as the precise cause.  Most if not all of these difficulties are within the realm of FreeBSD ports and not kernel or world related at all.

Somehow after updating fvwm3 to its most recent commit, what will be the next release version, I exited out to start the new version and couldn't get back.  I was getting a very odd message related to some sort of X windows thing which at this moment I cannot recall.  It was something like how it wasn't sure which amdgpu driver or definition or something to use.  This was odd as when I looked at what I thought was pertinent, there was only one related to this.  I think it was related to drivers again, or I assumed it was, so back to that list and reinstalling.  I eventually added new things to the list after finally getting fvwm up again, mostly.

The trouble was, even though I had fvwm up, my mouse cursor was stuck at the center of the screen and things felt stalled, I think they *were* stalled.  I tried reinstalling the mouse driver and then went back to desktop-installer for the essentials and hopefully a fix.  Once more, desktop-installer cured the issue.  I looked at what drivers were now installed and added more to the list.

x11/libXxf86dga
x11/libXxf86vm
x11-drivers/xf86-input-libinput
x11/xf86dga
x11/xmodmap

I believe now after desktop-installer saved me yet one more time, everything should be working and I would have no troubles.  Except that I was wrong of course.  Now I am unsure how but something with my keyboard configuration now lacked anything for arrow keys.  After a lot of digging around I finally found how to fix them.  What I didn't discover until in the process of writing this is that my delete key does not function (fixed in below list).  I had to edit /usr/local/etc/Xmodmap to add some lines.  Later with xmodmap -pke > ~/.Xmodmap I told xmodmap to copy the file to my home directory.  Vermaden's blog post on FreeBSD desktop (part 9) was also useful to help me remember xev and its function, and some other things.

keycode 111 = 0xFF52
keycode 116 = 0xFF99
keycode 114 = 0xFF98
keycode 113 = 0xFF96
keycode 119 = BackSpace

I keep adding to my 'kernel rebuilt redo GUI' file list, at present this is the correct listing, missteps left out but vulkan stuff added just in case as it is an AMD card.

misc/pciids
graphics/libdrm
graphics/linux-c7-libdrm
graphics/libglvnd
graphics/libGLU
graphics/gstreamer1-plugins-vulkan
graphics/realesrgan-ncnn-vulkan
graphics/realsr-ncnn-vulkan
graphics/vapoursynth-waifu2x-ncnn-vulkan
graphics/vulkan-caps-viewer
graphics/vulkan-extension-layer
graphics/vulkan-headers
graphics/vulkan-loader
graphics/vulkan-tools
graphics/vulkan-validation-layers
graphics/gpu-firmware-kmod
x11/xorg
graphics/mesa-libs
graphics/mesa-dri
graphics/drm-515-kmod
devel/libudev-devd
x11-drivers/xf86-input-mouse
x11-drivers/xf86-input-keyboard
x11/libXxf86dga
x11/libXxf86vm
x11-drivers/xf86-input-libinput
x11-drivers/xf86-video-amdgpu
x11/xf86dga
x11/xmodmap

They always say what does not kill you makes you stronger, well, in the software/computer world it might be more like perseverence along with a little knowledge will always lead to a victory against those inanimate objects.  There is definitely an aspect of the former saying, in that, the greater the struggle, the more extreme the frustration, surpassing both will lead to greater knowlege and understanding, but in a nutshell: just don't give up.  Look for answers, look for help, embrace the puzzle.

Writing about our difficulties, successes, and failures, will help others who find the content.  You too could start a FreeBSD blog.

Sunday, September 3, 2023

Revised minetest-dev: fonts!

Anyone watching my git repo for minetest-dev or any of the others will know that I have expanded a bit on what I attempt to maintain in a reasonably timely manner.  Not too long ago, during a period when my FreeBSD install was functioning without hiccups, I eventually figured out another improvement to the minetest-dev Makefile.  I also have done a little cleanup for the options exposed to those who wish to install it.

Something I have wanted to do but hadn't the motivation or free time simultaneously is to have a way for the fonts installed on the system to be used by minetest, instead of duplicating them.  The solution I worked out is definitely not the best and actually begs upstream developers to include a proper mechanism to do this properly.  I understand that the minetest client should be guaranteed to have fonts it expects but without a mechanism within its build structure or configuration, we are stuck with a duplication of files for no real purpose.

minetest-dev:

PORTNAME=       minetest
DISTVERSION=    g20230902
CATEGORIES=     games
PKGNAMESUFFIX=  -dev
DISTNAME=       ${PORTNAME}-${GH_TAGNAME}
DIST_SUBDIR=    ${PORTNAME}${PKGNAMESUFFIX}

MAINTAINER=     nope@nothere
COMMENT=        Near-infinite-world block sandbox game
WWW=            https://www.minetest.net/

LICENSE=        LGPL21+

LIB_DEPENDS=    libIrrlichtMt.so:x11-toolkits/irrlicht-minetest libzstd.so:archivers/zstd

USES=           cmake compiler:c++14-lang iconv:wchar_t sqlite luajit
# Upstream requires Clang 3.5+ which for our criteria matches c++14-lang
# since https://en.cppreference.com/w/cpp/compiler_support
# lists "C++14 library support status (complete as of 3.5)"
# All other dependency version numbers are more direct and obvious, and surpass requirement.

CONFLICTS=      minetest

USE_GITHUB=     nodefault
GH_ACCOUNT=     minetest
GH_PROJECT=     minetest
GH_TAGNAME=     83b85ba16aaab7bddc479beada4ca3af9fa54a1b

CMAKE_ARGS=     -DCMAKE_BUILD_TYPE="MinSizeRel" \
                -DCUSTOM_EXAMPLE_CONF_DIR="${PREFIX}/etc" \
                -DCUSTOM_MANDIR="${PREFIX}/man"

WRKSRC=         ${WRKDIR}/${PORTNAME}-${GH_TAGNAME}

OPTIONS_DEFINE=                 CURL DOCS FREETYPE LUAJIT NCURSES NLS SOUND SPATIAL TOUCH PROMETHEUS
OPTIONS_DEFAULT=                CURL DOCS FREETYPE LUAJIT SOUND SPATIAL SYSTEM_FONTS SYSTEM_GMP SYSTEM_JSONCPP CLIENT GLVND

OPTIONS_SINGLE=                 GRAPHICS
OPTIONS_MULTI=                  SYSTEM COMP
OPTIONS_GROUP=                  BUILD DATABASE

COMP_DESC=                      Software components
OPTIONS_MULTI_COMP=             CLIENT SERVER

SYSTEM_DESC=                    System subsitutes
OPTIONS_MULTI_SYSTEM=           SYSTEM_GMP SYSTEM_JSONCPP SYSTEM_FONTS

SYSTEM_GMP_DESC=                Use gmp from ports (ENABLE_SYSTEM_GMP)
SYSTEM_GMP_CMAKE_BOOL=          ENABLE_SYSTEM_GMP
SYSTEM_GMP_CMAKE_ON=            -DGMP_INCLUDE_DIR="${PREFIX}/include"
SYSTEM_GMP_LIB_DEPENDS=         libgmp.so:math/gmp

SYSTEM_JSONCPP_DESC=            Use jsoncpp from ports (ENABLE_SYSTEM_JSONCPP)
SYSTEM_JSONCPP_CMAKE_BOOL=      ENABLE_SYSTEM_JSONCPP
SYSTEM_JSONCPP_CMAKE_ON=        -DJSON_INCLUDE_DIR="${PREFIX}/include/jsoncpp"
SYSTEM_JSONCPP_LIB_DEPENDS=     libjsoncpp.so:devel/jsoncpp

SYSTEM_FONTS_DESC=              Use or install default fonts from ports
SYSTEM_FONTS_RUN_DEPENDS=       ${LOCALBASE}/share/fonts/ChromeOS/Arimo-Bold.ttf:x11-fonts/croscorefonts-fonts-ttf \
                                ${LOCALBASE}/share/fonts/Droid/DroidSans.ttf:x11-fonts/droid-fonts-ttf

BUILD_DESC=                     Dev Build options
OPTIONS_GROUP_BUILD=            BENCHMARKS EXAMPLES UNITTESTS DEVTEST

BENCHMARKS_DESC=                Build benchmark sources (BUILD_BENCHMARKS)
BENCHMARKS_CMAKE_BOOL=          BUILD_BENCHMARKS
EXAMPLES_DESC=                  BUILD_EXAMPLES
EXAMPLES_CMAKE_BOOL=            BUILD_EXAMPLES
DEVTEST_DESC=                   Install Development Test game also (INSTALL_DEVTEST)
DEVTEST_CMAKE_BOOL=             INSTALL_DEVTEST
UNITTESTS_DESC=                 Build unittest sources (BUILD_UNITTESTS)
UNITTESTS_CMAKE_BOOL=           BUILD_UNITTESTS

OPTIONS_SINGLE_GRAPHICS=        GLVND LEGACY
GRAPHICS_DESC=                  Graphics support

GLVND_DESC=                     Use libOpenGL or libGLX
GLVND_CMAKE_BOOL=               ENABLE_GLVND
GLVND_CMAKE_ON=                 -DOPENGL_GL_PREFERENCE="GLVND" -DOPENGL_xmesa_INCLUDE_DIR="${PREFIX}/lib"
GLVND_USE=                      GL+=opengl

LEGACY_DESC=                    Use libGL - where GLVND may be broken on nvidia
LEGACY_CMAKE_BOOL=              ENABLE_LEGACY
LEGACY_CMAKE_ON=                -DOPENGL_GL_PREFERENCE="LEGACY" -DOPENGL_xmesa_INCLUDE_DIR="${PREFIX}/lib"
LEGACY_USE=                     GL+=opengl

DATABASE_DESC=                  Database support
OPTIONS_GROUP_DATABASE=         LEVELDB PGSQL REDIS

OPTIONS_SUB=                    yes

CLIENT_DESC=                    Build client
CLIENT_CMAKE_BOOL=              BUILD_CLIENT
CLIENT_LIB_DEPENDS=             libIrrlichtMt.so:x11-toolkits/irrlicht-minetest \
                                libpng.so:graphics/png

CLIENT_USES=                    gl xorg
CLIENT_USE=                     jpeg GL=gl,glu \
                                XORG=ice,sm,x11,xext,xcb,xres,xshmfence,xau,xaw,xcomposite,xcursor,xdamage,xdmcp,\
                                xfixes,xft,xi,xinerama,xkbfile,xmu,xpm,xrandr,xrender,xscreensaver,xt,xtst,xv,xxf86vm

SERVER_DESC=                    Build server
SERVER_CMAKE_BOOL=              BUILD_SERVER

CURL_DESC=                      Enable cURL support for fetching media
CURL_CMAKE_BOOL=                ENABLE_CURL
CURL_LIB_DEPENDS=               libcurl.so:ftp/curl

SOUND_DESC=                     Enable sound via openal-soft
SOUND_CMAKE_BOOL=               ENABLE_SOUND

FREETYPE_DESC=                  Support for TrueType fonts with unicode
FREETYPE_CMAKE_BOOL=            ENABLE_FREETYPE
FREETYPE_LIB_DEPENDS=           libfreetype.so:print/freetype2

NCURSES_DESC=                   Enables server side terminal (cli option: --terminal)
NCURSES_CMAKE_BOOL=             ENABLE_CURSES
NCURSES_USES=                   ncurses

# This option is becoming uncertain, though it does something, is it useful?
LUAJIT_DESC=                    Require LUAJIT (always enabled)
LUAJIT_CMAKE_BOOL_ON=           REQUIRE_LUAJIT
LUAJIT_CMAKE_BOOL_OFF=          ENABLE_LUAJIT

LEVELDB_DESC=                   Enable LevelDB backend
LEVELDB_CMAKE_BOOL=             ENABLE_LEVELDB
LEVELDB_LIB_DEPENDS=            libleveldb.so:databases/leveldb

PGSQL_DESC=                     Enable PostgreSQL map backend
PGSQL_USES=                     pgsql
PGSQL_CMAKE_BOOL=               ENABLE_POSTGRESQL

REDIS_DESC=                     Enable Redis backend
REDIS_CMAKE_BOOL=               ENABLE_REDIS
REDIS_LIB_DEPENDS=              libhiredis.so:databases/hiredis

SPATIAL_DESC=                   Enable SpatialIndex (Speeds up AreaStores)
SPATIAL_LIB_DEPENDS=            libspatialindex.so:devel/spatialindex
SPATIAL_CMAKE_BOOL=             ENABLE_SPATIAL

NLS_DESC=                       Native Language Support (ENABLE_GETTEXT)
NLS_CMAKE_BOOL=                 ENABLE_GETTEXT
NLS_USES=                       gettext
NLS_LDFLAGS=                    -L${LOCALBASE}/lib

TOUCH_DESC=                     Build with touch interface support
TOUCH_CMAKE_BOOL=               ENABLE_TOUCH
# dependency?

PROMETHEUS_DESC=                Build with Prometheus metrics exporter
PROMETHEUS_CMAKE_BOOL=          ENABLE_PROMETHEUS
# dependency?

.include 

.if ${PORT_OPTIONS:MCLIENT} && ${PORT_OPTIONS:MSOUND}
USES+=                  openal
LIB_DEPENDS+=           libogg.so:audio/libogg \
                        libvorbis.so:audio/libvorbis \
                        libvorbisfile.so:audio/libvorbis
.endif

.if ${PORT_OPTIONS:MSERVER}
#USE_RC_SUBR=   ${PORTNAME}
#USERS=         ${PORTNAME}
#GROUPS=        ${PORTNAME}
#USE_RC_SUBR=           minetest/ERX
USE_RC_SUBR=            minetest
USERS=                  minetest
GROUPS=                 minetest
.endif

# Exactly why this must be done this way eludes me but this works and satisfies the install needs.
.if ${PORT_OPTIONS:MSYSTEM_FONTS}
pre-install:
        ${MKDIR} ${LOCALBASE}/share/minetest/fonts
        ${RM} ${LOCALBASE}/share/minetest/fonts/Arimo-Bold.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Arimo-BoldItalic.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Arimo-Italic.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Cousine-Bold.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Cousine-BoldItalic.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Cousine-Italic.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/Cousine-Regular.ttf
        ${RM} ${LOCALBASE}/share/minetest/fonts/DroidSansFallbackFull.ttf
        ${MKDIR} ${LOCALBASE}/share/minetest/fonts
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Arimo-Bold.ttf ${LOCALBASE}/share/minetest/fonts/Arimo-Bold.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Arimo-BoldItalic.ttf ${LOCALBASE}/share/minetest/fonts/Arimo-BoldItalic.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Arimo-Italic.ttf ${LOCALBASE}/share/minetest/fonts/Arimo-Italic.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Cousine-Bold.ttf ${LOCALBASE}/share/minetest/fonts/Cousine-Bold.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Cousine-BoldItalic.ttf ${LOCALBASE}/share/minetest/fonts/Cousine-BoldItalic.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Cousine-Italic.ttf ${LOCALBASE}/share/minetest/fonts/Cousine-Italic.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/ChromeOS/Cousine-Regular.ttf ${LOCALBASE}/share/minetest/fonts/Cousine-Regular.ttf
        ${LN} -s ${LOCALBASE}/share/fonts/Droid/DroidSansFallbackFull.ttf ${LOCALBASE}/share/minetest/fonts/DroidSansFallbackFull.ttf
.endif

post-install:
        @${ECHO_MSG} " "
        @${ECHO_MSG} "-->  "${PREFIX}/etc/"minetest.conf.example explains options and gives their default values. "
        @${ECHO_MSG} " "
        @${ECHO_MSG} "-->  Local network issues could cause singleplayer to fail. "
        @${ECHO_MSG} " "
        @${ECHO_MSG} "-->  Alternate graphics driver may be set in client config, must be set to get used."
        @${ECHO_MSG} " "
        @${ECHO_MSG} " "

# hacky way to not bring irrlicht and X11 depends for server only
#.if ! ${PORT_OPTIONS:MCLIENT} && ${PORT_OPTIONS:MSERVER}
#BROKEN= server only hack fails for irrlicht fork
#.endif
# From wiki:
#  Building without Irrlicht / X dependency
# You can build the Minetest server without library dependencies to Irrlicht or any graphical stuff.
# You still need the Irrlicht headers for this, so first, download the irrlicht source to somewhere.
#
# When invoking CMake, use -DBUILD_CLIENT=0 -DIRRLICHT_SOURCE_DIR=/wherever/you/unzipped/the/source.

#----------------------------------------------------------------------
#
# Strangely network issues can prevent singleplayer from functioning.
# GCC           7.5+    or Clang 6.0+
# CMake         3.5+
# IrrlichtMt    -       Custom version of Irrlicht, see https://github.com/minetest/irrlicht
# Freetype      2.0+
# SQLite3       3+
# Zstd          1.0+
# LuaJIT        2.0+    Bundled Lua 5.1 is used if not present
# GMP           5.0.0+  Bundled mini-GMP is used if not present
# JsonCPP       1.0.0+  Bundled JsonCPP is used if not present
# Curl          7.56.0+ Optional
# gettext       -       Optional
#----------------------------------------------------------------------
.include 

The post-install message is there for information which I believe an official port wouldn't even place within a pkg-message file, and why must I bother with another file if this remains the same much of the time. It would be one more file to track and possibly update.

Once more: I am not an official port maintainer of official ports, and although I may have based my initial Makefiles upon prior works, these are different, unique, and likely never to be annointed in their format or style or technique.  Use them as you choose, the risk is yours regardless of whether they seem safe and effective on my system, caveat emptor for both and all.  I will attempt to keep both updated nearly as frequently as I discover new commits.

Repo urls:

Maintaining these unofficial ports [tree leaf contents files] is an enjoyable hobby which ensures that I also have the very most current software.  The process to do this officially or not is not particularly difficult but it can certainly be a challenge.  I have yet to completely beat dbeaver into utter submission, but the datestamp solution is less than obvious even with a partial answer I am uncertain how to adapt it.  You will never know how you could succeed if you never attempt it.  If there is ANYTHING out there in the Linux world which you want to have for yourself, you can try to port it.  Read the porter's handbook, my own blog posts relate my experiences.

Frequently viewed this week