Warning: there may be occasional oddness due to css and blog edits. **Active remodel of blog posts - mostly starting with most recent** 70/86
Showing posts with label fail. Show all posts
Showing posts with label fail. Show all posts

Wednesday, November 26, 2025

flameshot qt6 conversion fail

I had hoped to jump ahead with my flameshot-dev repo, since I let it languish for such a long time, but just as FVWM3 made changes that stymie me at present, flameshot has as well.  The much newer version of flameshot now uses qt:6 but although I have tried to implement things in the Makefile to comply with this change, after each adjustment I reach another qt6 issue which feels like its not handled by our make build structure completely somehow.

Because the most recent version gave me troubles, I backtracked to earlier commits, and ever earlier commits, until I found hopefully the last or nearly last commit that used qt5.  I am doing all of this because once more something strange has occurred.  I would assume it is xlibre related, but it could be anything else.  When I try to open the Flameshot Launcher, I get a very small window and even when expanded seems to contain nothing.  Flameshot does have a maintainer but at present the version in ports is a number of releases behind, and this means I cannot cheat with my flameshot-dev repo and simply copy what I haven't been able to figure out for the qt6 stuff.

My main hope is that firstly I will be able to rebuild flameshot to be more current from source.  Second, I would really like to be able to start with a Makefile that works with the last changes that includes qt5, and try to slowly adapt and figure things out, possibly eventually jumping ahead if I can get all of the qt6 things to succeed.  It may be humorous to see the long series of commits in my flameshot-dev repo for the 'catching up' process beginning with Feb 2025.

I figured out some things for the qt6 requirements, one method was searching freshports for the missing item in a pkg-plist there, which gave me the needed dependency line.  My main trouble, since the first commit that included it, is 'singleapplication' by whatever variation that was used.  Looking at some other ports that happened to involve singleapplication (or a variant) helped me discover other Use_QT items such as 5compat, tools:build, and translations:build which are different for qt6 but I didn't notice in the handbook.  I probably could have found them if I looked in the /usr/port/Mk/Uses at the qt.mk or qt-dist.mk file.

I don't know why I struggle so much to pass a variable toggle to cmake but I do.  After once more looking at the CMakeLists.txt file I finally notice USE_KDSINGLEAPPLICATION which I can set to some variation of "do not use" which for me and my struggle to set that turns out to be:

make
                -DUSE_KDSINGLEAPPLICATION="false" \
                -DUSE_KDSINGLEAPPLICATION="FALSE" \
                -DUSE_KDSINGLEAPPLICATION="off" \
                -DUSE_KDSINGLEAPPLICATION="OFF"

And then I make clean and try make install once more.  This time something above is set to avoid 'singleapplication' and everything builds beyond that requirement.  It keeps happily chugging along until we reach the QtColorWidgets requirement.

ld: error: unable to find library -lQtColorWidgets
c++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped making "install" in /home/tigersharke/flameshot-dev

I would have expected by now that everything would have been satisfied, but I look at the qt.mk file again and wonder if upstream is mixing qt5 and qt6 as well as using the 5compat shim package.  It may be that I add a different line to include the Qt5 QtColorWidgets package and see if that cures it.  I'd believe upstream should find a Qt6 replacement for it.

make
# The Qt components supported by qt.mk: list of shared, and version specific ones
_USE_QT_COMMON=     3d charts connectivity datavis3d declarative doc \
            examples imageformats location multimedia networkauth \
            quick3d quicktimeline remoteobjects scxml sensors \
            serialbus serialport speech svg virtualkeyboard wayland \
            webchannel webengine websockets webview

_USE_QT5_ONLY=      assistant buildtools concurrent core dbus \
            declarative-test designer diag gamepad \
            graphicaleffects gui help l10n linguist linguisttools \
            network opengl paths pixeltool plugininfo printsupport \
            qdbus qdbusviewer qdoc qdoc-data qev qmake quickcontrols \
            quickcontrols2 script scripttools sql sql-mysql sql-odbc \
            sql-pgsql sql-sqlite2 sql-sqlite3 sql-tds testlib uiplugin \
            uitools webglplugin websockets-qml \
            widgets x11extras xml xmlpatterns

_USE_QT6_ONLY=      5compat base coap graphs grpc httpserver languageserver \
            lottie mqtt pdf positioning quick3dphysics quickeffectmaker \
            shadertools tools translations sqldriver-sqlite \
            sqldriver-mysql sqldriver-psql sqldriver-odbc

It turns out that I had already included that extra line for QtColorWidgets and cmake is simply not finding the requisite files.  So this time I go into the work/flameshot directory to search:

packaging/rpm/opensuse/flameshot.spec:rm -rf %{buildroot}%{_includedir}/QtColorWidgets
packaging/rpm/opensuse/flameshot.spec:rm -rf %{buildroot}%{_libdir}/cmake/QtColorWidgets
packaging/rpm/opensuse/flameshot.spec:rm -f %{buildroot}%{_libdir}/libQtColorWidgets.*
packaging/rpm/opensuse/flameshot.spec:rm -f %{buildroot}%{_libdir}/pkgconfig/QtColorWidgets.pc
packaging/rpm/fedora/flameshot.spec:rm -rf %{buildroot}%{_includedir}/QtColorWidgets
packaging/rpm/fedora/flameshot.spec:rm -rf %{buildroot}%{_libdir}/cmake/QtColorWidgets
packaging/rpm/fedora/flameshot.spec:rm -f %{buildroot}%{_libdir}/libQtColorWidgets.*
packaging/rpm/fedora/flameshot.spec:rm -f %{buildroot}%{_libdir}/pkgconfig/QtColorWidgets.pc
src/widgets/panel/sidepanelwidget.h:#include "QtColorWidgets/color_wheel.hpp"
src/CMakeLists.txt:        QtColorWidgets
src/config/colorpickereditor.h:#include "QtColorWidgets/color_wheel.hpp"
src/config/uicoloreditor.h:#include "QtColorWidgets/color_wheel.hpp"
root@ichigo:/home/tigersharke/flameshot-dev/work/flameshot-0f37bf09972c7e66ed78ad0460d2ebce97b82809 #

After a long list of translations I see what it is attempting to find.  What we have for our path may be a bit different though.  The pkg-plist for x11-toolkits/color-widgets-qt5 gives me this:

tigersharke@ichigo:~ % more /usr/ports/x11-toolkits/color-widgets-qt5/pkg-plist
include/QtColorWidgets/AbstractWidgetList
include/QtColorWidgets/BoundColorSelector
include/QtColorWidgets/ColorDelegate
include/QtColorWidgets/ColorDialog
include/QtColorWidgets/ColorListWidget
include/QtColorWidgets/ColorPreview
include/QtColorWidgets/ColorSelector
include/QtColorWidgets/ColorWheel
include/QtColorWidgets/GradientEditor
include/QtColorWidgets/GradientListModel
include/QtColorWidgets/GradientSlider
include/QtColorWidgets/HarmonyColorWheel
include/QtColorWidgets/HueSlider
include/QtColorWidgets/QtColorWidgets_version.h
include/QtColorWidgets/abstract_widget_list.hpp
include/QtColorWidgets/bound_color_selector.hpp
include/QtColorWidgets/color_2d_slider.hpp
include/QtColorWidgets/color_delegate.hpp
include/QtColorWidgets/color_dialog.hpp
include/QtColorWidgets/color_line_edit.hpp
include/QtColorWidgets/color_list_widget.hpp
include/QtColorWidgets/color_names.hpp
include/QtColorWidgets/color_palette.hpp
include/QtColorWidgets/color_palette_model.hpp
include/QtColorWidgets/color_palette_widget.hpp
include/QtColorWidgets/color_preview.hpp
include/QtColorWidgets/color_selector.hpp
include/QtColorWidgets/color_utils.hpp
include/QtColorWidgets/color_wheel.hpp
include/QtColorWidgets/color_wheel_private.hpp
include/QtColorWidgets/colorwidgets_global.hpp
include/QtColorWidgets/gradient_delegate.hpp
include/QtColorWidgets/gradient_editor.hpp
include/QtColorWidgets/gradient_helper.hpp
include/QtColorWidgets/gradient_list_model.hpp
include/QtColorWidgets/gradient_slider.hpp
include/QtColorWidgets/harmony_color_wheel.hpp
include/QtColorWidgets/hue_slider.hpp
include/QtColorWidgets/swatch.hpp
lib/cmake/QtColorWidgets-Qt52/QtColorWidgets-Qt52-%%CMAKE_BUILD_TYPE%%.cmake
lib/cmake/QtColorWidgets-Qt52/QtColorWidgets-Qt52.cmake
lib/cmake/QtColorWidgets-Qt52/qtcolorwidgets-qt52-config-version.cmake
lib/cmake/QtColorWidgets-Qt52/qtcolorwidgets-qt52-config.cmake
lib/cmake/QtColorWidgets-Qt52/qtcolorwidgets-qt52-targets-%%CMAKE_BUILD_TYPE%%.cmake
lib/cmake/QtColorWidgets-Qt52/qtcolorwidgets-qt52-targets.cmake
lib/libQtColorWidgets-Qt52.so
lib/libQtColorWidgets-Qt52.so.2
lib/libQtColorWidgets-Qt52.so.2.2.0
libdata/pkgconfig/QtColorWidgets-Qt52.pc
tigersharke@ichigo:~ % 

The error told us that it couldn't find the directory for the includes, so perhaps we need to be more explicit or modify the src/CMakeLists.txt file in work/flameshot.  After no progress with plenty of effort, I decided to patch a file and this got the build process going a bit farther forward but still quit with errors.

patch-src_CMakeLists.txt
root@ichigo:/home/tigersharke/flameshot-dev # cat files/patch-src_CMakeLists.txt 
--- src/CMakeLists.txt.orig     2025-11-21 21:00:13 UTC
+++ src/CMakeLists.txt
@@ -226,7 +226,7 @@ target_link_libraries(
         Qt${QT_VERSION_MAJOR}::Svg
         Qt${QT_VERSION_MAJOR}::Network
         Qt${QT_VERSION_MAJOR}::Widgets
-        QtColorWidgets
+        /usr/local/include/QtColorWidgets
 )
 if (UNIX)
 target_link_libraries(

I succeeded to get cmake to finally find the QtColorWidgets but now it seems like it still isn't what is expected because some things are different or missing.

slations/Internationalization_zh_HK.qm && cd /home/tigersharke/flameshot-dev/work/.build/src && /usr/local/bin/cmake -E copy /home/tigersharke/flameshot-dev/work/.build/src/Internationalization_zh_TW.qm /home/tigersharke/flameshot-dev/work/.build/src/translations/Internationalization_zh_TW.qm
ld: error: undefined symbol: color_widgets::ColorWheel::ColorWheel(QWidget*)
>>> referenced by colorpickereditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/colorpickereditor.cpp.o:(ColorPickerEditor::ColorPickerEditor(QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))
>>> referenced by uicoloreditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/uicoloreditor.cpp.o:(UIcolorEditor::initColorWheel())

ld: error: undefined symbol: color_widgets::ColorWheel::setColor(QColor)
>>> referenced by colorpickereditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/colorpickereditor.cpp.o:(ColorPickerEditor::ColorPickerEditor(QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::onColorChanged(QColor const&))
>>> referenced 4 more times

ld: error: undefined symbol: color_widgets::ColorWheel::colorSelected(QColor)
>>> referenced by colorpickereditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/colorpickereditor.cpp.o:(ColorPickerEditor::ColorPickerEditor(QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))
>>> referenced by uicoloreditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/uicoloreditor.cpp.o:(UIcolorEditor::initColorWheel())

ld: error: undefined symbol: color_widgets::ColorWheel::staticMetaObject
>>> referenced by colorpickereditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/colorpickereditor.cpp.o:(ColorPickerEditor::ColorPickerEditor(QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))
>>> referenced by uicoloreditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/uicoloreditor.cpp.o:(UIcolorEditor::initColorWheel())

ld: error: undefined symbol: color_widgets::ColorWheel::colorChanged(QColor)
>>> referenced by uicoloreditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/uicoloreditor.cpp.o:(UIcolorEditor::initColorWheel())
c++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped making "install" in /home/tigersharke/flameshot-dev

I should not have needed to patch the file to convince cmake to find QtColorWidgets but even this was not entirely successful.  Something seems strange with this project.  Does it use Qt5 or Qt6 and does a mish mosh of the two affect how everything fits together?  How do I manipulate all of this to make a nice smooth build?  Since I am making only limited progress and possibly making a few missteps that may be temporary improvements, I'll try another method.

Since I do not have an updated ports tree member for deskutils/flameshot, I will look at what archlinux has done in their AUR and specifically commentary and changes in their commit

PKGBUILD
+ 13
− 1
@@ -5,7 +5,7 @@

pkgname=flameshot
pkgver=13.0.0
pkgrel=2
pkgrel=3
pkgdesc="Powerful yet simple to use screenshot software"
url="https://github.com/flameshot-org/flameshot"
arch=('x86_64')
@@ -25,10 +25,22 @@ build() {
		-DCMAKE_INSTALL_PREFIX=/usr \
		-DUSE_WAYLAND_CLIPBOARD=ON \
		-DDISABLE_UPDATE_CHECKER=ON \
		-DUSE_KDSINGLEAPPLICATION=OFF \
		-Wno-dev
	cmake --build build
}

package() {
	DESTDIR="${pkgdir}" cmake --install build

	# Remove vendored static builds of shared libraries (used for single application, which we disable)
	# See https://github.com/flameshot-org/flameshot/blob/v13.0.0/packaging/rpm/fedora/flameshot.spec#L64-L70
	rm -rf "${pkgdir}/usr/include/QtColorWidgets/" \
		"${pkgdir}/usr/include/kdsingleapplication-qt6/" \
		"${pkgdir}/usr/lib/cmake/KDSingleApplication-qt6/" \
		"${pkgdir}/usr/lib/cmake/QtColorWidgets/" \
		"${pkgdir}/usr/lib/libQtColorWidgets.a" \
		"${pkgdir}/usr/lib/libkdsingleapplication-qt6.a" \
		"${pkgdir}/usr/lib/pkgconfig/QtColorWidgets.pc"
	rmdir "${pkgdir}/usr/include/" "${pkgdir}/usr/lib/"{cmake,pkgconfig,}
}

What I have at this point is certainly a lot more "junk" than is necessary in my attempts to satisfy requirements and shut up the complaints about missing stuff.  According to the arch folks, the single application part of the build includes the QtColorWidgets and a number of other things.  Maybe I can do something similar for my unofficial port.  I am unsure but it may be that all those deleted things mentioned above are part of a tarball of some sort, so it will probably look differently for me, unless if they're among what is in the initial extracted tarball.

I am at an impasse.  The best I can do is the error output below and I have no idea how to get beyond it.

ld: error: undefined symbol: color_widgets::ColorWheel::ColorWheel(QWidget*)
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))

ld: error: undefined symbol: color_widgets::ColorWheel::setColor(QColor)
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::onColorChanged(QColor const&))
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::onColorGrabAborted())

ld: error: undefined symbol: color_widgets::ColorWheel::colorSelected(QColor)
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))

ld: error: undefined symbol: color_widgets::ColorWheel::staticMetaObject
>>> referenced by sidepanelwidget.cpp
>>>               src/CMakeFiles/flameshot.dir/widgets/panel/sidepanelwidget.cpp.o:(SidePanelWidget::SidePanelWidget(QPixmap*, QWidget*))

ld: error: undefined symbol: UIcolorEditor::UIcolorEditor(QWidget*)
>>> referenced by visualseditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/visualseditor.cpp.o:(VisualsEditor::initWidgets())

ld: error: undefined symbol: ColorPickerEditor::ColorPickerEditor(QWidget*)
>>> referenced by visualseditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/visualseditor.cpp.o:(VisualsEditor::initWidgets())

ld: error: undefined symbol: UIcolorEditor::updateComponents()
>>> referenced by visualseditor.cpp
>>>               src/CMakeFiles/flameshot.dir/config/visualseditor.cpp.o:(VisualsEditor::updateComponents())
c++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make: stopped making "install" in /home/tigersharke/flameshot-dev

I am patching a few files which seems to help me get as far as this in the build but I cannot seem to get the build to find exactly what it needs.  If I didn't need to deal with the Qt or Qt6 stuff, I would not but it is integral to how flameshot interfaces with the desktop.  I don't know how the patch I mentioned earlier fails to help the build find the QtColorWidgets since the listing of that directory is below.

root@ichigo:/home/tigersharke/flameshot-dev # ls /usr/local/include/QtColorWidgets
abstract_widget_list.hpp color_dialog.hpp         color_palette.hpp        ColorDelegate            colorwidgets_global.hpp  GradientEditor           HueSlider
AbstractWidgetList       color_line_edit.hpp      color_preview.hpp        ColorDialog              gradient_delegate.hpp    GradientListModel        QtColorWidgets_version.h
bound_color_selector.hpp color_list_widget.hpp    color_selector.hpp       ColorListWidget          gradient_editor.hpp      GradientSlider           swatch.hpp
BoundColorSelector       color_names.hpp          color_utils.hpp          ColorPreview             gradient_helper.hpp      harmony_color_wheel.hpp
color_2d_slider.hpp      color_palette_model.hpp  color_wheel_private.hpp  ColorSelector            gradient_list_model.hpp  HarmonyColorWheel
color_delegate.hpp       color_palette_widget.hpp color_wheel.hpp          ColorWheel               gradient_slider.hpp      hue_slider.hpp
root@ichigo:/home/tigersharke/flameshot-dev #

The other patch is below, that I thought would either cause another build error or avoid the part that seems to need the QtColorWidgets.

patch-src_config_CMakeLists.txt
 
--- src/config/CMakeLists.txt.orig      2025-11-22 01:50:14 UTC
+++ src/config/CMakeLists.txt
@@ -3,8 +3,6 @@ target_sources(
   PRIVATE buttonlistview.cpp
           cacheutils.cpp
           clickablelabel.cpp
-          colorpickereditmode.cpp
-          colorpickereditor.cpp
           configerrordetails.cpp
           configresolver.cpp
           configwindow.cpp
@@ -15,6 +13,5 @@ target_sources(
           shortcutswidget.cpp
           strftimechooserwidget.cpp
           styleoverride.cpp
-          uicoloreditor.cpp
           visualseditor.cpp
 )

I am out of ideas, it seems this situation is unique to flameshot and how it may be using a mix of Qt5 and Qt6 along with the compat5 shim code.  I looked at the Gentoo build instructions and those are not any more help than those of Arch.  Sometimes these ports that I attempt to shoehorn into FreeBSD need time to adjust something or other which can be cured with our ports tree infrastructure, Mk scripts, and everything.  If anyone has any bright ideas how to get this working, they are welcome to make a PR on my github for flameshot-dev.  The flameshot in ports (12.1.0_1) is still considerably behind right now, release is 13.3.0 as of last month.

Thursday, October 9, 2025

Weird buildworld failure

Recently I ran into an unexpected issue when attempting to rebuild and reinstall world and kernel.  I was doing this to try to figure out why another install of xlibre was dropping me into VESA.  I'll spoil that cliffhanger to say that for whatever reason graphics/drm-kmod is not installed in that process and so my AMDGPU driver doesn't get used.  It is for the user to add graphics/drm-kmod even with xorg, not sure how this couldn't be automated in any way.  I cannot say why missing drm-kmod wasn't an issue before except if there was an update at xlibre-server upstream which is entirely possible, or very mysteriously it remained installed.  The output of the error-aborted buildworld is below.

root@ichigo:/usr/src # make -j `sysctl -n hw.ncpu` buildworld && make -j `sysctl -n hw.ncpu` kernel KERNCONF=ICHIGO KERNCONFDIR /root/kernconf/ && echo "go to singleuser"
--- buildworld ---
make[1]: /usr/src/Makefile.inc1:362: SYSTEM_COMPILER: Determined that CC=/usr/local/bin/ccache cc matches the source tree.   Not bootstrapping a cross-compiler.
make[1]: /usr/src/Makefile.inc1:367: SYSTEM_LINKER: Determined that LD=ld matches the source tree.   Not bootstrapping a cross-linker.
--- buildworld_prologue ---
--------------------------------------------------------------
>>> World build started on Sat Oct  4 21:19:21 CDT 2025
--------------------------------------------------------------
--- _cleanworldtmp ---
>>> Deleting stale files in build tree...
ld-elf.so.1: Shared object "libutil.so.9" not found, required by "env"
        0.00 real         0.00 user         0.00 sys
*** [_cleanworldtmp] Error code 1

make[1]: stopped making "buildworld" in /usr/src
make[1]: 1 error

make[1]: stopped making "buildworld" in /usr/src

make: stopped making "buildworld" in /usr/src

I had searched online for the issue but once more nothing pertinent appeared, or things that may have been vaguely pertinent were horribly outdated.  When I returned once more, after a few further updates via git to /usr/src, just to be sure that wasn't somehow the issue.  On a whim I checked to see if Grok might point me in any direction that could help me solve it.  I gave it a query: "How can I update world and kernel on FreeBSD 15-stable if it fails because libutil is missing?" and part of the answer is below, it seems to have searched 40 urls, 10 of them from reddit, part from freebsd forums and handbooks.

Step-by-Step Fix

Follow these steps to clean up and rebuild. This resolves most libutil-related failures by ensuring a consistent build tree.
  1. Update Source Tree Fetch the latest 15-STABLE sources to ensure you're building against current code: cd /usr/src git pull # If using Git (recommended; init with `git clone https://git.FreeBSD.org/src.git` if needed) # Or, if using SVN (legacy): svn update
  2. Clean Build Artifacts Remove stale object directories and temporary files that could reference old libutil versions: rm -rf /usr/obj/* cd /usr/src make clean # Cleans world build remnants
  3. Check and Clean Installed Headers/Libraries (If Mismatched) If /usr/include has mixed dates (e.g., old headers conflicting with new kernel), back it up and reinstall base headers: mv /usr/include /usr/include.old # Backup make -C /usr/src/etc/ maketmpenv # Recreates /usr/include from source
  4. Verify libutil header: ls /usr/include/libutil.h (should exist post-this step). If /lib/libutil.so.9 is missing or broken: ls -l /lib/libutil.so* # Should show libutil.so.9 -> libutil.so.9.x

The first step I had already done.  The second step I immediately followed with the next ones.  I was able to rename the /usr/include directory but maketmpenv doesn't seem to exist any longer which points to how AI (like Grok) is not always 100% accurate and its results should always be checked or verified.  After I tried having it build within /usr/src/etc and it failed after a little built, I tried buildworld again which also didn't work yet, and so did a cleanworld.  Of course I had to put things back as they were originally, so renamed /usr/include.old since the step after that couldn't create the fresh /usr/include.  With it all back as it should be, I tried buildworld again and it iterated through all the needed things to build, building as it went.

rm -rf /usr/obj/*
root@ichigo:/usr/src # make clean
mv /usr/include /usr/include.old
make -C /usr/src/etc/ maketmpenv
cd /usr/src/etc
make
cd /usr/src
make -j `sysctl -n hw.ncpu` buildworld && make -j `sysctl -n hw.ncpu` kernel KERNCONF=ICHIGO KERNCONFDIR /root/kernconf/ && echo "go to singleuser"
mv /usr/include.old /usr/include
make cleanworld
make -j `sysctl -n hw.ncpu` buildworld && make -j `sysctl -n hw.ncpu` kernel KERNCONF=ICHIGO KERNCONFDIR /root/kernconf/ && echo "go to singleuser"

Now that it seems everything is working, I should have tried buildworld after the rm -rf /usr/obj/* because that seems to have been likely what cured the weird state my build environment had.  Perhaps something in the newer build routine for 15-stable allowed things to remain in a condition that would not allow me to immediately buildworld as I had been doing so many times in the past.  I recall some old instructions, I believe from back when I was using or playing with PC-BSD, that included cleaning up /usr/obj/ before a fresh buildworld and buildkernel.  I always end with delete-old and delete-old-libs but do I then also remove /usr/obj?  I suspect that this is a step directly before building so that is where it will go.  It looks like I will have to add that step into my normal routine just to be sure, so that future attempts will not leave me with an error like Shared object "libutil.so.9" not found again in the future.

Had I not known, recalled that cleaning of /usr/obj/ as a step of the build process, I would not have recognized it was the cure.  Something like Grok could help spur an idea for a solution, it may give avenues to investigate and research, but be wary that it is pattern matching software and truly doesn't know like a human might.  An AI may very well give old advice, as we note it mentioned the legacy SVN.  I'll be adding more things to distilled and adjusting the buildworld + kernel updating info and probably blog posts as well.

There was also a situation where etcupdate needed to have an mtree change reconciled, but since I do not modify that myself and I am not a dev with any reason for it to be anything but vanilla, I just told it to accept 'their version' of the file.  The odd part was that etcupdate first gave me options that were not helpful because I did not understand what I was seeing, nor how that mtree file should look, so my choice of diff had to be cancelled.  Right after, I noticed there were other commands not immediately listed, one of them was reconcile and gave its options.  When I tried to resolve instead, it wouldn't take the command with 'theirs' as the option, so I had to exit etcupdate entirely, and when I retried etcupdate resolve with tf it worked properly.  I am not sure what this affected, of course it cured the conflict, aside from any future etcupdate attempt functioning properly and completely, not prevented by a needed though not initiated resolve.  Somehow I thought it said 'reconcile' somewhere in that tool's interaction but I cannot find proof of that now.  According to the

man 8 etcupdate
   Resolve Mode
     The resolve mode is used to resolve any conflicts encountered during a
     merge.  In this mode, etcupdate iterates over any existing conflicts
     prompting the user for actions to take on each conflicted file.  For each
     file, the following actions are available:

     (p) postpone      Ignore this conflict for now.
     (df) diff-full    Show all changes made to the merged file as a unified
                       diff.
     (e) edit          Change the merged file in an editor.
     (r) resolved      Install the merged version of the file into the
                       destination directory.
     (mf) mine-full    Use the version of the file in the destination
                       directory and ignore any changes made to the file in
                       the “current” tree.
     (tf) theirs-full  Use the version of the file from the “current” tree and
                       discard any local changes made to the file.
     (h) help          Display the list of commands.

This is not the first time that etcupdate had to correct something before other steps would succeed.  I am not sure how I could have missed that that previous installworld had a conflict and I passed that step seeming to succeed with a whole install.  I also have no idea what the change to that mtree file was or how it could affect anything itself.  How do I make certain that such a step is not missed?  Maybe I was tired or distracted and that caused me not to see it, so I need to do a buildworld and new kernel install when I am not sleepy and when I can focus.  The /usr/obj/ clean issue is an easy fix, just add it to steps taken when preparing to rebuild world.

I wish I could say that each of these missteps makes me less likely to repeat them, that I will learn and remember what I learned and correct things for all successive efforts.  I do my best to keep track of what I do and since writing this blog I have some notes in that form I can reference.  Surely even if I repeat a mistake, those who read of my trials will have the smallest chance to make my mistakes and will instead avoid them.

Saturday, February 26, 2022

Port breaks kernel breaks port

So many of us chug happily along without completely realizing or recognizing how some of the present FreeBSD build mechanisms have become a bit more complex.  Those who never have any need of graphics and remain in a text mode commandline interface for the duration of their use of FreeBSD would not know that there is indeed at least one situation, now, which ties a port and the kernel together.  When everything is working perfectly, this would likely never come up, but a relatively small problem inflated itself to cause my kernel build to fail.

During this troubleshooting quest, first I tried the obvious things, re-rebuild world just to be sure it was ok, then rebuild a GENERIC kernel instead of my custom kernconf and that after having re-enabled some possibly related things in that kernconf to no avail.  After beating my figurative head against the wall for quite a while, I went to twitter to see if @FreeBSDHelp had any ideas.

The details and comments on that discussion thread didn't solve my issue as I could not comprehend how our kernel build was now in any way tied to the build of a port, though the comment that was made may not have explicitly indicated this.  So my next thought was, if I could get a different, earlier version of the /usr/src from git in some way, then rebuild kernel from before the error seemed to appear in /usr/src.  I didn't know this was not the path to take to solve this, but I still wasted far too much time trying to go backward with git to an earlier commit.  I am definitely not a particularly big fan of git, and this exertion didn't help me love it any more.

The reason, besides that it had been in excess of 10 days since the last time I rebuilt my kernel and world, was to get virtualbox working which needed bits from the kernel build which I didn't have, and those need to be the same version as the running OS.  This meant my long journey to rebuild my kernel and world (multiple times each) so that I could use Virtualbox to try the game Veloren which due to whatever is different than expected (FVWM3 and Radeon graphics probably, or similar) does build and install but does NOT run.  I am sure that if I could startup Virtualbox, put tinycore Linux in there, and install Veloren for Linux, I would probably succeed where I was prevented otherwise.  I could not install another OS in Virtualbox because I could not boot the iso, and this due to the virtualbox kernel object not having been loaded.  I couldn't load the needed kernel object since it needed to be built, and now you know why I got stuck down this rabbit hole.

It has been some time since setting that whole "Play Veloren in Virtualbox via tinycore Linux" idea on a back burner or in a box on a shelf somewhere.  One of my incomplete projects is to get my port attempt of Reshade rebuilt, which I was attempting and it ran into some conflicts with python items.  The various python things it needed were installed as version 310 while I already had version 39 of those same ports.  The only way forward was to remove each of the python 3.9 ports to let the reshade build then install what it needed as version 3.10.  Among all of the things that were removed as a consequence of this, was vlc and firefox, both I use daily.  So I gave poudriere a gross list of everything I had installed on my system, let it build, and then discovered a number of things that failed.  The graphics/drm-fbsd13-kmod port was among the fairly long list of things that didn't get built, and it was in the smallish group of "lynchpin" ports, meaning that others failed (were skipped) because it failed.

And so, I thought that's not a problem, I'll go investigate what happened with graphics/drm-fbsd13-kmod to make it fail.  I remembered that poudriere keeps logs of all (or most everything) it does, and I just had to find it.  Since I often end up trying to remember where any certain important thing is and its path, I have been keeping a directory of symbolic links with sometimes more descriptive names.  The appropriate one was,

root@ichigo:~ # ls -l Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg
lrwxr-xr-x  1 root  wheel  66 May  5  2021 Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg -> /usr/local/poudriere/data/logs/bulk/13amd64-default/latest-per-pkg

and from there I could do

root@ichigo:~ # tail -n 15 Symbolic_Links/p-keg-logs_bulk_13amd64_latest-per-pkg/drm-fbsd13-kmod-5.4.144.g20220223.log
===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: %%KMODSRC%%/linuxkpi/dummy/include/linux/random.h
Error: Orphaned: %%KMODSRC%%/linuxkpi/dummy/include/linux/suspend.h
===> Checking for items in pkg-plist which are not in STAGEDIR
===> Error: Plist issues found.
*** Error code 1

Stop.
make: stopped in /usr/ports/graphics/drm-fbsd13-kmod
=>> Error: check-plist failures detected
=>> Cleaning up wrkdir
===>  Cleaning for drm-fbsd13-kmod-5.4.144.g20220223
build of graphics/drm-fbsd13-kmod | drm-fbsd13-kmod-5.4.144.g20220223 ended at Sat Feb 26 00:30:21 CST 2022
build time: 00:04:43
!!! build failure encountered !!!

Firstly, the build failure is due to my choice to be a bit more stringent on builds, to test for various things, so it is possible that this might not appear to most users, although it truly should be visible to all the port maintainers and various FreeBSD developers.  It tells me, as did the small highlighted concise reason in the failed build list output from after I ran poudriere, that it is an issue with the pkg-plist.  This I correctly believed was a simple issue, and easy to fix since this process is something I have repeated many times with my own repos for FreeBSD Port Tree Leaf items such as for Minetest-dev which I wrote about in another blog post.

What I needed to do was go to /usr/ports/graphics/drm-fbsd13-kmod and rename the pkg-plist to pkg-plist-old, and then do a fresh build of it.  Once the build completes, I create a fresh pkg-plist by
make makeplist > pkg-plist
in order to do a comparison between this new fresh list and the old original list.  This is accomplished by
diff -y pkg-plist pkg-plist-old | more
to step through the output, looking for something that is present or absent in the newly generated pkg-plist as compared to the old one.  Since a pkg-plist that is in the ports tree may have %%text%% type tags which are often not generated by the make makeplist script, I usually modify the pkg-plist-old to match the one freshly generated.  Once the edits are made, I rename the pkg-plist-old to pkg-plist and then rebuild once more to prove no errors related to the file remain.

Now that graphics/drm-fbsd13-kmod successfully builds and installs, I thought from the back of my mind, that I would try to update kernel and world, due to that vague mention of these two things being related.  World builds as expected, so I go on to the kernel, and then it fails.  It complains that kconfig.mk was missing.  I remember that that was one of the things that I had removed from the graphics/drm-fbsd13-kmod pkg-plist for a reason I am already uncertain about now-- and this is being written within hours of having done it.  I go back to that port tree directory and either the pkg-plist-old was still present or I went through the steps to generate fresh and make the needed edits to fix it.  Whatever actually happened seems to have fallen out of my mind but the result was "Hey! graphics/drm-fbsd13-kmod needs to be built in order for the kernel to build, gee that is weird."

I have been writing about all of this within a relatively short period after succeeding to build kernel and world when it had been broken some week(s) ago.  My new kernel has not yet been installed and I have to build the virtualbox thing(s) that are dependent upon the source.  It is nice now to have this mess cleared up and better understood.  I'll be adding this nit to the lists of build issues for kernel or world, and add emphasis on the relationship between this port and the kernel which likely many of us had not known.  The kernel failure meant the virtualbox port for a kernel object couldn't be built, but the kmod graphics port is what broke the kernel.

Saturday, November 14, 2020

Now I can't boot

There have been plenty of times by now that I have made some sort of adjustment on my system and then it doesn't boot.  We know the usual suspects are /etc/rc.conf and /boot/loader.conf but I'm sure there are others, possibly even a badly thought out recently built and installed custom kernel.  So now we are stuck, we have one box and it fails to boot but the way to solve the problem is to get online from it.  The situation with the broken kernel might be sidestepped easily, simply choose the option from the boot menu to use a different kernel.  If there were also mistakes with the buildworld, and lack of items means no booting, then there needs to be another way.

If you can get to single-user mode (another boot menu item), the changes will be easy to apply.  First, mount -u / and then if your filesystem is ZFS rather than UFS, zfs mount -a and now you can re-edit the typo out of your /boot/loader.conf or some other file, but what if your situation is a bit more complex?

You can use a usb stick to boot and from there mount the drives in your pc, make needed adjustments and get everything back to normal again.  This is where it can be fun, and when I say fun I mean not quite a nightmare though it is a real special pain.  You can probably use any bootable BSD which offers shell access to the machine, but since discovering NomadBSD, it has become my preference.  What NomadBSD has is a complete system which is self-contained within the usb media.  So if you are short of time and cannot fix your system, you can use it to get online to do something important, such as check your work schedule.  Of course, this immediate need situation means that you previously setup a web browser and installed and configured an addon (such as blur by Abine) which stores your passwords, and you had made any other needed adjustments to suit your needs.  So, getting online you have the solution to the problem and you've written it down and now you need to fix whatever is wrong on the HDD of your system.

What you need to do is mount the HDD of your system into the usb media that is loaded.  There should be a directory /media present already, if not, create it via mkdir /media because this will be the mount point to reach inside your system HDD.  I would assume that you are already in a shell window (xterm perhaps) or you used shell access from the menu when you booted the usb.  We have two pieces of the puzzle, the solution and the system running with a shell, what we need to do in order to make the changes is get to the drive.

With ZFS, there is a special command which will do what we need, change zroot to the name of your pool.
zpool import -f -R /media zroot

Many times the mistake was made or is corrected in either /etc or /boot, so now to reach those directories or any others on your HDD, you would prefix the desired directory with /media such as below.  Work slowly, re-read the command you've typed before committing to it by pressing return or enter.  While your HDD is mounted to your usb stick, /media is the root directory of your HDD, and / is the root directory of the usb stick itself.  Entering the specific directory in order to edit a file, such as rc.conf or loader.conf, may be better than remembering every time to prefix with /media, but always pay attention to your current working directory or path.
cd /media/etc
or
cd /media/boot

Only you can know what the problem and solution are.  Now that you have access  to your HDD you can make the corrections and reboot.  That zpool import command is only viable until you reboot and does not need to be turned off or disabled.  We have not made any permanent changes to how your drives are mounted in order to fix the problem, unless of course your problem and solution specifically involves a permanent adjustment to how your drives are mounted.  Right now I do not have any examples handy of the dumb things I have done which resulted in being unable to boot along with how they were fixed.

While tinkering with your system in ways that can only truly be done when it is open source, because you use FreeBSD for your need of control over all of it, you are setting yourself up with the potential for mistakes.  There is nothing wrong with unintentionally doing something incorrectly, it is the surest way to learn.  We may read somewhere how to do something but unknowingly miss a step or configure something wrong or assume their technique will work on our system.  The worst of these experiences involve Boot blockers and unfortunately they can at least temporarily halt all further progress.

Frequently viewed this week