Hi, Once again I got a massive amount of fixes from LLM in various stages of the pipeline. Things that used to be hard, now seem fixable. gcc PCH, ghc, cmake, meson... plenty toolchain fixes are possible and coming. The numbers start to improve already even when several patches are still going through upstream. Will have to figure out what to do about zig, who seem to not want to talk to anyone who has talked to an AI. I started to collect fixes in home:bmwiedemann:reproducible:distribution2026 as a continuation of my 2024 RBOS project - especially part3 that I did not complete back then. = https://en.opensuse.org/openSUSE:Reproducible_openSUSE This helps me keep an overview of what still needs debugging. As before, this will not receive ongoing maintenance updates, so don't use it in production (build on OBS is disabled anyway except for toolchain packages in the :ring1 subproject). Here are the autogenerated bits: last month's status: https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/JLAOJP7W6K3P2SL6XT6UYX444XZ5WQPN/ Last months' reproducible builds project updates (including my work): https://reproducible-builds.org/reports/2026-07/ I uploaded https://rb.zq1.de/compare.factory-20260903/ today https://rb.zq1.de/spec/glossar.txt explains the meaning of below values: total-packages: 16520 (+84) build-tried: 16450 (+81) build-failed: 65 (-13) build-n-a: 145 (+1) build-succeeded: 16240 (+93) build-official-failed+na: 94 (-92) build-compare-failed: 170 (-23) build-compare-succeeded: 16070 (+116) verify-failed: 399 (+35) verified-semi-reproducible: 15865 (+45) verified-bit-identical: 15598 (+68) bit-by-bit-identical: 16023 (+114) not-bit-by-bit-identical: 211 (-24) not-bit-by-bit-identicalcheck: 217 (-21) https://rb.zq1.de/compare.factory-20260903/graph.png shows the change over time https://rb.zq1.de/compare.factory-20260903/unreproduciblerings.txt lists very unreproducible core packages (bootstrap+DVD) Of the badly unreproducible packages, 1 were in ring0 15 were in ring1 That makes it 16/4154 => 0.39 % which is below the overall average of 170/16240 => 1.05 % 211/16240 => 1.30 % of packages are not perfectly reproducible package notes: ==> ./avr-libc/.rb.notes <== #=> https://src.opensuse.org/gcc/avr-libc/pulls/1 minor .a strange varying hardlinks from fdupes ==> ./boost/.rb.notes <== => SR 1374641 race /usr/lib64/libboost_mpi.so.1.91.0 oscbuildparams=" -M extra " years=13 parallelism=7 parallelism2=8 rbk ==> ./c2hs/.rb.notes <== ghc toolchain ==> ./cosmic-files/.rb.notes <== => https://github.com/pop-os/xdgen/pull/2 toolchain hash for cosmic-* see cosmic issue ==> ./cosmic-player/.rb.notes <== => https://github.com/pop-os/xdgen/pull/2 toolchain hash for cosmic-* /usr/share/metainfo/com.system76.CosmicPlayer.metainfo.xml random order ; rust ==> ./cosmic-settings/.rb.notes <== cosmic/rust issue ==> ./cosmic-store/.rb.notes <== => https://github.com/pop-os/xdgen/pull/2 toolchain hash for cosmic-* see cosmic issue ==> ./cosmic-term/.rb.notes <== => https://github.com/pop-os/xdgen/pull/2 toolchain hash for cosmic-* ==> ./cpio/.rb.notes <== => SR 1374640 mtime + unrelated bug ==> ./dmd/.rb.notes <== => https://github.com/dlang/dmd/pull/23746 sort ASLR ==> ./efl/.rb.notes <== => SR 1366589 1362390 => https://git.enlightenment.org/enlightenment/efl/pulls/133 race = https://git.enlightenment.org/enlightenment/efl/issues/41 = https://bugzilla.opensuse.org/show_bug.cgi?id=1052415 ==> ./elixir/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1205134 FTBFS-j1 stuck ; FTBFS-2038 found range good=2145916071 bad=2145917996 ==> ./erlang/.rb.notes <== => https://github.com/elixir-lang/ex_doc/pull/2255 https://github.com/elixir-lang/ex_doc/pull/2256 https://github.com/rpm-software-management/rpmlint/pull/1550 https://github.com/erlang/otp/pull/11394 https://github.com/erlang/otp/pull/11395 https://github.com/erlang/otp/pull/11396 https://github.com/elixir-makeup/makeup/pull/75 ==> ./esbuild/.rb.notes <== => SR 1373163 go1.27 https://github.com/golang/go/pull/81041 toolchain random = https://bugzilla.opensuse.org/show_bug.cgi?id=1234374 .note.go.buildid + .note.gnu.build-id from random go tmp path ==> ./eww/.rb.notes <== => SR 1368810 https://github.com/gtk-rs/gtk-rs-core/pull/1840 = https://github.com/elkowar/eww/issues/1334 rust1.76/llvm ==> ./falcosecurity-container-plugin/.rb.notes <== golang go-build-id / debug variations ==> ./forecast/.rb.notes <== low-entropy cosmic issue in binary ==> ./freeciv/.rb.notes <== https://redmine.freeciv.org/issues/2122 lua date+hash ==> ./fritzing/.rb.notes <== => https://github.com/fritzing/fritzing-app/pull/4335 = https://bugzilla.opensuse.org/show_bug.cgi?id=1239967 fritzing random parts.db ==> ./ghc-DAV/.rb.notes <== #=> https://gitlab.haskell.org/ghc/ghc/-/merge_requests/16457 sort, toolchain ghc = https://gitlab.haskell.org/ghc/ghc/-/work_items/27612 ==> ./ghc-cmdargs/.rb.notes <== ghc toolchain. draft fix exists ==> ./ghc-dbus/.rb.notes <== ghc toolchain. draft fix exists ==> ./ghc-ghc-lib-parser/.rb.notes <== ghc toolchain (draft fixes suffice) ==> ./ghc-yesod-core/.rb.notes <== ghc toolchain. draft fix exists ==> ./ghc/.rb.notes <== => https://gitlab.haskell.org/ghc/ghc/-/merge_requests/16457 sort ordering issue in /usr/lib64/ghc-9.8.3/lib/x86_64-linux-ghc-9.8.3/libHSCabal-3.10.3.0-8d80-ghc9.8.3.so ; nothing helps ==> ./git-annex/.rb.notes <== => SR 1373692 bug ==> ./grass/.rb.notes <== => https://github.com/OSGeo/grass/pull/7849 date/mtime ==> ./grok-build/.rb.notes <== some patches found via claude ; need review+submit ==> ./gstreamer-plugins-rs/.rb.notes <== => https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/3208 sort => SR 1314943 = https://bugzilla.opensuse.org/show_bug.cgi?id=1237097 = https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/599 date ; rust/llvm asm diff ==> ./gzip/.rb.notes <== 2026-02-11 new unreproducibility discovered - no source change. PGO+CPU gzip's CPU dependence is __builtin_cpu_supports("pclmul") && __builtin_cpu_supports("avx") in gnulib's ==> ./himalaya/.rb.notes <== => himalaya https://github.com/pimalaya/cli/pull/1 rust HashMap random +++ new /usr/bin/himalaya (objdump) ==> ./hlint/.rb.notes <== ghc toolchain. draft fix exists ==> ./intel-graphics-compiler/.rb.notes <== => https://github.com/intel/intel-graphics-compiler/pull/418 ASLR = https://github.com/intel/intel-graphics-compiler/issues/359 ASLR build has variations in some (likely generated) source files: /home/abuild/rpmbuild/BUILD/build/IGC/VectorCompiler/lib/BiF/VCBuiltins64.cpp /home/abuild/rpmbuild/BUILD/build/IGC/Release/bif/embedder/OCLBiFImpl__igc_bif_BIFBC_122.cpp ==> ./java-1_8_0-openjdk/.rb.notes <== => SR 1368504 various backports+fixes ==> ./java-21-openjdk/.rb.notes <== => https://bugzilla.opensuse.org/show_bug.cgi?id=1221224 java-21-openjdk race influence in libjvm.so debuginfo ==> ./jemalloc/.rb.notes <== CPU ==> ./kate/.rb.notes <== Binary files /var/tmp/build-root.11/.mount/home/abuild/rpmbuild/BUILD/kate-26.08.0-build/kate-26.08.0/build/addons/katesql/CMakeFiles/katesqlplugin.dir/queryhelpers/queryselectorpopup.cpp.o and /var/tmp/build-root.11b/.mount/home/abuild/rpmbuild/BUILD/kate-26.08.0-build/kate-26.08.0/build/addons/katesql/CMakeFiles/katesqlplugin.dir/queryhelpers/queryselectorpopup.cpp.o differ ==> ./ktextaddons/.rb.notes <== same issue as kate ==> ./libbluray/.rb.notes <== => https://github.com/mesonbuild/meson/pull/16145 meson jar zipfile, toolchain for libbluray ==> ./libpsl/.rb.notes <== #=> https://github.com/rockdaboot/libpsl/pull/297 date fixup ==> ./lilypond/.rb.notes <== #=> Already fixed by upstream commits 713cb338b6ce, 14e7b471ee6f, 35b761322e21, 822bf54c07fb, 5e659dbd7051 ==> ./llvm22/.rb.notes <== => SR 1374284 toolchain for cosmic-bg ==> ./lutgen-cli/.rb.notes <== => https://github.com/ozwaldorf/lutgen-rs/pull/46 nocheck ==> ./lutgen-studio/.rb.notes <== same issue as lutgen-cli ==> ./micro-editor/.rb.notes <== => https://src.opensuse.org/gbazzotti/micro-editor/pulls/5 ==> ./mingw32-gmp/.rb.notes <== parallelism ==> ./minizign/.rb.notes <== toolchain ASLR from zig ==> ./mono-core/.rb.notes <== => https://gitlab.winehq.org/mono/mono/-/work_items/33 various => https://github.com/mono/mono/pull/20690 date+time ==> ./msedit/.rb.notes <== rust debugsource random/ASLR? new//usr/src/debug/msedit-2.0.0/target/release/build/edit-6356a3e9c63f9c77/out/lsh_definitions.rs ==> ./nekobox/.rb.notes <== toolchain: CPU core count in .src.rpm -ninja -C "$(realpath build)" -v -j 4 ==> ./neochat/.rb.notes <== => neochat https://invent.kde.org/network/neochat/-/merge_requests/2987 race qmlcachegen = https://bugzilla.opensuse.org/show_bug.cgi?id=1248369 parallelism via Qt qmlcachegen https://bugreports.qt.io/browse/QTBUG-137440 ==> ./neovim/.rb.notes <== = https://github.com/neovim/neovim/issues/26387 - maybe toolchain https://github.com/LuaJIT/LuaJIT/issues/1008 ; draft patch exists = https://github.com/neovim/neovim/issues/32429 FTBFS-2038 ==> ./nst/.rb.notes <== => SR 1374393 toolchain scons, minor date in /usr/share/man/man5/nst.conf.5.gz from ["a2x", "-f", "manpage", "-D", "./doc", "doc/nst-msg.1.adoc"] ==> ./ocaml/.rb.notes <== => https://github.com/ocaml/ocaml/pull/14989 race ==> ./opencode/.rb.notes <== FTBFS-CPUv2 stuck building with cpu=Conroe failed ==> ./opentui/.rb.notes <== zig toolchain problem about build-id/debuginfo ==> ./os-autoinst/.rb.notes <== = https://bugzilla.opensuse.org/show_bug.cgi?id=1239686 FTBFS osc build --vm-type=kvm -j1 --clean --noservice standard ==> ./pandoc-cli/.rb.notes <== = ghc toolchain parallelism https://gitlab.haskell.org/ghc/ghc/-/issues/12935 ; fixed by draft patches ==> ./piglit/.rb.notes <== => SR 1372034 => https://gitlab.freedesktop.org/mesa/piglit/-/merge_requests/1193 race variations in /usr/lib64/piglit/tests/vulkan.xml.gz - maybe from %check ? or parallelism ==> ./prismlauncher-cracked/.rb.notes <== see prismlauncher ==> ./prismlauncher/.rb.notes <== jar from cmake toolchain ; draft patch exists ==> ./python-PyMuPDF/.rb.notes <== => SR 1373372 1373241 issue in .a file / RECORD ==> ./python-gguf/.rb.notes <== => SR 1373864 nocheck ==> ./python-hunter/.rb.notes <== toolchain cython ; draft patch exists RECORD file records hash of hunter/_predicates.c ==> ./python-kwant/.rb.notes <== toolchain Cython ; same as python-hunter ==> ./python-langgraph/.rb.notes <== => SR 1371636 nocheck ==> ./python-pook/.rb.notes <== => SR 1369446 nocheck ==> ./python-pyzmq/.rb.notes <== => SR 1369499 embedded random /tmp path in debuginfo ==> ./python-sentencepiece/.rb.notes <== parallelism/race ==> ./python-tokenizers/.rb.notes <== => python-tokenizer https://github.com/PyO3/pyo3/pull/6076 toolchain (rust HashMap) ==> ./python-vllm/.rb.notes <== toolchain: CPU core count in .src.rpm -export MAX_JOBS=4 ==> ./python-xgrammar/.rb.notes <== => SR 1371724 .a RECORD from LTO ==> ./python-xlsx2csv/.rb.notes <== => SR 1373248 python-xlsx2csv date ==> ./qgis/.rb.notes <== ASLR/parallelism in .note.gnu.build-id ; toolchain from gcc pch https://gcc.gnu.org/bugzilla/show_bug.cgi?id=124811 ==> ./radare2/.rb.notes <== #=> https://github.com/radareorg/radare2/pull/26625 date/random ==> ./rav1e/.rb.notes <== => https://github.com/medek/nasm-rs/pull/47 rav1e rust/nasm-rs toolchain ==> ./rocq-stdlib/.rb.notes <== ocaml toolchain ; parallelism from rocqworker ; draft patch exists --- old /usr/lib64/coq/user-contrib/Stdlib/Zmod/ZstarDef.vo (hex) ==> ./rpm/.rb.notes <== => 1375466 rpm delsign + Y2038 => https://github.com/rpm-software-management/rpm/pull/2762 ==> ./ruffle/.rb.notes <== #=> https://github.com/ruffle-rs/ruffle/pull/24360 rust HashMap #=> https://github.com/XAMPPRocky/fluent-templates/pull/112 readdir ==> ./sbcl/.rb.notes <== => SR 1371634 date ; CPU ; partial fix ==> ./scotch/.rb.notes <== #=> CPU cores influenced mpi builds ; patch sent per email (could not find codeberg repo) + accepted ==> ./taisei/.rb.notes <== => https://github.com/taisei-project/taisei/pull/441 zip mtime ==> ./virtualbox/.rb.notes <== => https://github.com/VirtualBox/virtualbox/pull/827 random UUID ==> ./winboat/.rb.notes <== => winboat https://github.com/winboat-org/winboat/pull/861 zip ==> ./wine/.rb.notes <== => https://gitlab.winehq.org/wine/wine/-/merge_requests/11754 => https://bugs.winehq.org/show_bug.cgi?id=60238 old: = https://bugzilla.opensuse.org/show_bug.cgi?id=1231620 wine parallelism ==> ./xpenguins/.rb.notes <== => SR 1373826 => https://gitlab.com/wvermin/xpenguins/-/merge_requests/1 ==> ./zabbix/.rb.notes <== => https://github.com/zabbix/zabbix/pull/184 date ==> ./zig0.15/.rb.notes <== => https://github.com/ziglang/zig/commit/ac38c7a6b55dd9c4f2c59361a0ac6b2ceba7d57f = https://codeberg.org/ziglang/zig/issues/36334 ==> ./zizmor/.rb.notes <== rust ==> ./zola/.rb.notes <== rust/llvm random order .rs generation from HashMap in vendor/minify-html-common-0.0.2/build.rs