In August, I had two weeks off with my family,
so I made fewer patches, but had the machines build all of ring0 reproducibly for the first time and did start builds of ring1 (500GB binaries each).

In ring0, I also managed to build packages with OBS, osc and pbuild and all methods agreed on the build-result. That is with debuginfo enabled everywhere to keep compatibility with Tumbleweed. Only pesign-obs-integration is still causing diffs in OBS.

With my reproducibleopensuse tools (devel branch), the lines to reproduce are:
osc co home:bmwiedemann:reproducible:distribution:ring0/zstd && cd $_
nachbau
# or for a double-build with more variations:
debuginfo="--debuginfo --baselibs" project=home:bmwiedemann:reproducible:distribution:ring0 repo=standard rbk

and with pbuild (patched with cd /usr/lib/build && curl https://github.com/openSUSE/obs-build/commit/f47374bc2027b3a21c2232e3d88b2b296ddd9483.patch | patch -p1 ), it is
osc co home:bmwiedemann:reproducible:distribution:ring0 && cd $_
ln -sf 000pbuildconf/_* .
pbuild --kvm --vm-memory=9000 --vm-disk-size=30000 --jobs 4 --buildjobs 4 --no-checks --release 1.1 --baselibs
sha256sums

To reach a 100% reproducible ring0, some trade-offs had to be accepted. One is that we use %do_profiling 0 which costs ~8% performance in gcc, bash, python and others that used profile-guided-optimization (PGO).
Another is that LTO triggered a reproducibility issue in llvm's libomp, so all versions of llvm build without LTO here, probably losing ~10% of compilation-performance with these as well.

There are also some unmerged patches integrated:
* SR 1192491 rpm-config-SUSE
* https://github.com/rpm-software-management/rpm/pull/2762
* https://github.com/apache/xmlgraphics-fop/pull/65 for pdf-rendering with a PoC/WIP portion for deterministic UUIDs

On another track, https://github.com/openSUSE/openSUSE-release-tools/pull/3129 will add extra test-builds for Factory submissions (one using a single-thread and another building 400 days in the future). This makes it simple to see which packages can produce bit-identical results under these variations.
Only CPU-type-variations cannot be explicitly tested with it. Those would help to catch programs that build with -march=native .


All this brings me close to the first goal of the project: to have all the 328 ingredient rpms for a minimalVM image build 100% bit-reproducible.
https://build.opensuse.org/package/show/home:bmwiedemann:reproducible:distribution:ring1/kiwi-templates-Minimal needs some parts from ring1 and those still need checking, but hopefully not much is left to fix.
Image-creation itself is not yet reproducible with random IDs in filesystems and maybe even mtimes. In past years, %post scripts also caused some non-determinism. However, this part is out-of-scope for this project.