From @ixdy on September 21, 2018 21:54

@bmwiedemann has done some work on this already in #48710.

For more increased reproducibility, we should probably be setting SOURCE_DATE_EPOCH in release builds (we already do this in CI), though there are still a few other bits missing.

From @bmwiedemann on September 22, 2018 5:44

Latest state can be seen in
http://rb.zq1.de/compare.factory/kubernetes-compare.out for openSUSE
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/kubernetes.html

@bmwiedemann i can't seem to tell if the results are from the latest k8s releases. if not is there a way to trigger these for say the v1.12-rc1 please?

From @bmwiedemann on September 22, 2018 20:2

openSUSE's diff is from 1.11.1 (and we do have SOURCE_DATE_EPOCH set)

Going to 1.12 is not that easy, because my reproducibility-test tools are designed around building of packages and there are usually so many changes and build is so slow (~20 minutes per try) that it can take hours to get right.

However, IMHO it would be a good start to find out if and how the two known issues in 1.11.1 have been addressed. If there are patches, I could apply them to 1.11.1 and see if anything remains there.

I'd prefer to not have to chase the master branch.

gotcha thanks @bmwiedemann

@bmwiedemann please see #68983 to see if it fixes the man page issue

On the buildid problem, looks like we may have to wait for next versions of golang:
golang/go#16860

See for example how others are trying to think about the same problem:

• Reproducible builds of statusgo status-im/status-go#1185
• https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/

We need to update k/release anago scripts to set SOURCE_DATE_EPOCH and save the information somewhere (in generated tarballs? release notes?)

From @bmwiedemann on September 26, 2018 4:30

regarding random build-ids:
https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/ seems to imply that it is already possible to generate reproducible go binaries and indeed our openSUSE "docker" package is already reproducible (we always build in the same path)

strace showed me

execve("/usr/lib64/go/1.10/pkg/tool/linux_amd64/compile", ["/usr/lib64/go/1.10/pkg/tool/linux_amd64/compile", "-o", "/tmp/go-build336594203/b073/pkg.a", "-trimpath", "/tmp/go-build336594203/b073", "-p", "k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets", "-complete", "-buildid", "SUvgWqVQmIZMGMbPSYtX/SUvgWqVQmIZMGMbPSYtX", "-goversion", "go1.10.3", "-D", "", "-importcfg", "/tmp/go-build336594203/b073/importcfg", "-pack", "-c=4", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/byte.go", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/doc.go", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/empty.go", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/int.go", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/int64.go", "/home/abuild/rpmbuild/BUILD/kubernetes-1.11.1/_output/local/go/src/k8s.io/kubernetes/vendor/k8s.io/gengo/examples/set-gen/sets/string.go"],

so my guess is that a part of the build system generates explicit random buildids instead of using something reproducible (e.g. a constant or the shasum of the source(s))
Maybe go would even do the right thing (like gcc) when no buildid is given?

@bmwiedemann i could get reproducible builds with latest master. PR is here. here's what i had to do.

• make quick-release builds stuff inside a docker container, so we control a lot of the things including go version, paths etc.

But that was not enough. then i had to add trimpath

• 1f74a33#diff-dac3f0d616ae45f98d45d6420ed1be65R671

And then added -s -w to remove the symbol table

• 1f74a33#diff-dac3f0d616ae45f98d45d6420ed1be65R670

and finally pass the SOURCE_DATE_EPOCH into the container where the builds happen.

• 1f74a33#diff-0ee501c7b4b1ab61678579826c6d6534R603

Finally tested the build process under my laptop (MacOS) and ubuntu and verified the build id of the kubeadm binary.

/sig release

@dims should we stop zero-ing out the buildid now?

possibly @BenTheElder but we have to verify :(

@dims should we stop zero-ing out the buildid now?

May I ask why?

@saschagrunert apparently there are fixes in golang compiler itself to better compute the buildid, so we should check it out when we have time.

the buildid is a cache key, and the buildid should be reproducible now I think. but we need to confirm. I thought we'd checked that in KIND but it seems we're still zero-ing it.

Got it, did some research and proposed the change: #101411

@dims: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

/close

please reopen if needed @bmwiedemann

@dims: Closing this issue.