marfrit/marfrit-packages
0
0
Fork 1
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

distcc-avahi: package missing /usr/lib/distcc/bin/{gcc,g++,cc,c++} masquerade symlinks #7

New Issue
Closed
opened 2026-05-09 18:53:44 +00:00 by marfrit · 1 comment
marfrit commented 2026-05-09 18:53:44 +00:00
Owner
Copy Link
Copy Source

Symptom

On a build host with distcc-avahi 3.4-17 installed and BUILDENV=(distcc color !ccache check !sign) + DISTCC_HOSTS="+zeroconf" in /etc/makepkg.conf (or ~/.makepkg.conf), distcc never engages for native gcc/g++ invocations. All compilation runs locally; remote workers (dcc1/dcc2/tesla discovered via avahi) sit idle.

Observed on boltzmann (aarch64) during the kernel-pkgrel=5 build of linux-pinetab2-danctnix-besser. Load average 16+ from local cc1 processes, distccmon-text empty, avahi-browse _distcc._tcp showed dcc1/dcc2/tesla all reachable. Same situation in pkgrel=1..4 builds previously — distcc was never offloading on this host even though everything looked configured.

Root cause

distcc-avahi ships only the binaries themselves:

$ pacman -Ql distcc-avahi | grep -E "lib/distcc|bin/distcc"
distcc-avahi /usr/bin/distcc
distcc-avahi /usr/bin/distccd
distcc-avahi /usr/bin/distccmon-text

It does not populate /usr/lib/distcc/bin/ with symlinks like

/usr/lib/distcc/bin/gcc -> /usr/bin/distcc
/usr/lib/distcc/bin/g++ -> /usr/bin/distcc
/usr/lib/distcc/bin/cc  -> /usr/bin/distcc
/usr/lib/distcc/bin/c++ -> /usr/bin/distcc

makepkg's distcc integration prepends /usr/lib/distcc/bin to PATH when distcc is in BUILDENV. Without those symlinks, plain gcc/g++/cc/c++ lookups fall through to /usr/bin/gcc (etc.) and compile locally. The cross-prefix symlinks (aarch64-linux-gnu-gcc) that boltzmann had were created manually out-of-band and so don't help when the build invokes plain gcc (which is the case for native kernel builds).

Reproducer

$ ls -la /usr/lib/distcc/bin/
total 0
drwxr-xr-x 1 root root 106 ...
drwxr-xr-x 1 root root 252 ...
# (empty or only cross-prefix symlinks)

$ pacman -Qo /usr/lib/distcc/bin/gcc
error: No package owns /usr/lib/distcc/bin/gcc

$ which gcc
/usr/bin/gcc
# Should be /usr/lib/distcc/bin/gcc -> /usr/bin/distcc when in a makepkg distcc env

Kick a kernel build, watch distccmon-text — empty.

Fix

PKGBUILD::package() should create the four standard masquerade symlinks. Reference: Arch upstream distcc package does this via:

install -d "$pkgdir/usr/lib/distcc/bin"
for c in gcc g++ cc c++; do
  ln -s ../../../bin/distcc "$pkgdir/usr/lib/distcc/bin/$c"
done

Non-x86 hosts that also need cross-prefix names can extend the list (e.g. aarch64-linux-gnu-gcc, arm-linux-gnueabihf-gcc), but the four basic names cover the native-build common case which is what makepkg expects.

Workaround

Until packaged: sudo ln -sf /usr/bin/distcc /usr/lib/distcc/bin/{gcc,g++,cc,c++}. Verified to immediately enable distcc — distccmon-text starts showing remote compile slots within seconds of the next cc1 invocation.

Where this matters

Every aarch64 build host running distcc-avahi: boltzmann, fermi, fresnel (if used as builder), tesla, dcc1, dcc2, ampere. They've all been compiling locally instead of sharing load. Kernel builds are the most visible case (~50min becomes ~25-30min when cluster engages); package builds in general also benefit.

Doesn't affect builds that explicitly use the cross-prefix wrapper (e.g. cross-arch builds). Affects any native-aarch64 build that invokes gcc / cc.

Related

  • Already-known marfrit-packages distcc bug: #5 (DISTCC_ARGS vs DISTCC_OPTS naming).
  • See also: reference_distcc_kernel_builds.md notes in claude-noether memory.
## Symptom On a build host with `distcc-avahi 3.4-17` installed and `BUILDENV=(distcc color !ccache check !sign)` + `DISTCC_HOSTS="+zeroconf"` in `/etc/makepkg.conf` (or `~/.makepkg.conf`), distcc never engages for native gcc/g++ invocations. All compilation runs locally; remote workers (dcc1/dcc2/tesla discovered via avahi) sit idle. Observed on boltzmann (aarch64) during the kernel-pkgrel=5 build of `linux-pinetab2-danctnix-besser`. Load average 16+ from local cc1 processes, `distccmon-text` empty, `avahi-browse _distcc._tcp` showed dcc1/dcc2/tesla all reachable. Same situation in pkgrel=1..4 builds previously — distcc was *never* offloading on this host even though everything looked configured. ## Root cause `distcc-avahi` ships only the binaries themselves: ``` $ pacman -Ql distcc-avahi | grep -E "lib/distcc|bin/distcc" distcc-avahi /usr/bin/distcc distcc-avahi /usr/bin/distccd distcc-avahi /usr/bin/distccmon-text ``` It does not populate `/usr/lib/distcc/bin/` with symlinks like ``` /usr/lib/distcc/bin/gcc -> /usr/bin/distcc /usr/lib/distcc/bin/g++ -> /usr/bin/distcc /usr/lib/distcc/bin/cc -> /usr/bin/distcc /usr/lib/distcc/bin/c++ -> /usr/bin/distcc ``` `makepkg`'s distcc integration prepends `/usr/lib/distcc/bin` to `PATH` when `distcc` is in `BUILDENV`. Without those symlinks, plain `gcc`/`g++`/`cc`/`c++` lookups fall through to `/usr/bin/gcc` (etc.) and compile locally. The cross-prefix symlinks (`aarch64-linux-gnu-gcc`) that boltzmann had were created manually out-of-band and so don't help when the build invokes plain `gcc` (which is the case for native kernel builds). ## Reproducer ``` $ ls -la /usr/lib/distcc/bin/ total 0 drwxr-xr-x 1 root root 106 ... drwxr-xr-x 1 root root 252 ... # (empty or only cross-prefix symlinks) $ pacman -Qo /usr/lib/distcc/bin/gcc error: No package owns /usr/lib/distcc/bin/gcc $ which gcc /usr/bin/gcc # Should be /usr/lib/distcc/bin/gcc -> /usr/bin/distcc when in a makepkg distcc env ``` Kick a kernel build, watch `distccmon-text` — empty. ## Fix `PKGBUILD::package()` should create the four standard masquerade symlinks. Reference: Arch upstream `distcc` package does this via: ```bash install -d "$pkgdir/usr/lib/distcc/bin" for c in gcc g++ cc c++; do ln -s ../../../bin/distcc "$pkgdir/usr/lib/distcc/bin/$c" done ``` Non-x86 hosts that also need cross-prefix names can extend the list (e.g. `aarch64-linux-gnu-gcc`, `arm-linux-gnueabihf-gcc`), but the four basic names cover the native-build common case which is what makepkg expects. ## Workaround Until packaged: `sudo ln -sf /usr/bin/distcc /usr/lib/distcc/bin/{gcc,g++,cc,c++}`. Verified to immediately enable distcc — distccmon-text starts showing remote compile slots within seconds of the next cc1 invocation. ## Where this matters Every aarch64 build host running `distcc-avahi`: boltzmann, fermi, fresnel (if used as builder), tesla, dcc1, dcc2, ampere. They've all been compiling locally instead of sharing load. Kernel builds are the most visible case (~50min becomes ~25-30min when cluster engages); package builds in general also benefit. Doesn't affect builds that explicitly use the cross-prefix wrapper (e.g. cross-arch builds). Affects any native-aarch64 build that invokes `gcc` / `cc`. ## Related - Already-known marfrit-packages distcc bug: #5 (DISTCC_ARGS vs DISTCC_OPTS naming). - See also: `reference_distcc_kernel_builds.md` notes in claude-noether memory.
claude-noether commented 2026-05-09 19:24:27 +00:00
Contributor
Copy Link
Copy Source

Shipped in distcc-avahi 3.4-20 (skipped the broken 19 slot — the previous publish left a stale db entry pointing at a missing pkg).

package() now installs masquerade symlinks for c++ c89 c99 cc clang clang++ cpp g++ gcc plus ${CHOST}-g++/gcc/gcc-$(version) into /usr/lib/distcc/bin/.

Verified on boltzmann post-install: /usr/lib/distcc/bin/gcc owned by distcc-avahi 3.4-20. Roll out via pacman -S distcc-avahi on each affected host (boltzmann ✓; fermi/ampere/dcc1/dcc2/tesla/fresnel: pending).

Bonus diagnostic finding: the in-flight linux-fresnel-fourier 7.0-1 build that ran 17:30-19:32 on boltzmann was never actually using distcc despite the load-32 visual — same root cause (no masquerade symlinks, gcc resolved to /usr/bin/gcc). Pure local 8-core. Future kernel-agent builds are no-distcc by policy anyway, so the only practical effect is on the historical record.

Shipped in `distcc-avahi 3.4-20` (skipped the broken 19 slot — the previous publish left a stale db entry pointing at a missing pkg). `package()` now installs masquerade symlinks for `c++ c89 c99 cc clang clang++ cpp g++ gcc` plus `${CHOST}-g++/gcc/gcc-$(version)` into `/usr/lib/distcc/bin/`. Verified on boltzmann post-install: `/usr/lib/distcc/bin/gcc` owned by `distcc-avahi 3.4-20`. Roll out via `pacman -S distcc-avahi` on each affected host (boltzmann ✓; fermi/ampere/dcc1/dcc2/tesla/fresnel: pending). Bonus diagnostic finding: the in-flight `linux-fresnel-fourier 7.0-1` build that ran 17:30-19:32 on boltzmann was never actually using distcc despite the load-32 visual — same root cause (no masquerade symlinks, gcc resolved to /usr/bin/gcc). Pure local 8-core. Future kernel-agent builds are no-distcc by policy anyway, so the only practical effect is on the historical record.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
marfrit (Markus Fritsche)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: marfrit/marfrit-packages#7
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?