After Patch C v3 closed (PR #5 merged, Phase 7 N=3 verified at +73%
throughput vs Patch B baseline), the post-13-min RX-degradation
pattern remains. Reproduces on Patch B, F, and v3 alike — independent
of the relay/race issues v3 addressed. Side-effect that was masked
by the throughput floor while v2's race was the dominant variable.
Research question (locked):
Why does the bes2600 RX path collapse from ~2 MB/s sustained @
fresh-chip uptime to ~180 B/s @ ~28-min uptime, with periodic
wsm_generic_confirm failed for request 0x0007 + ieee80211 phy0:
[SCAN] Scan failed (-22) every 300 s in the intervening window?
Phase 0 protocol:
- long-capture rig armed on ohm at uptime 0 (fresh boot 23:13 CEST)
- ftrace events: workqueue, mac80211, cfg80211, mmc, sdhci, power
- iw event (cfg80211 reason codes), dmesg follow, per-30s netdev
counter snap, 5 stress probes at T+5/10/15/20/25 min
Phase 0 will:
- re-anchor the predecessor data via the long capture (in-session
N=1; re-run if anomalous)
- characterize state transitions (first scan-fail, first throughput
drop) via cfg80211/mac80211 ftrace + iw event correlation
- feed Phase 1 metric formulation
Mechanism candidates (Phase 4 will discriminate):
1. Firmware-side resource exhaustion (per-scan accumulator)
2. NetworkManager scan-fail recovery loop competing with data
3. AP-side rate limiting / fairness probation
4. PSM state machine deadlock (c7 latch stale)
5. SDIO bus retune interaction
6. Power-management busy-event accumulator leak
Out of scope: Patch C2/D/E, higher-rate ramp, reproducing on different
APs. Independent campaign from Patch C closure.
Supersedes v2 (PR #10). cw1200 mining (~/src/linux-rockchip, 228
cw1200 commits) confirmed: upstream cw1200 has no sdio_rx_work
workqueue at all. IRQ handler bumps bh_rx + wakes bh_wq; bh thread
does the SDIO read inline via cw1200_bh_rx_helper. Single thread =
single writer for hw_bufs_used = no race by construction. Same int
hw_bufs_used as bes2600, never atomic_t'd in 16 years upstream.
v3 brings bes2600 into that shape:
- delete sdio_rx_work, self->rx_work, self->rx_queue,
self->rx_queue_lock, bes2600_sdio_pipe_read
- GPIO IRQ handler calls self->irq_handler directly (matches
cw1200_sdio_irq_handler shape)
- bes2600_bh_rx_helper's BES_SDIO_RX_MULTIPLE_ENABLE branch
replaced with inline SDIO read + extract_packets + per-skb
delivery via new bes2600_bh_handle_rx_skb()
- GPIO wake-flag bracketing moves into bh thread
§5 shared-state delta table (the v2 lesson, applied): zero fields
require new locking. hw_bufs_used / hw_bufs_used_vif / wsm_tx_pending
all stay single-writer-from-bh. v2's atomic_t prep is mooted.
§6 risk #6 is the open question for reviewer: bes2600's
__bes2600_irq_enable(1) call is commented out in the BH-loop done:
label with an asm volatile("nop") in its place. Either SDIO IRQ
is auto-managed (so commenting out is fine) or the current code
relies on sdio_rx_work being queued regardless of driver-side IRQ
flag. Block Phase 6 on this audit.
Patch F (PR #4 merged) is the new baseline. v3 will branch off
F-merged cleanups. Phase 7 N=3 stress ramp uses wired enu1 path
(192.168.88.80) for wedge-resilient telemetry collection.
Phase 7 of Patch C (PR #9 → bes2600-dkms PR #3 → boot -1 of ohm
20:18:10) failed with a thread-safety race: wsm_release_tx_buffer's
unlocked R-M-W on hw_bufs_used races against wsm_alloc_tx_buffer in
the bh thread when Patch C moved the RX-confirm decrement into
sdio_rx_work. WARN storm at +13s under stress, chip wedges, host
off-network.
Phase 6 contract analysis cited wsm_handle_rx's sleepability and
held-lock invariants but stopped at the function signature. Did not
enumerate hw_bufs_used as shared state mutated by the callee. Lesson
saved as feedback_phase6_contract_threadsafety memory.
Phase 4 v2 designs around that gap. Two-step:
1. Patch C-prep: NFC refactor — convert hw_bufs_used,
hw_bufs_used_vif[], wsm_tx_pending[] from int / int[] to atomic_t /
atomic_t[]. Use atomic_fetch_sub_release in wsm_release_tx_buffer
(returns prior value for the >= numInpChBufs - 1 predicate).
Mechanical atomic_read swap at ~58 read sites. Lands first;
Phase 7 should show zero delta from baseline.
2. Patch C v2: re-apply the sdio_rx_work direct-deliver on top of
C-prep. Identical structural change to the closed PR #3, but now
the racing counter is safe. Contract block in
bes2600_bh_handle_rx_skb expanded to include the shared-state
delta table.
Plan §2 is the shared-state delta table — every field
bes2600_bh_handle_rx_skb mutates directly or transitively, with
current protection and required action. 3 fields need atomic_t,
the rest are already concurrency-safe or stay single-writer.
Plan §6 lists 6 risks including memory-ordering choices, the
inc/dec_pending_count timer-decision race, and the new wired-rig
fallback (enu1 192.168.88.80) that survives bes2600 wedges so Phase 7
can capture dmesg / ftrace from a wedged ohm without reboot.
PR superseded #3 closed with full verdict comment. Phase B rolled
back on ohm at /lib/modules/.../extra/bes2600.ko. Markus's reboot
button to land Patch B again before C-prep work begins.
Per merged PR #8 inline review: items 1 and 2 split, sequential. Patch C
is item-1-only (collapse the sdio_rx_work → rx_queue → bh_work
indirection). Patch C2 (ieee80211_rx_list batch delivery) is split out
and gated on Task #19 kerneldoc contract verification.
Approach choice: Option A (sdio_rx_work delivers directly into
wsm_handle_rx, removing rx_queue and its two synchronization points per
frame) over Option B (subsume into bh thread). Option A has a smaller
diff and clearer bisection story; the residual per-IRQ workqueue
dispatch is preserved as a measurable Phase 7 data point that motivates
or doesn't motivate a follow-on Option-B patch.
Predicted delta in Phase 3 units, with confidence levels stated
explicitly. §4.6 lists 6 risks, of which 2 require Phase 6 contract
citations (wsm_handle_rx callability from sdio_wq context;
wsm_release_tx_buffer's bh-wake invariant). §4.8 mandates a stress
ramp in Phase 7, not a steady cap, per feedback_phase7_stress_ramp.
Symptom-shaped findings (asm nop, commented-out IRQ re-enable, BUG_ON
in hot path) explicitly deferred to Task #24 per
feedback_dont_patch_downstream_artifacts.
Awaiting Phase 5 second-model review on DokuWiki.
Independent code-review writeup (Opus 4.7) against Sonnet's review of the
same tree. Concurs with Sonnet on items 1+2 (RX relay, batch delivery)
and items 4+5 (ba_lock atomics, ps_state_lock skip-when-pm_unsupported);
pushes back on the "9 workqueue events per frame" quantification and
records BES_SDIO_OPTIMIZED_LEN as hard-baked rather than togglable.
New findings: cw12xx-not-bes2600 genealogy still active in source, ~700
lines of #if 0 fossil in bh.c, Allwinner-specific sw_mci_check_r1_ready
in the SDIO bus path, asm volatile("nop") placeholder where IRQ re-enable
used to live, BUG_ON in steady-state hot path, vendor-SDK Makefile shape
that pollutes every diff, 8 EXPORT_SYMBOLs from a nominally-single-binary
module.
Recommends ordering: Patch C (1+2 wrapped) high-risk-first, Patches D+E
as small individually-verifiable cleanups, explicit don't-touch list.
Notes ieee80211_rx_list contract verification (task #19) blocks Patch C.
Sonnet (general-purpose subagent, model=sonnet) reviewed
~/src/besser/bes2600-dkms-mobian/bes2600/ given the Phase 0 measurement
context. Output: 8-item ranked restructuring map, file:line cited.
Headline:
- Item 1: collapse sdio_rx_work relay into BH loop (~5x workqueue
dispatch reduction, medium effort)
- Item 2: batch deliver via ieee80211_rx_list (small effort, removes
per-frame softirq)
- Items 1 + 2 together collapse "9 workqueue events per delivered
frame" to ~1.
Items 3-5 clean up next-layer overhead (TX-side queue_work,
per-frame ba_lock, ps_state_lock under known-dead PSM). Items 6-8
are follow-ons to be re-measured after 1-3 land.
Phase 4 plan locking the lead candidate(s) follows in a separate PR.
Follow-up ftrace measurement (post-reboot, 3-min 4MB/s capture):
- workqueue_execute_start: 5,643/sec ← dominates
- wsm_cmd_send: only 13/sec (host-to-chip command path NOT the hotspot)
- lock contention: 50/sec (modest)
The throughput floor is set by per-SDIO-transaction workqueue dispatch
overhead. Surgical patches B5-1/B5-2/B5-3 from the prior Phase 4 plan
all targeted the wrong layer; deferring those until an architectural
restructuring map is produced.
Promoting the Sonnet architect review from "backlog" to
"blocking on Bug #5" — the next step is a restructuring assessment,
not another patch.
Phase 0 anchored at N=3 reps (10min @ 4MB/s pv-cap on 2.4GHz):
- rep1+2: ~700 KB/s sustained (10% of link capacity)
- rep3: link death at ~9 min in (passive mode, beacon-loss cascade)
Hot symbol identified: _raw_spin_unlock_irqrestore at ~20% CPU in both
healthy and failed reps, callstack process_one_work → wsm_configuration
→ wsm_cmd_send → bes2600_bh.isra.0 → spin-unlock.
Phase 1 metric locked: ≥2 MB/s sustained throughput, <10% CPU in lock-
cycling, no link death under 30 min continuous load.
Three Phase 4 candidates drafted (B5-1: shrink wsm_cmd_send lock scope;
B5-2: coalesce vif_list_lock in BH dispatcher; B5-3: SPSC ringbuffer for
WSM commands). Locking pending review.
Phase 7 verification of cleanups + Patch A + Patch B (srcversion
1B3B3ED0) on ohm 2026-05-07 12:48 → 15:13 CEST under netcat load
ramped 1 MB/s → 4 MB/s on 2.4GHz newton.
Patch A: predicted delta CONFIRMED at N=2 reproductions.
- 13:47:56 storm → 1 s reassoc, no AP-deauth-6 escalation
- 13:49:26 storm → 1 s reassoc, no AP-deauth-6 escalation
Patch B: installed, untriggered. 2 api_connection_loss events spaced
91 s apart, never tripping the 3-in-60s threshold. No false positives,
no spurious bus_resets. Recovery delta unobserved (no harm done).
Trigger C: 17-frame AP-deauth-6 cluster at 12:53 with no patch hooks
firing — bes2600 TX-side glitch suspect. Recovery via mac80211 reauth
in ~4 s. New backlog item.
Bug #5 documented separately (RX path degrades under throughput
pressure; possible root of the original Phase-0 YouTube frame drops).
Observed 2026-05-07: bumping the netcat sender from 1 MB/s to 4 MB/s
DECREASED ohm's observed RX rate (1015 KB/s → 563 KB/s) and degraded
the link (signal -57 → -67 dBm, MCS 4 → 3). Chip can't sustain near-
link-rate RX even though theoretical capacity is ~8 MB/s.
Hypothesis: driver/firmware lock contention or busy-wait on the RX
SDIO path. Plausibly explains the original Phase-0 observation that
YouTube DASH chunks drop ~10 frames per chunk fetch — chunk fetch is
a brief near-line-rate burst that this bug would be triggered by.
Drafts Patch A (decrypt-storm fast-recover, Trigger B) at txrx.c:1696
with sliding-window threshold + ieee80211_connection_loss reassoc.
Patch B (beacon-loss / Trigger A) parked behind one more diagnostic
rep with 10s snap-loop cadence on the beacon-loss counter.
Folds reviewer feedback from PR #3 + the new Trigger-A finding
(post-resume P1 = api_connection_loss-driven, two reps captured today
at 17:23 and 18:03) into a revised Phase 1 metric counting three
event classes.
Pending Phase 5 second-model review of the plan before Phase 6
implementation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claude-noether
test0r