Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 02d3f4b222 | |||
| 3d63ec0a35 | |||
| 722434414a | |||
| fc88ff41c3 | |||
| fde41fcdd4 | |||
| 6bae531917 | |||
| 3a38286e6f | |||
| 1e408c9d33 | |||
| d01400140b | |||
| 993117a108 | |||
| 0b63ca3c24 | |||
| 4666e03254 | |||
| f232476240 | |||
| 08c7aafb48 |
@@ -0,0 +1,108 @@
|
||||
# Bug #5 RX-degradation campaign — Phase 0
|
||||
|
||||
**Date:** 2026-05-07
|
||||
**Module under test:** v3 + F (`bes2600.ko` srcversion `371C6606B73AF19299228CA`)
|
||||
**Hardware:** ohm (PineTab2, RK3566 + BES2600 SDIO), wired enu1 fallback path live.
|
||||
|
||||
---
|
||||
|
||||
## Research question (locked)
|
||||
|
||||
> **Why does the bes2600 RX path collapse from ~2 MB/s sustained @ fresh-chip uptime to ~180 B/s @ ~28-min uptime, with periodic `wsm_generic_confirm failed for request 0x0007` + `ieee80211 phy0: [SCAN] Scan failed (-22)` every 300 s in the intervening window?**
|
||||
|
||||
Reproduces on Patch B, Patch F, and Patch C v3 alike — independent of the relay/race issues v3 addressed. Side-effect that was masked by the throughput floor while v2's race was the dominant variable.
|
||||
|
||||
## Predecessor data (reference, not anchor)
|
||||
|
||||
| source | observation |
|
||||
|---|---|
|
||||
| Patch C v3 N=3 (uptime 200/391/582 s) | mean 2.352 MB/s @ 4 MB/s sender |
|
||||
| v3 single rep at uptime ~28 min (rep 2 of 2026-05-07 22:23) | 180 KB / 5 min = 600 B/s, sender saw "Connection reset by peer" |
|
||||
| v3 single rep at uptime ~47 min (N=3 first attempt 22:42) | 55 KB / 5 min = 180 B/s, sender timed out (exit 124) |
|
||||
| dmesg pattern observed at 47-min uptime | scan failures every 301-302 s starting at uptime 778 s (~13 min) |
|
||||
|
||||
The shape: **fresh chip → linear data flow at ~2 MB/s sustained → sometime around 13 min uptime, NetworkManager-triggered scans start failing → sometime around 28 min uptime, data throughput collapses to <1 KB/s while link still shows associated.**
|
||||
|
||||
Predecessor data is reference. Phase 0 will re-anchor at N=1 long-trace + 5 in-window stress probes; if the pattern doesn't reproduce, that's the campaign result.
|
||||
|
||||
## Mechanism candidates (Phase 4 will discriminate)
|
||||
|
||||
1. **Firmware-side resource exhaustion.** Per-scan or per-WSM-event accumulation in chip-side state. Scan-failed -22 (EINVAL) suggests firmware refusing the request — possibly out of scan handles, scan-buffer slots, or some other limit.
|
||||
2. **NetworkManager scan-fail recovery loop.** Each failed scan triggers NM retry. If retry overhead dominates the bh thread, data path starves. Verifiable by suppressing NM scans.
|
||||
3. **AP-side rate limiting.** Newton (AVM) AP could be applying QoS / fairness / probation after sustained 4 MB/s burst. Verifiable by Fritz!Box log access (Markus has it) or by switching to a different AP.
|
||||
4. **PSM state machine deadlock.** c7's `pm_unsupported` self-detect was supposed to handle this, but the latch state could become stale if a real PM_IND arrives mid-operation. Verifiable by `chip_pm_state` debugfs read at degradation onset.
|
||||
5. **SDIO bus clock degradation / mmc retune.** SDIO retune with `retune_protected` flag interacts with bes2600's data path. Verifiable by ftrace `mmc/mmc_request_*` event correlation with throughput drop.
|
||||
6. **Power-management busy-event accumulation.** `bes2600_pwr_set_busy_event` counters might leak — busy events not cleared lock the chip awake (no PSM) but also exhaust event capacity. Verifiable by `bes2600_pwr_busy_event_record` dump.
|
||||
|
||||
## Phase 0 measurement protocol (rig armed 2026-05-07 23:18:58 CEST, T0=1778188738)
|
||||
|
||||
Capturing for 35 minutes from fresh boot. All capture lives in `/root/bes2600-samples/run-20260507-bug5-degradation-rig/` on ohm.
|
||||
|
||||
### Always-on streams
|
||||
|
||||
| stream | tool | output |
|
||||
|---|---|---|
|
||||
| ftrace events | per-event `enable=1` | `trace.log` (via `trace_pipe`) |
|
||||
| cfg80211 events | `iw event -t -f` | `iw-event.log` |
|
||||
| kernel printks | `dmesg -wT` | `dmesg.log` |
|
||||
| netdev counters | per-30s shell loop | `snap.log` |
|
||||
|
||||
### ftrace event set
|
||||
|
||||
- `workqueue/workqueue_execute_start` — work dispatches
|
||||
- `workqueue/workqueue_queue_work` — work submissions
|
||||
- `mac80211/api_beacon_loss` — driver beacon-loss events
|
||||
- `mac80211/api_connection_loss` — driver-side conn-loss
|
||||
- `mac80211/api_disconnect` — driver-side disconnect
|
||||
- `mac80211/drv_hw_scan` — mac80211 → driver scan dispatch
|
||||
- `mac80211/drv_set_key` — key state changes
|
||||
- `cfg80211/rdev_assoc` — assoc requests
|
||||
- `cfg80211/rdev_deauth` — deauth requests
|
||||
- `cfg80211/rdev_disassoc` — disassoc requests
|
||||
- `cfg80211/cfg80211_assoc_comeback` — AP-side assoc-busy throttling
|
||||
- `cfg80211/cfg80211_send_auth_timeout` — auth timeouts
|
||||
- `cfg80211/cfg80211_scan_done` — scan completions
|
||||
- `power/suspend_resume` — PM transitions
|
||||
- `mmc/mmc_request_start` / `mmc_request_done` — bus-level transactions
|
||||
|
||||
### Scheduled stress probes
|
||||
|
||||
Sender on boltzmann (`/tmp/bug5-probe-loop.sh`) fires `pv -L 4m | nc ohm 12345` for 30 s at T+5/10/15/20/25 min. Each probe brackets uptime, RX-bytes pre, RX-bytes post, elapsed. Throughput-vs-uptime curve falls out of the snap.log + probe boundaries.
|
||||
|
||||
Probe markers logged via `logger -t bes2600-bug5 PROBE_N_START/END` so they appear in dmesg.log timeline.
|
||||
|
||||
## Anti-theatre receipts (must tick before claiming Phase 0 done)
|
||||
|
||||
- [ ] In-session baseline: long-capture across degradation window, N=1 for now; re-run if anomalous
|
||||
- [ ] ftrace events actually firing (verify by tail of trace.log mid-capture)
|
||||
- [ ] dmesg captures the scan-failure pattern timestamp (expected ~uptime 778 s)
|
||||
- [ ] Probes actually transferred data at fresh chip (T+5 should be > 1 MB/s)
|
||||
- [ ] At least one probe in-window after scan-failure onset (expected: T+15 or T+20)
|
||||
- [ ] Snap.log shows monotonic counter behaviour (no rx_bytes going backwards)
|
||||
|
||||
## Phase 1 hypothesis (provisional, refine after Phase 3 data)
|
||||
|
||||
Metric candidate: **probe throughput as function of uptime, with state-transition markers (first `wsm_generic_confirm 0x0007 failed`, first `[SCAN] Scan failed (-22)`, first NetworkManager-deauth-and-reassociate)**.
|
||||
|
||||
Discriminator question: does throughput collapse abruptly at the first scan failure, or gradually over a window? Abrupt = single-event causation; gradual = accumulator.
|
||||
|
||||
## Phase 4 candidates (post-Phase-3)
|
||||
|
||||
Depending on which mechanism (1-6) Phase 3 surfaces:
|
||||
- (1) firmware resource exhaustion: report to upstream; possibly disable NetworkManager scans pending firmware fix.
|
||||
- (2) NM scan-fail loop: configure `wpa_supplicant` to skip scans; or add scan-failure handling in driver to dampen retry cascade.
|
||||
- (3) AP-side: switch APs for testing; report to AVM if reproducible.
|
||||
- (4) PSM deadlock: extend c7 latch with timeout-or-progress recovery.
|
||||
- (5) SDIO retune: ftrace correlation guides the lock-ordering fix.
|
||||
- (6) PWR busy-event leak: audit set/clear pairs; add a warning-when-stale.
|
||||
|
||||
## Out-of-scope
|
||||
|
||||
- Patch C v3 closure (PR #5 merged, Phase 7 done).
|
||||
- Patch C2 (`ieee80211_rx_list` batch) — gated on Task #19 kerneldoc.
|
||||
- Patch D / E independent.
|
||||
- Reproduction at higher rates (8 MB/s ramp) — defer to Phase 4 once mechanism identified.
|
||||
|
||||
---
|
||||
|
||||
*Phase 0 plan written 2026-05-07 23:21 CEST by Claude (noether), at the close of Patch C v3 Phase 7. Rig armed; long capture in flight; probes scheduled at T+5/10/15/20/25 min. Post-capture analysis will populate Phase 3 results before Phase 4 plan branches off.*
|
||||
@@ -0,0 +1,184 @@
|
||||
# Patch C — Phase 4 Plan: collapse sdio_rx_work into BH
|
||||
|
||||
**Author:** Claude (noether)
|
||||
**Status:** Phase 4 — pending Phase 5 second-model review before any Phase 6 code.
|
||||
**Scope:** **item 1 only** (per merged PR #8 inline review: "do it sequentially; we're not on the clock").
|
||||
**Item 2** (batch deliver via `ieee80211_rx_list`) splits to **Patch C2**, gated on Task #19 kerneldoc verification.
|
||||
|
||||
---
|
||||
|
||||
## §0 Substrate — anchored
|
||||
|
||||
Bug #5 anchor (recorded 2026-05-07, see `notes/phase1-bug5-2026-05-07.md`):
|
||||
|
||||
- Sender: netcat-over-WiFi, 4 MB/s cap, 2.4 GHz, AVM AP, single-STA
|
||||
- Receiver: ohm (PineTab2, RK3566 + BES2600WM-SDIO)
|
||||
- N=3 baseline reps: 725 / 663 / 75 KB/s (rep 3 saw link-death at ~9 min)
|
||||
- `perf record -g` during 4MB/s window: `_raw_spin_unlock_irqrestore` ≈ 20% CPU
|
||||
- ftrace lock-instrumentation, system-wide: `workqueue_execute_start` ≈ 5,643/sec
|
||||
- Driver-side count: `wsm_cmd_send` 13/sec — wsm command path is *not* the dispatch source; the contributor is the per-SDIO-transaction relay.
|
||||
|
||||
Root cause traced in PR #7 (Sonnet review) and concurred in PR #8 (Opus review): RX path adds two synchronization points per frame and one wait-queue wake-up per IRQ batch via `sdio_rx_work` → `rx_queue` → `bh_work` indirection.
|
||||
|
||||
## §1 Goal (locked)
|
||||
|
||||
Reduce per-RX-frame overhead enough that observed receive ≥ 1.0 MB/s sustained @ 4 MB/s sender, with `_raw_spin_unlock_irqrestore` < 15 % CPU during the 4 MB/s window. No 30-min cascade to link-death.
|
||||
|
||||
(This is a partial step toward Phase 1's full target of ≥ 2 MB/s sustained @ 4 MB/s with < 10 % lock CPU. The full target is jointly addressed by Patch C + Patch C2; Patch C alone should *cross half the gap*.)
|
||||
|
||||
## §2 Situation
|
||||
|
||||
- `bes2600.ko` srcversion `1B3B3ED0…` deployed on ohm (c-stack + Patch A + Patch B).
|
||||
- `cleanups` branch on `marfrit/bes2600-dkms` is the current source-of-truth.
|
||||
- Build sandbox `/var/tmp/c6-sandbox/` on ohm, native `make -j4`.
|
||||
- `BES2600_RX_IN_BH` is **defined** in Makefile — `bes2600_bh_rx_helper` is the active RX consumer.
|
||||
- ohm reachable. Markus pushes the reboot button; never me.
|
||||
- Test rig under `/root/bes2600-samples/` — `rep-trace.sh` per-rep capture script.
|
||||
|
||||
## §3 Baseline measurements
|
||||
|
||||
Reused from Bug #5 Phase 0 (above). No re-anchor needed for Patch C — same regime.
|
||||
|
||||
**Specific Phase-3-units that this plan's predictions reference:**
|
||||
|
||||
| metric | tool | current value (4MB/s window) |
|
||||
|---|---|---|
|
||||
| observed receive throughput | netcat receiver byte-count | 75–725 KB/s, rep-variance high |
|
||||
| `_raw_spin_unlock_irqrestore` CPU% | perf record / report | ~20% |
|
||||
| `workqueue_execute_start`/sec | ftrace `workqueue:workqueue_execute_start` | ~5,643/sec system-wide |
|
||||
| `bes_sdio` workqueue dispatches | `cat /sys/kernel/tracing/events/workqueue/.../filter` filtered by `bes_sdio` | not measured pre-patch — **TODO before Phase 6** |
|
||||
| RX SKB rate at mac80211 boundary | trace `mac80211:drv_rx_irqsafe` count | not measured pre-patch — **TODO before Phase 6** |
|
||||
|
||||
Phase 6 must not start until the two TODOs above are filled in — otherwise Phase 7 has no reference point for the predicted-delta comparison.
|
||||
|
||||
## §4 Plan
|
||||
|
||||
### §4.1 What will be touched
|
||||
|
||||
- `bes2600_sdio.c::sdio_rx_work` — the relay loop. After this patch, it still drains the SDIO bus into SKBs but **delivers SKBs directly into `wsm_handle_rx`** instead of `skb_queue_tail`-ing them onto `self->rx_queue`.
|
||||
- `bes2600_sdio.c::bes2600_sdio_extract_packets` — the inner per-SKB extractor. Changes the in-loop action from `skb_queue_tail(&self->rx_queue, skb)` to a direct call (or callback) into the wsm dispatcher.
|
||||
- `bes2600_sdio.c::bes2600_sdio_pipe_read` — becomes unused, removed.
|
||||
- `bh.c::bes2600_bh_rx_helper` — its `BES_SDIO_RX_MULTIPLE_ENABLE` branch is no longer reachable for RX (RX path no longer feeds bh). Either gate the helper, or remove the helper outright if `bh_rx` atomic is no longer raised on RX.
|
||||
|
||||
### §4.2 What will NOT be touched
|
||||
|
||||
- `ieee80211_rx_irqsafe()` call sites — that's Patch C2 (item 2).
|
||||
- TX path — `sdio_tx_work`, `bes2600_bh_tx_helper`, etc. Untouched.
|
||||
- `sdio_wq` workqueue alloc — stays. After patch it hosts only `tx_work` + `scan_work` + (briefly during patch) `rx_work`. Renaming is cosmetic and out of scope.
|
||||
- The bh thread itself — still runs, still handles TX, still watches the timeouts.
|
||||
- `bh.c` `#if 0` graveyard — separate hygiene patch, not bundled.
|
||||
- `__bes2600_irq_enable(1)` commented-out / `asm volatile("nop")` placeholder — **deferred** per `feedback_dont_patch_downstream_artifacts`. These are symptom-shaped; Patch C may dissolve them. Re-evaluate at Task #24 (post-Patch-E observation).
|
||||
- `bh_rx` / `bh_tx` atomic split — out of scope.
|
||||
|
||||
### §4.3 Approach choice — Option A (sdio_rx_work direct delivery)
|
||||
|
||||
Two structural options surveyed in PR #8 §2.1; recap:
|
||||
|
||||
| | Option A: direct delivery from sdio_rx_work | Option B: subsume sdio_rx_work into bh thread |
|
||||
|---|---|---|
|
||||
| diff size | small | medium |
|
||||
| eliminates `rx_queue->lock` × 2 per frame | yes | yes |
|
||||
| eliminates `sdio_wq.rx_work` workqueue dispatch per IRQ | no | yes |
|
||||
| changes who calls `wsm_handle_rx` | sdio_wq context (already process context) | bh thread |
|
||||
| TX/RX SDIO bus contention | unchanged (sdio_rx_work and sdio_tx_work already share `bes2600_sdio_lock`) | adds bh ↔ sdio_tx_work contention on the SDIO mutex |
|
||||
| bisection isolation | clean: only the rx_queue handoff is removed | mixes "remove handoff" with "subsume thread" |
|
||||
|
||||
**Choosing Option A.** Reasons:
|
||||
1. Smaller diff = clearer Phase-7 attribution. If RX KB/s rises, we know it was the rx_queue handoff, not the workqueue topology.
|
||||
2. Per Markus's PR #8 review: split was for bisection clarity. Option A is narrower than Option B.
|
||||
3. The remaining cost (per-IRQ `sdio_wq.rx_work` dispatch) is ≤ 1 dispatch per IRQ batch; multi-RX coalescing means several frames per dispatch. If Phase 7 of Patch C shows that dispatch IS the residual cost, that becomes a concrete data point and motivates a *measured* Option-B follow-up, not a speculative one.
|
||||
|
||||
### §4.4 Implementation sketch (preview — actual code in Phase 6)
|
||||
|
||||
**Today** (`bes2600_sdio.c:783–831`):
|
||||
```c
|
||||
static int bes2600_sdio_extract_packets(...) {
|
||||
for each packet:
|
||||
skb = dev_alloc_skb(...);
|
||||
memcpy(skb->data, &data[pos], packet_len);
|
||||
spin_lock(&self->rx_queue_lock);
|
||||
skb_queue_tail(&self->rx_queue, skb); // ← handoff
|
||||
spin_unlock(&self->rx_queue_lock);
|
||||
}
|
||||
static void sdio_rx_work(...) {
|
||||
bes2600_sdio_extract_packets(...);
|
||||
self->irq_handler(self->irq_priv); // ← wakes bh_wq
|
||||
}
|
||||
// bh thread later: pipe_read = skb_dequeue(rx_queue) → wsm_handle_rx(skb)
|
||||
```
|
||||
|
||||
**After patch** (sketch):
|
||||
```c
|
||||
static int bes2600_sdio_extract_packets(struct sbus_priv *self, u32 ctrl_reg, u8 *data) {
|
||||
for each packet:
|
||||
skb = dev_alloc_skb(...);
|
||||
memcpy(skb->data, &data[pos], packet_len);
|
||||
ret = wsm_handle_rx(self->core, wsm_id_from(skb), wsm_hdr_of(skb), &skb);
|
||||
if (skb) dev_kfree_skb(skb);
|
||||
// no rx_queue, no spinlock, no wake-up
|
||||
}
|
||||
static void sdio_rx_work(...) {
|
||||
bes2600_sdio_extract_packets(...);
|
||||
// self->irq_handler(...) is no longer called for RX-only wakes
|
||||
// (it remains called for TX-confirm-completion paths, if any)
|
||||
}
|
||||
```
|
||||
|
||||
Caveats discovered during sketch:
|
||||
- `wsm_handle_rx`'s signature wants `(hw_priv, id, wsm_hdr*, **skb)`. `extract_packets` doesn't currently parse the wsm header — we either parse it inline (cheap; the cost is one `__le16_to_cpu`) or defer parsing into a new `bes2600_sdio_deliver_rx(skb)` helper that wraps it.
|
||||
- `hw_priv` is reachable as `self->core`.
|
||||
- Need to verify `wsm_handle_rx` is callable from sdio_wq context. **Hypothesis:** yes, because today's bh thread is also process-context-via-workqueue and that's where wsm_handle_rx already runs. Phase 6 contract-cite from `wsm.h` / call-graph confirms.
|
||||
- The `irq_handler(self->irq_priv)` wakeup at sdio_rx_work:902 — keep it, but confirm whether bh actually has remaining work after RX is gone. Possibilities: TX-confirm completions (`wsm_release_tx_buffer`) still need a bh wake. Verify in Phase 6.
|
||||
|
||||
### §4.5 Predicted delta (Phase 3 units)
|
||||
|
||||
Conservative because Patch C is item 1 only, not items 1+2.
|
||||
|
||||
| metric | predicted change | confidence |
|
||||
|---|---|---|
|
||||
| `rx_queue->lock` acquire/release rate | → 0 (lock is removed entirely; struct field deleted) | high |
|
||||
| RX-path wait-queue wakes (`bh_wq` from sdio_rx_work for RX) | → 0 (TX-confirm wakes remain) | high |
|
||||
| `_raw_spin_unlock_irqrestore` CPU% | 20 % → 12–15 % | **medium** — the rx_queue lock is one of several contributors; I don't have per-lock breakdown pre-patch |
|
||||
| `workqueue_execute_start`/sec | marginal change (≤ 5 %) | high — sdio_wq dispatch still happens per IRQ |
|
||||
| observed receive @ 4 MB/s | floor lifts from 75 KB/s → ≥ 1.0 MB/s; rep-variance shrinks | **medium** — rep 3's link death has multiple causes (decrypt-storm path is Patch A's territory; AP-side `aid 30` rejection is also possible) |
|
||||
| Phase 7 N=3 outcome | all reps ≥ 1 MB/s sustained for 30 min @ 4 MB/s | **medium** |
|
||||
|
||||
**Honest acknowledgement:** the medium-confidence predictions are the ones where Phase 7 either confirms the model or surfaces a new bug. If `_raw_spin_unlock_irqrestore` only drops to 18 %, the next-largest contributor was something else — `pool->lock` (workqueue infrastructure) or `ba_lock` — and Patch D/E/C2 become the answer.
|
||||
|
||||
### §4.6 Risks
|
||||
|
||||
1. **`wsm_handle_rx` not callable from sdio_wq**: low probability (process context, same shape as today's bh), but a cite-failure here means revert to Option B. **Phase 6 must produce a `wsm.h` contract citation** before code lands.
|
||||
2. **TX-confirm wake-ups stop firing**: if `wsm_handle_rx` was the only thing that ultimately bumped `bh_tx`, removing it from bh's input causes TX-confirm starvation. Mitigation: keep `irq_handler(irq_priv)` call in sdio_rx_work for now; let the bh's wait_event re-evaluate `bh_tx` on every wake. **Verify in Phase 6 that `wsm_release_tx_buffer` still wakes bh.**
|
||||
3. **SKB allocation under memory pressure**: `dev_alloc_skb` in extract_packets currently `msleep(100)` retries up to 10×. Calling `wsm_handle_rx` directly from extract_packets keeps us in sdio_wq context during sleep; that's the same as today, so no new risk.
|
||||
4. **rcu / locking invariants in `wsm_handle_rx`**: it traverses `priv->vif_list`, may grab `priv->vif_lock`. Currently called from bh thread. After patch: called from sdio_wq context. Both are process context, both can sleep. No new risk *unless* there's a held lock at sdio_wq level that wsm_handle_rx tries to re-acquire. **Phase 6 lock-graph audit required.**
|
||||
5. **`bes2600_chrdev_is_bus_error()` early-return**: currently checked in `pipe_read`. After patch, must move into `extract_packets` or `sdio_rx_work` so RX during a bus-error window still gets dropped, not passed to mac80211.
|
||||
6. **Multi-vif RX serialization**: the `rx_queue` is per-sbus_priv, not per-vif. After patch, multi-vif demux happens inside `wsm_handle_rx` (same as today). No new risk; same ceiling.
|
||||
|
||||
### §4.7 Phase 5 review handover
|
||||
|
||||
Goal/Situation/Measurements/Plan paste verbatim into DokuWiki when Markus initiates handover. **Do not curate** the plan for the reviewer — including the "medium-confidence" predictions and the §4.6 risk list verbatim. Reviewer should see the same uncertainty I have.
|
||||
|
||||
### §4.8 Phase 7 protocol (after Phase 6 lands)
|
||||
|
||||
Per `feedback_phase7_stress_ramp.md` — **stress ramp, not steady cap**:
|
||||
|
||||
1. Pre-patch baseline (re-anchor): 5 min @ 1 MB/s, 10 min @ 2 MB/s, 30 min @ 4 MB/s. Capture ftrace `workqueue/`, `lock/`, `mac80211/`, `mmc/`. perf record during the 4 MB/s window.
|
||||
2. Apply Patch C, install, reboot (Markus pushes).
|
||||
3. Post-patch: identical ramp, identical instrumentation.
|
||||
4. Compute deltas in **the same units** as §3 baseline. Compare to §4.5 predictions. Any unexplained delta is a finding, not a footnote — log it and loop back to Phase 4 if the model is wrong.
|
||||
5. **N=3 reps** post-patch. The user's stress-ramp memory and the receipts checklist both require this.
|
||||
6. Capture `sdio_work_debug` output and `dmesg` if any storm fires (Patch A's counter should hold steady).
|
||||
7. If Phase 7 numbers match prediction → Phase 8 memory update + proceed to Patch C2.
|
||||
8. If they don't match → loop back to Phase 4. Don't paper-fix.
|
||||
|
||||
## §5 Out-of-scope items recorded for follow-on patches
|
||||
|
||||
- **Patch C2**: items 2 — `ieee80211_rx_list` batch delivery. Gated on Task #19 kerneldoc verification.
|
||||
- **Patch D**: ba_lock atomicization at `txrx.c:998-1005, 1632`. Independent.
|
||||
- **Patch E**: ps_state_lock skip when `pm_unsupported = true` at `txrx.c:1942-1948`. Independent, gated on c7 latch.
|
||||
- **Task #24**: post-Patch-E observation of bh.c `asm volatile("nop")`, commented-out `__bes2600_irq_enable(1)`, BUG_ON in steady-state hot path. Symptom-shaped; observe before patching.
|
||||
- **Task #25**: measure `sw_mci_check_r1_ready` on RK3566 during testing.
|
||||
|
||||
---
|
||||
|
||||
*Plan written 2026-05-07 by Claude (noether). Awaiting Phase 5 second-model review on DokuWiki, initiated by Markus.*
|
||||
@@ -0,0 +1,136 @@
|
||||
# Patch C v2 — Phase 4 Plan: atomic_t prep + direct-deliver
|
||||
|
||||
**Author:** Claude (noether)
|
||||
**Status:** Phase 4 v2 — Phase 7 of Patch C (notes/patch-c-phase4-plan-2026-05-07.md, PR #9 merged) failed with a thread-safety race; this is the redesign.
|
||||
**Decision:** Option B from PR #3 close-out comment — `atomic_t` prep refactor first, direct-deliver on top.
|
||||
|
||||
---
|
||||
|
||||
## §0 What just happened (Phase 7 of Patch C)
|
||||
|
||||
Reproduced verbatim from boot -1 of ohm 2026-05-07 20:18:10 CEST, ~13 s into a 4 MB/s nc stress:
|
||||
|
||||
```
|
||||
WARNING: at wsm_release_tx_buffer+0x84/0xa0 [bes2600], CPU#0: kworker/0:3H/3912
|
||||
Workqueue: bes_sdio sdio_rx_work [bes2600]
|
||||
pc : wsm_release_tx_buffer+0x84/0xa0 [bes2600]
|
||||
lr : bes2600_bh_handle_rx_skb+0x134/0x370 [bes2600]
|
||||
sdio_rx_work+0x2a8/0x540 [bes2600]
|
||||
bes2600_wlan: wsm_release_tx_buffer failed: -1
|
||||
```
|
||||
|
||||
Storm continued; chip wedged; ohm fell off the WiFi (wlan0). Patch C module preserved at `/var/tmp/bes2600.patchC-broken.ko` for forensics. Patch B rolled back, currently on disk on ohm. Lesson saved as `feedback_phase6_contract_threadsafety` memory.
|
||||
|
||||
## §1 Why it failed
|
||||
|
||||
`wsm_release_tx_buffer()` (bh.c:222–243) does **unlocked** read–modify–write on `hw_priv->hw_bufs_used`. Pre-Patch-C invariant was single-writer = BH thread; the lock that mattered was structural, not annotated. Patch C's direct-deliver moved one writer (RX-confirm decrement) into `sdio_rx_work` workqueue context. BH thread + sdio_rx_work race on the int counter; underflow below zero, WARN, return -1, bookkeeping corrupt, TX wedges.
|
||||
|
||||
Phase 6 contract block correctly cited `wsm_handle_rx`'s sleepability and held-lock invariants — but stopped at the called function's signature. It did not enumerate `hw_bufs_used` as shared state mutated by the callee. That's the gap.
|
||||
|
||||
## §2 Shared-state delta table (the thing missing from Patch C)
|
||||
|
||||
Every field that `bes2600_bh_handle_rx_skb` mutates either directly or transitively, with current protection and required action:
|
||||
|
||||
| field | declared at | written by (today) | written by (after Patch C v2) | current protection | action needed |
|
||||
|---|---|---|---|---|---|
|
||||
| `hw_priv->hw_bufs_used` | bes2600.h | `wsm_alloc_tx_buffer` (bh thread, TX submit), `wsm_release_tx_buffer` (bh thread, RX confirm), `main.c:543` (init) | + `wsm_release_tx_buffer` from sdio_rx_work | single-writer = BH thread (structural) | **convert to `atomic_t`** |
|
||||
| `hw_priv->hw_bufs_used_vif[i]` | bes2600.h | `wsm_release_vif_tx_buffer` (bh thread), `bh.c:1271` (vif TX submit), init | + `wsm_release_vif_tx_buffer` from sdio_rx_work | single-writer = BH thread | **convert to `atomic_t [N]`** |
|
||||
| `hw_priv->wsm_rx_seq[i]` | bes2600.h | bh thread RX | sdio_rx_work only | single-writer = BH/sdio_rx context (was BH, now is sdio_rx_work, but still **one writer**) | OK — single writer |
|
||||
| `hw_priv->wsm_tx_pending[i]` | bes2600.h | `bes2600_bh_inc_pending_count` (TX submit, BH thread), `bes2600_bh_dec_pending_count` (RX confirm) | dec moves to sdio_rx_work; inc stays BH | single-writer = BH | **also needs `atomic_t`** |
|
||||
| `hw_priv->lmac_mon_timer` / `mcu_mon_timer` | bes2600.h | mod_timer / del_timer_sync from BH | ditto from sdio_rx_work | timer API is internally locked | OK — `mod_timer` is concurrency-safe |
|
||||
| `hw_priv->wsm_cmd.lock` (taken inside wsm_handle_rx) | wsm_buf | bh thread (today) | sdio_rx_work | spinlock | OK — already protected |
|
||||
| `hw_priv->vif_lock` (taken inside wsm_handle_rx for some paths) | per vif | bh thread today | sdio_rx_work | spinlock | OK |
|
||||
| `priv->bh_evt_wq` wake-up | bes2600.h | wsm_release_tx_buffer when count hits 0 | ditto from sdio_rx_work | wake_up is concurrency-safe | OK |
|
||||
| `bes2600_pwr_clear_busy_event` (called inside release) | bes_pwr | bh thread | sdio_rx_work | internal locking via `bes_power.lock` | OK |
|
||||
| `hw_priv->buf_released` | bes2600.h | only `wsm_release_buffer_to_fw` (MCAST_FWDING ifdef, AP-only) | unchanged — BH only | single-writer = BH | OK — not on Patch C v2 hot path |
|
||||
|
||||
**Three fields require atomic_t conversion:** `hw_bufs_used`, `hw_bufs_used_vif[]`, `wsm_tx_pending[]`. Everything else is already concurrency-safe or moves cleanly to single-writer-in-sdio_rx_work.
|
||||
|
||||
## §3 Read-site survey (the rest of the work — atomic_read swaps)
|
||||
|
||||
`grep -hE "hw_bufs_used\b|hw_bufs_used_vif\b" *.c *.h | wc -l` = **57 references** across the source tree:
|
||||
|
||||
- 5 writers (above)
|
||||
- 52 readers — converted mechanically to `atomic_read()`. Distribution:
|
||||
- `bh.c`: 22 read sites (most in the bh main loop, BUG_ON gates, idle / suspend predicates)
|
||||
- `sta.c`: 3 sites (PM idle check at sta.c:1231–1253)
|
||||
- `bes2600_sdio.c`: 1 site (PM idle check at line 958)
|
||||
- `main.c`: 2 sites (init zero, teardown wait)
|
||||
- `debug.c`: 1 site (debugfs stats)
|
||||
- `itp.c`: 1 site (test mode)
|
||||
|
||||
`wsm_tx_pending[i]` site count is smaller — ~6 references, all in bh.c and the timer monitors. Same mechanical conversion.
|
||||
|
||||
## §4 Plan v2 — two-step
|
||||
|
||||
**Patch C-prep** (NFC, lands first):
|
||||
|
||||
- Convert `hw_bufs_used` from `int` → `atomic_t`.
|
||||
- Convert `hw_bufs_used_vif[CW12XX_MAX_VIFS]` from `int[]` → `atomic_t[]`.
|
||||
- Convert `wsm_tx_pending[2]` from `int[]` → `atomic_t[]`.
|
||||
- Update writers:
|
||||
- `wsm_alloc_tx_buffer`: `atomic_inc(&hw_priv->hw_bufs_used)`.
|
||||
- `wsm_release_tx_buffer`: rewrite with `atomic_fetch_sub_release(count, &hw_priv->hw_bufs_used)` — returns prior value. Re-derive the "tx restart" predicate (`prior >= numInpChBufs - 1`) and the "wake bh_evt_wq + clear busy" predicate (`prior - count == 0`) from that. WARN if `prior - count < 0`.
|
||||
- `wsm_release_vif_tx_buffer`: same pattern on the array element.
|
||||
- `bes2600_bh_inc/dec_pending_count`: use `atomic_inc` and `atomic_dec_return` (need post-decrement value to decide whether to del_timer).
|
||||
- Update all 52+6 read sites: mechanical `atomic_read()` swap.
|
||||
- `main.c:543` init: `atomic_set(&hw_priv->hw_bufs_used_vif[i], 0)`.
|
||||
|
||||
**Patch C-prep does NOT change behaviour.** Same atomic ordering (`_release` / `_acquire` chosen to match the implicit memory ordering the BH-only path had). Phase 7 of C-prep alone should show **identical** numbers to pre-patch baseline (`run-20260507-patchC-preflight`): 1.36 MB/s, 86.4 sdio_rx_work/sec, 90.3 dispatches per 1000 RX pkts, 0 bh_work redispatches. If Phase 7 of C-prep shows a delta, the atomic ordering is wrong and we loop back here, not to C v2.
|
||||
|
||||
**Patch C v2** (the actual structural change, lands on top of C-prep):
|
||||
|
||||
- Identical to Patch C as merged in PR #3 (since closed): direct-deliver from `bes2600_sdio_extract_packets` into `bes2600_bh_handle_rx_skb`, no `rx_queue` indirection, no bh wake-up for RX.
|
||||
- The contract block in `bh.c::bes2600_bh_handle_rx_skb` is **expanded** to include the shared-state delta table from §2 of this plan, with explicit citations.
|
||||
- Same minimum-diff scope as Patch C: keep `rx_queue`, `pipe_read`, `bh_rx_helper` for clean bisection; remove in a follow-up hygiene patch.
|
||||
|
||||
## §5 What will NOT be touched (deferred or out of scope)
|
||||
|
||||
- mac80211-side `ieee80211_rx_irqsafe` → `ieee80211_rx_list` migration: that's Patch C2, gated on Task #19 kerneldoc verification.
|
||||
- The `#if 0` graveyard in bh.c, the `asm volatile("nop")` placeholder, the BUG_ON in steady-state hot path: still symptom-shaped per `feedback_dont_patch_downstream_artifacts`. Re-evaluate at Task #24 after C v2 / D / E land.
|
||||
- `ba_lock` (Patch D) and `ps_state_lock` (Patch E): independent.
|
||||
|
||||
## §6 Risk list (per Phase 6 contract-thread-safety memory)
|
||||
|
||||
1. **C-prep memory ordering**: I've chosen `atomic_fetch_sub_release` for `wsm_release_tx_buffer` to mirror the implicit BH-thread ordering (release before subsequent atomic ops on `bh_evt_wq` / `bes_power`). If the BH thread or other readers expect `_acquire` semantics on the value, we get reordering bugs that are hard to reproduce. **Mitigation:** pair with `_acquire` reads where the read-then-decision pattern is critical (e.g., the bh main loop's `if (!hw_priv->hw_bufs_used)` idle predicate). Cite the kerneldoc reference for `atomic_fetch_sub_release` in the commit message.
|
||||
|
||||
2. **`wsm_tx_pending[]` decrement-side timer interaction**: `bes2600_bh_dec_pending_count` does `if (--hw_priv->wsm_tx_pending[idx] == 0) del_timer_sync(timer); else mod_timer(timer, ...)`. After atomic_t conversion: `if (atomic_dec_return(&hw_priv->wsm_tx_pending[idx]) == 0) ...`. But *another* thread could `atomic_inc` between our dec and the timer call, racing the del_timer. `del_timer_sync` is internally safe (it can be called concurrently with `mod_timer`), but the **decision** "whether to delete vs mod" is racy. **Mitigation:** even after atomic conversion, this function still needs to be called from a single context. Verify `inc/dec_pending_count` callers — if both sides only fire from BH and sdio_rx_work and never overlap on the same idx, we're fine; if not, this needs a lock.
|
||||
|
||||
3. **`hw_bufs_used_vif[]` array vs `wsm_alloc_tx_buffer`**: vif counter increment lives at bh.c:1271, called from bh thread TX-submit path. Decrement (`wsm_release_vif_tx_buffer`) called from RX-confirm. After Patch C v2 the decrement is in sdio_rx_work — same race shape as the global counter. Already covered by the atomic_t array conversion.
|
||||
|
||||
4. **PM idle predicate at sta.c:1239**: reads `hw_priv->hw_bufs_used_vif[priv->if_id]` to decide can-sleep. Currently racy (was already reading BH-mutated state from a non-BH PM context). Atomic conversion makes the read coherent. PM context's read-then-decide is still fundamentally a snapshot — no change in semantics, just no torn-read.
|
||||
|
||||
5. **Reboot / module-unload teardown** (`main.c:840`): `wait_event_timeout(... !hw_priv->hw_bufs_used ...)`. Becomes `... !atomic_read(...)`. No semantic change — the wait_event macro re-evaluates the predicate on each wake.
|
||||
|
||||
6. **Phase 7 rig: Patch C v2 still wedges chip if I missed anything**: now mitigated by ohm's new wired interface (enu1, 192.168.88.80) — survives bes2600 wedges, lets us collect dmesg / ftrace / journalctl from a wedged ohm without reboot. See `reference_ohm_wired_iface` memory.
|
||||
|
||||
## §7 Phase 5 review handover
|
||||
|
||||
PR on git.reauktion.de/marfrit/besser, this file as the artifact (per `feedback_phase5_surface_is_pr`). Specifically request reviewer focus on §2 shared-state delta table — that's the part that should have caught Patch C's bug. Don't curate.
|
||||
|
||||
## §8 Phase 6 implementation order
|
||||
|
||||
1. Branch off `cleanups` on bes2600-dkms-mobian: `bes2600/atomic-tx-buf-counters` (= Patch C-prep).
|
||||
2. Mechanical refactor: `int hw_bufs_used` → `atomic_t hw_bufs_used`, all reads → `atomic_read`, all writes → atomic ops. Same for vif array and tx_pending array. No other changes.
|
||||
3. Build, install, smoke-test. Phase 7 of C-prep. Should be a no-op delta.
|
||||
4. PR + Phase 5 review + merge.
|
||||
5. Branch off C-prep: `bes2600/sdio-rx-direct-deliver-v2` (= Patch C v2).
|
||||
6. Re-apply the Patch C delta (3 files: bh.h, bh.c, bes2600_sdio.c — same edits as PR #3).
|
||||
7. Build, install, Phase 7 N=3 stress ramp.
|
||||
8. PR + Phase 5 review + merge.
|
||||
|
||||
## §9 Phase 7 v2 protocol (per `feedback_phase7_stress_ramp` + wired-rig)
|
||||
|
||||
1. Pre-C-prep baseline rep N=3 (re-anchor, since current N=1 baseline is from `run-20260507-patchC-preflight`).
|
||||
2. Apply C-prep, N=3. Compare to pre. Expect: zero meaningful delta. If non-zero → memory-ordering bug, loop back to §4 atomic-ordering choice.
|
||||
3. Apply C v2, N=3. Compare to C-prep baseline. Expect: §4.5 of original Patch C plan's predicted delta (rx_queue lock acquires → 0, observed RX KB/s lifts toward ≥1 MB/s sustained @ 4MB/s).
|
||||
4. **All Phase 7 stress runs use the wired path (`ssh mfritsche@192.168.88.80`) for telemetry collection.** When the chip wedges (it shouldn't this time, but planning for it), wlan0 stops responding but enu1 stays alive. Collect dmesg / ftrace / journalctl over enu1 BEFORE rebooting. This is the data we lost in Patch C boot -1 because wlan0 was the only path.
|
||||
5. N=3 reps per phase per `feedback_phase7_stress_ramp`. Don't accept N=1 as verification.
|
||||
|
||||
## §10 Closeout
|
||||
|
||||
If C-prep + C v2 both pass Phase 7: proceed to D (ba_lock atomicization), E (ps_state_lock skip). Markus's "we're not on the clock" applies — sequencing per bisection clarity, not delivery deadline.
|
||||
|
||||
---
|
||||
|
||||
*Plan written 2026-05-07 by Claude (noether), in response to Patch C Phase 7 failure. Phase 5 review = PR comments on this artifact at git.reauktion.de/marfrit/besser. Don't curate the shared-state delta table for the reviewer — that's the part the previous round's reviewer should have caught me on.*
|
||||
@@ -0,0 +1,127 @@
|
||||
# Patch C v3 — Phase 4 Plan: drop sdio_rx_work, match cw1200 architecture
|
||||
|
||||
**Author:** Claude (noether)
|
||||
**Status:** Phase 4 v3 — supersedes v2 (PR #10) after cw1200 mainline survey showed the race-free path is structural, not lock-based.
|
||||
**Decision:** drop the `sdio_rx_work` workqueue entirely; SDIO IRQ wakes `bh_wq`; bh thread does the SDIO read inline. Restores single-writer-from-bh invariant on `hw_bufs_used` *by construction*. No `atomic_t` prep needed.
|
||||
|
||||
---
|
||||
|
||||
## §0 Why v3 supersedes v2
|
||||
|
||||
PR #10's plan was: convert `hw_bufs_used` etc. to `atomic_t` (prep), then direct-deliver from `sdio_rx_work` (structural). That was a workaround for the race that *only existed because of the relay*.
|
||||
|
||||
The cw1200 mining (`~/src/linux-rockchip`, 228 cw1200 commits) showed the upstream answer: there is no relay. cw1200's IRQ handler bumps `bh_rx` and wakes the bh thread; the bh thread does the SDIO read itself inside `cw1200_bh_rx_helper` (`drivers/net/wireless/st/cw1200/bh.c:233`). Single thread = single writer for `hw_bufs_used` = no race. Same `int hw_bufs_used` as bes2600, never atomic_t'd in 16 years upstream because it never needed to be.
|
||||
|
||||
Patch C v3 brings bes2600 into that shape. The structural simplification is bigger than v2's diff but lands the right architecture in one move.
|
||||
|
||||
## §1 Goal
|
||||
|
||||
Same as Patch C v2 §1: ≥ 1 MB/s sustained receive @ 4 MB/s sender, < 15 % `_raw_spin_unlock_irqrestore` CPU%, no 30-min cascade to link-death. Stretch toward Phase 1's full 2 MB/s once Patch C2 (rx_list batch) lands separately.
|
||||
|
||||
## §2 Situation
|
||||
|
||||
- Cleanups branch is at Patch F merged (commit `b717251`). All Phase 5 reviews of the F series merged via PR #4.
|
||||
- ohm rebooted with F module live (srcversion `A9438692D6A8698F92AEEA1`) — F is the new baseline for Patch C v3 Phase 7 comparison.
|
||||
- Wired path `enu1` at `192.168.88.80` survives bes2600 wedges; lmcp `ohm` still goes through wlan0. Phase 7 telemetry collection over enu1.
|
||||
- Reboot-permission override active (ohm dev-allocated; I can `sudo reboot` directly — `feedback_user_pushes_reboot_button` override clause).
|
||||
|
||||
## §3 Baseline measurements
|
||||
|
||||
Carry forward from `run-20260507-patchC-preflight/baseline.tsv` (N=1, F-less Patch B module):
|
||||
|
||||
| metric | value |
|
||||
|---|---|
|
||||
| observed receive @ 4 MB/s | 1.362 MB/s |
|
||||
| sdio_rx_work dispatches | 86.4/s = 90.3 per 1000 RX packets |
|
||||
| sdio_tx_work dispatches | 276.1/s |
|
||||
| bes2600_bh_work redispatches | 0 (single long-lived) |
|
||||
|
||||
**Phase 6 prereq:** capture an N=3 baseline ON THE F MODULE before Patch C v3 code lands. Same instrumentation, same stress ramp. This is the post-F / pre-v3 reference. Without it, Phase 7's delta is C+F vs B+nothing — confounded.
|
||||
|
||||
## §4 Plan v3
|
||||
|
||||
### §4.1 What gets eliminated
|
||||
|
||||
- **`sdio_rx_work` (bes2600_sdio.c:829)** — function deleted. No longer queued, no longer runs.
|
||||
- **`self->rx_work` work_struct** — field deleted from `struct sbus_priv`. `INIT_WORK` removed.
|
||||
- **`self->rx_queue` + `self->rx_queue_lock`** — fields deleted. `skb_queue_head_init` removed. No SKB ever queued there.
|
||||
- **`bes2600_sdio_pipe_read`** — function deleted. No callers after this patch.
|
||||
- **`sbus_ops->pipe_read`** — sbus op slot deleted (or kept and stubbed; tx_loop.c also implements it for the test-loop bus, has to stay if test-loop is preserved).
|
||||
- **`queue_work(self->sdio_wq, &self->rx_work)`** at the 3 call sites in `bes2600_sdio.c` (lines 416, 941, 1199) — removed.
|
||||
|
||||
### §4.2 What gets added
|
||||
|
||||
- **A new `bes2600_bh_handle_rx_skb()`** in bh.c (same shape as Patch C added, same contract block; no longer needs to also wake the bh thread because we ARE the bh thread).
|
||||
- **A new helper `bes2600_sdio_read_rx_batch()`** in bes2600_sdio.c, exported, that does what `sdio_rx_work` used to do MINUS the queuing: lock → read ctrl_reg → memcpy_fromio → packets_check → for-each-frame extract+deliver. Called from bh.
|
||||
|
||||
### §4.3 What gets rewired
|
||||
|
||||
- **`bes2600_gpio_irq_handler`** in bes2600_sdio.c:413 (the GPIO-IRQ path used when CONFIG_BES2600_USE_GPIO_IRQ is set): drop `queue_work(self->sdio_wq, &self->rx_work)`; instead call `self->irq_handler(self->irq_priv)` directly (which is `bes2600_irq_handler` in bh.c, bumps `bh_rx` + wakes `bh_wq`). Matches cw1200_sdio_irq_handler shape.
|
||||
- **`bes2600_bh_rx_helper`** (bh.c:961, BES_SDIO_RX_MULTIPLE_ENABLE branch): instead of `pipe_read`-ing one SKB from the (now-gone) rx_queue, call the new `bes2600_sdio_read_rx_batch()` which does the SDIO read AND delivers each frame inline via `bes2600_bh_handle_rx_skb()`. Returns count delivered, or negative on error.
|
||||
- **`bes2600_bh()` outer loop**: after a successful rx_batch read, the helper signals whether to continue draining (more frames pending) — same shape as today's `BH_RX_CONT_LIMIT=3` outer loop.
|
||||
- **`bes2600_gpio_wakeup_mcu(SDIO_RX)`** + **`bes2600_gpio_allow_mcu_sleep(SDIO_RX)`** brackets: currently called inside sdio_rx_work. Move into bh thread around the `bes2600_sdio_read_rx_batch()` call. Same wake-flag bracketing, just from a different thread.
|
||||
- **`sdio_wq` workqueue**: keeps `tx_work` and (briefly) `scan_work`. Renamed or kept — cosmetic. Don't touch in this patch.
|
||||
|
||||
### §4.4 What stays untouched
|
||||
|
||||
- TX path (`sdio_tx_work`, `bes2600_bh_tx_helper`, `wsm_alloc_tx_buffer`). Independent.
|
||||
- WSM protocol layer (`wsm.c`, `wsm_handle_rx`). Same callees, just from bh thread now.
|
||||
- mac80211 RX delivery (`ieee80211_rx_irqsafe`). That's Patch C2.
|
||||
- `BES2600_RX_IN_BH` ifdef gate. Stays defined; the gated branch is now the only RX path.
|
||||
- Symptom-shaped artifacts (asm nop, BUG_ON in hot path) — still deferred, see task #24 post-cleanup.
|
||||
|
||||
## §5 Shared-state delta table (the v2 lesson, applied)
|
||||
|
||||
Every field `bes2600_bh_handle_rx_skb` mutates directly or transitively, with the v3 protection:
|
||||
|
||||
| field | written by (today) | written by (after v3) | concurrency | required action |
|
||||
|---|---|---|---|---|
|
||||
| `hw_priv->hw_bufs_used` | bh thread (TX submit + RX confirm), main.c init | **bh thread only** (RX moves into bh) | single-writer | none — `int` is fine, race-free by construction |
|
||||
| `hw_priv->hw_bufs_used_vif[i]` | bh thread (TX vif submit + RX vif confirm), main.c init | **bh thread only** | single-writer | none |
|
||||
| `hw_priv->wsm_rx_seq[i]` | sdio_rx_work today | bh thread | single-writer | none — moves cleanly between contexts |
|
||||
| `hw_priv->wsm_tx_pending[i]` | bh thread (inc on TX submit), bh+sdio_rx_work (dec on RX confirm) | **bh thread only** | single-writer | none |
|
||||
| `hw_priv->lmac_mon_timer` / `mcu_mon_timer` | mod_timer / del_timer_sync from bh + sdio_rx_work | bh thread only | timer API safe anyway | none |
|
||||
| `hw_priv->wsm_cmd.lock` | spinlock taken inside wsm_handle_rx | same | already protected | none |
|
||||
| `priv->bh_evt_wq` wake-up | wsm_release_tx_buffer when count→0 | same | wake_up is concurrency-safe | none |
|
||||
| `bes_pwr.lock` (inside bes2600_pwr_clear_busy_event) | bh thread (today) | bh thread | already protected | none |
|
||||
| `self->rx_data_cnt` etc. (sbus_priv stats) | sdio_rx_work | bh thread | single-writer | none |
|
||||
|
||||
**Zero fields require new locking.** The architectural pivot eliminates the race v2's atomic_t was working around.
|
||||
|
||||
## §6 Risks
|
||||
|
||||
1. **bh thread now holds the SDIO bus mutex during read** (currently held by sdio_rx_work). TX work in the same bh thread is unaffected (sdio_tx_work runs on a separate workqueue and shares the same mutex anyway). The sdio_lock contention pattern doesn't change.
|
||||
2. **Loss of "parallelism" between sdio_rx_work and bh TX**: sdio_rx_work and bh thread *appeared* to run in parallel today, but both serialize through `bes2600_sdio_lock(self)` for the actual bus operations. The parallelism was illusory. Net throughput should not regress.
|
||||
3. **bh thread CPU-busy-time per RX batch increases**: inline SDIO read is the same cost, just charged to bh instead of sdio_wq's worker. Mitigation: the per-IRQ workqueue dispatch cost (~86/s) is what we trade for it. Net: -86 dispatches/s, +0 µs per frame.
|
||||
4. **Multi-RX coalescing (BES_SDIO_RX_MULTIPLE_NUM=16)** stays. bes2600_sdio_extract_packets parses the multi-frame buffer same as before, just inline now. No functional change to chip-side behaviour.
|
||||
5. **GPIO wake-flag bracketing**: `bes2600_gpio_wakeup_mcu(SDIO_RX)` and `bes2600_gpio_allow_mcu_sleep(SDIO_RX)` currently bracket sdio_rx_work. Move them to bracket the new bh-side read. If the wake-flag accounting is sub-system-scoped (it is — flag bits per subsystem), this is a clean move.
|
||||
6. **IRQ re-enable in bh thread**: cw1200's bh re-enables IRQ via `__cw1200_irq_enable(priv, 1)` after each round. bes2600 has the analogous `__bes2600_irq_enable(0/1)` (commented out as the `asm volatile("nop")` symptom in `bh.c:1518-1520`). This patch does NOT re-engage the commented-out re-enable — that's still task #24's call. But if the IRQ stays disabled across rounds, we'd never receive the next IRQ. **Investigate before Phase 6 lands**: where does IRQ re-enable happen in the current bes2600 hot path? The sdio_func IRQ may be auto-managed by sdio core differently. Block Phase 6 on this audit.
|
||||
7. **Phase 7 wedge resilience**: if v3 has a different bug shape than v2's race (which it shouldn't, since the race is gone by construction), the wired path lets us collect telemetry from a wedged ohm.
|
||||
|
||||
## §7 Phase 5 / 6 / 7
|
||||
|
||||
- **Phase 5**: PR on `git.reauktion.de/marfrit/besser` with this artifact. Specifically request reviewer focus on §6 risk #6 (IRQ re-enable mechanism).
|
||||
- **Phase 6**: branch off cleanups (post-F): `bes2600/sdio-rx-no-relay`. Implement the file changes per §4. Build, install, smoke-test.
|
||||
- **Phase 7**:
|
||||
- First: N=3 stress-ramp **on F module** (post-F pre-v3 baseline). 10 min @ 1, 30 min @ 2, 30 min @ 4 MB/s. Use wired path for telemetry.
|
||||
- Then: install v3 module, identical N=3 ramp. Compare deltas.
|
||||
- Predicted: sdio_rx_work dispatch rate → 0/s (was 86/s). observed receive lifts toward ≥ 1.0 MB/s sustained. `_raw_spin_unlock_irqrestore` drops by the rx_queue lock contribution (was 1914/s acquires).
|
||||
|
||||
## §8 What gets dropped from v2 plan
|
||||
|
||||
- atomic_t prep refactor (`hw_bufs_used` → `atomic_t`): not needed. Single-writer invariant preserved structurally. Still a defensible standalone hardening patch *if mainlining bes2600 ever requires defense-in-depth*, but not on the Bug-#5 critical path.
|
||||
- `wsm_tx_pending[]` decrement-decision race (v2 risk #2): also moots. Both sides single-thread under v3.
|
||||
- v2 Phase 7's "C-prep should show zero delta" gate: replaced by "v3 should match cw1200's structural shape" gate.
|
||||
|
||||
## §9 Open question for reviewer
|
||||
|
||||
The big one is §6 risk #6 — IRQ re-enable. cw1200 explicitly does `__cw1200_irq_enable(priv, 1)` from bh after each round; bes2600 has the call **commented out** with an `asm volatile("nop")` placeholder. Either:
|
||||
|
||||
(a) bes2600's SDIO IRQ is level-triggered + auto-acked by SDIO core, so re-enable isn't needed (that would explain the nop).
|
||||
(b) The current code happens to work because sdio_rx_work is queued by the IRQ regardless of whether IRQ is "enabled" by the driver-side flag. After v3 we have to manually re-enable like cw1200 does.
|
||||
|
||||
Need to confirm (a) vs (b) before Phase 6 lands. Plan to grep for `__bes2600_irq_enable` callsites and trace back to whether it's load-bearing.
|
||||
|
||||
---
|
||||
|
||||
*Plan written 2026-05-07 by Claude (noether), after Patch F merged and Patch C v2 (PR #10) was superseded by the cw1200 architectural mining finding. Phase 5 review on PR. Don't curate.*
|
||||
@@ -0,0 +1,171 @@
|
||||
# Patch C2 — Phase 4 Plan: migrate ieee80211_rx_irqsafe → ieee80211_rx_list
|
||||
|
||||
**Author:** Claude (noether)
|
||||
**Status:** Phase 4 — pending Phase 5 PR review before any Phase 6 code.
|
||||
**Predecessor:** Patch C v3 (PR #5 merged, +73% throughput, no-relay architecture); Patch D + E + F + G also landed. Cleanups branch tip = 42fd0ce.
|
||||
**Task #19 contract**: `ieee80211_rx_list` callable from process context, **requires `local_bh_disable()` + `rcu_read_lock()` wrap**, **cannot mix with `ieee80211_rx_irqsafe()` for the same hardware** → all 6 sites convert in one shot.
|
||||
|
||||
---
|
||||
|
||||
## §0 Substrate
|
||||
|
||||
After Patch C v3:
|
||||
- bh thread is the sole RX-delivery context (no relay, no sdio_rx_work)
|
||||
- Per-frame work runs in process context (sleepable)
|
||||
- Single-writer-from-bh invariant covers `hw_bufs_used` and friends
|
||||
|
||||
`ieee80211_rx_irqsafe` is currently called from process context. Per kerneldoc (`include/net/mac80211.h:5399-5411`):
|
||||
|
||||
> **Like ieee80211_rx() but can be called in IRQ context** (internally defers to a tasklet.)
|
||||
|
||||
The tasklet hop is the cost we pay today for delivering each RX frame from process context. `ieee80211_rx_list` is the process-context replacement.
|
||||
|
||||
## §1 Goal
|
||||
|
||||
Per-frame: skip the tasklet hop. Batch: process multiple SKBs from one SDIO read inside a single `local_bh_disable()`/`rcu_read_lock()` window.
|
||||
|
||||
Phase 1 metric: **RX throughput @ 4 MB/s sender**, with v3 N=3 baseline = 2.352 MB/s. Hypothesis: small to moderate uplift (<10%) from removing the tasklet deferral. Larger improvement would be surprising — if observed, that's a finding to investigate.
|
||||
|
||||
## §2 Situation
|
||||
|
||||
- 6 call sites in bes2600 currently use `ieee80211_rx_irqsafe`:
|
||||
- `ap.c:96` (AP-mode link-id RX queue drain)
|
||||
- `sta.c:1487` (link-id rx_queue drain in ?)
|
||||
- `txrx.c:1960` (early-data + pm_unsupported branch — Patch E added)
|
||||
- `txrx.c:1967` (early-data + LINK_SOFT-not-set branch)
|
||||
- `txrx.c:1971` (normal RX path)
|
||||
- `wsm.c:2415` (beacon SKB delivery from `bes2600_beacon_handler`?)
|
||||
- All 6 must convert together (kerneldoc: cannot mix per hardware)
|
||||
- bh thread is single-writer post-v3 → `_rx_list`'s "calls must be synchronized" satisfied trivially
|
||||
- bh thread is process context → `_rx_list` callable
|
||||
|
||||
## §3 Baseline (carry forward)
|
||||
|
||||
From `notes/phase7-v3-2026-05-07.md` (v3 N=3 ramp, Phase 7 closed):
|
||||
|
||||
| metric | v3 fresh-chip N=3 |
|
||||
|---|---|
|
||||
| RX throughput @ 4 MB/s | mean 2.352 MB/s, min 2.102, max 2.590 |
|
||||
| sdio_rx_work dispatches | 0/s |
|
||||
| bh_work redispatches | 0 |
|
||||
|
||||
Phase 7 of C2 will compare against this baseline.
|
||||
|
||||
## §4 Plan
|
||||
|
||||
### §4.1 Conversion shape
|
||||
|
||||
Per call site:
|
||||
```c
|
||||
ieee80211_rx_irqsafe(priv->hw, skb);
|
||||
```
|
||||
becomes:
|
||||
```c
|
||||
ieee80211_rx_list(priv->hw, NULL, skb, &priv->rx_list);
|
||||
```
|
||||
|
||||
Where `priv->rx_list` is a `struct list_head` initialized once.
|
||||
|
||||
**Wrap requirement:** `local_bh_disable()` + `rcu_read_lock()` must be held across the call. Per the kerneldoc, that's also needed for batch correctness.
|
||||
|
||||
### §4.2 Wrap placement (the design decision)
|
||||
|
||||
**Option A — per-call wrap.** Wrap each individual `ieee80211_rx_list()` call. Simple but loses the batch benefit (each call's wrap+unwrap costs as much as the avoided tasklet defer).
|
||||
|
||||
**Option B — per-batch wrap.** Wrap the OUTER frame-iteration loop (e.g., the `for` in `bes2600_sdio_extract_packets`). All 16 SKBs from one SDIO read get delivered inside one wrap. This is the upstream-idiomatic pattern (mt76, iwl_pcie do this).
|
||||
|
||||
Choosing **Option B**. Concrete shape:
|
||||
|
||||
- `bes2600_sdio_read_rx_batch` (the per-SDIO-batch entry point added in Patch C v3) wraps the read+extract+deliver phase:
|
||||
```c
|
||||
rcu_read_lock();
|
||||
local_bh_disable();
|
||||
// existing read + extract_packets that calls bh_handle_rx_skb per frame
|
||||
local_bh_enable();
|
||||
rcu_read_unlock();
|
||||
```
|
||||
- Inside `bes2600_bh_handle_rx_skb`, the single `ieee80211_rx_irqsafe` swap becomes `ieee80211_rx_list(priv->hw, NULL, skb, &priv->rx_list)`.
|
||||
- The OTHER 5 call sites (in `ap.c`, `sta.c`, `txrx.c`'s branches, `wsm.c`) need the same treatment, but they're called from the bh thread (post-v3) so they're already in the right context. Each gets its own narrow wrap (Option A applied selectively because those paths process one frame at a time, not a batch).
|
||||
|
||||
### §4.3 The `rx_list` field
|
||||
|
||||
Add `struct list_head rx_list` to either `struct bes2600_common` (driver-wide) or `struct bes2600_vif` (per-vif). Per-vif is cleaner because the existing `priv->hw` parameter implies vif scope.
|
||||
|
||||
`INIT_LIST_HEAD(&priv->rx_list)` at vif setup; no teardown needed (mac80211 owns the SKBs once handed off).
|
||||
|
||||
**Open question for reviewer:** does the `rx_list` need to be drained explicitly after the batch (e.g., via a `list_for_each_entry_safe` + `netif_receive_skb_list_internal`)? Looking at mainline mt76 / iwl_pcie usage will clarify. Phase 6 must answer this before code lands.
|
||||
|
||||
### §4.4 What will NOT be touched
|
||||
|
||||
- The 6 call sites change atomically (all-or-nothing per kerneldoc) — no per-site progressive migration
|
||||
- `wsm.c:2415` beacon path: same conversion shape, but beacon delivery is once-per-beacon-interval (not hot path); could stay `_irqsafe` if upstream allows mixing per-SKB-type. Re-read kerneldoc carefully — it says "per hardware", not per-call-site, so we can't keep _irqsafe even on the slow paths.
|
||||
- bh thread structure (Patch C v3 stands)
|
||||
- atomic_t counters from Patch D
|
||||
- `pm_unsupported` lock-skip from Patch E
|
||||
- mac80211 batch-delivery semantics (mainline owns this; we just call the API)
|
||||
|
||||
### §4.5 Predicted delta in Phase 3 units
|
||||
|
||||
| metric | predicted |
|
||||
|---|---|
|
||||
| `rx_irqsafe` tasklet schedule rate | → 0 (function no longer called) |
|
||||
| RX throughput @ 4 MB/s sustained | 2.352 → +5-15% (medium confidence) |
|
||||
| `_raw_spin_unlock_irqrestore` CPU% | small drop (no tasklet schedule lock contribution) |
|
||||
|
||||
**Honest acknowledgment:** I don't have data on how much the tasklet hop actually costs. The improvement might be smaller than predicted if tasklet defer was already cheap on this kernel. If <2%, Phase 7 says "marginal but no regression" and we ship anyway for upstream-cleanliness.
|
||||
|
||||
### §4.6 Risks
|
||||
|
||||
1. **`ieee80211_rx_list` semantics surprise.** mainline drivers I have access to (mt76, iwl_pcie) use this via NAPI infrastructure. bes2600 doesn't have NAPI; we're doing process-context-direct. The kerneldoc says callable that way but we should verify a few mainline drivers actually do it. **Phase 6 contract-cite from at least one upstream caller** before code lands.
|
||||
|
||||
2. **`rx_list` lifetime in cross-batch / cross-vif scenarios.** Multiple vifs (P2P_MULTIVIF=y in Makefile) might race on the same hw's `rx_list`. The kerneldoc says "for a single hardware" — the list is per-call destination, which means each call appends to its argument list. Per-vif `rx_list` per-call is the natural shape. No per-hw aggregator needed.
|
||||
|
||||
3. **`local_bh_disable` cost in batch wrap.** Not free. If the batch is small (1-2 SKBs), the wrap might dominate. Estimated breakeven: 2-3 SKBs per wrap. Phase 7 should look at SKB-per-batch distribution to confirm.
|
||||
|
||||
4. **`rcu_read_lock` across SDIO read.** SDIO read can take multi-ms (multi-block transfers). RCU reader-cs across that is fine (no preemption blocked) but it's a longer reader-cs than typical. Verifiable but not a blocker — kerneldoc requires it.
|
||||
|
||||
5. **wsm.c:2415 (beacon) is a different SKB lifecycle** — `hw_priv->beacon` is owned by hw_priv, not allocated per-call. After `_rx_list` consumes it (by passing ownership to mac80211), `hw_priv->beacon` is dangling. **Phase 6 must verify the beacon path either reallocates after delivery or wasn't actually transferring ownership.** Risk #5 is the biggest open question.
|
||||
|
||||
### §4.7 Phase 5 review handover
|
||||
|
||||
PR on `git.reauktion.de/marfrit/besser` with this artifact. Specifically request reviewer focus on:
|
||||
- §4.2 wrap-placement choice (Option B vs A)
|
||||
- §4.3 rx_list scoping (per-vif)
|
||||
- §4.6 risks #1 (mainline-caller verification) and #5 (beacon path SKB ownership)
|
||||
|
||||
Don't curate.
|
||||
|
||||
### §4.8 Phase 6 implementation order
|
||||
|
||||
1. Branch off cleanups: `bes2600/rx-list-batch-delivery`
|
||||
2. Add `struct list_head rx_list` to `struct bes2600_vif`, `INIT_LIST_HEAD` in vif setup
|
||||
3. Convert all 6 call sites: `ieee80211_rx_irqsafe(...)` → `ieee80211_rx_list(...)`
|
||||
4. Wrap `bes2600_sdio_read_rx_batch` outer loop with `rcu_read_lock + local_bh_disable / local_bh_enable + rcu_read_unlock`
|
||||
5. For the non-bh-thread call sites (ap.c, sta.c, wsm.c beacon): per-call narrow wrap
|
||||
6. Verify beacon path in wsm.c:2415 (Risk #5)
|
||||
7. Build, install, smoke-test
|
||||
8. Phase 7 N=3 stress ramp — compare to v3 baseline
|
||||
|
||||
### §4.9 Phase 7 protocol (per `feedback_phase7_stress_ramp`)
|
||||
|
||||
- N=3 reps, 30s each at 4 MB/s, fresh-chip (uptime <15 min)
|
||||
- Use wired path (`ssh mfritsche@192.168.88.80`) for telemetry
|
||||
- Fresh nc listener per rep (per `feedback_rig_failure_is_finding`)
|
||||
- Compare: throughput delta + tasklet schedule rate (ftrace `irq:tasklet_*` events)
|
||||
- If predicted delta met → close C2 + memory entry
|
||||
- If NO delta → marginal patch but no regression; ship for upstream-cleanliness
|
||||
|
||||
## §5 Out of scope
|
||||
|
||||
- Patch D / E already shipped (PR #7, #8 merged)
|
||||
- Patch G already shipped (PR #6 merged)
|
||||
- bh.c `#if 0` graveyard removal (Task #24 hygiene)
|
||||
- Allwinner `sw_mci_check_r1_ready` (Task #25)
|
||||
|
||||
## §6 Summary
|
||||
|
||||
C2 is a 6-site mechanical migration with ONE design decision (per-batch wrap), TWO open questions for the reviewer (rx_list draining + beacon path SKB ownership), and SMALL expected throughput delta (<15%). Risk-low, upstream-prep-high. Worth shipping for the kernel.org submission story even if the throughput delta is marginal.
|
||||
|
||||
---
|
||||
|
||||
*Plan written 2026-05-08 by Claude (noether). Phase 5 review on PR. Phase 6 contingent on review passing.*
|
||||
@@ -0,0 +1,63 @@
|
||||
# Patch C2 Phase 7 — N=3 ramp results
|
||||
|
||||
**Date:** 2026-05-08
|
||||
**Module:** `bes2600.ko` srcversion `619A51E61BF5479AAC146E6` (cleanups + F + G + D + E + C2)
|
||||
**Rig:** ohm fresh boot, wired enu1 path for control, wlan0 for data probes
|
||||
**Stress:** netcat sender, `pv -L 4m`, 30 s per rep
|
||||
|
||||
---
|
||||
|
||||
## Results table
|
||||
|
||||
| rep | uptime (s) | rate (MB/s) |
|
||||
|---:|---:|---:|
|
||||
| 1 | 544 | **2.289** |
|
||||
| 2 | 716 | **2.165** |
|
||||
| 3 | 750 | **2.376** |
|
||||
|
||||
**N=3:** mean 2.277, median 2.289, min 2.165, max 2.376
|
||||
|
||||
## Comparison to baselines
|
||||
|
||||
| series | mean MB/s | Δ vs Patch B | Δ vs v3 |
|
||||
|---|---:|---:|---:|
|
||||
| Patch B (run-20260507-patchC-preflight, N=1) | 1.362 | — | -42% |
|
||||
| Patch C v3 N=3 (run-20260507-N3v3-rep*) | 2.352 | +73% | — |
|
||||
| Patch C v3 + F + G + D + E + C2 N=3 (this rep set) | 2.277 | +67% | -3% |
|
||||
|
||||
Δ vs v3 is **within rep variance** (v3 N=3 had min 2.102, max 2.590 → spread ±20%; this set's spread is similar). Statistically indistinguishable.
|
||||
|
||||
## Verdict: no measurable C2 throughput delta
|
||||
|
||||
The tasklet hop in `ieee80211_rx_irqsafe` was apparently cheap on this kernel. Migrating 6 sites from `_irqsafe` to `_rx_ni` (synchronous-from-process-context, internal `local_bh_disable` wrap) preserves throughput but doesn't measurably improve it.
|
||||
|
||||
**This was a predicted outcome.** The C2 Phase 4 plan §4.5 said:
|
||||
> "If <2%, Phase 7 says 'marginal but no regression' and we ship anyway for upstream-cleanliness."
|
||||
|
||||
Observed: -3% (within noise) → falls into the "marginal but no regression" bucket. Ship for the kernel.org submission story (no `_irqsafe` from process context = upstream-idiomatic) even though performance is unchanged.
|
||||
|
||||
## Receipts checklist
|
||||
|
||||
- [x] N=3 reps captured at fresh-chip uptime (544/716/750 s — within first 13 min, before scan-failure-cadence onset)
|
||||
- [x] All reps under same conditions: same fresh boot, same nc listener, same AP (newton, BSSID c0:25:06:e6:61:b0 on chan 1)
|
||||
- [x] No WARN/BUG/oops on any rep
|
||||
- [x] dmesg pattern: only the pre-existing wsm_generic_confirm 0x0007 noise — same on Patch B / Patch F / Patch C v3 / D / E / C2 (firmware-side, independent of all our patches)
|
||||
- [x] Wired-rig telemetry collection — would have caught any wedge that wlan0 ate
|
||||
- [x] Rig-failure-is-finding: an early "0-throughput" set of reps was rig artifact (nc-loop race, port-binding state from a prior session) — caught and discounted per `feedback_rig_failure_is_finding`. The recovered N=3 reps used setsid-detached listener + post-reboot fresh state.
|
||||
|
||||
## Phase 8 lesson
|
||||
|
||||
**Drop-in replacements with the right kerneldoc reading still need Phase 7 measurement.** I expected +5-15% from removing the tasklet schedule. Got -3% (noise). The cost we were saving was already amortised by something else (NAPI infra? per-CPU softirq scheduling?). The kerneldoc-correctness story stands; the perf story does not.
|
||||
|
||||
**Memory entry:** the perf-vs-correctness distinction is worth keeping. `_irqsafe → _rx_ni` is a CORRECTNESS / API-cleanliness move, not a performance optimization. Don't oversell predicted deltas without baseline measurement.
|
||||
|
||||
## Out-of-scope follow-ups
|
||||
|
||||
- Patch C v3 architectural win is the durable +73%. C / D / E / C2 / F / G are smaller cleanups that don't compound visibly.
|
||||
- Bug #5 RX-degradation campaign already closed (hypothesis falsified).
|
||||
- Task #24 (post-cleanup observation of bh.c symptom-shaped artifacts): mostly answered.
|
||||
- Task #25 (Allwinner sw_mci_check_r1_ready measurement): can be done during any future stress run; not on critical path.
|
||||
|
||||
---
|
||||
|
||||
*Phase 7 captured 2026-05-08 by Claude (noether). Patch C2 closes the post-Bug-#5 cleanup track. Throughput ceiling on this hardware = ~2.4 MB/s sustained @ 4 MB/s sender, fresh chip; further improvement would need firmware-side fixes (the wsm_generic_confirm 0x0007 path), not driver-side.*
|
||||
@@ -0,0 +1,94 @@
|
||||
# Patch C v3 Phase 7 — N=3 verification results
|
||||
|
||||
**Date:** 2026-05-07
|
||||
**Module:** `bes2600.ko` srcversion `371C6606B73AF19299228CA` (cleanups+F+v3)
|
||||
**Rig:** ohm (PineTab2, RK3566 + BES2600 SDIO), wired enu1 path for telemetry
|
||||
**Stress:** netcat sender from boltzmann, `pv -L 4m` rate cap (4 MB/s), 3-min window per rep
|
||||
**Boot:** fresh — uptime 200 s / 391 s / 582 s at rep 1/2/3 starts (all within fresh-chip window before the ~13-min Bug #5 RX-degradation point)
|
||||
|
||||
---
|
||||
|
||||
## Results table
|
||||
|
||||
| rep | elapsed (s) | RX bytes | RX MB | MB/s | sdio_rx_work | sdio_tx_work | bes2600_bh_work redispatches |
|
||||
|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||
| 1 | 180.72 | 447,758,333 | 427.0 | **2.363** | 0 | 368 | 0 |
|
||||
| 2 | 180.67 | 490,669,836 | 467.9 | **2.590** | 0 | 20 | 0 |
|
||||
| 3 | 180.69 | 398,224,992 | 379.8 | **2.102** | 0 | 39 | 0 |
|
||||
|
||||
**N=3 stats:** mean 2.352 MB/s · median 2.363 MB/s · min 2.102 MB/s · max 2.590 MB/s
|
||||
|
||||
## Comparison to baselines
|
||||
|
||||
### vs Patch B baseline (`run-20260507-patchC-preflight`, N=1, 5 min @ 4 MB/s, fresh chip)
|
||||
|
||||
| | Patch B | v3 mean | Δ |
|
||||
|---|---:|---:|---:|
|
||||
| throughput | 1.362 MB/s | 2.352 MB/s | **+73%** |
|
||||
|
||||
### vs original Bug #5 baseline (`run-20260506-0659-fresh`, N=3, decay over time)
|
||||
|
||||
Bug #5 anchor was 725 / 663 / **75** KB/s — rep 3 saw link-death at ~9 min.
|
||||
|
||||
| | Bug #5 floor (rep 3) | v3 floor (rep 3) | Δ |
|
||||
|---|---:|---:|---:|
|
||||
| throughput | 0.075 MB/s | 2.102 MB/s | **28× improvement** |
|
||||
|
||||
### vs Phase 4 v3 plan §4.5 predictions
|
||||
|
||||
| metric | predicted | observed | verdict |
|
||||
|---|---|---|---|
|
||||
| sdio_rx_work dispatch rate | → 0/s (high confidence) | 0/s all 3 reps | ✅ |
|
||||
| `bes2600_bh_work` redispatches | → 0 (high confidence) | 0 all 3 reps | ✅ |
|
||||
| observed RX @ 4 MB/s | floor lifts toward ≥ 1 MB/s sustained (medium) | 2.10 MB/s floor | ✅ exceeds prediction |
|
||||
| `_raw_spin_unlock_irqrestore` CPU% | 20% → 12-15% (medium) | not measured | deferred — perf-record run can confirm |
|
||||
|
||||
## Workqueue dispatch rate collapse
|
||||
|
||||
Patch B baseline (per `run-20260507-patchC-preflight`):
|
||||
- sdio_rx_work: 86.4/s
|
||||
- sdio_tx_work: 276.1/s
|
||||
- bes2600_bh_work redispatches: 0
|
||||
|
||||
v3 N=3 mean:
|
||||
- **sdio_rx_work: 0.0/s** (function deleted)
|
||||
- **sdio_tx_work: 0.8/s** (post-tx queue_work → self->irq_handler call; the chip-side TX driver no longer needs to wake a separate workqueue)
|
||||
- bes2600_bh_work redispatches: 0 (preserved invariant; bh thread still single long-lived work item)
|
||||
|
||||
The 99.7% reduction in `sdio_tx_work` dispatch rate is a side-effect of v3's IRQ→bh-direct rewiring: the post-TX `queue_work(self->sdio_wq, &self->rx_work)` call I replaced with `self->irq_handler()` was actually firing more often than I'd assumed (276/s on Patch B). Folding it into the bh wake-up cuts 275/s of workqueue dispatches that weren't doing anything useful.
|
||||
|
||||
## Risks observed
|
||||
|
||||
- **Bug #5 RX-degradation after ~13-min uptime is independent of v3.** Same scan-failure pattern observed (`wsm_generic_confirm failed for request 0x0007` + `[SCAN] Scan failed (-22)` every 300s) on v3 as on Patch B. v3 did NOT fix Bug #5; it fixed the v2-race that was ALSO present. RX-degradation is firmware-side, likely needs a separate campaign.
|
||||
- **N=3 reps were 3 minutes each instead of 5** to fit within the fresh-chip window. Direct comparison with Patch B's 5-min baseline is approximate; chip-side throughput in 3-min vs 5-min should be similar given the bug fires on uptime, not on transferred-bytes.
|
||||
- **No regression observed in 3×3 min = 9 min of stress.** The v2 race that wedged Patch C v1 within 13 s did NOT reproduce. v3's structural fix held.
|
||||
|
||||
## Phase 8 — lesson distilled
|
||||
|
||||
**The cw1200 mining was decisive.** Patch C v2 (atomic_t prep + direct-deliver on top of relay, PR #10 closed) would have worked correctly but kept the structural relay that was the source of the race. v3 removed the relay entirely — restoring single-writer-from-bh invariant by construction, no atomic_t needed, and delivering a 73% throughput improvement as side benefit.
|
||||
|
||||
Without the cw1200 history mine (`~/src/linux-rockchip`, 228 cw1200 commits over 16 years), v2's atomic_t prep would have shipped. The structural fix is upstream-grade because it matches the reference driver. v2's atomic_t wrapper would have been bes2600-specific bookkeeping with no upstream parallel — defensible as a fix, but worse to maintain.
|
||||
|
||||
**Memory entry:** *When you have an upstream-ancestral driver still in the kernel tree, mine its bug-fix history before patching the inherited fork. The architectural answer may already be there; you just have to look.*
|
||||
|
||||
## Receipts checklist (Phase 7 done)
|
||||
|
||||
- [x] N=3 reps captured at fresh-chip uptime (200/391/582 s)
|
||||
- [x] Same instrumentation pre/post (workqueue ftrace + rx_packets/rx_bytes counters)
|
||||
- [x] Predicted delta matched (sdio_rx_work → 0; bh redispatches → 0; throughput ≥ 1 MB/s sustained)
|
||||
- [x] No WARN/BUG/oops during stress on any rep
|
||||
- [x] Wired-rig telemetry collection (would have caught a wedge if v3 had one)
|
||||
- [x] Receiver `nc` listener restarted fresh per rep (avoiding rep-2-style TCP race)
|
||||
- [x] Stress-ramp memory honored: not steady-state low-rate; saw 4 MB/s saturate
|
||||
|
||||
## Out-of-scope follow-ups
|
||||
|
||||
- Patch C2 — `ieee80211_rx_list` batch delivery — gated on Task #19 kerneldoc verification.
|
||||
- Patch D — ba_lock atomicization — independent.
|
||||
- Patch E — ps_state_lock skip when pm_unsupported — independent.
|
||||
- Bug #5 RX-degradation after 13-min uptime — separate campaign, scan-failure pattern is the entry point.
|
||||
- Task #24 — observe whether `bh.c` `asm volatile("nop")` / commented-out `__bes2600_irq_enable(1)` / BUG_ON in hot path are still load-bearing post-v3. Already partially answered: `__bes2600_irq_enable` is a stub (PR #11 comment). The other artifacts can be re-read fresh.
|
||||
|
||||
---
|
||||
|
||||
*Phase 7 results captured 2026-05-07 by Claude (noether). v3 (PR #5) closes Patch C campaign with structural improvement + race fix + measurable throughput win.*
|
||||
Reference in New Issue
Block a user