From fde8a25779ae26475df9344f2d0744ff87a547fc Mon Sep 17 00:00:00 2001 From: Markus Fritsche Date: Thu, 14 May 2026 15:32:27 +0000 Subject: [PATCH] Update handoff doc: HEVC Bug 5 fully fixed (3/3 PASS) --- PRE_COMPACT_HANDOFF.md | 184 ++++++++++++++++++----------------------- 1 file changed, 79 insertions(+), 105 deletions(-) diff --git a/PRE_COMPACT_HANDOFF.md b/PRE_COMPACT_HANDOFF.md index 0c4d3a3..af9f7e2 100644 --- a/PRE_COMPACT_HANDOFF.md +++ b/PRE_COMPACT_HANDOFF.md @@ -1,155 +1,129 @@ -# Pre-Compact Handoff — Session 2026-05-14 +# Pre-Compact Handoff — Session 2026-05-14 (updated post iter31) Use this doc to resume the fresnel-fourier campaign after Claude context compaction. ## TL;DR (read first) -- **Bug 4 (H.264 keyframe-partial): FIXED** — H.264 10F byte-equal to SW reference. -- **Bug 5 (HEVC libva all-zero): PARTIAL** — frame 1 byte-equal to SW; frame 2+ diverges (separate ffmpeg-vaapi slice_data inflation bug, deferred). -- **VP9**: unchanged (HW=SW byte-equal). +- **Bug 4 (H.264 keyframe-partial): FIXED iter25 α-25** — H.264 10F byte-equal to SW reference. +- **Bug 5 (HEVC libva all-zero / frame 2+ divergence): FULLY FIXED** — frame 1 via α-25, frames 2+ via iter31 α-29. HEVC 10F byte-equal to SW. +- **VP9**: unchanged (HW=SW byte-equal, no regression from α-29). - **MPEG-2 / VP8**: untestable through libva on current kernel boot (pre-existing libva single-device profile-probe limitation; auto-select picks rkvdec which doesn't expose those profiles). -- Root cause identified after 6 kernel-printk iterations: `rkvdec_s_ctrl` returns -EBUSY when first SPS triggers `image_fmt` reset on a busy CAPTURE queue. Fixed by synthetic SPS injection at libva CreateContext. + +Final score on rkvdec-routed anchors: **3/3 PASS**. MPEG-2/VP8 path orthogonal to Bug 4/5. ## Substrate state (where things live) | Component | Location | Tip | |---|---|---| -| Campaign repo (this) | `/home/mfritsche/src/fresnel-fourier/` | `c15fc6c` on gitea master | -| Libva backend fork (noether) | `/home/mfritsche/src/libva-multiplanar/libva-v4l2-request-fourier/` | `6646b16` on gitea master | -| Libva backend (fresnel deploy) | `/home/mfritsche/src/libva-v4l2-request-fourier/` | sync to gitea master, run `ninja -C build` | -| Kernel source (boltzmann) | `~/src/kernel-agent-bootstrap/build/marfrit-packages/arch/linux-fresnel-fourier/` | pkgrel=9 with iter17/20/21/22/23/27 diag printks | -| Kernel running on fresnel | `linux-fresnel-fourier 7.0-9` | diagnostic build; revert to clean 7.0-X before any production work | +| Campaign repo (this) | `/home/mfritsche/src/fresnel-fourier/` | `c1f9738` on gitea master | +| Libva backend fork (noether) | `/home/mfritsche/src/libva-multiplanar/libva-v4l2-request-fourier/` | `23eb1bd` on gitea master | +| Libva backend (fresnel deploy) | `/home/mfritsche/src/libva-v4l2-request-fourier/` | sync to gitea master, `ninja -C build` | +| Kernel source (boltzmann) | `~/src/kernel-agent-bootstrap/build/marfrit-packages/arch/linux-fresnel-fourier/` | pkgrel=10 with iter17/20/21/22/23/27/31 diag printks | +| Kernel running on fresnel | `linux-fresnel-fourier 7.0-10` | diagnostic build; revert to clean 7.0-N before any production work | | Test fixtures (fresnel) | `/home/mfritsche/fourier-test/bbb_*.{mp4,ts,webm}` | 5 codecs at 720p10s or 1080p30 | -| OUTPUT-buffer dumps (fresnel) | `/tmp/out_dump/output_*.bin` | from α-16 env `LIBVA_V4L2_DUMP_OUTPUT` | -| Memory | `/home/mfritsche/.claude/projects/-home-mfritsche-src-fresnel-fourier/memory/` | `feedback_rkvdec_image_fmt_pre_seed.md` is the key new entry | +| Anchors (fresnel) | `/tmp/iter31/{libva,sw}_{h264,hevc,vp9}_10f.yuv` | per-frame SHA match SW | +| Memory | `~/.claude/projects/-home-mfritsche-src-fresnel-fourier/memory/` | new: `feedback_va_st_rps_bits_is_slice_field.md` | ## Identity for gitea pushes All `git.reauktion.de` interactions use `claude-noether` identity (per memory `feedback_gitea_as_claude_noether.md`). Backend remote URL: `ssh://gitea@git.reauktion.de.claude-noether/marfrit/libva-v4l2-request-fourier.git`. -## Backend commits delivered this session +## Backend commits delivered (chronological, this campaign) ``` -6646b16 Revert iter28b DIAG: trim=40 universal-trim breaks IDR frame 1 -c555788 iter28b DIAG: env-gated trim of HEVC slice_data trailing N bytes (reverted) -cd286d9 iter28 α-28: bit_size = (slice_data_size - data_byte_offset) * 8 for HEVC (no-op, rkvdec ignores) +23eb1bd iter31 α-29: slice_params.short_term_ref_pic_set_size = picture->st_rps_bits ← Bug 5 remainder fix +68dbbdd iter30 DIAG: LIBVA_TS_SCALE env-gated timestamp multiplier (env-gated, no-op default) +0eca3ff iter29 DIAG: env-gated dump of HEVC slice_data trailing 80 bytes +6646b16 Revert iter28b DIAG (universal trim=40 broke IDR) +cd286d9 iter28 α-28: bit_size = (slice_data_size - data_byte_offset) * 8 for HEVC 754be1d iter27 diag: env-gated VAAPI slice fields dump -c9bfa21 iter27: remove request_log diag -719d813 iter27 α-27: populate slice_params.num_entry_point_offsets from VAAPI (no-op) -66ef848 iter26 α-26: populate decode_params.short_term_ref_pic_set_size from VAAPI st_rps_bits +719d813 iter27 α-27: populate slice_params.num_entry_point_offsets (no-op, rkvdec ignores) +66ef848 iter26 α-26: populate decode_params.short_term_ref_pic_set_size (mis-routed; rkvdec ignores) d062fec iter25 α-25 fix: add FRAME_MBS_ONLY to H264 dummy SPS -db0b7f9 iter25 α-25: inject synthetic SPS before cap_pool_init to seed image_fmt ← THE FIX +db0b7f9 iter25 α-25: inject synthetic SPS before cap_pool_init to seed image_fmt ← Bug 4 + Bug 5 frame 1 fix ``` ## Campaign repo commits delivered ``` +c1f9738 iter31 α-29 close: HEVC Bug 5 remainder FIXED — 3/3 PASS +422ecaf Add pre-compact handoff doc for session resumption c15fc6c iter28b DIAG documented: universal trim=40 breaks IDR (reverted) 8b17bf7 Final session summary: H264 + VP9 + HEVC frame 1 byte-equal to SW -02c4192 iter27/28: probe HEVC frame 2+ divergence; α-27/α-28 no-op; ffmpeg-vaapi slice_data inflation localized +02c4192 iter27/28: probe HEVC frame 2+ divergence; α-27/α-28 no-op bf67900 iter20-26: kernel-side root-cause localization, α-25/α-26 fix Bug 4, partial Bug 5 ``` -Phase docs (chronological): `phase4_iter21_plan.md`, `phase4_iter22_plan.md`, `phase8_iteration20_close.md` … `phase8_iteration27_close.md`, `CAMPAIGN_SESSION_2026_05_14.md`. +Phase docs (chronological): `phase4_iter21_plan.md`, `phase4_iter22_plan.md`, `phase8_iteration20_close.md` … `phase8_iteration27_close.md`, `phase8_iteration31_close.md`, `CAMPAIGN_SESSION_2026_05_14.md`. ## How to verify the current state Run on fresnel after `git pull` + `ninja -C build` in `~/src/libva-v4l2-request-fourier`: ```bash -# H.264 — should be byte-equal to SW (Bug 4 fixed) -env LIBVA_DRIVER_NAME=v4l2_request \ - LIBVA_DRIVERS_PATH=/home/mfritsche/src/libva-v4l2-request-fourier/build/src \ - ffmpeg -hide_banner -loglevel error -y -hwaccel vaapi -hwaccel_output_format vaapi \ - -i /home/mfritsche/fourier-test/bbb_1080p30_h264.mp4 \ - -vf "hwdownload,format=nv12,crop=1920:1080:0:0" \ - -frames:v 10 -f rawvideo -pix_fmt nv12 /tmp/libva_h264.yuv -ffmpeg -hide_banner -loglevel error -y \ - -i /home/mfritsche/fourier-test/bbb_1080p30_h264.mp4 \ - -frames:v 10 -f rawvideo -pix_fmt nv12 /tmp/sw_h264.yuv -cmp /tmp/libva_h264.yuv /tmp/sw_h264.yuv # SHOULD print nothing (equal) - -# HEVC — frame 1 byte-equal, frames 2+ differ -env LIBVA_DRIVER_NAME=v4l2_request \ - LIBVA_DRIVERS_PATH=/home/mfritsche/src/libva-v4l2-request-fourier/build/src \ - ffmpeg -hide_banner -loglevel error -y -hwaccel vaapi -hwaccel_output_format vaapi \ - -i /home/mfritsche/fourier-test/bbb_720p10s_hevc.mp4 \ - -vf "hwdownload,format=nv12,crop=1280:720:0:0" \ - -frames:v 1 -f rawvideo -pix_fmt nv12 /tmp/libva_hevc1.yuv -ffmpeg -hide_banner -loglevel error -y \ - -i /home/mfritsche/fourier-test/bbb_720p10s_hevc.mp4 \ - -frames:v 1 -f rawvideo -pix_fmt nv12 /tmp/sw_hevc1.yuv -cmp /tmp/libva_hevc1.yuv /tmp/sw_hevc1.yuv # SHOULD print nothing +for codec in h264:bbb_1080p30_h264.mp4 hevc:bbb_720p10s_hevc.mp4 vp9:bbb_720p10s_vp9.webm ; do + name="${codec%%:*}"; fixture="${codec#*:}" + env LIBVA_DRIVER_NAME=v4l2_request \ + LIBVA_DRIVERS_PATH=/home/mfritsche/src/libva-v4l2-request-fourier/build/src \ + LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 \ + LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0 \ + ffmpeg -hide_banner -loglevel error -y \ + -hwaccel vaapi -hwaccel_output_format vaapi \ + -i "/home/mfritsche/fourier-test/$fixture" \ + -vf "hwdownload,format=nv12" -frames:v 10 \ + -f rawvideo -pix_fmt nv12 "/tmp/libva_${name}.yuv" + ffmpeg -hide_banner -loglevel error -y \ + -i "/home/mfritsche/fourier-test/$fixture" \ + -frames:v 10 -f rawvideo -pix_fmt nv12 "/tmp/sw_${name}.yuv" + if cmp -s "/tmp/libva_${name}.yuv" "/tmp/sw_${name}.yuv"; then + echo "$name: PASS" + else + echo "$name: FAIL" + fi +done ``` -## Root cause (saved to memory) +Expect: 3× PASS. -`rkvdec_s_ctrl` on first HEVC_SPS / H264_SPS: +## Root cause summary -```c -image_fmt = desc->ops->get_image_fmt(ctx, ctrl); -if (rkvdec_image_fmt_changed(ctx, image_fmt)) { - vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); - if (vb2_is_busy(vq)) - return -EBUSY; // ← THIS - ctx->image_fmt = image_fmt; - rkvdec_reset_decoded_fmt(ctx); -} -``` +**Bug 4 (H.264) + Bug 5 frame 1 (HEVC IDR)**: `rkvdec_s_ctrl` returned -EBUSY when first SPS set tried to reset `image_fmt` on a busy CAPTURE queue. libva pre-allocated CAPTURE in CreateContext (iter5b-β design) before per-frame S_EXT_CTRLS. Fix: synthetic SPS injection pre-cap_pool_init so reset succeeds while queue empty. Source: `db0b7f9` + `d062fec`. -`ctx->image_fmt` defaults to `RKVDEC_IMG_FMT_ANY` at open. First per-frame SPS resolves to a concrete value (e.g., `RKVDEC_IMG_FMT_420_8BIT`); since ANY ≠ concrete, `image_fmt_changed` returns true → tries reset → `vb2_is_busy` returns true (libva pre-allocated 24 CAPTURE buffers at CreateContext) → -EBUSY → setup loop breaks → SPS never committed to ctx->ctrl_hdl → rkvdec_hevc_run reads zero → all-zero CAPTURE. - -## The fix (α-25) - -In `src/context.c::RequestCreateContext`, BEFORE `cap_pool_init` (line ~215), inject one synthetic SPS via non-request `v4l2_set_controls`: - -- VAProfileHEVCMain: synthetic `v4l2_ctrl_hevc_sps` with chroma_format_idc=1 (4:2:0), bit_depth=0 (8-bit), width/height from caller. -- VAProfileH264*: synthetic `v4l2_ctrl_h264_sps` with same chroma+bit_depth + `V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY` flag set (else rkvdec_h264_validate_sps doubles height and rejects). - -At this point CAPTURE is empty → `vb2_is_busy=false` → rkvdec_s_ctrl succeeds → `ctx->image_fmt = RKVDEC_IMG_FMT_420_8BIT`. From then on per-frame SPS finds `image_fmt_changed=false` → skip reset → commits successfully. - -Source: see commit `db0b7f9` for the full diff. +**Bug 5 frame 2+ (HEVC non-IDR)**: libva backend set `slice_params->short_term_ref_pic_set_size = 0` (with stale "VAAPI doesn't expose" comment). rkvdec's `assemble_sw_rps` (rkvdec-hevc.c:386-389) reads this field to compute long-term-RPS bit offset; when zero AND `num_short_term_ref_pic_sets <= 1`, falls back to 0 → HW entropy decoder consumes slice-header bits as long-term-RPS → garbage state for every non-IDR slice. IDR is gated out (`!IDR_PIC` flag) so frame 1 unaffected. Fix: `slice_params->short_term_ref_pic_set_size = picture->st_rps_bits` (VAAPI doc says st_rps_bits IS the slice-header bit count — α-26 mis-routed it into decode_params with same field name but different semantics). Source: `23eb1bd`. ## Open items (deferred) -### 1. HEVC frame 2+ divergence +### 1. Kernel substrate cleanup -For non-IDR HEVC frames, libva's `slice_data_size` from VAAPI is consistently 40 bytes larger than `ffmpeg-v4l2request`'s `size` parameter (= the `nal->raw_size` value libavcodec dispatches). The 40 extra bytes inflate libva's OUTPUT buffer → rkvdec reads past the slice payload → wrong reference → frame 2+ visual garbage. +`linux-fresnel-fourier 7.0-10` has 5+ accumulated `pr_info` diagnostic patches in: +- `drivers/media/v4l2-core/v4l2-ctrls-request.c` (iter21-24 setup/clone/loop traces) +- `drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c` (iter17/20/27/31 SPS/DP/slice dumps) -Evidence (from iter27 kernel printk): - -``` -libva frame 2 OUTPUT = 5552 bytes (3 prefix + 5549 slice_data) -kdirect frame 2 OUTPUT = 5512 bytes (3 prefix + 5509) -diff = 40 bytes per slice, P/B-frame specific (IDR is correct) -``` - -Universal trim=40 tested as iter28b → broke IDR (frame 1) which was correct → reverted. Real fix requires: - -**Option A**: Rebuild ffmpeg with `fprintf(stderr, "size=%u\n", size)` at top of `v4l2_request_hevc_decode_slice` in `libavcodec/v4l2_request_hevc.c:564` to confirm what `size` libavcodec actually dispatches. Probe added during session, build was killed mid-link due to context length. To redo: source is at `/home/mfritsche/src/aur/ffmpeg-git/src/FFmpeg/` on fresnel, restored to clean state. Add the probe, `nohup make -j4 ffmpeg > /tmp/log 2>&1 &`, wait ~25 min, then run libva HEVC and see actual size values dispatched by libavcodec. - -**Option B**: Write a libva-side HEVC slice trailing-bits parser to find the rbsp_stop_one_bit position dynamically. Scan slice_data buffer backwards, identify the byte containing the stop bit (pattern: data...1 0...0), trim slices_size to that position. Complicated by the fact that the 40 trailing bytes for BBB frame 2 look like real entropy data (not zeros), so simple "trim trailing zeros" doesn't work. - -**Option C**: Patch ffmpeg-vaapi to ensure slice_data_size matches `nal->raw_size` from libavcodec exactly (suspect there's some internal inflation in `vaapi_hevc_decode_slice`/`ff_vaapi_decode_make_slice_buffer` path). Upstream ffmpeg work. +Before any production work, revert to clean 7.0-N (i.e., apply only the 3 PBP DTS patches + RFC v2 fence series, without diagnostics). Bump pkgrel to 11 and ship clean. ### 2. MPEG-2 / VP8 untestable through libva on current kernel boot -Libva backend's `find_codec_device` (in `src/request.c:427`) selects ONE device for the entire session. On RK3399 with both rkvdec (`/dev/media0`+`/dev/video1` this boot) and hantro (`/dev/media1`+`/dev/video2`+`/dev/video3`), the backend picks rkvdec — which exposes H264/HEVC/VP9 only, not MPEG-2/VP8. +Libva backend's `find_codec_device` (`src/request.c:427`) selects ONE device for the entire session. On RK3399 with both rkvdec (`/dev/media0`+`/dev/video1`) and hantro (`/dev/media1`+`/dev/video2`+`/dev/video3`), the backend picks rkvdec — which exposes H264/HEVC/VP9 only. -Override with `LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video3 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media1` to force hantro for MPEG-2/VP8 testing. But that disables H264/HEVC/VP9 simultaneously, and the unconditional HEVC DECODE_MODE/START_CODE controls libva sets at CreateContext (context.c:343-379) fail on hantro with `Unable to set control(s): Invalid argument` — pre-existing, not iter25 regression. +Override with `LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video3 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media1` to force hantro for MPEG-2/VP8 testing. But that disables H264/HEVC/VP9 simultaneously, and the unconditional HEVC DECODE_MODE/START_CODE controls libva sets at CreateContext (`context.c:343-379`) fail on hantro with `Unable to set control(s): Invalid argument` — pre-existing, orthogonal to Bug 4/5. Fix would require either: - Libva backend multi-device probe + per-codec dispatch (~200-400 LOC, called out in `phase0_findings_iter7.md`). - Conditional codec-init controls (skip controls hantro doesn't support). -### 3. Kernel substrate cleanup +### 3. iter29/iter30 env-gated diagnostics in backend -`linux-fresnel-fourier 7.0-9` has 5+ accumulated `pr_info` diagnostic patches in `drivers/media/v4l2-core/v4l2-ctrls-request.c` and `drivers/media/platform/rockchip/rkvdec/rkvdec-hevc.c`. Before any production work, revert to clean 7.0-X (i.e., apply only the 3 PBP DTS patches + RFC v2 fence series, without diagnostics). Or just bump to 7.0-X and ship without diagnostics. +`LIBVA_HEVC_DUMP_SLICE_TAIL=1` and `LIBVA_TS_SCALE=N` are present in the backend but env-gated (no behavior change without env set). Could clean up to keep ship-ready source minimal. Or leave them — useful for future regression debugging. Low priority either way. -## Memory entries this session +### 4. α-26 dead-code -- New: `feedback_rkvdec_image_fmt_pre_seed.md` — root cause + α-25 fix summary. -- Updated: `feedback_libva_byte_correct_kernel_bug.md` flagged as partially overturned (the byte-correctness claim was right; the kernel-side bug claim was misleading — actual bug was libva-side CAPTURE-pool timing interacting with kernel state). +`decode_params->short_term_ref_pic_set_size = picture->st_rps_bits` was mis-routed (right value to wrong field). rkvdec doesn't use decode_params's same-named field. Could revert α-26 to set 0 (which is correct per V4L2 spec when SPS-defined RPS bit count is unknown). Cosmetic. + +## Memory entries (this session arc) + +- **New**: `feedback_va_st_rps_bits_is_slice_field.md` — VAAPI's `picture->st_rps_bits` belongs in `slice_params`, not `decode_params`. Same field name, different semantics. +- **Updated**: `feedback_rkvdec_image_fmt_pre_seed.md` — note Bug 5 remainder is now fixed (not via image_fmt; see new entry). +- **Updated**: `feedback_libva_byte_correct_kernel_bug.md` — FULLY OVERTURNED (both Bug 4 and Bug 5 are libva-side fixes). ## Key commands quickreference @@ -157,22 +131,22 @@ Fix would require either: # Sync backend on fresnel + rebuild ssh fresnel 'cd ~/src/libva-v4l2-request-fourier && git fetch && git reset --hard origin/master && ninja -C build' -# Run libva HEVC + capture rkvdec kernel printk +# 3-codec smoke (above script). Each codec ~5s. + +# Run libva HEVC + capture rkvdec kernel iter27/31 printk ssh fresnel 'sudo dmesg -C; env LIBVA_DRIVER_NAME=v4l2_request \ LIBVA_DRIVERS_PATH=/home/mfritsche/src/libva-v4l2-request-fourier/build/src \ + LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video1 \ + LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0 \ ffmpeg -hide_banner -loglevel error -y -hwaccel vaapi -hwaccel_output_format vaapi \ -i /home/mfritsche/fourier-test/bbb_720p10s_hevc.mp4 \ - -vf "hwdownload,format=nv12,crop=1280:720:0:0" -frames:v 3 -f rawvideo -pix_fmt nv12 /tmp/x.yuv; - sudo dmesg | grep -E "rkvdec|iter2[0-9]_"' + -vf "hwdownload,format=nv12" -frames:v 3 -f rawvideo -pix_fmt nv12 /tmp/x.yuv; + sudo dmesg | grep -E "rkvdec_iter2[07]|rkvdec_iter31"' # kdirect (ffmpeg-v4l2request) reference -ssh fresnel 'ffmpeg -hide_banner -loglevel error -y \ - -hwaccel v4l2request -hwaccel_output_format drm_prime \ +ssh fresnel 'ffmpeg -hide_banner -loglevel error -y -hwaccel v4l2request \ -i /home/mfritsche/fourier-test/bbb_720p10s_hevc.mp4 \ - -vf "hwdownload,format=nv12,crop=1280:720:0:0" -frames:v 3 -f rawvideo -pix_fmt nv12 /tmp/y.yuv' - -# Force hantro path (untested with backend, see open-item 2) -env LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video3 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media1 ... + -frames:v 10 -f null -' # decode-only, dmesg has iter27/31 entries # Reboot fresnel (sddm autologin reseats mfritsche per /etc/sddm.conf.d/20-autologin.conf) ssh fresnel 'sudo systemctl reboot'; sleep 60 @@ -189,6 +163,6 @@ ssh fresnel 'sudo systemctl reboot'; sleep 60 ## What needs user confirmation -- Significant ffmpeg rebuild (~25 min CPU time). -- Reverting kernel-substrate diagnostics to ship a clean kernel. -- Decisions on whether to invest in HEVC frame 2+ fix or MPEG-2/VP8 multi-device probe. +- Significant rebuild (~25-30 min CPU time on boltzmann, e.g., ffmpeg or fresh kernel build). +- Reverting kernel-substrate diagnostics to ship a clean kernel (mechanical but heavy). +- Architectural change to libva multi-device probe (Item 2) — affects libva backend design.