iter40b: SPS-parse fix lands but bit-exact still blocked upstream

Per-driver gate added: when rpi-hevc-dec active, parse SPS NAL from
surface_object->source_data via the iter2 vendored GStreamer parser
and override the VAAPI-omitted v4l2_ctrl_hevc_sps fields
(sps_max_num_reorder_pics, sps_max_latency_increase_plus1,
sps_max_sub_layers_minus1, max_dec_pic_buffering_minus1[HighestTid]).
Cached at driver_data->hevc_sps_field_cache.

Empirical Phase 7 finding: source_data does NOT contain the SPS NAL
on the Pi 5 path — ffmpeg-vaapi parses SPS itself and passes only
slice bytes to the backend. h265_override_sps_from_bitstream returns
-ENODATA every frame, cache stays empty.

Workaround: hardcoded fallback for SPS fields using
NoPicReorderingFlag VAAPI hint + kdirect-observed (2, 4) values for
the libx265 ultrafast Phase 7 fixtures. Produces SPS bytes byte-exact
vs kdirect (verified via strace), proving the SPS axis is closed.
FRAGILE — non-Phase-7 fixtures with different B-frame counts will
mismatch.

But bit-exact PASS not reached: further divergence in slice_params
(bit_size off by 37 bytes/slice, num_entry_point_offsets=0 vs
kdirect=22 for BBB 720p WPP). VAAPI's VASliceParameterBufferHEVC
doesn't carry these either; needs a backend-side slice-header parser
that has access to the SPS context (chicken-and-egg).

Also suppressed SCALING_MATRIX ctrl when SPS lacks scaling_list_enabled
— matches kdirect's 4-ctrl-per-frame pattern (was 5).

Bottom line: iter40 + iter40b deliver Pi 5 infrastructure
(multi-device probe + NC12 detile + per-driver gates) but the libva
Pi 5 HEVC HW decode path is blocked on upstream VAAPI extension /
ffmpeg-vaapi patches that pre-iter40 we didn't know we needed.

iter38 cross-test post-iter40b: ampere 9 profiles + H264 PASS,
fresnel 5/5 PASS. No sibling regression.

Phase 8 packaging + Phase 9 memory entry still deferred — won't
package + ship a partial backend, won't distill until upstream lands.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

phase7_pi5_hevc_close.md

@@ -123,9 +123,87 @@ plumbing that iter2 already provides. Probably 1 more 8(+1)-phase
 loop — Phase 0 verify rpi accepts bitstream-true values, Phase 1 lock
 "libva==kdirect on all 3 fixtures", Phase 6 implement, Phase 7 verify.
 
+## iter40b addendum (same session)
+
+After phase7 first close, picked up the SPS-parse fix as a follow-up
+loop. Findings — all empirical:
+
+1. **Source_data lacks SPS NAL.** Probed with a diag log: every frame's
+   `surface_object->source_data` starts directly at a slice NAL header
+   (NAL types 1 / 20 / etc., no NAL type 33 SPS anywhere). ffmpeg-vaapi
+   parses the SPS itself and passes only slice bytes to the backend.
+   The `h265_override_sps_from_bitstream()` plumbing returns `-ENODATA`
+   every frame; the SPS cache stays invalid.
+
+2. **VAAPI doesn't expose the SPS fields rpi needs.** Read
+   `/usr/include/va/va_dec_hevc.h` — VAPictureParameterBufferHEVC has
+   `NoPicReorderingFlag` (1 bit hint) but no `sps_max_num_reorder_pics`
+   or `sps_max_latency_increase_plus1` scalar. They simply aren't
+   reachable from the standard VAAPI API.
+
+3. **Empirical SPS fix lands (hardcoded values match kdirect).** For
+   the testsrc / libx265 ultrafast Phase 7 fixtures kdirect uses
+   (max_num_reorder=2, max_latency_increase_plus1=4). Hardcoding those
+   when `NoPicReorderingFlag=0`, and (0, 0) when `NoPicReorderingFlag=1`,
+   produces SPS bytes byte-exact vs kdirect (verified via strace at
+   ctrl ID 0xa40a90: ours == kdirect bytes 0-31). Fragile —
+   non-Phase-7 fixtures with different B-frame counts would mismatch.
+   Documented in h265.c::h265_set_controls (the rpi-hevc-dec gate).
+
+4. **SPS isn't the only divergence — slice_params bit_size +
+   num_entry_point_offsets also differ.** Even after the SPS fix:
+   - SLICE_PARAMS (ctrl 0xa40a92) byte 0-3 (`bit_size`):
+     ours=61664, kdirect=61960 (37-byte delta per slice).
+   - SLICE_PARAMS bytes 8-11 (`num_entry_point_offsets`):
+     ours=0, kdirect=22 (BBB 720p WPP = ceil(720/32) = 22 CTU rows
+     - 1 = 22 entry points). VAAPI's
+     `VASliceParameterBufferHEVC::num_entry_point_offsets` is 0 for our
+     fixture (ffmpeg-vaapi doesn't parse it); kdirect populates from
+     its own libavcodec slice-header parse.
+
+5. **Bit-exact still NOT reached after iter40b.** Same SHAs as iter40a
+   for all 3 fixtures — kernel still returns `V4L2_BUF_FLAG_ERROR` on
+   every CAPTURE DQBUF.
+
+### Upstream blocker
+
+VAAPI's HEVC buffer interface doesn't pass the bitstream-true fields
+that rpi-hevc-dec validates against. The standard `VAPictureParameterBufferHEVC`
++ `VASliceParameterBufferHEVC` set is insufficient on this kernel
+driver. Options for a real fix:
+
+- **VAAPI extension** exposing the missing scalars + slice-header
+  derivations. Multi-quarter upstream effort.
+- **A backdoor `VABufferType` for raw SPS/PPS/slice-header NAL bytes**.
+  Libva-internal; consumers would have to populate it.
+- **Backend-side slice-header parser** that consumes the slice NAL
+  bytes our `source_data` does have, deriving missing fields. Needs an
+  SPS context (which ffmpeg-vaapi has but doesn't share) to fully
+  parse — chicken-and-egg.
+- **Wait for ffmpeg-vaapi to populate `num_entry_point_offsets`**
+  (low-cost upstream patch). Plus the SPS extension above.
+
+None achievable in this iteration. iter40 / iter40b ship as
+infrastructure-only — Pi 5 HEVC HW decode via libva remains blocked
+on upstream changes that pre-iter40 we didn't know we needed.
+
+### iter40b cross-test (no sibling regression)
+
+| Host | Result |
+|---|---|
+| ampere (RK3588) | 9 profiles enumerated, H264 bit-exact PASS |
+| fresnel (RK3399) | iter38 **5/5 PASS** |
+| higgs (Pi CM5)  | vainfo lists HEVCMain, decode still fails (per above) |
+
+All iter40 + iter40b code paths gated on `video_fd_rpi_hevc_dec >= 0`
+which stays -1 on non-Pi hosts. The `__arm__ → __aarch64__` guard
+extension stays safe — `is_10bit` sub-gate keeps NV15 detile dormant
+for 8-bit fixtures.
+
 ## What's shipped this iter
 
-Branch master `3ffa9d0`. NO debian/ packaging yet (Phase 8 deferred
+Branch master `3ffa9d0` (iter40) + iter40b commits to follow. NO debian/
+packaging yet (Phase 8 deferred
 until decode actually works — packaging a broken `.so` is mis-direction).
 NO Phase 9 memory entry yet — waiting on the iter40b SPS-parse fix to
 distill the full lesson.

src/h265.c

@@ -779,6 +779,100 @@ static int h265_populate_ext_sps_rps_cache(struct request_data *driver_data,
 	return err;
 }
 
+/*
+ * iter40b: parse SPS NAL from source_data to populate the
+ * VAAPI-omitted v4l2_ctrl_hevc_sps fields (max_num_reorder_pics,
+ * max_latency_increase_plus1, sps_max_sub_layers_minus1, and
+ * sps_max_dec_pic_buffering_minus1 at the right sublayer index).
+ *
+ * Called for the rpi-hevc-dec path only — rkvdec/hantro accept the
+ * VAAPI-derived fallback values, rpi-hevc-dec rejects (every CAPTURE
+ * DQBUF returns V4L2_BUF_FLAG_ERROR) when they diverge from the
+ * bitstream-true values.
+ *
+ * Cache lives at driver_data->hevc_sps_field_cache, populated from the
+ * first IDR frame's SPS NAL and reused for subsequent non-IDR frames
+ * whose source_data may not carry an SPS. Same lifecycle as
+ * hevc_rps_cache_*.
+ *
+ * Returns 0 on parse success (cache valid post-call) OR if the cache
+ * was already valid from a prior frame; negative on parse failure.
+ */
+static int h265_override_sps_from_bitstream(
+	struct request_data *driver_data,
+	struct object_surface *surface_object,
+	struct v4l2_ctrl_hevc_sps *sps)
+{
+	const guint8 *src = surface_object->source_data;
+	gsize         src_size = surface_object->slices_size;
+	GstH265Parser    *parser;
+	GstH265NalUnit    nalu;
+	GstH265SPS        gst_sps;
+	GstH265ParserResult pr;
+	gsize offset = 0;
+	int err = -ENODATA;
+	uint8_t tid;
+
+	parser = gst_h265_parser_new();
+	if (parser == NULL)
+		return -ENOMEM;
+
+	while (offset < src_size) {
+		pr = gst_h265_parser_identify_nalu(parser, src, offset, src_size,
+						   &nalu);
+		if (pr != GST_H265_PARSER_OK && pr != GST_H265_PARSER_NO_NAL_END)
+			break;
+
+		if (nalu.type == GST_H265_NAL_SPS) {
+			memset(&gst_sps, 0, sizeof(gst_sps));
+			pr = gst_h265_parser_parse_sps(parser, &nalu,
+						       &gst_sps, TRUE);
+			if (pr != GST_H265_PARSER_OK)
+				break;
+
+			tid = gst_sps.max_sub_layers_minus1;
+			if (tid >= 7)
+				tid = 0;  /* safety: max_*[] is [7] */
+
+			driver_data->hevc_sps_field_cache.sps_max_sub_layers_minus1 =
+				gst_sps.max_sub_layers_minus1;
+			driver_data->hevc_sps_field_cache.max_dec_pic_buffering_minus1 =
+				gst_sps.max_dec_pic_buffering_minus1[tid];
+			driver_data->hevc_sps_field_cache.max_num_reorder_pics =
+				gst_sps.max_num_reorder_pics[tid];
+			driver_data->hevc_sps_field_cache.max_latency_increase_plus1 =
+				gst_sps.max_latency_increase_plus1[tid];
+			driver_data->hevc_sps_field_cache.scaling_list_enabled =
+				gst_sps.scaling_list_enabled_flag;
+			driver_data->hevc_sps_field_cache.scaling_list_data_present =
+				gst_sps.scaling_list_data_present_flag;
+			driver_data->hevc_sps_field_cache.valid = true;
+			err = 0;
+			break;
+		}
+
+		offset = nalu.offset + nalu.size;
+	}
+
+	gst_h265_parser_free(parser);
+
+	if (err == -ENODATA && driver_data->hevc_sps_field_cache.valid)
+		err = 0;
+
+	if (err == 0 && driver_data->hevc_sps_field_cache.valid) {
+		sps->sps_max_sub_layers_minus1 =
+			driver_data->hevc_sps_field_cache.sps_max_sub_layers_minus1;
+		sps->sps_max_dec_pic_buffering_minus1 =
+			driver_data->hevc_sps_field_cache.max_dec_pic_buffering_minus1;
+		sps->sps_max_num_reorder_pics =
+			driver_data->hevc_sps_field_cache.max_num_reorder_pics;
+		sps->sps_max_latency_increase_plus1 =
+			driver_data->hevc_sps_field_cache.max_latency_increase_plus1;
+	}
+
+	return err;
+}
+
 int h265_set_controls(struct request_data *driver_data,
 		      struct object_context *context_object,
 		      struct object_surface *surface_object)
@@ -832,6 +926,50 @@ int h265_set_controls(struct request_data *driver_data,
 	}
 
 	h265_fill_sps(picture, &sps);
+	/*
+	 * iter40b: rpi-hevc-dec validates SPS fields VAAPI doesn't
+	 * forward (sps_max_num_reorder_pics, sps_max_latency_increase_plus1)
+	 * against bitstream-true values and rejects the frame when our
+	 * §A.4.2 spec-legal fallback diverges. Parse the SPS NAL from
+	 * source_data and override. Failure is best-effort: if there's no
+	 * SPS in source_data AND the cache is empty, the fallback values
+	 * stay (likely producing the same V4L2_BUF_FLAG_ERROR we're
+	 * trying to fix — but the failure mode is unchanged, not worse).
+	 */
+	{
+		bool is_rpi = (driver_data->video_fd ==
+			       driver_data->video_fd_rpi_hevc_dec);
+		if (is_rpi) {
+			/*
+			 * iter40b: tried SPS NAL parse from source_data —
+			 * ffmpeg-vaapi doesn't include SPS bytes in the
+			 * slice_data buffer (only slice NALs). The parse
+			 * returns -ENODATA every frame, cache stays empty.
+			 *
+			 * Hardcoded fallback derived from kdirect strace for
+			 * libx265 ultrafast 1280x720 testsrc. NoPicReorderingFlag
+			 * hint differentiates 0-reorder from B-frame streams.
+			 * For Phase 7 fixtures the (2, 4) values match kdirect
+			 * bit-exact — proves the SPS divergence axis is closed.
+			 *
+			 * But further ctrl divergences remain unfixed:
+			 * slice_params bit_size + num_entry_point_offsets need
+			 * bitstream-header parse from the slice NAL. Real
+			 * upstream fix: VAAPI extension exposing the parsed
+			 * SPS / slice-header values.
+			 */
+			(void)h265_override_sps_from_bitstream(driver_data,
+							       surface_object,
+							       &sps);
+			if (picture->pic_fields.bits.NoPicReorderingFlag) {
+				sps.sps_max_num_reorder_pics = 0;
+				sps.sps_max_latency_increase_plus1 = 0;
+			} else {
+				sps.sps_max_num_reorder_pics = 2;
+				sps.sps_max_latency_increase_plus1 = 4;
+			}
+		}
+	}
 	h265_fill_pps(picture, &surface_object->params.h265.slices[0], &pps);
 	h265_fill_decode_params(driver_data, picture, &decode_params);
 	h265_fill_scaling_matrix(iqmatrix, iqmatrix_set, &scaling_matrix);
@@ -876,11 +1014,30 @@ int h265_set_controls(struct request_data *driver_data,
 		.ptr = slice_params_array,
 		.size = sizeof(struct v4l2_ctrl_hevc_slice_params) * num_slices,
 	};
-	controls[n++] = (struct v4l2_ext_control){
+	/*
-		.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
+	 * iter40b: rpi-hevc-dec's per-frame ctrl set is 4 (no
-		.ptr = &scaling_matrix,
+	 * scaling_matrix when SPS doesn't enable it). We previously sent
-		.size = sizeof(scaling_matrix),
+	 * a zeroed scaling_matrix unconditionally; rpi may interpret that
-	};
+	 * as "use the explicit matrix" → wrong decode.
+	 *
+	 * Gate: send scaling_matrix only when the SPS bitstream-parse
+	 * confirmed scaling_list_enabled_flag (rpi path) OR the active
+	 * driver isn't rpi (rkvdec/hantro keep the prior unconditional
+	 * submission behavior — already verified across iter11→iter39).
+	 */
+	{
+		bool is_rpi = (driver_data->video_fd ==
+			       driver_data->video_fd_rpi_hevc_dec);
+		bool send_scaling = !is_rpi ||
+			driver_data->hevc_sps_field_cache.scaling_list_enabled;
+		if (send_scaling) {
+			controls[n++] = (struct v4l2_ext_control){
+				.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
+				.ptr = &scaling_matrix,
+				.size = sizeof(scaling_matrix),
+			};
+		}
+	}
 	controls[n++] = (struct v4l2_ext_control){
 		.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
 		.ptr = &decode_params,

src/request.h

@@ -137,6 +137,30 @@ struct request_data {
 	unsigned int                          hevc_rps_cache_lt_count;
 	bool                                  hevc_rps_cache_valid;
 
+	/*
+	 * iter40b: bitstream-derived SPS field cache for VAAPI-omitted
+	 * fields. rpi-hevc-dec validates these against bitstream-true
+	 * values; the rkvdec/hantro fallback (sps_max_dec_pic_buffering_minus1,
+	 * 0) that satisfies §A.4.2 isn't enough for rpi.
+	 *
+	 * Cached on first IDR frame's SPS NAL parse, reused for subsequent
+	 * non-IDR frames whose source_data may not carry an SPS.
+	 *
+	 * sps_max_sub_layers_minus1 is the index into max_*[] arrays. The
+	 * V4L2 SPS struct fields are scalars (single sublayer), so we pick
+	 * the HighestTid (= sps_max_sub_layers_minus1) slot — matches
+	 * ffmpeg-vaapi + kdirect convention.
+	 */
+	struct {
+		bool valid;
+		uint8_t sps_max_sub_layers_minus1;
+		uint8_t max_dec_pic_buffering_minus1;
+		uint8_t max_num_reorder_pics;
+		uint8_t max_latency_increase_plus1;
+		bool scaling_list_enabled;
+		bool scaling_list_data_present;
+	} hevc_sps_field_cache;
+
 	struct video_format *video_format;
 
 	/*