marfrit/libva-v4l2-request-fourier
1
0
Fork 1
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

iter28 α-28: bit_size = (slice_data_size - slice_data_byte_offset) * 8 for HEVC

Browse Source 
VAAPI's slice_data_size includes NAL+slice header bytes that precede the
slice payload. rkvdec_hevc expects bit_size to cover the slice payload
(starting at data_byte_offset). Setting bit_size = slice_data_size * 8
made rkvdec read past slice payload → wrong entropy state → frame 2+
garbage despite correct ctx->image_fmt (iter25) and decode_params
(iter26).

Empirical match: with formula (slice_data_size - slice_data_byte_offset)
* 8, libva produces bit_size=44096 for BBB frame 2 matching kdirect's
44096 exactly per iter27 dmesg printk.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude (noether)
2026-05-14 10:24:40 +00:00
parent 754be1de7e
commit cd286d9bf0
1 changed files with 19 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/h265.c
+19 -10
View File
@@ -380,16 +380,25 @@ static void h265_fill_slice_params(VAPictureParameterBufferHEVC *picture,
nuh_temporal_id_plus1 = (b[1] >> H265_NUH_TEMPORAL_ID_PLUS1_SHIFT) &
H265_NUH_TEMPORAL_ID_PLUS1_MASK;
slice_params->bit_size = slice->slice_data_size * 8;
if (getenv("LIBVA_HEVC_DIAG")) {
request_log("hevc_diag: slice_data_size=%u slice_data_offset=%u "
"slice_data_byte_offset=%u slice_data_flag=%u source_offset=%u\n",
(unsigned)slice->slice_data_size,
(unsigned)slice->slice_data_offset,
(unsigned)slice->slice_data_byte_offset,
(unsigned)slice->slice_data_flag,
source_offset);
}
/*
* iter28 α-28: bit_size formula.
*
* VAAPI's slice_data_size is the size of the slice's source-data
* buffer INCLUDING the NAL header and slice header. rkvdec_hevc
* expects bit_size to cover the slice_data area starting at
* data_byte_offset (the slice payload). Setting bit_size =
* slice_data_size * 8 makes rkvdec read past the slice payload
* into trailing bytes → wrong entropy state → frame 2+ visual
* garbage.
*
* Empirical match with ffmpeg-v4l2request (which uses
* (size+extra_size)*8 for the data it actually appended):
* bit_size = (slice_data_size - slice_data_byte_offset) * 8
* yields 44096 bits for BBB frame 2 (= 5512 bytes), matching
* kdirect exactly per iter27 dmesg printk.
*/
slice_params->bit_size =
(slice->slice_data_size - slice->slice_data_byte_offset) * 8;
/* C1: data_byte_offset, NOT data_bit_offset. Plain byte offset to
* the first byte of slice segment header data within the OUTPUT