[flac-dev] C API: How to get a seektable for very long files?

[Stefan Oltmanns]

Am 14.10.24 um 09:11 schrieb Martijn van Beurden:
> Op zo 13 okt 2024 om 22:33 schreef Stefan Oltmanns <stefan-oltmanns at gmx.net>:
>>
>
>>
>> The signal is the FM-modulated video signal of video tapes (like VHS).
>> The idea is to capture the signal directly from the video head amplifier
>> in the VCR and later demodulate/decode it in software, providing higher
>> quality than traditional capture of analog video. See this project:
>> https://github.com/oyvindln/vhs-decode/
>> I started to design a capture device, as there is no 40 MHz continuous
>> sampling hardware available at consumer prices:
>> https://github.com/Stefan-Olt/MISRC
>>
>
> I've seen similar uses before. Maybe this one can serve as some
> inspiration: https://www.youtube.com/watch?v=ZrEFU22C8l8 According to
> that guy, he used cheap hardware.

That's indeed an interesting device.

>
> Op ma 14 okt 2024 om 00:09 schreef Stefan Oltmanns <stefan-oltmanns at gmx.net>:
>>
>> I think there is another major issue for me: In
>> METADATA_BLOCK_STREAMINFO the field for the length is only 36 bit,
>> that's not even half an hour at 40 MHz sample rate, resulting in that
>> the encoder sets it to 0 for longer captures. In the seekpoint the
>> sample number is 64 bit, which is more than enough.
>> But how does the decoder handle the seektable when the total number of
>> samples is unknown? Or does the seektable override the info from
>> METADATA_BLOCK_STREAMINFO?
>
> When a suitable seektable is found, it overrides the information from
> streaminfo, yes.

Unfortunately that doesn't seem to be the case. I just made a capture
that is > 30 Minutes with total samples set to 0 and a seek table: All
players I tried cannot seek in the file and cannot determine it's
length: VLC, Celluloid and DeaDBeef

I wondered why I can seek files I manually compressed with flac from
raw: The total samples field is just wrong, the seek table is correct,
having sample_numer values a lot higher than total samples. The players
display the wrong length. Seems to be a bug in flac.

>
>>
>> I used this functions now to add seekpoints, but all remain placeholders
>> according to metaflac:
>>
>> FLAC__metadata_object_new
>> FLAC__metadata_object_seektable_template_append_placeholders
>> FLAC__stream_encoder_set_metadata
>> (encoder init & loop)
>> FLAC__metadata_object_seektable_template_sort
>>
>
> Yes, that is correct, because you asked for placeholder points. You
> should ask for spaced points. I just tested what happens if you make a
> seek table template with a total_samples that is bigger than the
> eventual total_samples that is encoded, and found a bug in the
> encoder. It works, but the resulting seek table isn't valid. You could
> try to use that approach anyway in the meantime, perhaps it works just
> fine?

I fixed that manually after calling
FLAC__metadata_object_seektable_template_sort, according to metaflac
everything is correct:

     for(int i = seektable->data.seek_table.num_points-1; i>=0; i--) {
         if (seektable->data.seek_table.points[i].stream_offset != 0) break;
         seektable->data.seek_table.points[i].sample_number =
0xFFFFFFFFFFFFFFFF;
     }


>
> I'd say, have your implementation prepare a seek table template for 5
> hours of recording (I assume that is above the upper bound of such
> captures?), the stream encoder will fill in those seek points when it
> reaches them, and leaves the unused ones unfilled. I will work on a
> fix, so the stream encoder converts those unused points to placeholder
> points in the future.

I have done that now, 2^18 seekpoints for a maximum length of 2^41
samples, that sould be enough.

>
> The stream encoder works pretty simple: you give it metadata to add to
> the start of the stream, it adds those verbatim. After encoding is
> finished, it will update streaminfo and seektable. There is no
> metadata at the end of the stream, and the metadata blocks should not
> be changed during encoding.
>
> I see the difficulty here now by the way: metaflac also refuses to
> write a seektable when the streaminfo metadata block specifies 0 total
> samples, which is unavoidable in your case.

Just an idea: Of course the size of the total samples field cannot be
changed, and a seektable currently doesn't seem to work with total
samples = 0.

Is it possible to extend the specification with full backward
compatibility in a way like this?

If total samples is 0, the last seek point in the seek table that is not
a place holder can indicate the total number of samples:
In case frame_samples is 0, stream_offset is pointing to the last frame
and sample_number is the total number of samples.

FLAC__stream_decoder_get_total_samples would check that and return the
correct value.

Not sure if that breaks any old versions / 3rd party decoder. Another
option would be to use the first place holder, set stream_offset to the
total samples and some magic value in frame_samples. That should
definitly be backward compatible as these values are undefined for
placeholders.

Another option would of course be to add a new type of
metadata, like METADATA_BLOCK_STREAMINFO2, but I don't know how
backwards compatible that is.

Best regards
Stefan