[flac-dev] C API: How to get a seektable for very long files?

[brianw]

I am not an SDR user, but I think it would be a great idea to design and register an application metadata block for FLAC that supports these needs.

The fields in this application-specific metadata block could document how many files there are in total; which file in the sequence the current file is; what the actual sample rate is; how many bits in each sample are valid (to a finer degree than FLAC registers).

One thing to do is start this SDRm block with a revision number, so, if the community adds more fields later, then supporting software will know which fields to expect.

I don't think that extra metadata can get around limitations in total samples or seek table fields. One topic that I never studied is how FLAC differs between files and streams. For a file, any sizes or references need to have limits, otherwise handling an unlimited value can be difficult to verify on all compatible software. For a stream, though, total samples or seek information would seem completely unusable for a stream. So, I'm wondering what a pure FLAC stream should even do with such information.

I encourage you to put something together. I would certainly be willing to review it on general principles. Then, you could at least start using something and submit it for consideration as an official addition to the FLAC application metadata standards.

Brian


On Oct 15, 2024, at 12:26 PM, Alistair Buxton wrote:
> FLAC performs extremely well on SDR samples, both speed and compression ratio. In my testing it outperforms any other free lossless codec by a large margin, being 20% smaller and 10% faster than the next best (which was ffv1). The problem is the metadata, and not just total samples. We also can't put true values in the sample rate field because it doesn't have enough bits (I have files with 35468950MHz nominal sample rate for example), and there is no way to record that samples have been padded eg from 10 bits to 16 bits, which seems to be very common in SDR applications. These are just two examples off the top of my head - there are probably more.
> 
> The problems around total samples and seek table allocation could be alleviated by using multiple files as mentioned previously, but that introduces a new problem: how do you know if you have the full set of files? How do you know which file contains the nth sample? This would still require extra metadata somewhere.
> 
> I would like to see this kind of thing put into a secondary metadata block aimed specifically at SDR. This could be completely ignored by regular audio players - these files are not meant to be listened to anyway. I could probably figure out how to implement that, I even started looking into it once, but I realised that 1. nobody would adopt it if it is just me behind it and 2. I don't know enough to make it suitable for all use cases. So I cannot and should not do this alone, as it would just be yet another half-baked adhoc "standard" that causes more problems than it solves. For example I had not even considered the idea of using multiple files.