[flac-dev] identical audio but not identical unrecognized chunks

[Brian Willoughby]

On Apr 29, 2020, at 4:46 PM, Federico Miyara <fmiyara at fceia.unr.edu.ar> wrote:
> Dear all,
> 
> I've converted a wav file to flac but during the process three wrnings were casted. One of them I recall, it said that a BEXT chunk is unrecognized and thus ignored.
> 
> Wouldn't it be posible to keep the unrecognized information as is when decoding the file?
> 
> In general the extra chunks are located before or after the audio content, and represents a tiny part of the whole file, so it oould affect only very slightly the compression ratio attained.
> 
> Regards,
> 
> Federico Miyara

Federico,

flac compresses an audio file by using characteristics of the audio waveform. The algorithm cannot compress non-audio data. You probably already knew this. I just wanted to point out that flac completely ignores the non-audio data, so it does not even factor in to the operation, whether there are warnings, BEXT chunks, or not.

However, I suggested to the team many years ago that FLAC APPLICATION metadata blocks, which are intended for third-party applications, could be used for specific audio file formats such as AIFF and WAVE in order to preserve the non-audio chunks. As you mentioned, these extra chunks represent a small fraction of the total size, so adding them does not have much affect on the final file size.

I am happy that the suggestion was taken seriously and implemented. You will find that the flac command line supports an option `--keep-foreign-metadata` which does exactly what you seem to be suggesting, and it has been part of flac for many years. One of the things we discussed was the fact that some chunks occur before the audio chunk, and some occur after, so preserving the order might have been difficult. I forget the details of the discussions, but I believe that we eventually decided on a technique that also preserves the order of non-audio chunks relative to the audio chunk. This feature treats AIFF differently from WAVE, because it's not really possible to distinguish audio from non-audio without knowing what format the file is in.

A minor caveat is that you'll see a warning when using `--keep-foreign-metadata` - because this is a relatively new feature, the original designer of flac decided to let users know that it might not work perfectly. In my experience, it has always worked.

Another caveat is that you cannot combine `--delete-input-file` with `--keep-foreign-metadata` - the reason being that it's not necessarily safe to delete the original in the same command that you're compressing with a new feature. In these cases, I just manually delete the original file after compressing.

Yet another issue is that you cannot compress a WAVE file with --keep-foreign-metadata and then uncompress it into AIFF format while expecting the non-audio data to be preserved. flac cannot convert non-audio data from one format to another. Similarly, you cannot compress an AIFF file with --keep-foreign-metadata and then uncompress it into WAVE format while preserving non-audio data. flac is perfectly capable of translating the audio between AIFF and WAVE, but the metadata will get lost.

As a side-note, my mobile recording gear puts time stamps and other information into the BEXT chunk, and I usually want to preserve that. flac --keep-foreign-metadata does that perfectly.

On another note, there are applications like Logic Pro that add considerable amounts of non-audio data to a chunked file - typically these are graphical waveforms of the audio - and you probably do not want to waste the space needed to preserve these large chunks. My habit is to back up the original audio recordings with flac, using --keep-foreign-metadata, before loading them into Logic where the extra chunks will be added. This gives me the archive that I need without wasting extra space. Logic Pro will automatically add the waveform image to a file that's been freshly restored from a flac backup.

Brian Willoughby