[Flac-dev] Re: Problem with CRAM and flac-1.1.2

[Josh Green]

Thanks for the reply, I'll try to keep it to the point, since it sounds
as if you are quite busy.

The issue I was experiencing I don't think is related to the
sample/frame number stored at the end of a FLAC frame.  Its rather
related to the block size and what is stated below (from the FLAC format
spec):

<NOTES>

The blocksize bits 0000-0101 and 1000-1111 may only be used if the
blocksize is fixed throughout the entire stream. Blocksize bits
0110-0111 may be used in any case but the decoder will have to
pessimistically guess that it is a variable-blocksize stream unless it
has STREAMINFO metadata and the min_blocksize and max_blocksize values
in it match.

</NOTES>

I had not noticed this clause before.  CRAM stores several streams in a
single custom container (CRAM) file.  The FLAC main header and
STREAMINFO are not stored but instead forged in the callbacks.  The CRAM
decoder was telling the FLAC decoder to expect the entire range of block
sizes, since it did not know in advance what it would be (was
convenient) and individual streams could in fact differ in block size
(though a single stream is fixed).

Some logic was changed in the libFLAC stream_decoder.c in flac-1.1.2
which caused things to no longer work.  This is because the streams are
actually stored as fixed block size streams, but the forged STREAMINFO
was indicating otherwise.  So the fixed blocksize bit numbers were being
used, but the flac decoder was now assuming it was a variable block size
stream.  I did try flushing the decoder for each stream, but this
resulted in a 25% increase in time required and needed some hacks to
manually look at the blocksize field of the first frame to fill in a
fixed blocksize into the STREAMINFO.

At this point I've decided to distribute CRAM with a minimal version of
libFLAC (containing only the stream encoder/decoder).  This will also
allow for custom changes and optimizations particular to this
application (multiple streams in a single file).  In particular it would
be nice to be able to quickly seek within an individual stream and then
change streams (CRAM has some idea of where the streams are) without too
much overhead of flushing or creating a new FLAC decoder.

I think this is probably the best solution for now.  I'm open to any
suggestions though.

Best regards,
	Josh Green


On Thu, 2006-08-03 at 02:50 -0700, Josh Coalson wrote:
> sorry if I'm not reading this close enough, I'm ploughing through
> a bunch of emails here, but the source of this problem is what I
> consider a design defect in FLAC which uses frame numbers to save
> space, and confusing logic for determining whether the frame number
> or sample number is stored.  if I had it to do again I would just
> have it store the sample number always.
> 
> if you are going to store the sample number, this complex logic
> must be observed exactly, and even then, it may not be suitable
> (e.g. cannot go back to using frame numbers after using sample
> numbers in a frame).
> 
> --- Josh Green <josh at resonance.org> wrote:
> 
> > Replying to myself once again, since I seem to have found the answer
> > I
> > was looking for.  Sorry for the noise.  It seems CRAM is indeed
> > misusing
> > FLAC, since stated in the Notes section of the FLAC format for the
> > FRAME_HEADER is the fact that only block size values 0110 and 0111
> > can
> > be used for variable block size data (i.e., the block size is
> > specified
> > as an 8 or 16 bit value at the end of the header):
> > http://flac.sourceforge.net/format.html#frame_header
> > 
> > 
> > Not quite sure the best way to fix this.  I'd love to hear any
> > ideas..
> > 
> > CRAM files usually consist of several audio samples all packed into
> > the
> > same file (with binary segments compressed with zlib).  FLAC is used
> > to
> > store only the compressed audio and no other FLAC headers
> > (STREAMINFO,
> > etc) are actually found in a CRAM file.  A single sample could (and
> > is
> > currently) compressed using a single block size.  It might be
> > advantageous to change the block size between audio samples though,
> > depending on the individual sample parameters (sample rate, length,
> > etc), I have yet to determine a good method for guessing a good block
> > size though.
> > 
> > Seems like it might be a little bit of a waste to add 2 bytes to
> > every
> > frame in this case (but I imagine that's probably a trivial amount). 
> > I
> > suppose the FLAC stream decoder could be re-created for each sample
> > with
> > the appropriate fixed block size STREAMINFO which is forged, but that
> > leaves it up to CRAM to determine what block size is stored in the
> > FLAC
> > FRAME_HEADER and seems a little messy anyways.  Any input
> > appreciated.
> > 
> > Best regards,
> > 	Josh Green
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
>