[Flac-dev] alternate compression

[Brian Willoughby]

On Aug 8, 2009, at 23:11, Didier Dambrin wrote:
> Electronic music quite often doesn't leave a computer these days.  
> And it
> mainly consists of drums, synths & vocals/effects. Drums are often  
> samples
> sequenced at sample (not sub-sample) accuracy, thus repeated (of  
> course if
> the song was post-resampled, there will be sub-sample times).

Good point.  I have certainly seen songs which were at a fixed tempo,  
say 128 BPM, and were so precise that you could cut and paste pieces  
of the song without glitches.  Every measure lined up closely enough  
with the others that you could separate instruments from each other  
by subtracting out the repeated patterns.


> Synths are a problem, as the riffs will have more variations, and  
> also free-running
> oscillators will give troubles.

Not only that, but some synths are oversampled, thus you have the  
"analog" problem of subsampled waveforms.


> Anyway, right now I get what I wanted somewhat working, well enough
> considering I've only spent a couple of hours.

Excellent!


> ..But sadly none of FLAC, WavPack or OptimFrog could compress the
> pre-processed song better, or hardly. And considering you'd also  
> have to add
> the pool of frames, it would end up worse.

This surprises me.  Have you tried aligning your frames to the  
standard FLAC frame size?

As for the pool, it seems like the first occurrence of a repetition  
would compress like usual, and the subsequent ones would compress  
more than usual.

> The problem is the discontinuities I think. Say you work with little,
> non-tempo-synced frames, and you find a matching frame, which you  
> subtract
> from the song at the places it matches. You'll have a discontinuity  
> around
> it. If the frames around this one also match, it doesn't matter as  
> they will
> be subtracted as well. But if they don't (enough), the  
> discontinuity will
> stay.

You may be right about the discontinuities.  Have you tried making  
your transitions only at zero-crossings?

> I also tried windowing the frame before subtracting it, no more
> discontinuity but with small frames it's not very useful anymore.

Windowing may seem like a good idea, but remember that your decoder  
will have to recreate every step that your encoder uses so that it  
can be undone.  Thus, windowing may make it difficult to be lossless.

> But if I run the pre-processing on something perfectly repeated  
> several
> times, it really finds the frames, and it doesn't require knowing  
> the tempo.
> If you don't know the tempo, the only problem will be misalignment,  
> which
> will leave little bits of audio that were too short to find  
> matching frames,
> but most of the processed waveform will still be silence.

Seems like knowing the tempo would allow the encoding phase to take  
far less time.  It makes sense that you don't absolutely "need" it,  
but you did say it takes a really long time to find matches.


> Btw, are all lossless compression methods working in the time domain?

I would guess that most lossless audio compression methods are time  
domain.  However, LJPG (lossless JPEG) uses a very efficient lossy  
compression followed by lossless compression of the difference.  I  
wouldn't be surprised if there is an audio codec which combines lossy  
frequency domain compression with lossless compression of the  
difference between the lossy version and the original.  If there  
isn't then I'll just patent that...

Brian Willoughby
Sound Consulting