[vorbis-dev] Re: lossless ogg encoding with good compression rate

[General purpose mail]

thanks for your interest Dimitry but you must keep in mind two things:
1 - the wav must be generated from the same program, also a shift of one
byte or something like that
   generate a dister
2 - in binary a sum is done in 2's complement notation

follow the complete procedure I've followed (with only 4 files sorry)
also excuse my english I'm not fluent and is too late in the morning here

I've chosen 4 file compressed with format .ape, .mp3, .ogg ,
converted back to wav with dbPowerAmp like all the conversions here.
REMEMBER TO UNCHECK VOLUME NORMALIZE
so we have
in01.wav
in02.wav
in03.wav
in04.wav

convert them to ogg always with quality Q8 256 Kb/s

in01.ogg
in02.ogg
in03.ogg
in04.ogg

rename them to

out01.ogg
out02.ogg
out03.ogg
out04.ogg

convert them to

out01.wav
out02.wav
out03.wav
out04.wav

apply the following code:
(I know that php It's not the faster and the better for this but I feel
comfortable with it ;)

<?php

// rm *.tmp *.tmp.bz2; php elab.php && bzip2 *.tmp ; ls -l

diff2lossless('in01.wav', 'out01.wav');
diff2lossless('in02.wav', 'out02.wav');
diff2lossless('in03.wav', 'out03.wav');
diff2lossless('in04.wav', 'out04.wav');

function diff2lossless($filename1, $filename2) {

 $blockSize = max(filesize($filename1) -2, 0);
 $lossbit = 0;

 $f1 = fopen($filename1, "rb");
 $f2 = fopen($filename2, "rb");
 $fo1 = fopen($filename2.'.tmp', "wb");

 //while(!feof($f1)) {
  $c1 = fread ($f1, $blockSize);
  $c2 = fread ($f2, $blockSize);
  $tmps = '';
  for($i=0; $i < $blockSize -2; $i += 2) {
   $tmp1 = (ord($c1{$i+1} << (8-$lossbit))) & (ord($c1{$i}) >> $lossbit);
   $tmp2 = (ord($c2{$i+1} << (8-$lossbit))) & (ord($c2{$i}) >> $lossbit);
   $tmp3 = $tmp1 -$tmp2;
   $tmps .= chr($tmp3 >> 8) . chr($tmp3 & 0xFF);
  }
  fwrite($fo1, $tmps);
 //}

 fclose($f1);
 fclose($f2);
 fclose($fo1);

 echo "\n\n.";
}
?>

you must obtain compression rate around 35% (28.53% -> 34.58% for me)

I can also send all the stuff if someone need it but keep in mind that they
are about 360 Mb

P.S. total elapsed time is more than a quarter of hour with php on a athlon
xp 1.8

+-----------+------------+------------+------------+-----------+
| n.        | 1          | 2          | 3          | 4         |
+-----------+------------+------------+------------+-----------+
| ORIG      | 49.153.582 | 36.463.150 | 36.350.206 | 39.866.446|
+-----------+------------+------------+------------+-----------+
| ogg       | 8.031.545  | 5.675.553  | 5.717.410  | 6.155.756 |
| bz2 diff  | 8.961.170  | 6.934.106  | 6.713.344  | 5.216.487 |
+-----------+------------+------------+------------+-----------+
| SUM       | 16.992.715 | 12.609.659 | 12.430.754 | 11.372.243|
+-----------+------------+------------+------------+-----------+
| %ogg      | 16,34%     | 15,57%     | 15,73%     | 15,44%    |
| %bz2 diff | 18,23%     | 19,02%     | 18,47%     | 13,08%    |
+-----------+------------+------------+------------+-----------+
| %sum      | 34,57%     | 34,58%     | 34,20%     | 28,53%    |
+-----------+------------+------------+------------+-----------+

cheers

Francesco Riosa

<p>----- Original Message -----
From: "Dimitry Andric" <dim at xs4all.nl>
To: "General purpose mail" <maillist at pnpitalia.it>
Cc: "Vorbis Development List" <vorbis-dev at xiph.org>
Sent: Thursday, June 26, 2003 3:41 AM
Subject: Re: lossless ogg encoding with good compression rate

On 2003-06-26 at 02:40:30 General purpose mail wrote:

> 1) create a wav file [A.wav]:
> 2) encode it in ogg format (my best result with high 7) [B.ogg]
> 3) decode the ogg into a wav again [C.wav] (the size will be the same as
A.wav)
> 4) create a file with the diff of each sample from A.wav
> with B.wav [D.tmp] also with the  same size at least of the
> header.
> 5) bzip2 the D.tmp [D.tmp.bz2]
> 6) keep B.ogg and D.tmp.bz2 (the size will be around 17% of the original
for each file)

> simply is't it?

I'm afraid this will not work, except in some special cases. It's
probably best to go read some books on data compression and
information theory. ;)

For example:

1) I start with test.wav here (5:59.200/63,362,924 bytes). It's just a
music track, nothing weird.
2) Encoding to ogg with -q7 gives me test.ogg, 10,472,702 bytes.
3) Decoding this ogg gives me test2.wav, 63,362,924 bytes again.
4) I make a diff with Cool Edit, by mix pasting an inverted copy of
test2.wav into test.wav. This gives diff.wav, also 63,362,924 bytes.
5) However, compressing diff.wav with bzip2 -9, results in
diff.wav.bz2, which is 39,858,199 bytes, so only ~62.9% compression.
6) The end size is thus: 10,472,702 bytes ogg + 39,858,199 bytes bz2,
totaling 50,330,901 bytes, so ~79.4% compression.
7) This is much worse than FLAC, for example, which manages to
compress the original test.wav to 37,728,889 bytes, so ~59.5%
compression. Monkey's Audio even manages to get it to 34,971,640
bytes, so ~55.2% compression.

The problem is that you cannot guarantee that step 5) will compress
the diff so much. If it were so simple, I guess FLAC/APE/etc would
already be using this for a long time. :)

<p>--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.