[vorbis-dev] questions re residuevqtrain

[Mike Coleman]

I'm trying to understand the residuevqtrain program, and I have some questions
for Monty, Erik, or anyone that understands how it's supposed to work.

I captured TRAIN_RES data from an encoding of a single track (about 4:43),
producing two files, residue_0.vqd (3727 lines, = 3727 points?) and
residue_1.vqd (huge).  I then did a run with the parameters from the usage
message

   residuevqtrain test_256_6_8_01_0 -p 256,6,8 -e .01 residue_0.vqd

(with the version of residuevqtrain at the CVS head, last changed around 11/9
or so).

1.  I'm thinking that this program is basically supposed to solve the VQ
    design problem, as documented on 'http://data-compression.com/vq.html'
    (with some variations for vorbis).

    In that problem, the goal is to choose the codevectors to minimize average
    distortion (average distance between each training vector and its
    associated codevector).  This measure, or something similar, is given in
    the residuevqtrain output as 'metric error'.  In the run I did, though,
    the value of metric error actually *increases* over the 1000 passes (see
    output below).  Isn't this a bad thing, or am I missing something?

2.  The residuevqtrain algorithm actually seems to be trying to minimize a
    slightly different measure, marked as 'dist' in the output.  If I
    understand the idea correctly, the idea is to choose codevectors so that a
    nearly equal number of training vectors will be associated with each one.
    The 'dist' measure is a measure of how much this is so, smaller values
    being better.  Why would this be better than just minimizing distortion?
    Although I can imagine the two metrics being highly correlated, it looks
    to me like the former would be better when they differ (think of what
    happens if you have a group of training vectors clumped together).

3.  The program does seem to reduce 'dist', but I notice that the lowest value
    seen for it and for 'metric error' in this run was actually at pass 0.
    Does this mean that we should just stop at pass 0, or that the metrics are
    wrong, or is something else going on here?

4.  If I'm reading the 'quantized entries' from the output .vqi file
    correctly, it looks as if there are a large number of duplicate entries
    (maybe because of quantization?).  Isn't this bad?  Or am I misreading?
    (In my example, I'm reading the first six lines as the first codevector,
    and so on.)

I have more questions, but I'll stop here in case this is all just
cluelessness on my part.

--Mike

$ residuevqtrain test_256_6_8_01_0 -p 256,6,8 -e .01 residue_0.vqd 
128 colums per line in file residue_0.vqd
reseeding with quantization....
Pass #0... : dist 0.361175(305.73) metric error=1.73526 
cells shifted this iteration: 4
cell diameter: 4.66::10.3::36.8 (0 unused/79 dup)
Pass #1... : dist 1.82539(305.73) metric error=9.36826 
cells shifted this iteration: 32
cell diameter: 4.57::28.6::43.1 (5 unused/77 dup)
Pass #2... : dist 1.70066(305.73) metric error=9.79896 
cells shifted this iteration: 5
cell diameter: 4.57::29.9::46.5 (3 unused/76 dup)
[...]
Pass #995... : dist 0.554347(305.73) metric error=17.324 
cells shifted this iteration: 0
cell diameter: 4.4::26.7::48.8 (0 unused/70 dup)
Pass #996... : dist 0.554347(305.73) metric error=17.324 
cells shifted this iteration: 0
cell diameter: 4.4::26.7::48.8 (0 unused/70 dup)
Pass #997... : dist 0.554347(305.73) metric error=17.324 
cells shifted this iteration: 0
cell diameter: 4.4::26.7::48.8 (0 unused/70 dup)
Pass #998... : dist 0.554347(305.73) metric error=17.324 
cells shifted this iteration: 0
cell diameter: 4.4::26.7::48.8 (0 unused/70 dup)
Pass #999... : dist 0.554347(305.73) metric error=17.324 
cells shifted this iteration: 0
cell diameter: 4.4::26.7::48.8 (0 unused/70 dup)


-- 
[O]ne of the features of the Internet [...] is that small groups of people can
greatly disturb large organizations.  --Charles C. Mann

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to 'vorbis-dev-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.