[xiph-commits] r17679 - trunk/squishyball

[xiphmont at svn.xiph.org]

Author: xiphmont
Date: 2010-11-29 06:43:55 -0800 (Mon, 29 Nov 2010)
New Revision: 17679

Modified:
   trunk/squishyball/main.c
   trunk/squishyball/squishyball.1
Log:
Don't randomiz the A/B samples in A/B/X testing.  Revert A/B/X to the 
way other apps do it, and point out in the manpage that X/X/Y provides 
full randomization.



Modified: trunk/squishyball/main.c
===================================================================
--- trunk/squishyball/main.c	2010-11-29 14:34:48 UTC (rev 17678)
+++ trunk/squishyball/main.c	2010-11-29 14:43:55 UTC (rev 17679)
@@ -78,8 +78,8 @@
           "USAGE:\n"
           "  squishyball [options] fileA [fileB [[-c] fileN...]]\n\n"
           "OPTIONS:\n"
-          "  -a --ab                : Perform randomized A/B test\n"
-          "  -b --abx               : Perform randomized A/B/X test\n"
+          "  -a --ab                : Perform A/B test\n"
+          "  -b --abx               : Perform A/B/X test\n"
           "  -B --beep-flip         : Mark transitions between samples with\n"
           "                           a short beep\n"
           "  -c --casual            : casual mode; load up to ten\n"
@@ -116,7 +116,7 @@
           "                           playback.\n"
           "  -v --verbose           : Produce more progress information.\n"
           "  -V --version           : Print version and exit.\n"
-          "  -x --xxy               : Perform randomized X/X/Y test.\n"
+          "  -x --xxy               : Perform X/X/Y (triangle) test.\n"
           "\n"
           "INTERACTION:\n"
           "    a b x    : Switch playback between A, B [and X] samples.\n"
@@ -180,8 +180,8 @@
 void randomize_samples(int *r,int *cchoice, int test_mode){
   switch(test_mode){
   case 1:
-    r[0] = random()&1;
-    r[1] = 1-r[0];
+    r[0] = 0;
+    r[1] = 1;
     r[2] = random()&1;
     *cchoice = (r[1]==r[2] ? 1 : 0);
     break;

Modified: trunk/squishyball/squishyball.1
===================================================================
--- trunk/squishyball/squishyball.1	2010-11-29 14:34:48 UTC (rev 17678)
+++ trunk/squishyball/squishyball.1	2010-11-29 14:43:55 UTC (rev 17679)
@@ -26,7 +26,7 @@
 
 .SH TEST TYPES
 .IP "\fB-a --ab"
-Perform randomized A/B test on two input samples.
+Perform A/B test on two input samples.
 
 A/B testing randomizes the order of two input samples and presents
 them, unnamed, as sample 'A' and sample 'B'.  In each trial the user
@@ -34,15 +34,21 @@
 re-randomized for the next trial.  This test is useful for
 establishing relative or preferred quality between two samples.
 .IP "\fB-b --abx"
-Perform randomized A/B/X test on two input samples.
+Perform A/B/X test on two input samples.
 
-A/B/X testing randomizes the order of two input samples and presents
-them, unnamed, as sample 'A' and sample 'B'.  A third sample 'X' is
-also chosen randomly from either either 'A' or 'B'.  In each trial, the
-user selects A or B as the sample believed to be the same as X. All
-samples are then re-randomized for the next trial. This test is useful
-for determining if any differences are audible between two samples and
-to what confidence level.
+A/B/X presents two input samples, unrandomized, as sample 'A' and
+sample 'B'.  A third sample 'X' is chosen randomly from either 'A'
+or 'B'.  In each trial, the user selects A or B as the sample believed
+to be the same as X. X is then re-randomized for the next trial. This
+test is useful for determining if any differences are audible between
+two samples and to what confidence level.
+
+Note that because the A and B samples are not randomized (they are
+presented in the order given on the command line as per standard
+industry practice), an A/B/X test does not eliminate ordering bias.
+A stronger version of this test that randomizes all samples is the
+X/X/Y test below.
+
 .IP "\fB-c --casual"
 Perform casual comparison of up to ten samples (default).
 
@@ -52,12 +58,15 @@
 .IP "\fB-x --xxy"
 Perform randomized X/X/Y test on two input samples.
 
-X/X/Y testing is a form of A/B/X testing in which the position of the 'X'
-sample is not known ahead of time to be in the third position. In each trial,
-the user selects which of sample 1, 2 or 3 is believed to be the
-sample that is different from the other two. This test is useful for
-determining if any differences are audible between two samples and to
-what confidence level.
+X/X/Y testing is a form of A/B/X testing in which the order of all
+samples is randomized and the position of the 'X' sample is not known
+ahead of time to be in the third position. In each trial, the user
+selects which of sample 1, 2 or 3 is believed to be the sample that is
+different from the other two. This test is useful for determining if
+any differences are audible between two samples and to what confidence
+level.  It is a stronger version of the A/B/X test that eliminates
+sample order bias.
+
 .SH OTHER OPTIONS
 .IP "\fB-B --beep-flip"
 Mark transitions between samples with a short beep.