[flac-dev] awk vs. mawk

[Jan Stary]

On Dec 10 23:04:44, mle+la at mega-nerd.com wrote:
> Definitely interested in a cross platform solution to this.

Diff below. Please test.

On Dec 03 13:21:16, hans at stare.cz wrote:
> On Nov 30 14:33:47, hans at stare.cz wrote:
> > On Nov 26 22:39:27, hans at stare.cz wrote:
> > > ./test_replaygain.sh fails for me in tonegenerator(), saying:
> > > 
> > > ./test_replaygain.sh[91]: mawk: not found
> > > Testing FLAC replaygain 8000 (8000 x 1) ... -: ERROR: got partial sample
> > > 
> > > Apparently, the tone-generating awk script does not work with
> > > my system's awk, which is "awk version 20110810" as distributed
> > > with current OpenBSD 5.6 GENERIC.MP#610 amd64.
> > > With mawk-1.3.4.20140914 it works fine
> > > (and ./test_replaygain.sh passes).
> > > 
> > > Can anyone please shed some light on what's mawk-spacific
> > > about that awk script? It seems to be just a sine-generator.
> > 
> > > Ha! If the final
> > > 	printf("%c", byte[bx]);
> > > is changed to
> > > 	printf("%d", byte[bx]);
> > > the results produced by mawk and awk are identical.
> > > So the diference must be just that mawk printf's "%c" differently.
> > 
> > > the two sequences of samples are identical as sequences of integers
> > > - they just differ in the way they are printed out with AWK's printf("%c")
> > > as opposed to MAWK's printf("%c").
> > > 
> > > I'll try to find time and look into it; this shouldn't be hard,
> > > we just want to generate a sine wave portably.
> > 
> > Indeed, the difference seems to be in how my systems AWK,
> > as opposed to MAWK, printf's zero bytes.
> > 
> > With mawk, a printf("%c") of a zero is always a zero byte (^@) in the output.
> > With my system awk, it's sometimes a zero byte, and sometimes nothing.
> 
> Turns out it's a 20 year old bug in OpenBSD's awk:
> http://marc.info/?l=openbsd-misc&m=141735117703872&w=2
> http://marc.info/?l=openbsd-tech&m=141739334413710&w=2

Now ./test_replaygain.sh seems to work the same with awk, mawk and gawk
on OpenBSD/amd64, OpenBSD/i386, and Debian/x86_64.

The diff below removes the check for mawk, and the comments
about different AWKs, and just calls 'awk' for the script.

gawk 4.0.1 and 4.1.1 works for me too.
Erik, do you please remember what exactly happened here:
https://git.xiph.org/?p=flac.git;a=commit;h=5797009fa2beb0426d74485e7624775e2e58e1d1
Does gawk >= 4.0.1 still fail for you?
With either the current script or with this diff?

While there, remove the *=2 loop, exponentiation is in the language.

> > Anyway, the whole point of that AWK script is to produce a raw sine wave.
> > Why are we doing it with AWK in the first place? Surely the FLAC
> > distrubution contains tools to produce test sine waves,
> > e.g. src/test_streams/main.c contains some, used in test_streams.sh.
> > Can we make test_replaygain.sh also use the produced sine waves,
> > instead of generating its own with AWK?

I still believe we don't need the awk script
to generate a sine wave in the first place
- we can ask test_streams; but I haven't looked into that yet.

Two points not addressed in the diff:

The current script stores the individual bytes
of the 24bit sample in an array, just to print them
back in reverse and feed them to flac --endian=big.
Why is that? Why can't we just print the bytes
from the lowest as they come by /256 and feed them to --endian=little?

The current script shifts the whole wave to the positive
and feeds it to --sign=unsigned; why can't we just leave
the wave as it is and feed it to --sign=signed?

	Jan


--- ./test_replaygain.sh.orig	Thu Dec 11 08:57:12 2014
+++ ./test_replaygain.sh	Wed Dec  3 13:11:33 2014
@@ -81,34 +81,19 @@ fi
 
 check_flac
 
-
-if mawk ; then
-	AWK=mawk
-else
-	# Really hope awk is not gawk, because the following AWK script doesn't
-	# work correctly with gawk 4.0.1 but did with earlier versions.
-	AWK=awk
-	fi
-
 # Replay gain tests - Test the rates which have specific filter table entries
 # and verify that harmonics can be processed correctly.
 
 tonegenerator ()
 {
-	# When using GAWK, use --lint=posix to identify non-POSIX awk usages.
-    $AWK -- '
+    awk -- '
     BEGIN {
             samplerate = '$1';
 
             tone = 1000;
             duration = 1;
             bitspersample = 24;
-
-            samplemidpoint = 1;
-			for (sps = 0 ; sps < bitspersample - 1 ; sps++) {
-				samplemidpoint *= 2;
-			}
-
+	    samplemidpoint = 2^(bitspersample - 1);
             samplerange = samplemidpoint - 1;
 
             pi = 4 * atan2(1,1);
@@ -127,7 +112,7 @@ tonegenerator ()
                     }
             }
 
-    }' /dev/null |
+    }' /dev/null | \
     flac${EXE} --silent --no-error-on-compression-fail --force-raw-format \
         --endian=big --channels=1 --bps=24 --sample-rate=$1 --sign=unsigned -
 }