[flac-dev] awk vs. mawk

[Jan Stary]

On Nov 30 14:33:47, hans at stare.cz wrote:
> On Nov 26 22:39:27, hans at stare.cz wrote:
> > ./test_replaygain.sh fails for me in tonegenerator(), saying:
> > 
> > ./test_replaygain.sh[91]: mawk: not found
> > Testing FLAC replaygain 8000 (8000 x 1) ... -: ERROR: got partial sample
> > 
> > Apparently, the tone-generating awk script does not work with
> > my system's awk, which is "awk version 20110810" as distributed
> > with current OpenBSD 5.6 GENERIC.MP#610 amd64.
> > With mawk-1.3.4.20140914 it works fine
> > (and ./test_replaygain.sh passes).
> > 
> > Can anyone please shed some light on what's mawk-spacific
> > about that awk script? It seems to be just a sine-generator.
> 
> > Ha! If the final
> > 	printf("%c", byte[bx]);
> > is changed to
> > 	printf("%d", byte[bx]);
> > the results produced by mawk and awk are identical.
> > So the diference must be just that mawk printf's "%c" differently.
> 
> > the two sequences of samples are identical as sequences of integers
> > - they just differ in the way they are printed out with AWK's printf("%c")
> > as opposed to MAWK's printf("%c").
> > 
> > I'll try to find time and look into it; this shouldn't be hard,
> > we just want to generate a sine wave portably.
> 
> Indeed, the difference seems to be in how my systems AWK,
> as opposed to MAWK, printf's zero bytes.
> 
> With mawk, a printf("%c") of a zero is always a zero byte (^@) in the output.
> With my system awk, it's sometimes a zero byte, and sometimes nothing.

Turns out it's a 20 year old bug in OpenBSD's awk:
http://marc.info/?l=openbsd-misc&m=141735117703872&w=2
http://marc.info/?l=openbsd-tech&m=141739334413710&w=2

Now ./test_replaygain.sh seems to work the same with awk, mawk and gawk
on OpenBSD/amd64, OpenBSD/i386, and Debian/x86_64.

The diff below removes the check for mawk, and the comments
about different AWKs, and just calls 'awk' for the script.

gawk 4.0.1 and 4.1.1 works for me too.
Erik, do you please remember what exactly happened here?
Does gawk >= still fail for you?
https://git.xiph.org/?p=flac.git;a=commit;h=5797009fa2beb0426d74485e7624775e2e58e1d1

While there, remove the *=2 loop, exponentiation is in the language.

> Anyway, the whole point of that AWK script is to produce a raw sine wave.
> Why are we doing it with AWK in the first place? Surely the FLAC
> distrubution contains tools to produce test sine waves,
> e.g. src/test_streams/main.c contains some, used in test_streams.sh.
> Can we make test_replaygain.sh also use the produced sine waves,
> instead of generating its own with AWK?

This I haven't looked at yet, but I believe we could
remove the awk sine generator altogether and just ask
test_streams to make the streams.

Also, the current script stores the individual bytes
of the 24bit sample in an array, just to print them
back in reverse and feed them to flac --endian=big.
Why is that? Why doesn't it just print the bytes
from the lowest and feed them to --endian=little?

Also, the current script shifts the whole wave to the positive
and feeds it to --sign=unsigned; why can't we just leave the wave
as it is and feed it to --sign=signed?


	Jan


--- test_replaygain.sh.orig	Wed Dec  3 12:11:46 2014
+++ test_replaygain.sh	Wed Dec  3 12:23:24 2014
@@ -81,34 +81,19 @@ fi
 
 check_flac
 
-
-if mawk ; then
-	AWK=mawk
-else
-	# Really hope awk is not gawk, because the following AWK script doesn't
-	# work correctly with gawk 4.0.1 but did with earlier versions.
-	AWK=awk
-	fi
-
 # Replay gain tests - Test the rates which have specific filter table entries
 # and verify that harmonics can be processed correctly.
 
 tonegenerator ()
 {
-	# When using GAWK, use --lint=posix to identify non-POSIX awk usages.
-    $AWK -- '
+    awk -- '
     BEGIN {
             samplerate = '$1';
 
             tone = 1000;
             duration = 1;
             bitspersample = 24;
-
-            samplemidpoint = 1;
-			for (sps = 0 ; sps < bitspersample - 1 ; sps++) {
-				samplemidpoint *= 2;
-			}
-
+	    samplemidpoint = 2^(bitspersample - 1);
             samplerange = samplemidpoint - 1;
 
             pi = 4 * atan2(1,1);
@@ -127,7 +112,7 @@ tonegenerator ()
                     }
             }
 
-    }' /dev/null |
+    }' /dev/null | \
     flac${EXE} --silent --no-error-on-compression-fail --force-raw-format \
         --endian=big --channels=1 --bps=24 --sample-rate=$1 --sign=unsigned -
 }