[flac-dev] awk vs. mawk

[Jan Stary]

(ping, if there is any interest).

On Dec 03 13:21:16, hans at stare.cz wrote:
> On Nov 30 14:33:47, hans at stare.cz wrote:
> > On Nov 26 22:39:27, hans at stare.cz wrote:
> > > ./test_replaygain.sh fails for me in tonegenerator(), saying:
> > > 
> > > ./test_replaygain.sh[91]: mawk: not found
> > > Testing FLAC replaygain 8000 (8000 x 1) ... -: ERROR: got partial sample
> > > 
> > > Apparently, the tone-generating awk script does not work with
> > > my system's awk, which is "awk version 20110810" as distributed
> > > with current OpenBSD 5.6 GENERIC.MP#610 amd64.
> > > With mawk-1.3.4.20140914 it works fine
> > > (and ./test_replaygain.sh passes).
> > > 
> > > Can anyone please shed some light on what's mawk-spacific
> > > about that awk script? It seems to be just a sine-generator.
> > 
> > > Ha! If the final
> > > 	printf("%c", byte[bx]);
> > > is changed to
> > > 	printf("%d", byte[bx]);
> > > the results produced by mawk and awk are identical.
> > > So the diference must be just that mawk printf's "%c" differently.
> > 
> > > the two sequences of samples are identical as sequences of integers
> > > - they just differ in the way they are printed out with AWK's printf("%c")
> > > as opposed to MAWK's printf("%c").
> > > 
> > > I'll try to find time and look into it; this shouldn't be hard,
> > > we just want to generate a sine wave portably.
> > 
> > Indeed, the difference seems to be in how my systems AWK,
> > as opposed to MAWK, printf's zero bytes.
> > 
> > With mawk, a printf("%c") of a zero is always a zero byte (^@) in the output.
> > With my system awk, it's sometimes a zero byte, and sometimes nothing.
> 
> Turns out it's a 20 year old bug in OpenBSD's awk:
> http://marc.info/?l=openbsd-misc&m=141735117703872&w=2
> http://marc.info/?l=openbsd-tech&m=141739334413710&w=2
> 
> Now ./test_replaygain.sh seems to work the same with awk, mawk and gawk
> on OpenBSD/amd64, OpenBSD/i386, and Debian/x86_64.
> 
> The diff below removes the check for mawk, and the comments
> about different AWKs, and just calls 'awk' for the script.
> 
> gawk 4.0.1 and 4.1.1 works for me too.
> Erik, do you please remember what exactly happened here?
> Does gawk >= still fail for you?
> https://git.xiph.org/?p=flac.git;a=commit;h=5797009fa2beb0426d74485e7624775e2e58e1d1
> 
> While there, remove the *=2 loop, exponentiation is in the language.
> 
> > Anyway, the whole point of that AWK script is to produce a raw sine wave.
> > Why are we doing it with AWK in the first place? Surely the FLAC
> > distrubution contains tools to produce test sine waves,
> > e.g. src/test_streams/main.c contains some, used in test_streams.sh.
> > Can we make test_replaygain.sh also use the produced sine waves,
> > instead of generating its own with AWK?
> 
> This I haven't looked at yet, but I believe we could
> remove the awk sine generator altogether and just ask
> test_streams to make the streams.
> 
> Also, the current script stores the individual bytes
> of the 24bit sample in an array, just to print them
> back in reverse and feed them to flac --endian=big.
> Why is that? Why doesn't it just print the bytes
> from the lowest and feed them to --endian=little?
> 
> Also, the current script shifts the whole wave to the positive
> and feeds it to --sign=unsigned; why can't we just leave the wave
> as it is and feed it to --sign=signed?
> 
> 
> 	Jan
> 
> 
> --- test_replaygain.sh.orig	Wed Dec  3 12:11:46 2014
> +++ test_replaygain.sh	Wed Dec  3 12:23:24 2014
> @@ -81,34 +81,19 @@ fi
>  
>  check_flac
>  
> -
> -if mawk ; then
> -	AWK=mawk
> -else
> -	# Really hope awk is not gawk, because the following AWK script doesn't
> -	# work correctly with gawk 4.0.1 but did with earlier versions.
> -	AWK=awk
> -	fi
> -
>  # Replay gain tests - Test the rates which have specific filter table entries
>  # and verify that harmonics can be processed correctly.
>  
>  tonegenerator ()
>  {
> -	# When using GAWK, use --lint=posix to identify non-POSIX awk usages.
> -    $AWK -- '
> +    awk -- '
>      BEGIN {
>              samplerate = '$1';
>  
>              tone = 1000;
>              duration = 1;
>              bitspersample = 24;
> -
> -            samplemidpoint = 1;
> -			for (sps = 0 ; sps < bitspersample - 1 ; sps++) {
> -				samplemidpoint *= 2;
> -			}
> -
> +	    samplemidpoint = 2^(bitspersample - 1);
>              samplerange = samplemidpoint - 1;
>  
>              pi = 4 * atan2(1,1);
> @@ -127,7 +112,7 @@ tonegenerator ()
>                      }
>              }
>  
> -    }' /dev/null |
> +    }' /dev/null | \
>      flac${EXE} --silent --no-error-on-compression-fail --force-raw-format \
>          --endian=big --channels=1 --bps=24 --sample-rate=$1 --sign=unsigned -
>  }
> _______________________________________________
> flac-dev mailing list
> flac-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/flac-dev