Yes that was it. I wasn't setting a variable correctly and it wad defaulting to 2 channels. Also, during decoding, my code wasn't setup to handle anything else other than 2 channels. Both of these things I looked at but I never thought to change them both at the same time.