<div dir="ltr">I have a project I'm investigating. The goal is to basically add lossless extensions to Opus. You have an Opus stream with standard packets, but interwoven in there are extension packets that contain the residuals. Ideally, compliant decoders play the stream back and ignore the extension packets. This (hopefully) makes the "lossless" stream compatible with existing players. Specialized decoders decode the standard Opus (CELT) packets, the residual extension packets, and then combines them to reproduce a bit-exact stream to the original PCM.<div>
<br></div><div>Now those of you familiar with how Opus works will readily identify some issues with why this might be completely impossible. Those problems are to be explored later. First, I need to know that current players can even handle streams with extension packets. The problem is that, currently, these packets are completely undefined.</div>
<div><br></div><div>My current idea is to have a packet with zero frames (illegal) and then put the extension data in that (illegal) packet's padding area. As I remember it, this is against the Opus RFC for at least two reasons:</div>
<div><br></div><div>1. An Opus packet must have at least one frame.</div><div>2. A packet's padding may not contain anything but null bytes.</div><div><br></div><div>However, the spec also says packets that violate this should simply be ignored, which is what I'm hoping players do. Another question is how streams with extension packets get muxed into containers like Matroska. Will mkvtools drop the extension packets?</div>
<div><br></div><div>So how should I go about structuring these things? I assume somewhere in the padding I need to place an identifier in there so that later decoders don't confuse my extension packets with other types.</div>
<div><br></div><div><br></div></div>