[Icecast-dev] Should Icecast send the "Connection: close" header?

[Spencer Russell]

I'm accessing a server using Icecast 2.4.0. It is streamed using HTTP 1.1 using the Transfer-Encoding: chunked - so far so good. The issue is that the HTTP library I'm using uses connection pooling by default and attempts to re-use connections if the server does not send the "Connection: close" header. As part of that re-use, when you end a request/response cycle, it tries to read from the socket through the end of the response, which doesn't work well when the stream is infinite. The idea is that it's setting things up so that there isn't data left on the socket and things are ready for the next transaction.

You can see the following conversation for more context:
https://github.com/JuliaWeb/HTTP.jl/issues/339

AFAICT the issue is that there's no way deterministically for the client to know whether the stream its receiving is infinite, so it doesn't know whether it's safe to try to read through the rest of the response to get the stream state ready for the next transaction. I imagine that putting Icecast behind a reverse proxy would have similar issues if they tried to reuse the connection, so I'm not sure if there's a standard client-side approach to handling this, or if it would be better for Icecast to send the "Connection: close" header.

One client-side workaround I can think of is to disable connection re-use for any chunked transfers, but that seems maybe overly-conservative.

Here's the transaction I'm seeing:

$ curl -v doppler.media.mit.edu/impoundment.ogg
*   Trying 18.85.59.166...
* TCP_NODELAY set
* Connected to doppler.media.mit.edu (18.85.59.166) port 80 (#0)
> GET /impoundment.ogg HTTP/1.1
> Host: doppler.media.mit.edu
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Mon, 29 Oct 2018 15:58:27 GMT
< Server: Icecast 2.4.0
< Content-Type: application/ogg
< Cache-Control: no-cache
< Expires: Sat, 26 Jul 1997 05:00:00 GMT
< Pragma: no-cache
< icy-name: no name
< icy-pub: 0
< Transfer-Encoding: chunked
< 
...