[Vorbis] libvorbisfile dynamic linkage with pkg-config - inter-library dependencies?

[Richard Ash]

On Mon, 11 Aug 2014 13:03:32 -0700
Ralph Giles <giles at thaumas.net> wrote:

> On 2014-08-09 3:04 PM, Richard Ash wrote:
> > vorbis_block_init() is called from ExportOGG.cpp, although that file
> > doesn't #include vorbis/codec.h - the only include from libvorbis is
> > #include <vorbis/vorbisenc.h>, which presumably brings the necessary
> > headers in indirectly?
> 
> Yes, vorbisenc.h includes codec.h.
That makes sense. It could be argued from that we are using chunks of
the libvorbis API and need to link explicitly to it, but it didn't look
like that when I first hit the problem.

> > The only solution I have found is to explicitly ask pkg-config for
> > the dependent libraries (libvorbis and libogg) when calling
> > pkg-config, so do
> >  pkg-config --libs --print-errors "vorbisfile vorbis ogg"
> > in order to get
> >  -lvorbisfile -lvorbis -logg
> 
> So vorbisfile.pc has vorbis as a Requires.private. I think the way
> this is supposed to work is that if you're linking to the shared
> libraries, transitive deps in the .so files will pull in vorbis when
> you '-l vorbisfile'. When you link statically (so pkg-config --libs
> --static vorbisfile) then you get -lvorbis -logg -lm, etc.
--static works, but this isn't intended to be a static build

> The question is why this isn't working for you. Are you linking
> statically? Is the build system passing --static to pkg-config? There
> are systems where the linker doesn't see dependencies inside .so
> files, but x86_64-pc-linux-gnu isn't one of them.
As these are system libraries, I wonder if the dependencies are getting
stripped / hidden at build time by the package management system
(Gentoo's Portage)?

There are a couple of patches being applied to the libvorbis source
before building, one of which is meant to make the -Wl,--as-needed
linker option work correctly in libtool environments (explanation here
http://www.cygwin.com/ml/cygwin/2012-10/msg00396.html, patch is
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/eclass/ELT-patches/as-needed/2.4.2?diff_format=s&view=markup)

On this system LDFLAGS="-Wl,-O1 -Wl,--as-needed" so the link command
run by libtool when building libvorbis looks like

x86_64-pc-linux-gnu-gcc -shared  -fPIC -DPIC  .libs/vorbisfile.o
-Wl,-rpath
-Wl,/var/tmp/portage/media-libs/libvorbis-1.3.4/work/libvorbis-1.3.4-amd64/lib/.libs
-Wl,--as-needed ./.libs/libvorbis.so -lm -logg  -O2 -march=native
-mtune=native -Wl,-O1   -Wl,-soname -Wl,libvorbisfile.so.3
-o .libs/libvorbisfile.so.3.3.6

I'm no expert, but this probably means that libvorbisfile.so has a
dependency on libvorbis.so, but the libvorbis.so symbols are not
re-exported by libvorbisfile.so. Hence my build error.

This is (in general terms) supposed to be a good thing, in that changes
to libvorbis do not break the libvorbisfile ABI if the latter does not
change, but it's causing a problem here, because historically anyone
linking libvorbisfile.so got the libvorbis.so symbols along for the
ride, and they have come to rely upon the freebies and miss them being
taken away. This isn't just Audacity - a number of other packages can
be found failing (recently) in this way on Gentoo by Googling the error
messages.

> For theora, we use a 'Requires' instead of a 'Requires.private' so it
> works unconditionally. If my understand above is correct, that's not
> optimal, but might work around your problem. Try that as well?
I found OpenBSD playing with that option
http://openbsd.7691.n7.nabble.com/audio-vorbis-reinstate-inter-library-dependencies-td225618.html
At the time I was skeptical about it, but I think it comes down to a
design question about the various vorbis APIs, which is where I am
unclear, because there are two possible views, and at the moment we
have a little of each:

1. If libvorbisfile is an extension of libvorbis, and users are
expected to pick and mix between them, then pulling in libvorbisfile
should bring libvorbis along. This happens in the headers (because
vorbisenc.h includes codec.h) and used to happen in the shared library
(because of, arguably, a libtool bug). With Gentoo's patch it doesn't
happen, and the future-proof solution is probably to set Requires=vorbis
in the pkg-config file, so that all users get the necessary linker
flags.

2. If libvorbisfile is supposed to encapsulate libvorbis, users are
not expected to know about the latter when using the former
libvorbisfile is encapsulating and hiding the libvorbis API. In this
case libvobisfile should not pull in libvorbis. The headers are
unhelpful (although probably hard to fix) to this, but the (Gentoo
patched) shared library is correct because it hides the libvorbis API
from users of libvorbisfile. The pkg-config file should not be changed,
because any users mixing and matching (like Audacity) should be
explictly linking to libvorbis, and those using pure libvorbisfile
as intended will not have a problem.

In the short term I will get Audacity to explicitly link to both
libraries by altering the pkg-config request (because this is the line
of least resistance), but I would be interested to know which of these
was the upstream developers intentions.

Richard