Many packages FTBFS with libxml 2.12
Void Linux identified that 62 of 304 packages using libxml2 fail to build with libxml 2.12.1, main reason seems to be shuffling prototypes in headers:
0ad
atril
audacious-plugins
badwolf
cegui
cloudfuse
compiz-plugins-experimental
compiz-plugins-extra
darktable
dia
eom
evolution-data-server
gdl
gnome-builder
gnumeric
goffice
grilo-plugins
gtkpod
gtranslator
gupnp
gupnp-av
gupnp-tools
gupnp1.2
inkscape
kdoctools
libcmis
libcroco
libgdal
libgsf
libiio
liblangtag
libmusicbrainz5
libpdal
libreoffice
libsoup
libvirt
libvirt-glib
libxml++
libxml++3.0
liferea
linphone
lxappearance-obconf
modsecurity
obconf
obconf-qt
openconnect
osinfo-db-tools
php8.1
php8.2
python3-html5-parser
qt5-webengine
qt6-webengine
recoll
s3fs-fuse
snapper
squid
swiften
tickr
vcdimager
webkit2gtk
xmlsec1
xmlstarlet
I think build-breaking refactorings of this kind are not appropriate for a project with the maturity of libxml2, and I hope we can find a solution to this that doesn't require us patching 62 packages.
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Maintainer
Versions before 2.12 had several complex circular dependencies in header files, for example:
tree.h -> xmlmemory.h -> globals.h -> parser.h -> tree.h
This only worked by carefully ordering declarations and inclusions and was an unmaintainable mess. I'm aware that this change breaks quite a few downstream projects that fail to include the required header files. Note that each declaration is still available from the header where it was originally documented. AFAICT, the main issue is projects including
libxml/tree.h
but forgetting to includelibxml/parser.h
.I have no idea how this could be fixed in a somewhat sane and fully backward compatible way, except maybe to merge many of the headers.
Edited by Nick Wellnhofer - Author
While I agree that cyclic header dependencies are not good, shouldn't include guards take care of that?
- Maintainer
Include guards only avoid infinite loops. They don't help with getting the order of declarations right.
Here's a somewhat hacky patch that should fix the
tree.h <-> parser.h
issue:diff --git a/include/libxml/entities.h b/include/libxml/entities.h index 615ead4de..f67937532 100644 --- a/include/libxml/entities.h +++ b/include/libxml/entities.h @@ -12,7 +12,9 @@ #define __XML_ENTITIES_H__ #include <libxml/xmlversion.h> +#define XML_TREE_INTERNALS #include <libxml/tree.h> +#undef XML_TREE_INTERNALS #ifdef __cplusplus extern "C" { diff --git a/include/libxml/parser.h b/include/libxml/parser.h index 44f1c2025..d8b38823f 100644 --- a/include/libxml/parser.h +++ b/include/libxml/parser.h @@ -11,7 +11,9 @@ #define __XML_PARSER_H__ #include <libxml/xmlversion.h> +#define XML_TREE_INTERNALS #include <libxml/tree.h> +#undef XML_TREE_INTERNALS #include <libxml/dict.h> #include <libxml/hash.h> #include <libxml/valid.h> diff --git a/include/libxml/tree.h b/include/libxml/tree.h index 2a6fef115..a90a174f0 100644 --- a/include/libxml/tree.h +++ b/include/libxml/tree.h @@ -9,6 +9,15 @@ * Author: Daniel Veillard */ +#ifndef XML_TREE_INTERNALS + +/* + * Emulate circular dependency for backward compatibility + */ +#include <libxml/parser.h> + +#else /* XML_TREE_INTERNALS */ + #ifndef __XML_TREE_H__ #define __XML_TREE_H__ @@ -1349,3 +1358,5 @@ XML_DEPRECATED XMLPUBFUN int #endif /* __XML_TREE_H__ */ +#endif /* XML_TREE_INTERNALS */ + diff --git a/include/libxml/valid.h b/include/libxml/valid.h index b19cbacb9..3e04b5521 100644 --- a/include/libxml/valid.h +++ b/include/libxml/valid.h @@ -13,7 +13,9 @@ #include <libxml/xmlversion.h> #include <libxml/xmlerror.h> +#define XML_TREE_INTERNALS #include <libxml/tree.h> +#undef XML_TREE_INTERNALS #include <libxml/list.h> #include <libxml/xmlautomata.h> #include <libxml/xmlregexp.h> diff --git a/include/libxml/xmlIO.h b/include/libxml/xmlIO.h index 7e88da5be..eaea07650 100644 --- a/include/libxml/xmlIO.h +++ b/include/libxml/xmlIO.h @@ -13,7 +13,9 @@ #include <stdio.h> #include <libxml/xmlversion.h> #include <libxml/encoding.h> +#define XML_TREE_INTERNALS #include <libxml/tree.h> +#undef XML_TREE_INTERNALS #ifdef __cplusplus extern "C" {
I hope this should fix the majority of build failures but I have no way to test it. Can you provide some diagnostics from the failed builds?
Collapse replies
- Author
Build logs are here: https://files.placeviolette.net/libxml2-logs/
I'll try your idea, but I probably don't have time before the weekend to get around to it.
- Maintainer
I had a look at the first 16 projects.
0ad: tree.h -> parser.h atril: tree.h -> parser.h audacious-plugins: tree.h -> parser.h badwolf: xpath.h -> parser.h cegui: const xmlError, see #629 cloudfuse: tree.h -> parser.h compiz-plugins-experimental: unrelated compiz-plugins-extra: unrelated darktable: XML schema, fixed upstream, see #624 dia: tree.h -> parser.h eom: tree.h -> parser.h evolution-data-server: parser.h -> globals.h gdl: xmlsave.h gnome-builder: xmlsave.h gnumeric: tree.h -> parser.h goffice: tree.h -> parser.h
With the patch above and another fix related to xmlsave.h, the majority of issues should be fixed.
Edited by Nick Wellnhofer - Author
That patch fixed 20 of 62: https://p.qrm.cat/lfVV86.txt (thanks to abby)
- Maintainer
I committed the patch to the 2.12 branch and added another fix: https://gitlab.gnome.org/GNOME/libxml2/-/commits/2.12?ref_type=heads
It would be great if you could provide updated build logs.
Collapse replies - Author
28 fails remain: https://files.placeviolette.net/libxml2-logs-2.12.2-patched/
- Maintainer
Thanks! I had a look at all the build logs and pushed some additional commits to the 2.12 branch:
https://gitlab.gnome.org/GNOME/libxml2/-/commits/2.12
Unfortunately, it's too late to revert the
xmlError
change.0ad: const xmlError *, must be fixed downstream cegui: const xmlError *, must be fixed downstream compiz-*: missing C stdlib #include, must be fixed downstream darktable: XML schema, fixed upstream, see #624 evolution-*: fixed in 7f767866 gtkpod: missing #include, must be fixed downstream inkscape: missing #include, already fixed downstream libgdal: const xmlError *, must be fixed downstream liblangtag: missing C stdlib #include, already fixed downstream libmusicbrainz5: const xmlError *, must be fixed downstream libpdal: const xmlError *, must be fixed downstream libreoffice: const xmlError *, must be fixed downstream libvirt-glib: fixed in 34a96657 libxml++: fixed in 34a96657 libxml++3.0: fixed in 34a96657 liferea: missing #include, must be fixed downstream obconf-qt: const xmlError *, must be fixed downstream openconnect: seems unrelated osinfo-db-tools: partially fixed in 34a96657 missing C stdlib #include, must be fixed downstream qt5-webengine: const xmlError *, must be fixed downstream qt6-webengine: const xmlError *, must be fixed downstream recoll: const xmlError *, must be fixed downstream squid: const xmlError *, must be fixed downstream swiften: const xmlError *, must be fixed downstream tickr: fixed in 23dd0b76 webkit2gkt: const xmlError *, already fixed downstream xmlsec1: const xmlError *, must be fixed downstream
With latest libxml2 942f5a93 I still get issues building 0ad-git:
In file included from ../../../source/ps/XML/RelaxNG.cpp:28: /usr/include/libxml2/libxml/relaxng.h:198:42: error: ‘xmlNodePtr’ has not been declared 198 | xmlNodePtr elem); | ^~~~~~~~~~ /usr/include/libxml2/libxml/relaxng.h:206:42: error: ‘xmlNodePtr’ has not been declared 206 | xmlNodePtr elem); | ^~~~~~~~~~ /usr/include/libxml2/libxml/relaxng.h:210:42: error: ‘xmlNodePtr’ has not been declared 210 | xmlNodePtr elem); | ^~~~~~~~~~ ../../../source/ps/XML/RelaxNG.cpp: In member function ‘bool RelaxNGValidator::ValidateEncoded(const std::string&, const std::string&) const’: ../../../source/ps/XML/RelaxNG.cpp:147:103: error: ‘XML_PARSE_NONET’ was not declared in this scope; did you mean ‘XML_FROM_NONE’? 147 | xmlDocPtr doc = xmlReadMemory(document.c_str(), (int)document.size(), filename.c_str(), NULL, XML_PARSE_NONET); | ^~~~~~~~~~~~~~~ | XML_FROM_NONE ../../../source/ps/XML/RelaxNG.cpp:147:25: error: ‘xmlReadMemory’ was not declared in this scope 147 | xmlDocPtr doc = xmlReadMemory(document.c_str(), (int)document.size(), filename.c_str(), NULL, XML_PARSE_NONET); | ^~~~~~~~~~~~~ ../../../source/ps/XML/RelaxNG.cpp:155:9: error: ‘xmlFreeDoc’ was not declared in this scope; did you mean ‘xmlDoc’? 155 | xmlFreeDoc(doc); | ^~~~~~~~~~ | xmlDoc ../../../source/ps/XML/RelaxNG.cpp: In member function ‘bool RelaxNGValidator::ValidateEncoded(xmlDocPtr) const’: ../../../source/ps/XML/RelaxNG.cpp:162:50: error: invalid conversion from ‘void (*)(void*, xmlErrorPtr)’ {aka ‘void (*)(void*, _xmlError*)’} to ‘xmlStructuredErrorFunc’ {aka ‘void (*)(void*, const _xmlError*)’} [-fpermissive] 162 | xmlRelaxNGSetValidStructuredErrors(ctxt, &relaxNGErrorHandler, NULL); | ^~~~~~~~~~~~~~~~~~~~ | | | void (*)(void*, xmlErrorPtr) {aka void (*)(void*, _xmlError*)} /usr/include/libxml2/libxml/relaxng.h:184:66: note: initializing argument 2 of ‘void xmlRelaxNGSetValidStructuredErrors(xmlRelaxNGValidCtxtPtr, xmlStructuredErrorFunc, void*)’ 184 | xmlStructuredErrorFunc serror, void *ctx); | ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~ In file included from ../../../source/ps/XML/RelaxNG.cpp:24: ../../../source/ps/XML/RelaxNG.cpp:172:77: error: invalid use of incomplete type ‘xmlDoc’ {aka ‘struct _xmlDoc’} 172 | LOGERROR("RelaxNGValidator: Validation failed for '%s'", doc->name); | ^~ ../../../source/ps/CLogger.h:36:67: note: in definition of macro ‘LOGERROR’ 36 | #define LOGERROR(...) g_Logger->WriteError (fmt::sprintf(__VA_ARGS__).c_str()) | ^~~~~~~~~~~ In file included from ../../../source/ps/XML/RelaxNG.cpp:20: ../../../source/ps/XML/RelaxNG.h:26:16: note: forward declaration of ‘xmlDoc’ {aka ‘struct _xmlDoc’} 26 | typedef struct _xmlDoc xmlDoc; | ^~~~~~~ make[1]: *** [engine.make:417: obj/engine_Release/RelaxNG.o] Error 1 make[1]: *** Waiting for unfinished jobs.... ../../../source/ps/XML/Xeromyces.cpp: In static member function ‘static void CXeromyces::Startup()’: ../../../source/ps/XML/Xeromyces.cpp:59:41: error: invalid conversion from ‘void (*)(void*, xmlErrorPtr)’ {aka ‘void (*)(void*, _xmlError*)’} to ‘xmlStructuredErrorFunc’ {aka ‘void (*)(void*, const _xmlError*)’} [-fpermissive] 59 | xmlSetStructuredErrorFunc(NULL, &errorHandler); | ^~~~~~~~~~~~~ | | | void (*)(void*, xmlErrorPtr) {aka void (*)(void*, _xmlError*)} In file included from /usr/include/libxml2/libxml/valid.h:15, from /usr/include/libxml2/libxml/parser.h:19, from ../../../source/ps/XML/Xeromyces.cpp:35: /usr/include/libxml2/libxml/xmlerror.h:898:57: note: initializing argument 2 of ‘void xmlSetStructuredErrorFunc(void*, xmlStructuredErrorFunc)’ 898 | xmlStructuredErrorFunc handler); | ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~ make[1]: *** [engine.make:423: obj/engine_Release/Xeromyces.o] Error 1 make: *** [Makefile:115: engine] Error 2
Collapse replies - Maintainer
The
xmlNodePtr
issue should be fixed here: 34a96657The
xmlErrorPtr
issue must be fixed in downstream code, see #622 (closed) and #629 (closed). - Maintainer
The
xmlErrorPtr
issue mostly affects C++ projects. See the WebKit fix, for example: https://github.com/WebKit/WebKit/commit/1bad176b2496579d760852c80cff3ad9fb7c3a4b
- Nick Wellnhofer closed
closed
- Developer
Two more problems:
-
mariadb
: SincexmlGetLastError
returns const pointerxmlResetError
can be no longer used on the result. (Apparently, they use it to make valgrind happy.) I opened https://github.com/MariaDB/server/pull/2983 switching toxmlResetLastError()
but not sure if there is not a better solution. -
libxml-ruby
:xmlNodePtr
(and others) still an issue on 2.12.3: https://github.com/xml4r/libxml-ruby/pull/209
Edited by Nick Wellnhofer -
I've updated libxml-ruby to compile with latest libxml2 and will also remove the usage of xmlGetLastError. Thanks for the heads up!
- Maintainer
switching to
xmlResetLastError()
but not sure if there is not a better solution.xmlResetLastError
is fine. - Maintainer
will also remove the usage of xmlGetLastError
The function doesn't have to be removed. It will only return a const xmlError now.
Sorry, I meant replacing
xmlLastError
withxmlGetLastError
. https://github.com/xml4r/libxml-ruby/commit/1476a2d86339f764c15e18bf8acd6e9e9937cc7d- Xi Ruoyao mentioned in issue #734 (closed)
mentioned in issue #734 (closed)
- Nick Wellnhofer mentioned in issue #751
mentioned in issue #751