GitLab

For https://bugzilla.gnome.org/show_bug.cgi?id=744980

There are times, like on unterminated entities that it's preferable to
stop parsing, even if that means less error reporting. Entities are
feeding the parser on further processing, and if they are ill defined
then it's possible to get the parser to bug. Also do the same on
Conditional Sections if the input is broken, as the structure of
the document can't be guessed.

For https://bugzilla.gnome.org/show_bug.cgi?id=738805

The fix for CVE-2014-3660 introduced a regression in some case
where entity substitution is required and the entity is used
first in anotther entity referenced from an attribute value

This reverts commit 054c716e.
As this break xmlcatalog command
https://bugzilla.redhat.com/show_bug.cgi?id=1153753

Issues related to the billion laugh entity expansion which happened to
escape the initial set of fixes

When the xml parser encounters an xml encoding in an xml header while
configured with option XML_PARSE_IGNORE_ENC, it fails to free memory
allocated for storing the encoding.
The patch below fixes this.
How to reproduce:
1. Change doc/examples/parse4.c to add xmlCtxtUseOptions(ctxt,
XML_PARSE_IGNORE_ENC); after the call to xmlCreatePushParserCtxt.
2. Rebuild
3. run the following command from the top libxml2 directory:
LD_LIBRARY_PATH=.libs/ valgrind --leak-check=full
./doc/examples/.libs/parse4 ./test.xml , where test.xml contains
following
input:
<?xml version="1.0" encoding="UTF-81" ?><hi/>
valgrind will report:
==1964== 10 bytes in 1 blocks are definitely lost in loss record 1 of 1
==1964==    at 0x4C272DB: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==1964==    by 0x4E88497: xmlParseEncName (parser.c:10224)
==1964==    by 0x4E888FE: xmlParseEncodingDecl (parser.c:10295)
==1964==    by 0x4E89630: xmlParseXMLDecl (parser.c:10534)
==1964==    by 0x4E8B737: xmlParseTryOrFinish (parser.c:11293)
==1964==    by 0x4E8E775: xmlParseChunk (parser.c:12283)

Signed-off-by: Bart De Schuymer <bart at amplidata com>

For https://bugzilla.gnome.org/show_bug.cgi?id=736417

basically a weird side effect and a failure
to properly parenthesize a boolean expression led to this bug

For https://bugzilla.gnome.org/show_bug.cgi?id=672539
Reported by Axel Miller <axel.miller@ppi.de>

Consider the following start-tag:
<x xmlns=""version="">

The start-tag does not conform to the rule

[40]       STag       ::=       '<' Name (S Attribute)* S? '>'

since there is no whitespace in front of the attribute "version".

Thus, libxml2 should reject the start-tag.
But it doesn't:

$ echo '<x xmlns=""version=""/>' | xmllint -
<?xml version="1.0"?>
<x xmlns="" version=""/>

The error seems to happen only if there is a namespace declaration in
front of
the attribute. A missing whitespace between other attributes is handled
correctly:

$ echo '<x someattr=""version=""/>' | xmllint -
-:1: parser error : attributes construct error
<x someattr=""version=""/>
              ^
[...]

For https://bugzilla.gnome.org/show_bug.cgi?id=734283

libxml2 reports wrong error column numbers (field int2 in xmlError)
in structured error handler, after parsing an end tag.

For https://bugzilla.gnome.org/show_bug.cgi?id=734280

libxml2 reports wrong error column numbers (field int2 in xmlError)
in structured error handler, after parsing XML attribute values.

Example XML:

<?xml version="1.0" encoding="UTF-8"?>
<root
xmlns="urn:colbug">&</root>
<!--
         1         2         3         4
1234567890123456789012345678901234567890
-->

Expected location of the error would be line 3, column 21.

The actual location of the error is line 3, column 9:

$ ./xmlparse colbug2.xml
colbug2.xml:3:9: xmlParseEntityRef: no name

The 12 characters of the xmlns attribute value "urn:colbug" are
not accounted for in the error column value.

For https://bugzilla.gnome.org/show_bug.cgi?id=734276

libxml2 reports wrong error column numbers (field int2 in xmlError)
in structured error handler, after an XML declaration containing
whitespace.

Example XML:

<?xml  version="1.0"  encoding="UTF-8"     ?><root>&</root>
<!--
         1         2         3         4         5         6
123456789012345678901234567890123456789012345678901234567890
-->

Expected location of the error would be line 1, column 53.

The actual location of the error is line 1, column 44:

$ ./xmlparse colbug1.xml
colbug1.xml:1:44: xmlParseEntityRef: no name

For https://bugzilla.gnome.org/show_bug.cgi?id=731511
Pointed byt vincent Lefevre

For https://bugzilla.gnome.org/show_bug.cgi?id=733040

xmlDictLookup() may return NULL in case of allocation error,
though very unlikely it need to be checked.

A number of issues have been raised after the fix, and this patch
tries to correct all of them, though most were related to
postvalidation.
https://bugzilla.gnome.org/show_bug.cgi?id=730290
and other reports on list, off-list and on Red Hat bugzilla

Unless explicitely asked for when validating or replacing entities
with their value. Problem pointed out by Daniel Berrange <berrange@redhat.com>

We really need to have ctxt->instate == XML_PARSER_CONTENT when
jumping in content parsing
Bug reported by Frank Gross

Apparently an verflow when comparing macro and unsigned long

As pointed out by "Tassyns, Bram <BramT@enfocus.com>" on the list
some call had it other didn't, clean it up and add to all missing
ones

Partially, a follow-up of 81d7a824.
Signed-off-by: Jan Pokorný <jpokorny@redhat.com>

https://bugzilla.gnome.org/show_bug.cgi?id=698550

Somehow the behaviour of the internal parser routine changed
slightly when encountering CR/LF, which led to a bug when
parsing document with non-ascii Names

The switch to xzlib had for consequence that the compression
level of the input was not gathered anymore in ctxt->input->buf,
then the parser compression flags was left to -1 and propagated
to the resulting document.
Fix the I/O layer to get compression detection in xzlib,
then carry it in the input buffer and the resulting document

  This should fix
    https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3456

https://bugzilla.gnome.org/show_bug.cgi?id=698582

xmlCleanupParser calls xmlCleanupGlobals() and then
xmlResetLastError() but the later reallocate the global
data freed by previous call. Just swap the two calls.

Error introduced in previous commit !

Add a specific parser error
Try to stop parsing as quickly as possible

slightly modify how ent->checked is used, and use the lowest bit to
keep the information

If an entity has already been checked for correctness no
need to check it on every reference

If entities expansion in the XML parser is asked for,
it is possble to craft relatively small input document leading
to excessive on-the-fly content generation.
This patch accounts for those replacement and stop parsing
after a given threshold. it can be bypassed as usual with the
HUGE parser option.

https://bugzilla.gnome.org/show_bug.cgi?id=692915

the new set of converting functions tried to limit the encoding
conversion of the raw buffer to the consumption one to work in
a more progressive fashion. Unfortunately this was bad for
performances and led to errors on progressive parsing when
a very large chunk was close to the end of the document. Fix
the new internal function and switch back to the old way of
converting. Fix another bug in the process.

https://bugzilla.gnome.org/show_bug.cgi?id=690202

Buffer overflow errors originating from xmlBufGetInputBase in 2.9.0
The pointers from the context input were not properly reset after
that call which can do reallocations.

Add missing cast for xmlNop to silence a compiler warning.

For https://bugzilla.gnome.org/show_bug.cgi?id=689958
We were looking for the wrong character in the input stream

https://bugzilla.redhat.com/show_bug.cgi?id=862969
The new code trying to detect excessive input lookup would
just get wrong sometimes in the case of very large file parsed
directly from memory.

Raised as a side effect of:
https://bugzilla.gnome.org/show_bug.cgi?id=663844

https://bugzilla.gnome.org/show_bug.cgi?id=684774
with help from Kjell Ahlstedt <kjell.ahlstedt@bredband.net>

Signed-off-by: Jan Pokorný <jpokorny@redhat.com>

Remove all space before tabs and space and tabs at end of lines.

Related to https://bugs.launchpad.net/lxml/+bug/502959

Basically the core of the issue is that if an entity references another
entity, then in case we are replacing entities content, we should always
do so by copying the referenced content as long as the reference is
done within the entity. Otherwise, if for some reason there is a later
parsing error that entity content may be freed.

Complex scenario exposed by command:
thinkpad:~/XML/diveintopython-5.4/xml -> valgrind --db-attach=yes
../../xmllint --loaddtd --noout --noent diveintopython.xml

  Document references &a;
  a references &b;
  we references b content directly in by linking in the a content
  a has an error further down
  we free a, freeing the chunk from b
  Document references &b; after &a;
  we try to copy b content, but it was freed already => segfault

* parser.c: never reference directly entity content without copying if
  we aren't in the document main entity