Fixes bug 783026.

Thanks to Pranjal Jumde for the report.

xmlSnprintfElementContent failed to correctly check the available
buffer space in two locations.

Fixes bug 781333 (CVE-2017-9047) and bug 781701 (CVE-2017-9048).

Thanks to Marcel Böhme and Thuan Pham for the report.

There were two bugs where parameter-entity references could lead to an
unexpected change of the input buffer in xmlParseNameComplex and
xmlDictLookup being called with an invalid pointer.

Percent sign in DTD Names
=========================

The NEXTL macro used to call xmlParserHandlePEReference. When parsing
"complex" names inside the DTD, this could result in entity expansion
which created a new input buffer. The fix is to simply remove the call
to xmlParserHandlePEReference from the NEXTL macro. This is safe because
no users of the macro require expansion of parameter entities.

- xmlParseNameComplex
- xmlParseNCNameComplex
- xmlParseNmtoken

The percent sign is not allowed in names, which are grammatical tokens.

- xmlParseEntityValue

Parameter-entity references in entity values are expanded but this
happens in a separate step in this function.

- xmlParseSystemLiteral

Parameter-entity references are ignored in the system literal.

- xmlParseAttValueCompl...

Check for overflow before casting double to int.

Found with afl-fuzz and UBSan.

For now this is mainly useful if you work on a fork of the libxml2
mirror on GitHub:

    https://github.com/GNOME/libxml2

Start with two build setups:

- GCC with as many GNU extensions disabled as possible, trying to
  emulate a C89 compiler on a POSIX system.

- clang with ASan and UBSan.

The Python tests don't set an exit code, so Travis won't detect
failures. The same goes for "make tests", but we only run "make check"
anyway.

The code in xmlParseStartTag2 must handle the case that the input
buffer was grown and reallocated which can invalidate pointers to
attribute values. Before, this was handled by detecting changes of
the input buffer "base" pointer and, in case of a change, jumping
back to the beginning of the function and reparsing the start tag.

The major problem of this approach is that whether an input buffer is
reallocated is nondeterministic, resulting in seemingly random test
failures. See the mailing list thread "runtest mystery bug: name2.xml
error case regression test" from 2012, for example.

If a reallocation was detected, the code also made no attempts to
continue parsing in case of errors which makes a difference in
the lax "recover" mode.

Now we store the current input buffer "base" pointer for each (not
separately allocated) attribute in the namespace URI field, which isn't
used until later. After the whole start tag was parsed, the pointers
to the attribute values are reconstructed using the offset between the
new and the old input buffer. This relies on arithmetic on dangling
pointers which is technically undefined behavior. But it seems like
the easiest and most efficient fix and a similar approach is used in
xmlParserInputGrow.

This changes the error output of several tests, typically making it
more verbose because we try harder to continue parsing in case of
errors.

(Another possible solution is to check not only the "base" pointer
but the size of the input buffer as well. But this would result in
even more reparsing.)

Remove some goto labels and deduplicate a bit of code after handling
namespaces.

Before:

    loop {
        parseAttribute
        if (ok) {
            if (defaultNamespace) {
                handleDefaultNamespace
                if (error)
                    goto skip_default_ns;
                handleDefaultNamespace
    skip_default_ns:
                freeAttr
                nextAttr
                continue;
            }
            if (namespace) {
                handleNamespace
                if (error)
                    goto skip_ns;
                handleNamespace
    skip_ns:
                freeAttr
                nextAttr;
                continue;
            }
            handleAttr
        } else {
            freeAttr
        }
        nextAttr
    }

After:

    loop {
        parseAttribute
        if (!ok)
            goto next_attr;
        if (defaultNamespace) {
            handleDefaultNamespace
            if (error)
                goto next_attr;
            handleDefaultNamespace
        } else if (namespace) {
            handleNamespace
            if (error)
                goto next_attr;
            handleNamespace
        } else {
            handleAttr
        }
    next_attr:
        freeAttr
        nextAttr
    }

The autogenerated API tests leak memory.

The API tests combine string buffers with arbitrary length values which
makes ASan detect out-of-bound array accesses. Even without ASan, this
could lead to unwanted test failures.

Add a check for "len", "size", and "start" arguments, assuming they
apply to the nearest char pointer. Skip the test if they exceed the
buffer size. This is a somewhat naive heuristic but it seems to work
well.

It's stupid, but the behavior of memcpy(NULL, NULL, 0) is undefined.

If available, use a C99 flexible array member to avoid spurious UBSan
errors.

Avoid undefined behavior and wrong results with huge exponents.

Found with afl-fuzz and UBSan.

Avoid undefined behavior when casting from double to int.

Found with afl-fuzz and UBSan.

Don't count leading zeros towards the fraction size limit. This allows
to parse numbers like

    0.0000000000000000000000000000000000000000000000000000000001

which is the only standard-conformant way to represent such numbers, as
scientific notation isn't allowed in XPath 1.0. (It is allowed in XPath
2.0 and in libxml2 as an extension, though.)

Overall accuracy is still bad, see bug 783238.

Use the C library's floor and ceil functions. The old code was overly
complicated for no apparent reason and could result in undefined
behavior when handling NaNs (found with afl-fuzz and UBSan).

Fix wrong comment in xmlXPathRoundFunction. The implementation was
already following the spec and rounding half up.

Quote echoed variable to avoid newlines being converted to space.

When traversing the "preceding" axis from an attribute node, we must
first go up to the attribute's containing element. Otherwise, text
children of other attributes could be returned. This made it possible
to hit a code path in xmlXPathNextAncestor which contained another bug:
The attribute node was initialized with the context node instead of the
current node. Normally, this code path is only hit via
xmlXPathNextAncestorOrSelf in which case the current and context node
are the same.

The combination of the two bugs could result in an infinite loop, found
with libFuzzer.

Traversing the "following" and the "preceding" axis from namespace nodes
should be handled similarly. This wasn't supported at all previously.

Commit c851970c introduced a spurious error message when evaluating
XPath expressions with xmlXPathCompiledEvalToBoolean.

Move the check for trailing characters from xmlXPathEval to
xmlXPathEvalExpr. Otherwise, a valid portion of a syntactically invalid
expression would be evaluated before returning an error.

Move cleanup of XPath stack to xmlXPathFreeParserContext. This avoids
memory leaks if valuePop fails in some error cases. Found with
libFuzzer and ASan.

Rework handling of the final XPath result object in
xmlXPathCompiledEvalInternal and xmlXPathEval to avoid useless error
messages.

Both functions are supposed to do exactly the same.

Found with libFuzzer and ASan.

Implement TODO block to free the arguments in error case.

Found with libFuzzer and ASan.

Found with libFuzzer and ASan.

Triggered in mixed content ELEMENT declarations if there's an invalid
name after the first valid name:

    <!ELEMENT para (#PCDATA|a|<invalid>)*>

Found with libFuzzer and ASan.

Found with libFuzzer and ASan.

Namespaces nodes must not be duplicated when merging.

Found with libFuzzer and ASan.

Namespace nodes must be freed when selecting the first or last element
of a node set.

Found with libFuzzer and ASan.

Found with libFuzzer and ASan.

David Tardon authored Apr 05, 2017 and Daniel Veillard committed Apr 07, 2017

For https://bugzilla.gnome.org/show_bug.cgi?id=780918

Neel Mehta authored Apr 07, 2017 and Daniel Veillard committed Apr 07, 2017

For https://bugzilla.gnome.org/show_bug.cgi?id=780691

* parser.c: add a specific check to avoid PE reference

For https://bugzilla.gnome.org/show_bug.cgi?id=780690

nanohttp.c: the code wrongly assumed a short int port value.

d-hat authored Apr 07, 2017 and Daniel Veillard committed Apr 07, 2017

For https://bugzilla.gnome.org/show_bug.cgi?id=772726

* include/libxml/parser.h: Add a new parser flag XML_PARSE_NOXXE
* elfgcchack.h, xmlIO.h, xmlIO.c: associated loading routine
* include/libxml/xmlerror.h: new error raised
* xmllint.c: adds --noxxe flag to activate the option

Found with afl-fuzz.

Fixes a `-pedantic` compiler warning.

Also fixes bug #768199:

https://bugzilla.gnome.org/show_bug.cgi?id=768199

Namespace nodes must be copied to avoid use-after-free errors.
But they don't necessarily have a physical representation in a
document, so simply disallow them in XPointer ranges.

Found with afl-fuzz.

Fixes CVE-2016-4658.