Skip to content

gh-149017: Upgrade bundled Expat to 2.8.0 #149020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
StanFromIreland wants to merge 5 commits into
base: main
Choose a base branch
from
Open

gh-149017: Upgrade bundled Expat to 2.8.0 #149020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
66f9412
Upgrade bundled Expat to 2.8.0
StanFromIreland Apr 26, 2026
9aa2874
Include the new `random_` headers
StanFromIreland Apr 26, 2026
89517f4
Revert "Include the new `random_` headers"
StanFromIreland Apr 26, 2026
b8a1bd4
We don't actually need 'em
StanFromIreland Apr 26, 2026
c80066b
And patch two includes out :-/
StanFromIreland Apr 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Security/2026-04-26-17-49-58.gh-issue-149017.EiVFPo.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update bundled `libexpat <https://libexpat.github.io/>`_ to version 2.8.0.
36 changes: 18 additions & 18 deletions Misc/sbom.spdx.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 14 additions & 2 deletions Modules/expat/expat.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
#ifndef Expat_INCLUDED
# define Expat_INCLUDED 1

# include <stdint.h> // for uint8_t
# include <stdlib.h>
# include "expat_external.h"

Expand Down Expand Up @@ -917,10 +918,21 @@ XML_SetParamEntityParsing(XML_Parser parser,
function behavior. This must be called before parsing is started.
Returns 1 if successful, 0 when called after parsing has started.
Note: If parser == NULL, the function will do nothing and return 0.
DEPRECATED since Expat 2.8.0.
*/
XMLPARSEAPI(int)
XML_SetHashSalt(XML_Parser parser, unsigned long hash_salt);

/* Sets the hash salt to use for internal hash calculations.
Helps in preventing DoS attacks based on predicting hash function behavior.
This must be called before parsing is started.
Returns XML_TRUE if successful, XML_FALSE when called after parsing has
started or when parser is NULL.
Added in Expat 2.8.0.
*/
XMLPARSEAPI(XML_Bool)
XML_SetHashSalt16Bytes(XML_Parser parser, const uint8_t entropy[16]);

/* If XML_Parse or XML_ParseBuffer have returned XML_STATUS_ERROR, then
XML_GetErrorCode returns information about the error.
*/
Expand Down Expand Up @@ -1081,8 +1093,8 @@ XML_SetReparseDeferralEnabled(XML_Parser parser, XML_Bool enabled);
See https://semver.org
*/
# define XML_MAJOR_VERSION 2
# define XML_MINOR_VERSION 7
# define XML_MICRO_VERSION 5
# define XML_MINOR_VERSION 8
# define XML_MICRO_VERSION 0

# ifdef __cplusplus
}
Expand Down
5 changes: 5 additions & 0 deletions Modules/expat/expat_config.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,10 @@
// bpo-30947: Python uses best available entropy sources to
// call XML_SetHashSalt(), expat entropy sources are not needed
#define XML_POOR_ENTROPY 1
#undef HAVE_ARC4RANDOM
Copy link
Copy Markdown
Member

@picnixz picnixz Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we actually are using a poor entropy source and later we will use the entropy source defined by Expat? The BPO issue seems also unsure on whether we wanted this for the C accelerated module. If we still want a poor entropy, I wonder why we need to undef those HAVE_* constants.

Because we would hit (in xmlparse.c):

#if ! defined(HAVE_ARC4RANDOM_BUF) && ! defined(HAVE_ARC4RANDOM)

on Windows I think? I'm confused here about (1) why we want poor entropy (2) why we undef those macros.

Copy link
Copy Markdown
Member Author

@StanFromIreland StanFromIreland Apr 26, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we actually are using a poor entropy source and later we will use the entropy source defined by Expat?

IIUC, not exactly. We skip all the "good" Expat entropy sources as we override it anyway with XML_SetHashSalt.

Because we would hit (in xmlparse.c):

It only uses gettimeofday/GetSystemTimeAsFileTime which are in the respective system libraries.

Copy link
Copy Markdown
Member

@picnixz picnixz Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But XML_SetHashSalt has poor entropy according to #149018 and we need to use the new interface, which uses the non-poor sources?

Copy link
Copy Markdown
Member Author

@StanFromIreland StanFromIreland Apr 26, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

XML_SetHashSalt16Bytes doesn’t use the new sources? It is just a setter.

Copy link
Copy Markdown
Member

@picnixz picnixz Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ok! so we produce the entropy and set it afterwards, om makes sense.

#undef HAVE_ARC4RANDOM_BUF
#undef HAVE_GETENTROPY
#undef HAVE_GETRANDOM
#undef HAVE_SYSCALL_GETRANDOM

#endif /* EXPAT_CONFIG_H */
5 changes: 3 additions & 2 deletions Modules/expat/expat_external.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@
Copyright (c) 2001-2002 Greg Stein <gstein@users.sourceforge.net>
Copyright (c) 2002-2006 Karl Waclawek <karl@waclawek.net>
Copyright (c) 2016 Cristian Rodríguez <crrodriguez@opensuse.org>
Copyright (c) 2016-2026 Sebastian Pipping <sebastian@pipping.org>
Copyright (c) 2016-2025 Sebastian Pipping <sebastian@pipping.org>
Copyright (c) 2017 Rhodri James <rhodri@wildebeest.org.uk>
Copyright (c) 2018 Yury Gribov <tetra2005@gmail.com>
Copyright (c) 2026 Matthew Fernandez <matthew.fernandez@gmail.com>
Licensed under the MIT license:

Permission is hereby granted, free of charge, to any person obtaining
Expand Down Expand Up @@ -48,7 +49,7 @@
/* Expat tries very hard to make the API boundary very specifically
defined. There are two macros defined to control this boundary;
each of these can be defined before including this header to
achieve some different behavior, but doing so it not recommended or
achieve some different behavior, but doing so is not recommended or
tested frequently.

XMLCALL - The calling convention to use for all calls across the
Expand Down
4 changes: 3 additions & 1 deletion Modules/expat/internal.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
Copyright (c) 2002-2003 Fred L. Drake, Jr. <fdrake@users.sourceforge.net>
Copyright (c) 2002-2006 Karl Waclawek <karl@waclawek.net>
Copyright (c) 2003 Greg Stein <gstein@users.sourceforge.net>
Copyright (c) 2016-2025 Sebastian Pipping <sebastian@pipping.org>
Copyright (c) 2016-2026 Sebastian Pipping <sebastian@pipping.org>
Copyright (c) 2018 Yury Gribov <tetra2005@gmail.com>
Copyright (c) 2019 David Loffredo <loffredo@steptools.com>
Copyright (c) 2023-2024 Sony Corporation / Snild Dolkow <snild@sony.com>
Expand Down Expand Up @@ -113,6 +113,7 @@
#if defined(_WIN32) \
&& (! defined(__USE_MINGW_ANSI_STDIO) \
|| (1 - __USE_MINGW_ANSI_STDIO - 1 == 0))
# define EXPAT_FMT_LLX(midpart) "%" midpart "I64x"
# define EXPAT_FMT_ULL(midpart) "%" midpart "I64u"
# if defined(_WIN64) // Note: modifiers "td" and "zu" do not work for MinGW
# define EXPAT_FMT_PTRDIFF_T(midpart) "%" midpart "I64d"
Expand All @@ -122,6 +123,7 @@
# define EXPAT_FMT_SIZE_T(midpart) "%" midpart "u"
# endif
#else
# define EXPAT_FMT_LLX(midpart) "%" midpart "llx"
# define EXPAT_FMT_ULL(midpart) "%" midpart "llu"
# if ! defined(ULONG_MAX)
# error Compiler did not define ULONG_MAX for us
Expand Down
18 changes: 15 additions & 3 deletions Modules/expat/refresh.sh
View file
Open in desktop
Comment thread
StanFromIreland marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ fi

# Update this when updating to a new version after verifying that the changes
# the update brings in are good. These values are used for verifying the SBOM, too.
expected_libexpat_tag="R_2_7_5"
expected_libexpat_version="2.7.5"
expected_libexpat_sha256="9931f9860d18e6cf72d183eb8f309bfb96196c00e1d40caa978e95bc9aa978b6"
expected_libexpat_tag="R_2_8_0"
expected_libexpat_version="2.8.0"
expected_libexpat_sha256="c7cec5f60ea3a42e7780781c6745255c19aa3dbfeeae58646b7132f88dc24780"

expat_dir="$(realpath "$(dirname -- "${BASH_SOURCE[0]}")")"
cd ${expat_dir}
Expand Down Expand Up @@ -64,6 +64,18 @@ This may be due to source changes and will require updating this script" >&2
exit 1
fi

# Step 4: Skip the Windows rand_s entropy path in xmlparse.c when
# XML_POOR_ENTROPY is set.
sed -z -i 's|#if defined(_WIN32)\n# include "random_rand_s\.h"\n#endif /\* defined(_WIN32) \*/|#if defined(_WIN32) \&\& ! defined(XML_POOR_ENTROPY)\n# include "random_rand_s.h"\n#endif /* defined(_WIN32) \&\& ! defined(XML_POOR_ENTROPY) */|' xmlparse.c
sed -z -i 's|# ifdef _WIN32\n if (writeRandomBytes_rand_s|# if defined(_WIN32) \&\& ! defined(XML_POOR_ENTROPY)\n if (writeRandomBytes_rand_s|' xmlparse.c

if ! grep -q '#if defined(_WIN32) && ! defined(XML_POOR_ENTROPY)' xmlparse.c; then
echo "
Error: rand_s gate not patched in xmlparse.c;
This may be due to source changes and will require updating this script" >&2
exit 1
fi

echo '
Updated! next steps:
- Verify all is okay:
Expand Down
Loading
Loading