GMS-2024-287: ReDos vulnerability of XMLFeedSpider
Impact
The following parts of the Scrapy API were found to be vulnerable to a ReDoS attack:
The
XMLFeedSpider
class or any subclass that uses the default node iterator:iternodes
, as well as direct uses of thescrapy.utils.iterators.xmliter
function.Scrapy 2.6.0 to 2.11.0: The
open_in_browser
function for a response without a base tag.
Handling a malicious response could cause extreme CPU and memory usage during the parsing of its content, due to the use of vulnerable regular expressions for that parsing.
Patches
Upgrade to Scrapy 2.11.1.
If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.11.1 is not an option, you may upgrade to Scrapy 1.8.4 instead.
Workarounds
For XMLFeedSpider
, switch the node iterator to xml
or html
.
For open_in_browser
, before using the function, either manually review the response content to discard a ReDos attack or manually define the base tag to avoid its automatic definition by open_in_browser
later.
Acknowledgements
This security issue was reported by @nicecatch2000 through huntr.com.
References
- docs.scrapy.org/en/latest/news.html
- docs.scrapy.org/en/latest/news.html
- github.com/advisories/GHSA-cc65-xxvf-f7r9
- github.com/scrapy/scrapy/commit/479619b340f197a8f24c5db45bc068fb8755f2c5
- github.com/scrapy/scrapy/commit/73e7c0ed011a0565a1584b8052ec757b54e5270b
- github.com/scrapy/scrapy/security/advisories/GHSA-cc65-xxvf-f7r9
Detect and mitigate GMS-2024-287 with GitLab Dependency Scanning
Secure your software supply chain by verifying that all open source dependencies used in your projects contain no disclosed vulnerabilities. Learn more about Dependency Scanning →