CVE-2026-126366: msgpack-ruby JRuby decoder makes uncapped header-count allocations; a 5-byte message forces multi-GiB OOM

The JRuby implementation of msgpack (msgpack gem, java platform — pure Java decoder under ext/java/) sizes containers directly from the element count written in the message header, with no cap at all: no equivalent of the C unpacker’s SHRT_MAX clamp (since 1.7.3) and none of the buffered-bytes cap that commit c8d42d8 added for CRuby in 1.8.6. A single 5-byte message decides how much memory a JVM process tries to allocate:

• array32 header declaring 0x7FFFFFFF elements → new IRubyObject[0x7FFFFFFF] (~8 GiB of reference slots) attempted immediately → java.lang.OutOfMemoryError: Requested array size exceeds VM limit.
• Counts below the VM’s max-array-size but above available heap (e.g. 5×10⁸ with a 512 MiB heap) produce a real allocation attempt — OutOfMemoryError: Java heap space — the JVM genuinely tries to take the memory before failing.
• str32/bin32 headers declaring 0x7FFFFFFF bytes do the same via readBytes(size) → new byte[size].
• Below the failure threshold there is a silent band: one streaming unpacker fed only an array32 header with count 2×10⁷ (5 wire bytes, zero elements) grows used heap +78 MiB immediately, no exception (each swallows the underflow), and the footprint stays elevated after Unpacker#reset + full GC. Linear in declared count (~4 B/element); nothing bounds it — not delivered bytes, not a per-container cap.

Only counts ≥ 2³¹ are refused (a clean MessagePack::UnderflowError from getUnsignedInt()), and only maps are safe (consumeHash grows per element instead of pre-sizing). Verified on the released gem msgpack 1.8.6 java, JRuby 9.4.12 / OpenJDK 25, 2026-10-30 — i.e. unfixed in the current release. Proposed fix mirrors c8d42d8: cap initial allocation by bytes actually buffered and grow as data arrives; read strings in bounded chunks instead of allocating the declared length up front.