• Stock JavaScript: replace raw CP437/Windows-1252 bytes with escapes, a

    From Rob Swindell@1:103/705 to GitLab issue in main/sbbs on Wed Nov 11 22:10:34 2026
    open https://gitlab.synchro.net/main/sbbs/-/work_items/1341

    ## Summary

    Thirteen tracked JavaScript files contain raw bytes in the 0x80-0xFF range that are not valid UTF-8: mostly CP437, some Windows-1252/Latin-1. Replace each with an equivalent escape sequence (`"\xNN"`), an `ascii(N)` call, or plain ASCII, so every stock script is either pure ASCII or valid UTF-8.

    Scan as of master 36867a9bbf (farm-3-array, 2026-11-11), over `git ls-files '*.js' '*.ssjs' '*.xjs' '*.inc'`.

    ## Why

    - **SpiderMonkey 128 compiles source as UTF-8.** The next-js branch uses `JS::CompileUtf8Path()` (`src/sbbs3/js_global.cpp:658`, `src/sbbs3/js_internal.cpp:470`) and `JS::SourceText<mozilla::Utf8Unit>` (`src/sbbs3/jsexec.cpp:1090`). A raw CP437 byte is malformed UTF-8 there: expect either a compile error or U+FFFD in place of the intended character. (Not yet verified which.) SpiderMonkey 1.8.5 reads source byte-for-byte, which is why these files work today.
    - Editors and tools that assume UTF-8 show these bytes as garbage, mangle them on save, or refuse to edit the file at all.
    - Some of the bytes are Windows-1252, not CP437, and already display wrong on a CP437 terminal (see hamtest below).

    ## Files

    | File | Bytes | Lines | What they are |
    |---|---|---|---|
    | `xtrn/mc/mc.js` | 0xBA x12, 0xBB, 0xBC, 0xC8, 0xC9, 0xCD x92, 0xDB x45, 0xDC x2, 0xDF x62 | 20 lines from 306 | CP437 box-drawing and block characters in `console.print()` strings |
    | `xtrn/hamtest/General.js` | 0x92 x12, 0x93 x14, 0x94 x14, 0xA0 x3 | 25 lines from 260 | **Windows-1252** curly quotes and no-break space in question text; on a CP437 terminal the quotes around `cold flow` show as o-circumflex and o-umlaut, and the no-break space as a-acute |
    | `xtrn/hamtest/Extra.js` | 0xFF x2 | 4675, 4680 | CP437 no-break space (`silicon<FF>photovoltaic`, `1.5<FF>V`) |
    | `xtrn/druglord/atm.js` | 0x9C x15 | 15 lines from 12 | CP437 pound sign in output strings |
    | `xtrn/druglord/druglord.js` | 0x9C x7 | 7 lines from 140 | CP437 pound sign | | `xtrn/druglord/Drug.js` | 0x9C x4 | 64, 80, 132, 144 | CP437 pound sign |
    | `xtrn/druglord/pocket.js` | 0x9C | 18 | CP437 pound sign |
    | `xtrn/lord/outhouse/outhouse.js` | 0xED x2, 0xF9 x2, 0xFA x12 | 6 lines from 75 | CP437 phi, bullet and middle-dot characters in output strings |
    | `xtrn/lord/gravyard/gravyard.js` | 0xED x2, 0xEF x2, 0xF9 x4, 0xFA x8 | 8, 336, 550, 714 | CP437 phi, intersection, bullet and middle-dot characters in output strings |
    | `xtrn/fatfish/Fish.js` | 0xF7 | 428 | CP437 almost-equal sign in a comment | | `exec/load/sha1.js` | 0xA7 x8 | 8 lines from 24 | **Latin-1** section sign (spec section numbers) in comments |
    | `exec/load/msgutils.js` | 0xFA | 115 | CP437 middle dot in a regex: `/[`<FA>]\[/g` converts a backquote or middle dot followed by `[` to `ESC[` |
    | `web/root/sajax-forum/client_functions.xjs` | 0xA9 | 49 | **Latin-1** copyright sign in a comment |

    ## How to replace

    - **Comments:** plain ASCII: the section sign -> `section`, the copyright sign -> `(c)`, the almost-equal sign -> `~=`.
    - **CP437 characters in strings sent to a terminal** (pound sign, box drawing, phi, middle dot, ...): an escape with the same byte value, `"\x9C"`, `"\xCD"`, so the output is byte-for-byte unchanged. For long runs such as `mc.js`'s 46-character box borders, build the string once (e.g. a repeat loop over `ascii(205)`) rather than writing 46 escapes in a row.
    - **CP437 bytes in a regex:** escape inside the pattern: `/[`\xFA]\[/g` matches exactly what the raw byte did.
    - **Windows-1252 bytes in displayed text** (hamtest `General.js`): replace with ASCII (`"`, `'`, a space). This also fixes how those questions display today.
    - **CP437 no-break space (0xFF)** between words: a plain space.

    The per-file escape/ASCII choice should leave each door's output identical on a CP437 terminal, except for the hamtest text, which should be corrected.

    ## Verification

    - Each changed file should be pure ASCII afterwards, i.e. this prints nothing:
    `LC_ALL=C grep -nP '[\x80-\xff]' <file>`
    - Run each touched door/module once and compare its output with the previous version (for the escapes, a byte-level diff of captured output should be empty).
    - Consider a check under `exec/tests/` that fails if any stock `.js`/`.ssjs`/`.xjs` file contains bytes that aren't valid UTF-8, so new ones don't creep back in.

    ## Out of scope: files that are valid UTF-8

    54 other tracked files contain non-ASCII characters that are valid UTF-8, mostly in `xtrn/zmachine` (11), `exec/tests` (8), `exec/load` (7), `xtrn/synchess` (4), `web/root` (3). These compile fine under both engines, but SpiderMonkey 1.8.5 reads each UTF-8 byte as a separate Latin-1 character while SpiderMonkey 128 decodes the sequence to one character, so string lengths and terminal output may differ between the two. That deserves its own review.

    *Authored by Claude (Claude Code), on behalf of @rswindell*
    --- SBBSecho 3.38-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab issue in main/sbbs on Fri Nov 13 20:36:23 2026
    close https://gitlab.synchro.net/main/sbbs/-/work_items/1341
    --- SBBSecho 3.38-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)