open
https://gitlab.synchro.net/main/sbbs/-/work_items/1340
## Summary
Several stock JavaScript services read a message base one record at a time, or look up every message number in a range, where one `MsgBase.get_index()` call (or clamping the range) would do. This is the same pattern c5f0ca87d9 (reliable-11-fear, 2026-11-11) fixed in webv4's mail page and #1339 describes in webv4's forum. Message numbers are sparse on most real bases (one production sub numbers 1,044 messages from 317,056 to 318,099), so a per-number loop mostly looks up numbers that don't exist, and every lookup is a separate locked index search.
The cost scales with per-call latency. On Linux, with the data on an SMB share served by the same host, a header lookup measured about 0.1 ms. Where the data directory is on an SMB share, every call is a network round trip: c5f0ca87d9 measured 8.6 ms per lookup on Windows.
Code references are as of master 36867a9bbf (farm-3-array, 2026-11-11).
## 1. NNTP: XOVER/XHDR look up every number in the client's range (high)
`exec/nntpservice.js:564` (`OVER`/`XOVER`) and `:618` (`HDR`/`XHDR`) call `get_msg_header(false, i)` for every `i` from `first` to `last`, where both come from the client and are not clamped to the group's `first_msg`/`last_msg`.
- `XOVER 1-` (an open range is `1` through `last_msg`) on the sub above makes about 318,000 lookups to return 1,044 messages.
- In the mail newsgroup (`include_mail`), only the user's own mail is listed, so even an exact range is mostly other users' messages: the case c5f0ca87d9 fixed in webv4 (24,870 lookups for 926 messages).
- A client can repeat this at will, and on a host whose data is on an SMB share each lookup is a round trip.
Suggested fix: clamp `first`/`last` to `first_msg`/`last_msg`, then walk the `get_index()` records inside the range (skipping `MSG_DELETE`/`MSG_VOTE`, and other users' mail by the index `to`/`from`), reading only those headers.
## 2. IMAP: the folder index is rebuilt one read per message (high)
`read_index()` (`exec/imapservice.js:1388`) calls `get_msg_index(true, i)` for every offset (`:1402`). `update_status()` rebuilds it whenever the base's `last_msg` changes (`:1345`). For INBOX that is the mail base, whose `last_msg` changes whenever anyone on the system receives mail, so every connected IMAP client re-reads the whole mail index on its next status update.
Measured on a 9,364-message mail base (Linux, same-host SMB share): 40 ms per rebuild with per-offset reads, 17 ms with one `get_index()`. With the data on an SMB share each of the 9,364 reads is a round trip.
Suggested fix: build the index from one `get_index()` call.
Related, minor: a `UID` sequence set such as `1:*` makes `:671` iterate every number from 1 to `last_msg` (752,280 on that mail base) in memory to find the user's messages. Walking `index.offsets` would avoid that.
## 3. NNTP: LISTGROUP reads the index one record per message (medium)
`exec/nntpservice.js:519` calls `get_msg_index(true, i)` per offset. `count_msgs()` in the same file (`:130`) already does this with one `get_index()` call; LISTGROUP can do the same.
## 4. Gopher: a sub listing reads and returns every message (medium)
`exec/gopherservice.js:298` calls `get_all_msg_headers()` and lists the entire sub with no paging. On a 159,537-message sub, `get_all_msg_headers()` alone took 1.9 s, and the response is one line per message. Each crawler request repeats it.
Suggested fix: list only the newest N messages (from `get_index()`, reading those headers), with a link to older ones.
## 5. msgutils.js getMessageThreads(): the max branch throws and mis-keys results (bug)
`exec/load/msgutils.js:360` `getMessageThreads(sub, max)`: when `max` is given and `last_msg - max` is above `first_msg`, it loops `m` from `last_msg - max` to `last_msg` (`:382`):
- `get_msg_header(m)` returns `null` for any missing or deleted number, and `header.num` then throws a TypeError.
- `header.num` is not a header property (the number is `header.number`), so every header is stored under the key `"undefined"`, and only the last one survives.
- On sparse numbering the window holds fewer than `max` messages.
Callers: `xtrn/ecreader/lib.js:376` (passes `settings.max`, normally unset, which takes the `get_all_msg_headers()` path) and the old web UI's `web/lib/forum.ssjs:119`, `:303` (`webIni.maxMessages`).
Suggested fix: take the newest `max` records from `get_index()` and read those headers by offset.
## 6. Lower priority: loops over every user record
`exec/load/birthdays.js:22`, `exec/load/consensus_vote.js:201`, `exec/hotline.js:970` and the old web UI's `web/root/ecwebv3/sidebar/002-whosOnline.ssjs:27` read every user record (1,457 records took 0.1 s on Linux). None runs per request in a default setup.
*Authored by Claude (Claude Code), on behalf of @rswindell*
--- SBBSecho 3.38-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)