open
https://gitlab.synchro.net/main/sbbs/-/work_items/1339
## Summary
webv4's forum code has the same problem c5f0ca87d9 (reliable-11-fear, 2026-11-11) fixed for the mail page: it looks up messages one number at a time, or reads every header in a sub, where an index read would do. Some of it runs on a timer for every logged-in viewer of a forum page.
Code references are as of master 36867a9bbf (farm-3-array, 2026-11-11); `webv4/lib/forum.js` is unchanged since c5f0ca87d9.
Measurements are read-only benchmarks against a production install's message bases (147 subs) using one real user's scan pointers. A header lookup there costs about 0.1 ms (Linux, loopback SMB mount). c5f0ca87d9 measured 8.6 ms per lookup for webv4 on Windows reading the message bases over an SMB share; the "at 8.6 ms" column uses that figure and is an estimate, not a measurement.
## 1. Unread counts: per-number loop, re-run every 60 s per viewer (high)
`getSubUnreadCount()` (`lib/forum.js:101`) calls `get_msg_header()` for every number from `scan_ptr + 1` to `last_msg`. `getSubUnreadCounts()` and `getGroupUnreadCounts()` call it for every sub in a group, or in every group.
The forum's group list and sub list (`pages/001-forum.xjs:382`, `:420`) subscribe to `groups_unread` / `subs_unread` on `api/events.ssjs`, and `lib/events/forum.js` re-runs those scans every `refresh_interval` (default 60000 ms), once per open page.
| | Count |
|---|---|
| Header lookups for one pass over all subs | 256,940 |
| Messages actually above the user's scan pointers | 58,681 |
| Lookups for numbers that don't exist | 77% |
| Worst sub: never read (`scan_ptr` 0), numbers 108531-113539 | 113,539 lookups for 5,009 messages |
| | Measured (0.1 ms/lookup) | At 8.6 ms/lookup |
|---|---|---|
| Current loop, one pass | about 26 s | about 37 min |
| `MsgBase.get_index()` on every sub | 0.9 s | |
`cycle()` in `lib/events/forum.js` sets `last_run` before it scans. A pass that takes longer than `refresh_interval` therefore starts the next one as soon as it ends, so a slow host keeps the loop running continuously for as long as the page is open (up to `event_stream_duration`, default one hour).
Suggested fix:
- Count from `get_index()`: records with `number > scan_ptr`, excluding `MSG_DELETE`/`MSG_NODISP`.
- For `SCAN_CFG_YONLY`, compare the index `to` field (`smb_name_crc()`, a CRC-16 of the lower-cased name) against `crc16_calc()` of the lower-cased alias and name. Read the header only on a match, to rule out CRC collisions.
- In `lib/events/forum.js`, skip a sub whose `last_msg` and the user's `scan_ptr` are unchanged since the previous pass, and set `last_run` after the scan rather than before.
## 2. Thread views read every header in the sub (high)
With `max_messages=0` (the stock `modopts.ini` value), `getMessageThreads()` (`lib/forum.js:917`) calls `get_all_msg_headers()` on the whole sub. It runs on each:
- thread list page (`pages/001-forum.ssjs:602`)
- thread view (`:387`, `pages/001-forum.xjs:24`)
- `list-threads` API page, i.e. each "load more" (`listThreads()`)
- vote refresh (`getVotesInThread()`, `getVotesInThreads()`)
- 60 s `scan_threads` cycle per viewer of a sub (`getThreadStats()`)
| Sub | Messages | `get_all_msg_headers()` |
|---|---|---|
| largest | 159,537 | 1.9 s |
| second largest | 67,247 | 0.9 s |
| third largest | 23,202 | 0.3 s |
The 60 s cycle can skip the work when the sub's `last_msg`/`total_msgs` are unchanged (votes are messages, so they change both). Removing the per-page-view cost needs the computed threads cached across requests, keyed on those values.
## 3. Correctness bugs in getMessageThreads()
- With `max_messages > 0`, `getSomeMessageHeaders()` (`lib/forum.js:973`) reads numbers `last_msg - max` through `last_msg`. When numbering is sparse that window holds fewer than `max` messages, so threads are silently dropped.
- `lib/forum.js:1006` calls `is_spam(header)`, but no `header` is defined in that scope (the record is `headers[h]`). With `forum_no_spam` set and `max_messages=0`, this throws a ReferenceError.
## 4. Poll data reads every header in the sub (medium)
`getUserPollData()` (`lib/forum.js:287`) calls `get_all_msg_headers()` to find one poll's `tally`; its comment calls this temporary. The tally is only assembled by `get_all_msg_headers()`, so fixing this needs a way to get one message's tally natively.
## 5. Lower priority
- `getNewestMessageInSub()` (`lib/forum.js:56`) walks down from `last_msg` past deleted messages and opens every sub in the group. Its only page caller is commented out; it is still reachable through the `get-newest-message-per-sub` API call.
- The user list page (`pages/More/001-userlist.xjs:143`) reads every user record on every view (1,457 records, 0.1 s here).
- The `node-list` call in `root/api/system.ssjs` reads every user record on every call. No webv4 page uses it.
Possibly related: #1129 (intermittent long pauses navigating webv4).
*Authored by Claude (Claude Code), on behalf of @rswindell*
--- SBBSecho 3.38-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)