Skip to content

feat(perf): memory census, Genesis frame-time breakdown, capture/report tools - #127

Merged
finger563 merged 3 commits into
mainfrom
feat/perf-instrumentation
Oct 12, 2026
Merged

finger563 merged 3 commits into
mainfrom
feat/perf-instrumentation

Conversation

@finger563

@finger563 finger563 commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Phase 1 of the Sonic performance / memory plan: instrumentation and tooling only — no behaviour change to any emulator. It makes the later phases (reclaiming internal RAM, Genesis VDP work) measurable with before/after numbers.

What's added

  • Memory census (components/statistics/memory_census.*, replaces main/heap_utils.hpp): one [mem] phase=boot|menu|launch|quit … line per phase with internal RAM / PSRAM free, largest free block and low-water mark, plus [mem] delta … lines at launch and quit against the menu snapshot. A leak or fragmentation across a play cycle is now a number instead of something to eyeball in the old Biggest / Free / Total table.
  • Genesis frame-time breakdown (GENESIS_PERF, default on): cycle-counter buckets per scanline — 68000, VDP render + RGB565 convert, core-0 barrier wait on the sound task, present/audio, pacing sleep, late-frame count, and the core-1 sound unit's frame and leash-wait time — printed as one [genesis perf] … line every 300 frames. One CCOUNT read per section per scanline; FPS is unchanged versus the uninstrumented build.
  • Internal-RAM block / heap-region / task-stack census (print_mem_blocks, second commit): at boot and launch, every internal block above a threshold (labelled with the task whose stack it is, or reserve_pool for the SPIRAM_MALLOC_RESERVE_INTERNAL chunks that esp_psram re-registers as nested internal/DMA-only heaps), each registered heap region's used/free/largest, and every task's stack high-water mark (repeated after a full play session at quit).
  • Tools: tools/serial_capture.py (plain-open, reconnecting console capture with --reset and --until REGEX) and tools/perf_report.py (summary of one capture, or a side-by-side diff of two including buffer-placement changes). README section "Measuring performance and memory".

First profile (Sonic 2, BOX-3)

[genesis perf] fps=53.1 frame=18298us 68k=7550us vdp=9992us barrier=563us present=73us other=120us sleep=517us late=300/300 | core1 snd=17754us leash=9092us | q_dropped=0 | int_free=14350 int_largest=7680
[mem] phase=launch internal: free=14350 largest=7680 min_free=12176 | psram: free=9461972 largest=9437184 | ...
[mem] delta phase=quit internal: free=-112 largest=+0 | psram: free=-256 largest=+0
  • Core 0 is the bottleneck: VDP render+convert 9.8 ms > 68000 7.1 ms per frame; the barrier is 0.6 ms and core 1 spends 9 ms/frame idle on the leash.
  • In-game internal RAM is nearly exhausted: 14 KB free, 7.7 KB largest block, 8 KB low-water. M68K_RAM takes the only large block; VRAM (64 KB) and lfo_pm_table (32 KB) fall back to PSRAM.
  • Quit returns the heap to within 112 B internal / 256 B PSRAM of the menu state; the 102,400 B block is fully restored.

Those two facts set the next phases: free ≥64 KB of contiguous internal RAM so VRAM lands internal, then the VDP line-render/convert hot path.

What the census found (menu, BOX-3)

The 256 KB internal reserve is two chunks (184 KB + 72 KB) carved out of the main heap; the 102 KB "largest block" lives in the 184 KB chunk. A 32 KB region at 0x3fcf0000 sits unused at the menu (744 B too small for lfo_pm_table). Task stacks hold ~75 KB (main 30 KB, esp_timer 17 KB, eight 4-5 KB espp tasks), the LCD line buffers 2 x 19 KB. After a full Sonic 2 session the stack high-water marks are: main 7.9 KB free of 30 KB, esp_timer 11.2 KB free of 16 KB, video task 2.3 KB free of 4 KB.

Testing

  • Built with ESP-IDF 6.1, flashed to a BOX-3, Sonic 2 launch → ~70 s play → quit captured with serial_capture.py --until phase=quit and summarised with perf_report.py (output above). Unthrottled statistics FPS 57.5, in line with the 56.5–57.7 measured on main before this change.
  • perf_report.py diff mode exercised on two synthetic captures.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

…rt tools

Phase 1 of the Sonic performance / memory plan: instrumentation only, no
behaviour change.

- components/statistics/memory_census: `[mem] phase=boot|menu|launch|quit`
  lines with internal/PSRAM free + largest block + low-water mark, and
  `[mem] delta` lines at launch and quit against the menu snapshot, so a leak
  or fragmentation across a play cycle is a number. Replaces main/heap_utils.
- genesis: GENESIS_PERF (default on) cycle-counter buckets per scanline --
  68k, VDP render+convert, core-0 barrier wait, present/audio, pacing sleep,
  late frames, and the core-1 sound unit's frame/leash time -- reported as one
  `[genesis perf]` line every 300 frames. One CCOUNT read per section per
  line; measured FPS unchanged vs the uninstrumented build.
- tools/serial_capture.py (reconnecting capture, --reset, --until REGEX) and
  tools/perf_report.py (summary of one capture or side-by-side diff of two,
  including buffer placement changes). README section on how to use them.

First profile (Sonic 2, BOX-3): VDP 9.8 ms + 68k 7.1 ms per frame on core 0,
core 1 idle 9 ms/frame on the leash; internal RAM at launch 14 KB free /
7.6 KB largest with VRAM and lfo_pm_table in PSRAM.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
Copilot AI balanced review requested due to automatic review settings October 10, 2026 04:31
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

✅Static analysis result - no issues found! ✅

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Performance counters can span sessions, and capture handling can silently produce stale or empty reports.

3 open findings
What changed in this PR

Adds performance and memory instrumentation to support measurable Genesis optimization work.

Changes:

  • Adds lifecycle memory census reporting.
  • Adds Genesis frame-time metrics.
  • Adds serial capture, reporting, and usage documentation.
File Description
tools/​serial_capture.py Captures reconnecting serial sessions.
tools/​perf_report.py Summarizes and compares captures.
README.md Documents profiling workflow.
main/​main.cpp Records lifecycle memory snapshots.
main/​heap_utils.hpp Removes legacy heap reporting.
components/​statistics/​src/​memory_census.cpp Implements memory census output.
components/​statistics/​include/​memory_census.hpp Defines the census API.
components/​statistics/​CMakeLists.txt Adds the heap dependency.
components/​genesis/​src/​genesis.cpp Instruments Genesis frame execution.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

uint32_t frames{0};
uint32_t late{0}; // frames that overran the native frame period
};
static GenesisPerf perf;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #128 (9ee78f5, the Phase 2 commit): init() now resets the perf window (perf = GenesisPerf{} plus the dropped/blit/Z80 baselines) when a Genesis instance starts, so a partial window from a previous session never leaks into the next launch's first report.

Comment thread tools/serial_capture.py
reset = args.reset
partial = b""

with open(out, "ab") as f:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #128 (4a3c0b4): the output is opened with wb, so one invocation is one capture and reusing before.log/after.log can no longer combine sessions. Reconnects still share the single handle.

Comment thread tools/serial_capture.py
Comment on lines +48 to +50
except Exception:
time.sleep(0.5)
continue

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #128 (4a3c0b4): open failures are reported to stderr as they happen (once per distinct error, with a 'connected' line on recovery), and if no connection ever succeeded the script prints the last error and exits 1 instead of claiming it wrote a capture.

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

ESP-IDF Size Report for 'Esp Box Emu'

Metric Base PR Delta
FLASH 4,031,956 bytes (64.09%) 4,034,260 bytes (64.12%) 🔺 +2,304 bytes (+0.03%)
DRAM 145,992 bytes (42.72%) 147,096 bytes (43.04%) 🔺 +1,104 bytes (+0.32%)
IRAM 0 bytes 0 bytes 0 bytes
RAM (DRAM+IRAM) 145,992 bytes 147,096 bytes 🔺 +1,104 bytes (+0.32%)

FLASH uses app .bin size or json2 flash sum. RAM sums DRAM+IRAM via idf_size. Percentages shown when totals are available.
DRAM/IRAM usage does not include memory used by the heap allocator at runtime.
This report was generated by esp-idf-size-delta.

Extend the memory census with `print_mem_blocks()`: at boot and launch it
lists every allocated internal-heap block above a threshold (labelled with the
task whose stack it is, or `reserve_pool` for the SPIRAM_MALLOC_RESERVE_INTERNAL
chunks that esp_psram re-registers as nested internal/DMA-only heaps), each
registered heap region's used/free/largest, and every task's stack high-water
mark -- repeated at quit so a full play session shows how much of each stack
was actually needed.

The heap walk callback runs with the heap locked, so it only records into
pre-reserved storage; printing happens afterwards (printing inside the walker
takes the stdout lock while the heap is locked and aborts).

tools/perf_report.py parses the new lines (blocks, regions, quit-time stack
marks become diffable rows). The capture stop pattern is now
`delta phase=quit`, which the firmware prints last.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
finger563 added a commit that referenced this pull request Oct 11, 2026
Phase 2 of the Sonic performance / memory plan, driven by the Phase 1 census
(PR #127). At the menu the largest contiguous internal block goes from 102 KB
to 131 KB and free internal RAM from 147 KB to 174 KB:

- CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL 256K -> 0. The reserve was realised by
  esp_psram carving the two largest free blocks (184 KB + 72 KB) out of the
  main heap and re-registering them as nested internal/DMA-only heaps, which
  capped the largest block at ~100 KB. Plain malloc() already goes to PSRAM
  first (ALWAYSINTERNAL=0) and every internal buffer here is requested with
  MALLOC_CAP_INTERNAL/DMA explicitly, so nothing relied on the reserve. As a
  side effect the ~110-190 B per play cycle the heap used to retain is gone
  (quit delta is now +0).
- esp_timer task stack 16 KB -> 8 KB (5.2 KB used after a full session).
- doom, msx and shared_memory .bss -> PSRAM via linker fragments (-13.6 KB of
  static DRAM; none of it is hot).
- Genesis: allocate the YM2612 tables (lfo_pm_table, tl_tab, sin_tab) right
  after M68K_RAM and VRAM last. The tables are read per sample on core 1 and
  thrash the shared data cache from PSRAM (measured: sound unit +0.5 ms/frame,
  audible crackle); moving VRAM internal measured no VDP speed-up, so it is
  the right buffer to spill. lfo_pm_table is now internal (was PSRAM); the
  sound unit's compute drops from ~9.0 to ~8.7 ms/frame. Every shared buffer
  logs its placement in the `[genesis mem]` format.
- Instrumentation: BoxEmu counts frames the video queue dropped and the video
  task's blit time; the `[genesis perf]` line reports `dropped=` and `blit=`.
  On the BOX-3 the full-frame blit is ~31 ms, i.e. the LCD path tops out
  around 31 fps and drops ~37% of emulated frames regardless of emulation
  speed.
- tools/serial_capture.py reports port-open failures instead of silently
  writing an empty capture.

Tried and reverted (documented in box-emu.hpp): LCD line buffers 30 -> 20
rows. It freed 12.8 KB but the 4 extra SPI transactions per frame cost core 1
~2 ms (sound-unit compute 8.7 -> 10.8 ms) and Sonic 2 lost ~3 fps, while the
blit time stayed ~31 ms. Running the sound task below the audio task was also
measured (+0.4 ms barrier, no audible gain) and reverted.

Sonic 2 (same ~1 min section): 55.7-57 fps unthrottled vs 57.5 before, within
run-to-run variance; VDP ~10.5 ms and 68k ~7.5 ms per frame unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
…ility

The box-emu example project does not enable configUSE_TRACE_FACILITY /
configTASKLIST_INCLUDE_COREID, so uxTaskGetSystemState() and
TaskStatus_t::xCoreID are unavailable there. Guard both: without the trace
facility the `[mem] task` lines are skipped and blocks are not attributed to
tasks; without the core-id field the task lines report core=-1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
@finger563
finger563 force-pushed the feat/perf-instrumentation branch from 1953029 to 6243b83 Compare October 11, 2026 19:59
finger563 added a commit that referenced this pull request Oct 11, 2026
Phase 2 of the Sonic performance / memory plan, driven by the Phase 1 census
(PR #127). At the menu the largest contiguous internal block goes from 102 KB
to 131 KB and free internal RAM from 147 KB to 174 KB:

- CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL 256K -> 0. The reserve was realised by
  esp_psram carving the two largest free blocks (184 KB + 72 KB) out of the
  main heap and re-registering them as nested internal/DMA-only heaps, which
  capped the largest block at ~100 KB. Plain malloc() already goes to PSRAM
  first (ALWAYSINTERNAL=0) and every internal buffer here is requested with
  MALLOC_CAP_INTERNAL/DMA explicitly, so nothing relied on the reserve. As a
  side effect the ~110-190 B per play cycle the heap used to retain is gone
  (quit delta is now +0).
- esp_timer task stack 16 KB -> 8 KB (5.2 KB used after a full session).
- doom, msx and shared_memory .bss -> PSRAM via linker fragments (-13.6 KB of
  static DRAM; none of it is hot).
- Genesis: allocate the YM2612 tables (lfo_pm_table, tl_tab, sin_tab) right
  after M68K_RAM and VRAM last. The tables are read per sample on core 1 and
  thrash the shared data cache from PSRAM (measured: sound unit +0.5 ms/frame,
  audible crackle); moving VRAM internal measured no VDP speed-up, so it is
  the right buffer to spill. lfo_pm_table is now internal (was PSRAM); the
  sound unit's compute drops from ~9.0 to ~8.7 ms/frame. Every shared buffer
  logs its placement in the `[genesis mem]` format.
- Instrumentation: BoxEmu counts frames the video queue dropped and the video
  task's blit time; the `[genesis perf]` line reports `dropped=` and `blit=`.
  On the BOX-3 the full-frame blit is ~31 ms, i.e. the LCD path tops out
  around 31 fps and drops ~37% of emulated frames regardless of emulation
  speed.
- tools/serial_capture.py reports port-open failures instead of silently
  writing an empty capture.

Tried and reverted (documented in box-emu.hpp): LCD line buffers 30 -> 20
rows. It freed 12.8 KB but the 4 extra SPI transactions per frame cost core 1
~2 ms (sound-unit compute 8.7 -> 10.8 ms) and Sonic 2 lost ~3 fps, while the
blit time stayed ~31 ms. Running the sound task below the audio task was also
measured (+0.4 ms barrier, no audible gain) and reverted.

Sonic 2 (same ~1 min section): 55.7-57 fps unthrottled vs 57.5 before, within
run-to-run variance; VDP ~10.5 ms and 68k ~7.5 ms per frame unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
@finger563
finger563 merged commit d394570 into main Oct 12, 2026
3 checks passed
@finger563
finger563 deleted the feat/perf-instrumentation branch October 12, 2026 15:03
finger563 added a commit that referenced this pull request Oct 12, 2026
Phase 2 of the Sonic performance / memory plan, driven by the Phase 1 census
(PR #127). At the menu the largest contiguous internal block goes from 102 KB
to 131 KB and free internal RAM from 147 KB to 174 KB:

- CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL 256K -> 0. The reserve was realised by
  esp_psram carving the two largest free blocks (184 KB + 72 KB) out of the
  main heap and re-registering them as nested internal/DMA-only heaps, which
  capped the largest block at ~100 KB. Plain malloc() already goes to PSRAM
  first (ALWAYSINTERNAL=0) and every internal buffer here is requested with
  MALLOC_CAP_INTERNAL/DMA explicitly, so nothing relied on the reserve. As a
  side effect the ~110-190 B per play cycle the heap used to retain is gone
  (quit delta is now +0).
- esp_timer task stack 16 KB -> 8 KB (5.2 KB used after a full session).
- doom, msx and shared_memory .bss -> PSRAM via linker fragments (-13.6 KB of
  static DRAM; none of it is hot).
- Genesis: allocate the YM2612 tables (lfo_pm_table, tl_tab, sin_tab) right
  after M68K_RAM and VRAM last. The tables are read per sample on core 1 and
  thrash the shared data cache from PSRAM (measured: sound unit +0.5 ms/frame,
  audible crackle); moving VRAM internal measured no VDP speed-up, so it is
  the right buffer to spill. lfo_pm_table is now internal (was PSRAM); the
  sound unit's compute drops from ~9.0 to ~8.7 ms/frame. Every shared buffer
  logs its placement in the `[genesis mem]` format.
- Instrumentation: BoxEmu counts frames the video queue dropped and the video
  task's blit time; the `[genesis perf]` line reports `dropped=` and `blit=`.
  On the BOX-3 the full-frame blit is ~31 ms, i.e. the LCD path tops out
  around 31 fps and drops ~37% of emulated frames regardless of emulation
  speed.
- tools/serial_capture.py reports port-open failures instead of silently
  writing an empty capture.

Tried and reverted (documented in box-emu.hpp): LCD line buffers 30 -> 20
rows. It freed 12.8 KB but the 4 extra SPI transactions per frame cost core 1
~2 ms (sound-unit compute 8.7 -> 10.8 ms) and Sonic 2 lost ~3 fps, while the
blit time stayed ~31 ms. Running the sound task below the audio task was also
measured (+0.4 ms barrier, no audible gain) and reverted.

Sonic 2 (same ~1 min section): 55.7-57 fps unthrottled vs 57.5 before, within
run-to-run variance; VDP ~10.5 ms and 68k ~7.5 ms per frame unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
finger563 added a commit that referenced this pull request Oct 12, 2026
…nnected

Addresses the Copilot review on #127: append mode let a reused before.log /
after.log silently combine two sessions (and perf_report.py then diffed stale
runs), and a bad or busy port produced an empty file with a success message
and exit 0. One capture per invocation now; if no connection ever succeeded
the last error is printed and the exit code is 1. (The third item -- resetting
the Genesis perf window when a core starts -- is already in this branch.)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1
finger563 added a commit that referenced this pull request Oct 12, 2026
…80 V-int fix, PICO-8 fast paths (#128)

* perf: reclaim internal RAM for the emulator cores

Phase 2 of the Sonic performance / memory plan, driven by the Phase 1 census
(PR #127). At the menu the largest contiguous internal block goes from 102 KB
to 131 KB and free internal RAM from 147 KB to 174 KB:

- CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL 256K -> 0. The reserve was realised by
  esp_psram carving the two largest free blocks (184 KB + 72 KB) out of the
  main heap and re-registering them as nested internal/DMA-only heaps, which
  capped the largest block at ~100 KB. Plain malloc() already goes to PSRAM
  first (ALWAYSINTERNAL=0) and every internal buffer here is requested with
  MALLOC_CAP_INTERNAL/DMA explicitly, so nothing relied on the reserve. As a
  side effect the ~110-190 B per play cycle the heap used to retain is gone
  (quit delta is now +0).
- esp_timer task stack 16 KB -> 8 KB (5.2 KB used after a full session).
- doom, msx and shared_memory .bss -> PSRAM via linker fragments (-13.6 KB of
  static DRAM; none of it is hot).
- Genesis: allocate the YM2612 tables (lfo_pm_table, tl_tab, sin_tab) right
  after M68K_RAM and VRAM last. The tables are read per sample on core 1 and
  thrash the shared data cache from PSRAM (measured: sound unit +0.5 ms/frame,
  audible crackle); moving VRAM internal measured no VDP speed-up, so it is
  the right buffer to spill. lfo_pm_table is now internal (was PSRAM); the
  sound unit's compute drops from ~9.0 to ~8.7 ms/frame. Every shared buffer
  logs its placement in the `[genesis mem]` format.
- Instrumentation: BoxEmu counts frames the video queue dropped and the video
  task's blit time; the `[genesis perf]` line reports `dropped=` and `blit=`.
  On the BOX-3 the full-frame blit is ~31 ms, i.e. the LCD path tops out
  around 31 fps and drops ~37% of emulated frames regardless of emulation
  speed.
- tools/serial_capture.py reports port-open failures instead of silently
  writing an empty capture.

Tried and reverted (documented in box-emu.hpp): LCD line buffers 30 -> 20
rows. It freed 12.8 KB but the 4 extra SPI transactions per frame cost core 1
~2 ms (sound-unit compute 8.7 -> 10.8 ms) and Sonic 2 lost ~3 fps, while the
blit time stayed ~31 ms. Running the sound task below the audio task was also
measured (+0.4 ms barrier, no audible gain) and reverted.

Sonic 2 (same ~1 min section): 55.7-57 fps unthrottled vs 57.5 before, within
run-to-run variance; VDP ~10.5 ms and 68k ~7.5 ms per frame unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* box-emu: log the LCD SPI clock the device actually got

The BSP asks for 60MHz but the ESP32-S3 SPI master only divides 80MHz by
integers, so the panel runs at 40MHz: 320x224x16bpp is 28.7ms of wire time per
frame, which is the display's ~31fps ceiling. 80MHz (the only faster setting)
leaves the BOX-3 panel white, so the ceiling is a hardware fact; log the real
clock at boot so it is not mistaken for software overhead again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* genesis: skip rendering frames the display cannot show

The LCD path takes ~31 ms per frame (40 MHz SPI, a hardware limit on the
BOX-3: Espressif's own BSP runs the ILI9341 at 40 MHz and 80 MHz leaves the
panel white), while a Genesis frame is 16.7 ms, so when the emulator runs at
speed roughly every other frame was discarded at the single-slot video queue
after ~10 ms of VDP render + RGB565 convert had been spent on it.

If the slot is still occupied when a frame starts, the frame could not be
shown: run the 68000/Z80/sound for it as before but skip the pixel work. The
VDP registers, DMA, interrupts and status still advance; only the per-line
sprite-overflow bookkeeping of an invisible frame is lost. The frame buffer
ping-pong only toggles on pushed frames, so screenshots still see the last
displayed frame. GENESIS_SKIP_UNDISPLAYABLE_FRAMES=0 restores the old path.

Sonic 2 on the BOX-3: in-play 49-51 -> 52-56 fps, frame 19.3 -> 15.6 ms,
dropped 37% -> 0 (skipped ~43%), unthrottled 56 -> 69 fps; the stutter in the
audio is audibly reduced. The [genesis perf] line now reports vdp (render) and
convert (RGB565 into PSRAM) separately -- ~8 ms and ~3.2 ms per rendered frame
-- plus skipped=N/300; BoxEmu gains frame_slot_free().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* genesis: 60 Hz pacing, time-aware frame skip, hold the Z80 V-int until taken

Three changes on top of the display-aware frame skip, each verified on the
BOX-3 with the [genesis perf] line and by ear:

- Pace against a running deadline instead of per frame. Rendered frames
  (~19 ms) overrun the 16.7 ms period and skipped frames (~10 ms) finish
  early; paced individually the early frames slept their slack away and the
  average sat at ~54 fps although the mean frame time fit. With a running
  deadline the early frames repay the overrun, so the average rate -- which is
  what keeps YM2612 sample production matched to the I2S clock -- holds 60 Hz
  whenever the mean fits. More than a frame behind resyncs to now (no bursts).
  Sonic 2: 52-56 -> 60.1-60.4 fps in play, skipped exactly 150/300 (a clean
  every-other-frame cadence on the ~31 fps panel), audio clean.

- Skip the next frame's render when the previous one ended past its deadline
  (at most GENESIS_MAX_CONSECUTIVE_SKIPS=2 in a row), so heavier games spend
  core 0 on emulation + audio before pixels nobody can see.

- Hold the Z80 V-int until the Z80 takes it. In the dual-core split the 68000
  runs each scanline before the Z80 does, so a 68k vblank handler that BUSREQs
  the Z80 immediately (Sonic 3 holds the bus ~80 lines for its DMA batch) left
  the Z80 halted across the whole vblank, and the V-int -- deasserted at end
  of frame -- was withdrawn before the bus came back. The Z80 driver got ~150
  of 300 ticks and the music ran slow and stuttered. On hardware the Z80 wins
  that race (it samples /INT within a few cycles). IAutoReset still clears the
  request on take, and a new assert replaces a pending one rather than
  queuing. Sonic 3: 298-301/300 ticks, "the best it has ever sounded"; Sonic 2
  unchanged at 300/300.

- Z80 diagnostics: z80inst.c counts accepted interrupts, BUSREQ assertions,
  slices lost to a held bus, 2 ms handshake timeouts and Z80->YM2612 writes;
  the [genesis perf] line reports per-window deltas (`z80 irq= busreq=
  halted= wto= ymw=`). These are what made the V-int loss visible.

Known remaining: Sonic 3's special stages run 56-59 fps with the core-1
sound unit as the bottleneck (3x the YM2612 writes per frame, barrier wait up
to 4 ms) and stutter slightly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* docs: describe the new perf-line fields and the LCD clock log

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* tools: serial_capture truncates its output and fails when it never connected

Addresses the Copilot review on #127: append mode let a reused before.log /
after.log silently combine two sessions (and perf_report.py then diffed stale
runs), and a bad or busy port produced an empty file with a success message
and exit 0. One capture per invocation now; if no connection ever succeeded
the last error is printed and the exit code is 1. (The third item -- resetting
the Genesis perf window when a core starts -- is already in this branch.)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* pico8: skip _draw for frames the display cannot show; latch gamepad presses

Same idea as the Genesis frame skip: femto8's main loop asks the box whether
the single-slot video queue is free before calling _draw. When it is not (the
LCD manages ~31-43 fps, so a 60 fps cart would have every other frame
discarded after _draw and the render had been paid for) the frame skips _draw
and the render -- PICO-8 itself drops _draw under load -- but still takes a
full frame period (p8_pace_and_post_flip(), split out of p8_flip()). The
[PICO8] report line counts them (`skipped N (display busy)`). 30 fps carts
such as Celeste never skip; the saving is for _update60 carts.

Gamepad presses are now latched between polls (BoxEmu::gamepad_state_latched,
rising edges accumulated by the 30 ms input timer). femto8 detects btnp() only
when the button is down at its once-per-update poll, so on a level that runs
at ~10 fps a normal tap fell between two polls and Celeste could not jump.
The SDL build has the same latch; the box build lacked it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* pico8: rectfill and sprite fast paths (Celeste's heavy levels 10-20 -> 30 fps)

Profiling (PICO8_PROFILE) showed Celeste's heavier levels at 10-20 fps with
rectfill at 27% of the frame: femto8 fills rectangles one pixel_set() at a
time, column-major and without clipping, so a full-screen background fill is
16k calls (~7 ms). draw_rectfill is now one draw_hline per visible row, which
clips once and writes screen nibbles directly in the common case (no fill
pattern, default masks) -- the path circfill/ovalfill already use.

The other hot entry was map() at 9%: 256 draw_sprite() calls per map(), each
64 pixels through gfx_get + color_get + pixel_set (camera, clip, fill-pattern
decode, gfx_set per pixel). draw_sprite now resolves camera, clip and the
draw palette once and copies nibbles directly when no fill pattern applies to
sprites and the read/write mask is default; transparency comes from the draw
palette's high nibble exactly as pixel_set() applies it. Flips, clipping and
the fill-pattern / rw-mask cases fall through to the original code.

Measured on the BOX-3, same level: 10-20 fps -> locked 30 fps; API share of
the frame 37% -> 8% (rectfill 27% -> 2%, map 9% -> 6%).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* main: log the reset reason at boot; optional main-task watchdog hooks

Silent hangs (the box freezes, no panic output) are indistinguishable from
crashes without this: the boot line now reports the hardware reset reason
via espp::SystemInfo (new espp/system dependency). When CONFIG_ESP_TASK_WDT_EN
is enabled in a local sdkconfig (it is off in the defaults), the main task
subscribes to the task watchdog and feeds it once per frame from Cart::run()
and from the menu wait loop, so a frozen main task prints its backtrace
instead of leaving a blank screen.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* genesis: degrade to PSRAM instead of failing init; quit to the menu on a failed init

A 64 KB data-cache experiment (reverted: no gain for the PICO-8 interpreter,
whose rate stayed at ~1.4 M instr/s) left the Genesis 32 KB short of internal
RAM: one of the small INTERNAL-only shared buffers returned NULL, init bailed
with 'Failed to allocate Genesis runtime memory', and the cart loop kept
running an uninitialised core -- a black screen the player read as a freeze.

The small state structs and line buffers (m68k core, YM2612, OPNREGS, render/
sprite line buffers, CRAM, SAT cache, VDP regs, FIFO, CRAM565, VSRAM) and the
audio mix buffers now prefer internal RAM and fall back to PSRAM, each logging
its placement; and GenesisCart::run() returns to the menu when init failed
(genesis_is_initialized()). Placement on the BOX-3 is unchanged by this.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

* review: bound the sprite fast path, drain the press latch at launch, keep VDP status on skipped frames, report failed placements

Copilot review of #128:
- draw_sprite's direct path only handles sprite ids 0..255 (inside the 128x128
  sheet); other values take the bounds-checked slow path as before.
- The gamepad press latch is global and accumulated since boot, so the cart's
  first poll could see menu-navigation presses: PICO-8 drains it at launch,
  and the existing release gate holds the launch press.
- Scanlines of skipped frames call gwenesis_vdp_skip_line(), which does the
  per-frame status bookkeeping (sprite-overflow / collision reset on line 0)
  the render would have done, so the 68000 never reads a stale overflow flag
  from the last drawn frame.
- [genesis mem] placement lines print FAILED when both allocations fail,
  instead of claiming a PSRAM placement for a null pointer.
- doom/linker.lf: note that the IRAM mappings it used to contain were never
  registered (no LDFRAGMENTS), so dropping them is not a regression.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AWhoGwibqcKDhcG17mjVT1

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants