Why this is hard at all為什麼這件事本來就難
The simulator advances one master-clock half-cycle (hc) at a time. Each hc it drains a chain of settle waves (mean 12.06, max 45): a wave re-evaluates the currently-dirty nodes, each doing a tiny group walk over the on pass-transistors. Two properties of that loop govern everything below:
模擬器一次推進一個主時脈半週期(hc)。每個 hc 排空一串安定波(settle wave)(平均 12.06 個、最多 45):一個波重算當下 dirty 的節點,每個節點沿導通的 pass-transistor 做一次極小的群組走訪。這個迴圈的兩個性質主宰了底下一切:
- Tiny work per event. ~418 node re-evaluations per hc; the average conducting group is ~1.4 nodes; per-event work is ~20 ns. The engine is dependent-load-latency bound (hot set ~15 KB, L1-resident), not compute-bound.
- 每個事件的工作極小。每 hc 約 418 次節點重算;平均導通群組 ~1.4 個節點;每事件約 20 ns。引擎受限於相依載入延遲(熱資料集 ~15 KB、L1 常駐),不是計算受限。
- Within-wave order is observable semantics. Processing a node mutates the states that decide whether other nodes' channels are on, so the Gauss-Seidel order within a wave changes the resulting groups. Reorder it and the simulation diverges. Bit-exactness (a full-state checksum) is non-negotiable. Remember this — it is the wall.
- 波內順序是可觀察語意。處理一個節點會改動「決定其他節點通道是否導通」的狀態,所以波內的 Gauss-Seidel 順序會改變最終的群組。一旦重排,模擬就發散。位元精確(全狀態 checksum)不可妥協。記住這點 —— 它就是那道牆。
Step 1 — does a clean split even exist?第一步 —— 乾淨的切分存在嗎?
The NES is two big chips (2A03 CPU, 2C02 PPU) plus board glue. They couple through exactly 14 wires: the 8-bit data bus, 3 register-address bits, the PPU chip-select io_ce, nmi, and rw. We flood-filled chip ownership through the transistor channels and got the key result:
NES 是兩顆大晶片(2A03 CPU、2C02 PPU)加上主機板膠合邏輯。它們只透過 14 條線耦合:8 位元資料匯流排、3 條暫存器位址、PPU 晶片選擇 io_ce、nmi、rw。我們沿電晶體通道做了 flood-fill 歸屬,得到關鍵結果:
contested = 0. Apart from those 14 cut wires, the CPU side and PPU side are completely channel-disjoint. A clean two-way partition genuinely exists — and because the sides are disjoint, two threads can write the shared state arrays with no lock at all (disjoint bytes; hardware coherence handles the rest). The only synchronization a split needs is a barrier.
contested = 0。除了那 14 條 cut 線,CPU 側與 PPU 側在通道上完全不相連。乾淨的二分切分確實存在 —— 而且因為兩側 disjoint,兩條執行緒寫共用狀態陣列完全不需要鎖(寫不同 byte;其餘交給硬體一致性)。切分唯一需要的同步是一個 barrier。
But the same census exposes the catch. Work splits 17% CPU / 54% PPU / 29% shared — the "shared" 29% is the clock distribution tree, which gates both chips and toggles every hc. The PPU is one big bidirectional SCC that can't be split further. And per hc, the PPU is active 100% of the time while the CPU is active only 12.5% — there is never an hc where the CPU works alone. So the perfect, sync-free Amdahl ceiling is only 1.20×, with the PPU as the permanent bottleneck.
但同一份普查也暴露了陷阱。工作分布是 CPU 17% / PPU 54% / 共用 29% —— 那 29% 的「共用」是時鐘分配樹,它 gate 兩顆晶片、每個 hc 都翻。PPU 是一整塊不可再分的雙向 SCC。而且每個 hc,PPU 100% 都在工作、CPU 只有 12.5% —— 從來沒有哪個 hc 是 CPU 單獨工作的。所以完美、零同步的 Amdahl 上限只有 1.20×,PPU 是永久瓶頸。
Step 2 — the cost of talking between cores第二步 —— 兩核之間「對話」的成本
A two-thread settle has to synchronize. We measured the raw cost on this machine (AMD Ryzen 7 3700X): two threads pinned to two P-cores, a cache-line-padded ping-pong, the round-trip latency of "signal the other core and wait for its reply":
兩執行緒的安定必須同步。我們在本機(AMD Ryzen 7 3700X)量了這個原始成本:兩條執行緒各釘一顆 P-core、做 cache-line padding 的 ping-pong,量「通知另一顆核心並等它回覆」的 round-trip 延遲:
Already the arithmetic looks grim: if you barrier once per wave, that is ~134 ns of pure sync against ~600 ns of work, twelve times per hc — a ~20% tax on a split whose best case is only 1.20×. (Gemini, consulted independently, put the C# floor at ~134 ns; native MWAITX might reach ~70 ns, same-CCX hardware ~40-50 ns — all still fatal against sub-microsecond work.)
光看算術就不妙:若每波都同步,那是 ~134 ns 純同步 vs ~600 ns 工作,每 hc 十二次 —— 對一個最佳只有 1.20× 的切分課了 ~20% 的稅。(獨立諮詢 Gemini 也把 C# 的地板定在 ~134 ns;原生 MWAITX 或許到 ~70 ns、同 CCX 硬體 ~40-50 ns —— 對次微秒的工作全都一樣致命。)
Step 3 — build it anyway, bit-exact第三步 —— 還是把它做出來,位元精確
We implemented the real thing: main thread = CPU side, worker thread = PPU side, each pinned to its own P-core, sharing the read-mostly arrays, with a lock-free per-wave barrier. A wave runs parallel only when it is provably cross-side-free (io_ce inactive, no cut node, no clock/shared node, both sides have work); otherwise it falls back to serial so the result stays exact.
我們把真版做出來了:主執行緒 = CPU 側、worker = PPU 側,各釘一顆 P-core,共用唯讀陣列,搭配一個無鎖的每波 barrier。一個波只在「可證明無跨側耦合」時才平行(io_ce 未啟用、無 cut 節點、無時鐘/共用節點、兩側都有工作);否則退回序列,結果才會精確。
| Variant變體 | hc/s | vs 1-thread | parallel waves平行波 | checksum |
|---|---|---|---|---|
| 1 thread單執行緒 (baseline) | 133,512 | 1.00× | — | golden |
| 2 threads, per-wave barrier2 執行緒、每波屏障 | 118,300 | 0.89× | 0.0% | golden ✓ |
It is bit-exact (checksum matches the golden value). But 0.0% of waves ran in parallel — so the two-thread engine is pure overhead and runs 11% slower. Why zero? The cause breakdown: ~92% of waves contain a clock/shared node, ~7% have io_ce active, ~0.2% a cut node. The clock tree is in almost every wave, and it gates both sides — so almost every wave is "coupled" and must run serially.
它是位元精確的(checksum 對到 golden)。但平行波 = 0.0% —— 所以兩執行緒引擎純粹是額外開銷,反而慢 11%。為什麼是零?原因分解:~92% 的波含有時鐘/共用節點、~7% 是 io_ce 啟用、~0.2% 含 cut 節點。時鐘樹幾乎在每個波裡、又 gate 兩側 —— 所以幾乎每個波都「耦合」,只能序列。
We tried to rescue it: process the clock/shared nodes serially first, then run CPU‖PPU in parallel (a "3-phase" wave). The checksum diverged immediately. That is the wall from Step 1 biting back — moving a clock node before the chip nodes changes the order in which it gates them, and within-wave order is the semantics. The 3-phase variant proved, by breaking, exactly why the safe version can't fire.
我們試圖搶救:把時鐘/共用節點先序列處理,再平行跑 CPU‖PPU(「三相」波)。checksum 立刻發散。這正是第一步那道牆反咬 —— 把時鐘節點移到晶片節點之前,就改變了它 gate 它們的順序,而波內順序就是語意。三相變體用「壞掉」精確證明了:為什麼安全版一個波都平行不起來。
Step 4 — what if we let it fire anyway?第四步 —— 那如果硬讓它平行呢?
To find out whether parallelism has any upside before investing more, we ran a speed probe: relax the clock-node rule (lump it onto the CPU thread, fire the wave in parallel anyway — not guaranteed bit-exact, measuring speed only).
為了在投入更多之前先確認平行到底有沒有上檔空間,我們跑了一個速度探針:放寬時鐘節點規則(把它併到 CPU 執行緒、硬讓波平行 —— 不保證位元精確,只量速度)。
| Variant變體 | hc/s | vs 1-thread | parallel waves平行波 |
|---|---|---|---|
| 2 threads, speed probe2 執行緒、速度探針 | 90,100 | 0.68× | 53.3% |
Parallel waves jumped from 0% to 53% — and the engine got even slower (0.68×). This is the decisive measurement: more parallelism made it slower. Each fired wave pays a ~134 ns barrier on ~600 ns of work, twelve times per hc; the sync tax simply swamps the work saved. This is the exact same wall that made an earlier per-chip threading attempt 15× slower — now pinned to a number with a clean partition.
平行波從 0% 跳到 53% —— 引擎卻更慢了(0.68×)。這是決定性的量測:越平行越慢。每個觸發的波要在 ~600 ns 的工作上付 ~134 ns 屏障,每 hc 十二次;同步稅直接淹掉省下的工作。這正是先前 per-chip 嘗試慢 15 倍的同一道牆 —— 現在用乾淨切分釘成了一個數字。
The two walls (and why coarse-grain doesn't escape them)兩道牆(以及粗粒度為何也逃不掉)
There are only two granularities, and each dies its own way:
只有兩種粒度,而每一種都用自己的方式死掉:
Fine grain (barrier per wave): dies on sync cost. Measured 0.68–0.89×. The per-wave barrier (~134 ns × 12/hc) is bigger than the work it coordinates. There is no granularity between "per wave" and "per hc" — clock edges define the boundaries of causality.
細粒度(每波一個屏障):死於同步成本。實測 0.68–0.89×。每波屏障(~134 ns × 12/hc)比它協調的工作還大。「每波」與「每 hc」之間沒有中間粒度 —— 時鐘邊界就是因果邊界。
Coarse grain (one barrier per hc): dies on imbalance + state-copy. The only >1× path is to give each thread a private copy of the clock tree so it settles independently within an hc (the "two copies" idea) and sync only at hc boundaries. But that needs a private copy of the hottest state array per thread (TLS cost on every access), plus snapshot/rollback for the ~1% of hc where io_ce actually bridges the bus. The model ceiling is ~1.16×, and those overheads very likely eat all 16%. You would also still be capped by the 1.20× work imbalance, since the PPU SCC can't be split (splitting it needs "ghost nodes" whose overlap approaches the size of the PPU itself — net more work).
粗粒度(每 hc 一個屏障):死於工作不平衡 + 狀態複製。唯一 >1× 的路是給每條執行緒一份私有的時鐘樹,讓它在一個 hc 內獨立安定(「兩份複本」構想),只在 hc 邊界同步。但那需要每執行緒一份最熱狀態陣列的私有複本(每次存取都有 TLS 成本),外加在 io_ce 真正橋接匯流排的那 ~1% hc 做快照/rollback。模型上限 ~1.16×,而這些開銷極可能把 16% 全吃光。而且你仍被 1.20× 的工作不平衡卡住,因為 PPU 那塊 SCC 切不開(切它需要「ghost node」,重疊量趨近 PPU 本身 —— 淨增工作)。
The verdict裁決
Under bit-exactness, this single-instance switch-level simulation cannot be multi-core-accelerated. The partition is clean (contested = 0) and needs no locks — but fine-grain dies on barrier cost, and coarse-grain dies on the 1.20× work imbalance plus state-replication overhead. The root cause is the engine's own defining property: the within-wave Gauss-Seidel order is the result, which makes each hc a strict dependency chain that the shared clock tree threads through nearly every wave. The only honest use of more cores here is running many independent NES instances — which was never this project's goal.
在維持位元精確下,這個單實例開關級模擬無法用多核加速。切分是乾淨的(contested = 0)、不需要任何鎖 —— 但細粒度死於屏障成本,粗粒度死於 1.20× 工作不平衡加狀態複製開銷。根因正是引擎自身的定義性質:波內 Gauss-Seidel 順序就是結果,這讓每個 hc 成為一條嚴格相依鏈,而共用時鐘樹幾乎穿過每一個波。這裡多核唯一誠實的用法是跑許多獨立的 NES 實例 —— 而那從來不是本專案的目標。
This is the parallelism entry in the engine's boundary map: not "we couldn't be bothered", but a chain of measured, falsifiable, bit-exact-verified results — every path priced with a real number, including an independent "stop" from Gemini. It is the multi-core companion to the single-core walls in the frontier and the P-5 case.
這是引擎邊界地圖裡的平行化條目:不是「懶得做」,而是一連串實測、可證偽、位元精確驗證過的結果 —— 每條路都標上了真實數字,還包含 Gemini 獨立給出的「停」。它是單核那幾道牆(前緣、P-5 案例)的多核版姊妹篇。
Where the numbers come from數字的出處
All measured on an AMD Ryzen 7 3700X (Zen 2), two threads pinned to two distinct top-class P-cores, unlocked clock. The 2-thread engine is a real implementation (WireCore.ThreadRun.cs) on a dedicated thread-experiment branch, gated against the golden full-state checksum; it is not merged into the shipping single-thread engine. Work-balance and cause breakdowns come from in-engine instrumentation; the ~134 ns barrier from a standalone two-core ping-pong microbenchmark. Bottleneck analysis cross-checked with an independent Gemini consult. Companion pages: the frontier, a correct optimization that still lost (P-5), BFS vs DFS, the ceiling.
全部量測於 AMD Ryzen 7 3700X(Zen 2),兩條執行緒各釘一顆最高級別 P-core、不鎖頻。兩執行緒引擎是真實作(WireCore.ThreadRun.cs),放在獨立的 thread-experiment 分支、以 golden 全狀態 checksum 把關;沒有併入出貨的單執行緒引擎。工作平衡與原因分解來自引擎內建的儀器化;~134 ns 屏障來自獨立的兩核 ping-pong 微基準。瓶頸分析另以 Gemini 獨立諮詢交叉驗證。延伸頁面:前緣、一個正確卻仍然輸的優化(P-5)、BFS vs DFS、天花板。