Skip to content

M2T6 — Route tokens through a toy MoE

Run ./run.ps1. The artifact records total versus active parameters and produces an expert-load histogram. Success means explaining why active parameters predict arithmetic but total parameters still predict weight capacity.


View the runnable source on GitHub