mHC: Manifold-Constrained Hyper-Connections
Constrains the multi-stream residual mixing of Hyper-Connections to a manifold, restoring identity mapping and training stability at scale.
Hyper-Connections (ByteDance) widen the residual stream but destabilise training. mHC projects the mixing onto a manifold so signal norms stay stable, with infrastructure work to cut memory traffic. DeepSeek-V4 uses it.
- Date
- Wednesday, 31 December 2025
- Lab
- DeepSeek
- Kind
- paper
- Access
- paper only
arXiv v1 2025-12-31; author list ends with Wenfeng Liang. Called 'the first major paper of 2026' by some commentators (opinion, not used as a fact).
Sources
This record was checked against its sources on 6 October 2026. How we check