mHC: Manifold-Constrained Hyper-Connections
Fixes the training instability of Hyper-Connections by projecting residual mixing matrices onto the doubly stochastic manifold with Sinkhorn-Knopp.
Widened residual streams (Hyper-Connections) improve quality but break the identity mapping that stabilizes deep nets. mHC restores it with a manifold constraint and fused-kernel infrastructure. It is one of three upgrades in V4.
- Date
- Wednesday, 31 December 2025
- Lab
- DeepSeek
- Kind
- paper
- Access
- research preview
Lead author Zhenda Xie; corresponding author Wenfeng Liang. arXiv v1 2025-12-31, v2 2026-01-05.
Sources
This record was checked against its sources on 6 October 2026. How we check