From HC to mHC: How DeepSeek Uses Manifold Constraints to Improve Large Model Training

DeepSeek shocked the world in 2025 with a high-cost-performance large model. Now entering 2026, the company continues to demonstrate resilience in technological innovation. On January 1st, DeepSeek published a new paper proposing the Manifold-Constrained Hyperconnection (mHC) architecture, which offers systematic improvements to address the stability issues of existing Hyperconnection (HC) networks in large model training. This not only reflects DeepSeek’s meticulous pursuit of technical details but also indicates that large model architecture design is entering a more refined optimization stage.

Hidden Challenges in Large Model Training

Hyperconnection (HC) technology is a great idea in itself, but it encounters key issues in practical applications. HC architecture enhances model performance by increasing network connections, but this process disrupts the identity mapping property — an important characteristic in neural network training that helps gradients flow better and maintains training stability.

This leads to two direct consequences:

• Unstable training: gradient flow is hindered, making convergence difficult
• Limited scalability: the larger the model, the more pronounced the problem, making it hard to support ultra-large-scale model training

For companies pursuing bigger and more powerful models, this is an unavoidable bottleneck.

The Solution Approach of mHC Architecture

DeepSeek’s solution is straightforward: since HC disrupts the identity mapping property, let’s restore it.

The core innovation of mHC lies in two aspects:

Theoretical Level

Mapping the residual connections of HC onto a specific manifold, restoring the identity mapping property within this particular geometric space. It sounds complex, but essentially, it uses mathematical constraints to allow the network to maintain training stability while increasing connections.

Engineering Level

Combining rigorous infrastructure optimization to ensure efficiency. It’s not just a theoretical improvement but also ensures that this architecture can operate efficiently in practical training scenarios.

According to the evaluation by the paper’s team, this improvement achieves “significant performance gains and superior scalability” — meaning models built with mHC are not only more stable during training but also better at scaling to larger sizes.

Why This Matters

On the surface, this is a technical paper. But there are several points worth considering:

Continuous Technical Refinement. Last year, DeepSeek shook the industry with its cost-effective advantage. This new paper shows that the company isn’t resting on its commercial success but continues to invest in fundamental technology. Such focus is rare.

Deepening Architecture Design. The competition in large models has shifted from “who has more parameters” to “who has a better architecture.” mHC represents this more refined direction — solving training challenges with smarter design rather than simply stacking resources.

Evolution of Foundation Models. DeepSeek explicitly states in the paper that mHC “will help deepen understanding of topological architecture design and point to promising directions for the evolution of foundation models.” This indicates they see this improvement as a reference for future large model development.

Summary

The release of the mHC architecture demonstrates DeepSeek’s ongoing commitment to technological innovation. By restoring the identity mapping property and combining engineering optimizations, this new architecture addresses practical pain points of HC technology in large model training. While such foundational architecture improvements may not be as eye-catching as new model releases, they are equally important for advancing large model technology. In the context of increasingly fierce global AI competition, such technical accumulation is becoming a core competitive advantage for enterprises.