ClusterTech Beijing Limited CHESS & Inspur KOS Co-build a Unified Multi-Paradigm Computing Management Platform for Scientific and AI Workloads
2025-10-15

Today, in fields such as engineering simulation, AI for Science (AI4S), energy exploration, and life sciences, the convergence of scientific and AI computing is accelerating. This poses a new challenge for computing management platforms: they must simultaneously support high-precision, highly stable scientific computing and large-scale, highly concurrent AI workloads. Balancing "stability" with "innovation" highlights the critical need for an underlying operating system capable of heterogeneous compatibility and flexible scheduling.

To address this, ClusterTech Beijing Limited and Inspur Information’s KeyarchOS (KOS) have jointly launched a unified multi-paradigm computing management platform. This solution enables efficient parallel processing and dynamic resource allocation for diverse tasks. By combining the heterogeneous resource management and job scheduling capabilities of its CHESS (ClusterTech HPC Environment Software) platform with KOS’s hardware-software synergy, container toolchain optimisation, and custom kernel enhancements, the platform simplifies the deployment of heterogeneous environments, eliminates performance bottlenecks, and meets the complex computing needs of multiple industries.

 

 

Challenges for Management Platforms: The Coexistence of Scientific and AI Computing

As research becomes increasingly intelligent, the coexistence of scientific and AI computing is the new normal. In automotive R&D, for instance, aerodynamic simulations (scientific computing) require sustained, high-precision CPU power over extended periods, while autonomous driving inference and battery system diagnosis (AI computing) demands peak GPU parallel processing while accommodating potentially sudden surges in workload.

Integrating these distinct paradigms on platform creates complex challenges:

  • Compatibility Challenges: Scientific tasks (e.g., CFD simulators) often rely on specific, stable Linux distributions and LTS kernels. AI frameworks need the latest GPU drivers and libraries. Reconciling "stability" with "innovation" on a single node is a major compatibility hurdle.
  • Resource Scheduling Conflicts: Running a CPU-intensive thermal simulation alongside a latency-sensitive AI inference task on the same node can lead to resource contention. The HPC job might starve the AI task of resources, causing unacceptable delays, or a burst of AI requests could degrade HPC performance.
  • Divergent Optimisation Paths: HPC tasks demand optimised CPU affinity, memory bandwidth, and parallel efficiency. AI tasks prioritise GPU utilisation and memory management. A unified platform must simultaneously satisfy these different needs on shared infrastructure.

 

The Solution: ClusterTech Beijing Limited CHESS + MetaBrain KOS

To address these challenges, ClusterTech Beijing Limited selected MetaBrain KOS as the system foundation for its unified management platform. This combination delivers heterogeneous compatibility, converged scheduling, and performance optimisation, precisely matching diverse tasks to the right compute resources.

 

 

  • Heterogeneous Compatibility: MetaBrain KOS serves as the base, supporting diverse chips (CPU, GPU, DPU) to create a unified resource pool via standardised interfaces. CHESS further optimises drivers for parallel storage (Lustre, BeeGFS, GPFS) and the InfiniBand stack to boost performance for large-scale data and parallel computing. Upwards, CHESS offers deep optimization for over 100 mainstream applications, from HPC tools like Star-CCM+ and WRF to AI models like Qwen and DeepSeek.
  • Converged Scheduling: The platform adopts a hybrid SLURM + K8s scheduling architecture, balancing the parallel processing needs of research tasks with the elastic scaling demands of AI. Leveraging MetaBrain KOS's optimised container toolchain, CHESS better supports containerised AI workloads, simplifying deployment and improving resource utilization for applications like engineering simulation software.
  • Performance Optimization: MetaBrain KOS features kernel customizations for mixed workloads. For HPC tasks, it optimises memory scheduling and CPU affinity. For AI inference, it enhances GPU memory management for dynamic allocation and reclamation, boosting utilization and accelerating model inference.

In the automotive sector, this unified platform effectively tackles the core challenges of hybrid deployment. It schedules HPC tasks (e.g., Star-CCM+, Abaqus) and AI tasks (e.g., autonomous driving model training/inference) onto a unified CPU/GPU pool. The SLURM+K8s architecture ensures stable, long-term CPU/memory allocation for simulations while providing elastic GPU resources for AI, enabling precise matching, efficient parallelism, and faster iteration cycles.

Looking ahead, ClusterTech Beijing Limited and Inspur Information will deepen their collaboration to deliver tailored computing solutions for more industries, reducing the complexity of heterogeneous environments, eliminating performance bottlenecks, and lowering management costs for secure and stable operations.