StepFun | CNCF

Enroll your company as a CNCF End User and save more than $10K in training and conference costs
StepFun develops multimodal foundation models spanning text, speech, images, video, music, and reasoning. With 22 foundation models, including 16 multimodal models, the company operates AI infrastructure supporting everything from data preparation and training to model distribution and inference.
As StepFun expanded across GPU clusters, private data centers, and multiple public cloud environments, its existing commercial storage system struggled to meet growing demands for scalability, performance, stability, and operational efficiency. Different stages of the AI lifecycle required different storage capabilities, from handling large numbers of small files and high-concurrency random reads to managing terabyte-scale checkpoints and distributing model files exceeding 100 GB.
To address these challenges, StepFun built a unified storage platform using CNCF graduated projects Kubernetes and TiKV alongside JuiceFS, an open source distributed file system. Kubernetes provides the orchestration foundation for containerized AI workloads, while TiKV supports the distributed metadata services used by JuiceFS.
Through metadata optimization, Kubernetes-based storage automation, and improvements to caching and data access, StepFun scaled its platform beyond 10,000 JuiceFS clients, improved targeted read-only operations by approximately 50%, and reduced safetensors read amplification by approximately 71%.
“What began as an inference solution has become our unified file access layer across data generation, training, and inference.”
— Changxin Miao, Head of File Storage, StepFun
As StepFun’s multimodal AI platform grew, so did the complexity of its underlying infrastructure. The company needed to support workloads with significantly different storage requirements across data generation, cleaning, pre-training, post-training, checkpointing, and inference.
Data processing required high bandwidth and high I/O throughput, while training workloads combined large numbers of small files with high-concurrency random access and terabyte-scale checkpoint operations. Inference presented another challenge: distributing and loading models exceeding 100 GB as GPU resources scaled across different environments.
StepFun’s original commercial storage platform became increasingly difficult to scale and operate across heterogeneous hardware, private data centers, and public cloud providers. The company needed a unified storage layer that could support existing applications through POSIX, HDFS, and S3-compatible interfaces while reducing operational complexity.
Metadata performance emerged as a significant bottleneck. At approximately 2,000 to 3,000 clients, a TiKV instance bound to a single disk could approach 6 GB/s of read bandwidth, saturating its NVMe device. Frequent Region splits, transaction conflicts, and prewrite errors further affected stability.
At the same time, safetensors workloads using memory-mapped random reads generated substantial read amplification. In some cases, the storage system retrieved more than ten times the amount of data requested by applications, consuming remote-cache bandwidth without delivering equivalent application throughput.
StepFun needed an architecture capable of supporting continued growth while improving storage efficiency and maintaining consistent operations across its expanding AI infrastructure.
StepFun adopted a unified storage architecture built around Kubernetes, TiKV, and JuiceFS, bringing together cloud native workload orchestration, distributed metadata services, and a consistent file access layer.
Kubernetes, a CNCF graduated project, provides the orchestration foundation for StepFun’s containerized AI workloads. To simplify storage operations across GPU clusters with different hardware configurations, StepFun enhanced the JuiceFS Container Storage Interface (CSI) implementation with a DaemonSet that detects each node’s disk type, capacity, and memory. The system then generates appropriate Mount Pod parameters, reducing the need for manual configuration as workloads move between nodes.
TiKV, another CNCF graduated project, serves as the distributed metadata engine for StepFun’s JuiceFS Community Edition deployments. To overcome metadata scalability challenges, StepFun investigated the sources of excessive TiKV load and optimized background scanning, metadata caching, and point-get operations.
By reducing unnecessary scans and using Linux kernel metadata caching for suitable read operations, the team lowered metadata pressure. Additional optimization of simple read-only transactions avoided unnecessary distributed-transaction and Placement Driver interactions, improving the affected operations by approximately 50%. These improvements were merged into JuiceFS Community Edition and became available from version 1.3.
JuiceFS provides the unified file access layer, connecting workloads to local NVMe, GPFS, and object storage while supporting POSIX, HDFS, and S3-compatible workflows. StepFun also extended its S3 Gateway to serve hundreds of independent volumes through a single endpoint and integrated it with the company’s identity and access management system for authentication and tenant isolation.
To improve inference performance, StepFun introduced multi-level caching and tuned readahead behavior for safetensors workloads. GPU-node memory provides a first-level cache, while stable general-purpose nodes host cache services independently of frequently changing GPU infrastructure. These changes reduced read amplification from 1,042% to approximately 300%.
The architecture also supports model distribution across clouds and data centers. StepFun synchronizes model files larger than 100 GB to destination environments and warms them into stable cache nodes before inference instances start, reducing the impact of cold model loading.
Together, Kubernetes, TiKV, and JuiceFS give StepFun a