Kubernetes Batch + HPC Day North America 2022

Kubernetes provides well-defined QoS Classes on pod as guaranteed, burstable, and best-effort. Users can colocate different QoS workloads to achieve resource overcommitment and improve cluster utilization. However, with scale increasing and workloads diversified, some limitations are becoming more: · Lower QoS will be easily throttled or killed once node runs out of resources · The noisy neighbor problem effects the performance of latency-sensitive application · Local hot spots affect the global We implements Koordinator based on Kubernetes with several add-ons to provide QoS-oriented scheduling enhancements: · Definition of sub-QoS classes for complex workloads in co-location scenarios and compatible with the Kubernetes existing QoS semantics · Using dynamic metrics of nodes and pod to provide a more reliable model for resource overcommitment, including resource usage profile and micro metrics such as CPU scheduling, memory allocate latency · Applying fine-grained resource orchestration and isolation mechanism on node to solve the noisy neighbor problem and improve the efficiency of latency-sensitive workloads and batch jobs

Adoption of cloud technologies by high performance computing (HPC) is accelerating, and HPC users want their applications to perform well everywhere. While container orchestration provides resiliency, elasticity, and declarative management, it is not designed to enable app performance like HPC schedulers. In particular, Kube-scheduler is not suited to scheduling emerging HPC workflows that require pods placed advantageously. In response to interest in scheduling flexibility, the K8s community developed the Scheduling Framework to integrate new policies and schedulers. KubeFlux, a Scheduling Framework plugin based on the Fluxion open-source HPC scheduler, provides HPC scheduling capability in K8s. We detail our improvements to the MPI Operator and demonstrate its scalability to 16,384 ranks. With the improved operator we compare the performance of HPC benchmark apps scheduled by Kube-scheduler and KubeFlux. We conclude that KubeFlux makes pod placements that enable much higher app performance than Kube-scheduler. KubeFlux is an example of the rich capability that can be added to K8s and paves the way to converged computing environments with the best capabilities of HPC and cloud.

While use of Kubernetes for various services is growing rapidly, it is still behind in the world of HPC and AI clusters. Part of the reason is that the lack of support for advanced features like multiple 100G networks available in HPC/AI Systems. Vast majority of AI systems in hyperscalers such as IBM Cloud, AWS, Azure, and Oracle Cloud come with two to 8 100G network interfaces on the A100 GPU nodes. However, by default in Kubernetes, a pod has only one network interface, but attaching multiple interfaces is often a requirement in the scenarios. Multus unlocks the potential of multi-networking feature in Kubernetes, but there are still challenges in usability, manageability, and scalability. We present Multi-NIC CNI, a new open-source project, to democratize multiple interfaces capability for everyone. This CNI saves users from the concerns regarding environment heterogeneity and acquiring CNI specific knowledge. This talk will introduce the architecture, use cases, and performance of the CNI, then show how beneficial it is for HPC/AI. We will demonstrate the CNI on a large scale GPU Cluster consisting of over 1400 GPUs and two 100G network interfaces that we build in IBM Cloud.

Thousands of GPUs. Hundreds of thousands of CPUs. Learn how (and why!) G-Research designed and built Armada - a system to enable massive throughput of batch jobs running on Kubernetes. In this session you’ll hear how we use large scale batch compute on Kubernetes to spot patterns in financial markets and predict the future. Armada enables us to schedule millions of batch jobs across many clusters and tens of thousands of nodes, getting optimum utilisation of our hardware to enable our researchers to run the latest machine-learning and advanced data science techniques across vast datasets. We’ll cover the architecture and approach of Armada, challenges and techniques for running Kubernetes at scale and some war stories and lessons learned along the way.

Academic institutions run their own e-infrastructure comprising HPC systems with batch scheduling, cloud infrastructure that allows users to run VMs, and in the last few years, a container infrastructure to run containers. Frequently utilized softwares across all environments are various interactive applications. However, they are not suitable for running in HPC or cloud (as VM), since setting them up requires infrastructural technical knowledge which poses unnecessary load on users. In the Czech NREN, we identified opportunities, situations, use cases where we leveraged K8s for academic use. We decided to offer managed K8s infrastructure for research where users focus solely on their application (container image). Integration with the rest of the infrastructures is DevOps team task. We come to present the progress we have made in this field since the first version of our infrastructure during KCNA 2021. We discuss what a managed K8s infrastructure can offer, e.g. traditional Jupyter notebooks, RStudio servers, but also true 3D game streaming, personalised storage, and much more. Lastly, we open a discussion on the challenges we had to face and issues that are yet to be solved.

Apache Spark on Kubernetes takes advantage of containers and the large, rapidly growing Kubernetes ecosystem to maximize the data processing capability on the cloud. However, running a large-scale production environment is not an effortless combination. Challenges at scale, dev-ops complexity, multi-cluster management, job scheduling, and autoscaling are all roadblocks that could quickly fail the mission. In this session, Bowen Li and Weiwei Yang will share their insights on leveraging open source technology such as Apache YuniKorn, Spark K8s operator, and Cloud primitives to evolve ML data infrastructure in the cloud, including considerations for multi-tenancy, observability, scalability, and cost-effectiveness.