Kubernetes Batch + HPC Day North America 2022

Since Kubernetes 1.0 until recently, the Kubernetes Job API had a rather limited feature set. Meanwhile, multiple batch-oriented frameworks were developed, each re-implementing their own job API and controller to manage pods, with their own advantages and limitations, leading to fragmentation of the ecosystem. Starting in Kubernetes 1.21, contributors to SIG Apps decided to evolve the Job API, implement common patterns and increase its scalability, so that it can become the defacto standard for running batch workloads or for building specialized frameworks on top of. They introduced features such as indexed Jobs, suspended jobs, tracking with finalizers, failure policies, etc. In this talk, Aldo will walk you through all these efforts, the challenges that we faced when implementing them and the remaining opportunities we have identified to make the API more comprehensive and useful for a wider range of applications.