Loading…
October 24, 2022 | Detroit, Michigan
View More Details & Registration Information 

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon North America 2022 - Detroit, MI + Virtual and add this Co-Located event to your registration to participate in these sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Eastern Daylight Time (EDT), UTC -4. To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date."

The schedule is subject to change.
Back To Schedule
Monday, October 24 • 3:25pm - 3:55pm
Beyond Experimental: Spark on Kubernetes - Weiwei Yang, Apple

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
Apache Spark on Kubernetes takes advantage of containers and the large, rapidly growing Kubernetes ecosystem to maximize the data processing capability on the cloud. However, running a large-scale production environment is not an effortless combination. Challenges at scale, dev-ops complexity, multi-cluster management, job scheduling, and autoscaling are all roadblocks that could quickly fail the mission. In this session, Bowen Li and Weiwei Yang will share their insights on leveraging open source technology such as Apache YuniKorn, Spark K8s operator, and Cloud primitives to evolve ML data infrastructure in the cloud, including considerations for multi-tenancy, observability, scalability, and cost-effectiveness.
Speakers
avatar for Weiwei Yang

Weiwei Yang

Staff Software Engineer, AIML Data Infra, Apple
Weiwei Yang is a staff engineer at Apple AIML Data Infra, his work is to build the data infra for batch processing and AI/ML workloads. He is the V.P of Apache YuniKorn, committer, and PMC member of Apache Hadoop, he had lots of experience in optimizing the data infra for big data... Read More →

Beyond experimental Spark on Kubernetes pdf
Monday October 24, 2022 3:25pm - 3:55pm EDT
Room 252 AB
  Sessions