Voxxed Days Melbourne 2019
from Monday 13 May to Tuesday 14 May 2019.
Wai Chee is a staff software engineer at Zendesk. She is a polyglot developer who loves working with data and machine learning. She has ten years experience in data processing, distributed systems, API, and web applications. She holds a PhD in computer vision. She is a mum to a cheeky miniature schnauzer. In her spare time she likes to explore dog training techniques and savour street food from all around the world.
See also https://medium.com/@wyau
- I will be co-presenting this talk with my colleague Derrick Cheng (Zendesk) *
This talk covers how we scaled our model building infrastructure at Zendesk with an aim to build at least 50,000 models a day. This is achieved as part of our efforts to deliver a machine learning (ML) product called Content Cues.
Content Cues summarises text from customers support tickets to form insightful topics. It combines multiple ML algorithms including deep learning, clustering and other natural language processing approaches. These ML algorithms are then run through tens of thousands of eligible Zendesk customer data every day.
We will talk about: - how we implement a horizontally scalable model building pipeline by combining AWS EMR, AWS Batch and Kubernetes - how to balance between model performance, scalability and computational efficiency - some real-world problems and scaling complexities you may encounter when building ML product at web scale