Devoxx UK 2019
from Wednesday 8 May to Friday 10 May 2019.
Paul is a Technical Lead at Auto Trader. Working in the Data Engineering team, he uses technologies including Spark, Kafka, Scala, Python and Java on a daily basis. Paul has a strong interest in applying lean/agile methodologies to Data Engineering. He received his PhD in Computer Science from the University of Liverpool in 2010.
Auto Trader is the UK’s leading digital automotive marketplace. We receive 60 million cross-platform visits each month, while our ML-powered car valuations provide 5.5 million valuations a month to both consumers and dealers.
Continuous delivery practices are well established at Auto Trader, especially when it comes to deploying more traditional web applications. We are therefore keen to ensure that any new machine learning models we develop fit this way of working; reducing the time to live allows for more experimentation and reduces the cost of getting machine learning models into production.
This talk describes how we launched a suite of new machine learning models with the ability serve low-latency predictions in real time. These models are automatically retrained and redeployed using continuous deployment pipelines in our existing deployment infrastructure, making use of technology including Apache Spark, Airflow, Docker and Kubernetes. Since models are deployed without manual intervention, we developed a robust testing strategy to ensure deployments will not cause a drop in model performance, including accuracy and coverage.