Experimentation for Engineers

Learn how to evaluate the changes you make to your system and ensure that your testing does not undermine revenue or other business metrics. Experimentation for Engineers: From A/B testing to Bayesian optimization delivers a toolbox of processes for optimising software systems. From learning the limits of A/B testing to advanced experimentation strategies involving machine learning and probabilistic methods, this practical guide will help you master the skills. It will also help you minimise the costs of experimentation and will quickly reveal which approaches and features deliver the best business results. What's inside Design, run, and analyse an A/B test Break the "feedback loops" caused by periodic retraining of ML models Increase experimentation rate with multi-armed bandits Tune multiple parameters experimentally with Bayesian optimisation About the reader For ML and software engineers looking to extract the most value from their systems. Examples are found in Python and NumPy.