Hidden Technical Debt in Machine Learning Systems - NIPS

← Back to document page

Hidden Technical debt in Machine Learning SystemsD. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Ebner, Vinay Chaudhary, Michael Young, Jean-Franc ois Crespo, Dan Learning offers a fantastically powerful toolkit for building useful com-plex prediction Systems quickly. This paper argues it is dangerous to think ofthese quick wins as coming for free. Using the software engineering frameworkoftechnical debt , we find it is common to incur massive ongoing maintenancecosts in real-world ML Systems . We explore several ML-specific risk factors toaccount for in system design.

As an example, suppose that to ease the transition from an old product numbering scheme to new product numbers, both schemes are left in the system as features. New products get only a new number, but old products may have both and the model continues to rely on the old numbers for some products.

  System, Model, Machine, Technical, Learning, Debt, Hidden, Numbering, Hidden technical debt in machine learning systems