Thomas Ristenpart (photo: supplied)
As the artificial intelligence boom started to pick up speed a decade ago, companies started to deploy machine learning (ML) in new ways. But with those deployments came new risks that caught the attention of security researchers like Thomas Ristenpart.
Ristenpart, a professor of computer science who joined the University of Toronto in 2026, co-authored a paper in 2016 that explored a new threat: the ability to “steal” the parameters that define ML models.
“Stealing Machine Learning Models via Prediction APIs” was presented at the 25th USENIX Security Symposium in 2016. At the time of the paper’s publication, Ristenpart was a faculty member at Cornell University.
Ten years on, the paper’s enduring impact on the field was recognized with a prestigious Test of Time Award at the 35th USENIX Security Symposium held in August 2026.
“For a lot of companies, parameters are meant to be kept confidential—we’ve seen this nowadays with proprietary AI systems like Claude and ChatGPT, where companies spend a lot of money to train them and want to charge people for use to make predictions, generate text, et cetera,” says Ristenpart. “What we realized at the time was that this introduces a new threat: that malicious users of ML systems may want to recover those confidential parameters, or ‘steal’ them, given access to the prediction interface.”
The research showed that this theft was possible for many types of ML models.
The attacks exploited nuances in the way prediction interfaces were designed. The more that interfaces revealed information about how a model reasoned its way to a prediction, the more that malicious users could learn from each query about the model’s underlying parameters, Ristenpart explains.
He notes the parameters could be used to avoid paying the original company for access to the model. There could also be privacy implications if information about training data can be inferred given the parameters. That led them to explore how to harden interfaces to make those attacks more difficult, a challenge that Ristenpart says is ongoing.
“The paper has been influential because so much work has arisen in the last 10 years focused on how we securely deploy ML and AI systems,” says Ristenpart. “The particular issue it focuses on has remained a key challenge for modern AI systems, presaging the issues that face big companies today, including how they design the chat interfaces for modern proprietary generative AI.”
Researchers and practitioners continue to grapple with secure ML and AI deployment that strikes a balance between functionality and confidentiality of parameters, he says.