When a deep learning model makes a decision in a high-stakes setting—say, a medical diagnosis or a financial risk assessment—researchers typically have to reverse-engineer a vast network of internal processing nodes to figure out what the model knows and how it will behave. Jacob Andreas, a newly tenured associate professor at MIT, thinks there is a better way: train the models to explain themselves.
Andreas, who works in the Department of Electrical Engineering and Computer Science (EECS) and the Computer Science and Artificial Intelligence Laboratory (CSAIL), develops techniques that enable AI systems to communicate accurately, in language humans can understand, about what they learned from their training data. "Rather than needing to use all these external tools to try and understand what is going on under the hood, maybe we can train models to talk about themselves in a way that is accurate," he told MIT News.
His group focuses on two main areas. One is the science of large language models: understanding the algorithms these models deploy as they learn and process information. The tools they develop help scientists predict how a trained model will behave in the real world. For instance, one family of techniques trains large language models to assign a natural language description to every information-processing node in a massive neural network. A description might label a node that identifies "patches of grass near animals" in input images, according to MIT News.
The other side of his team's work involves designing new algorithms for training machine-learning models and for predicting how specific datasets or examples will affect different aspects of model behavior. "We don't have a good scientific theory of what is going to happen when you train a model on a particular dataset. This is a big challenge for the field right now," Andreas said.
For university students studying computer science, AI, or related fields, Andreas's work points to a shift in how the next generation of engineers will interact with machine learning. Rather than treating models as black boxes, students may increasingly need to think about interpretability and communication as core design goals. Andreas himself co-teaches undergraduate and graduate courses in natural language processing at MIT (courses 6.461 and 6.861), and he told MIT News that the curriculum must be redesigned every year to keep pace with developments. He also noted a challenge familiar to students everywhere: in an era where a large language model can complete many assignments, a deep understanding of the underlying technology is what will set graduates apart. "I try to impress upon the students that having a really deep foundation in how all this technology works under the hood is the only way they are going to understand it, and will differentiate them in a world where everyone has access to a coding agent," he said.
Andreas's path to this research began with a love of language. Growing up in the San Francisco Bay Area, he was a voracious reader. A book passed along by a family friend, in which a character hacked into a computer system, sparked his interest in computers. He later studied computer science as an undergraduate at Columbia University, where he ended up in natural language processing after emailing faculty working on AI-related topics. "That is how I wound up in the field where I am now," he recalled.
He completed a master's degree at Cambridge University and a PhD at the University of California at Berkeley, where he worked in the lab of professor Dan Klein. His PhD studies coincided with the rise of neural networks, a seismic shift from the statistical methods that had dominated natural language processing. "It was fun to be there as we made this transition to the modern toolkit, and learning and inventing all of that stuff when it was still new," he said. He also collaborated with computer vision labs to design AI systems that could generate natural language by "seeing" and understanding the world around them.
After a year at the startup Semantic Machines, which spun out of his Berkeley lab, Andreas joined the MIT faculty in 2019. "It felt like this was a fantastic opportunity to be in the homeland of AI research and collaborate with all these amazing, established people, but also do something new and figure out what the modern era in machine learning was going to look like," he said.
Looking ahead, Andreas expects that interpretability will become even more critical as AI agents and models trained on raw scientific data are deployed in high-stakes situations. "Figuring out how to enable these models to verbalize all of this hidden knowledge they have about things people have never written about explicitly will be a huge challenge and opportunity for people who work in machine learning," he said, according to MIT News.