As generative AI models expand their capabilities, you may have seen them turn simple text prompts into hyper-realistic images or longer video clips.
More recently, generative AI has shown the potential to help chemists and biologists study static molecules like proteins and DNA. Models like AlphaFold can predict molecular structures to accelerate drug discovery, while the MIT-backed “RFdiffusion” can help design new proteins, for example. But one challenge is that molecules are constantly moving and shaking, which is important to model when creating new proteins and drugs. Simulating these movements on a computer using physics (a technique called molecular dynamics) would require billions of time steps on a supercomputer, making it prohibitively expensive.
As a step toward simulating these behaviors more efficiently, researchers at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) and Department of Mathematics have developed a generative model that learns from past data. The team’s system, called “MDGen,” can take one frame of a 3D molecule and simulate what happens next, like a video, stitching together separate still images and even filling in missing frames. By hitting a molecular “play button,” the tool could help chemists design new molecules and closely study how prototype drugs for cancer and other diseases interact with the molecular structures they target.
MDGen is an early proof of concept, but it suggests the beginnings of an exciting new research direction, says co-first author Bowen Jing SM ’22. “Initially, AI models were generating pretty simple videos of things like people blinking or dogs wagging their tails,” says Jing, a doctoral student at CSAIL. “If you go back a few years, we now have great models like Sora and Veo that are useful for a variety of interesting applications. We want to bring a similar vision to the molecular world, where dynamic trajectories become videos. For example, if you feed our model the first and tenth frames, it can animate what’s in between, or it can remove noise from molecular videos to infer what’s hidden.”
According to the researchers, MDGen marks a paradigm shift from previous work on generative AI that allows for a wider range of applications. Previous approaches have been “autoregressive,” creating a video sequence starting with an initial frame and relying on the previous still frame to build the next one. In contrast, MDGen generates frames that are parallel to the diffusion. This means that instead of just pressing play on the original frame, MDGen allows you to, for example, splice frames at endpoints or “upscale” low-frame-rate trajectories.
The research was presented as a paper at the Conference on Neural Information Processing Systems (NeurIPS) last December. Last summer, the work won an award at the International Machine Learning Conference’s ML4LMS workshop for its potential commercial impact.
A small step forward in molecular dynamics
In experiments, Jin and his colleagues found that MDGen simulations were comparable to running direct physics simulations, and generated orbits 10 to 100 times faster.
First, the team tested the model’s ability to take a 3D frame of a molecule and generate the next 100 nanoseconds. Their system chains together successive blocks of 10 nanoseconds so that these generations achieve that time interval. The team found that MDGen could complete the video generation process in about one minute, with accuracy comparable to the baseline model — a fraction of the time it took the baseline model three hours to simulate the same dynamics.
Given the first and last frames of a 1-nanosecond sequence, MDGen also models the steps in between. The researchers’ system demonstrated realism in more than 100,000 distinct predictions. In clips shorter than 100 nanoseconds, it simulated more likely molecular trajectories than the baseline. In these tests, MDGen also demonstrated the ability to generalize to peptides it had never seen before.
MDGen’s capabilities also include frame-within-frame simulations, “sampling” steps every nanosecond to capture faster molecular phenomena more completely. It can also “draw” the structure of a molecule, restoring information about a molecule that has been erased. These features may eventually be used to help researchers design proteins based on specifications for how different parts of the molecule should move.
Manipulating protein dynamics
MDGen is an early sign of progress toward creating more efficient molecular dynamics, Jin and co-first author Hannes Sterk said, but there is still not enough data for these models to have an immediate impact on designing drugs or molecules that produce the motions chemists want to see in target structures.
The researchers aim to expand MDGen from molecular modeling to predicting how proteins change over time. “We’re currently using a toy system,” says Stark, who is also a doctoral student at CSAIL. “To improve MDGen’s predictive capabilities in protein modeling, we need to build on existing architectures and data. There isn’t yet a YouTube-scale repository for these types of simulations, so we hope to develop a custom machine learning approach that can speed up the data collection process for our models.”
MDGen offers a revolutionary way to model molecular changes invisible to the naked eye. Chemists can also use these simulations to more deeply explore the behavior of drug prototypes for diseases such as cancer and tuberculosis.
“Machine learning methods that learn from physical simulations represent a new and growing area in AI for science,” said Bonnie Berger, Simons Professor of Mathematics at MIT, CSAIL principal investigator, and lead author on the paper. “MDGen is a versatile and versatile modeling framework that bridges these two domains, and we are excited to share our initial models in this direction.”
“Sampling realistic transition pathways between molecular states is a major challenge,” said senior author Tommy Jakkola, the Thomas Siebel Professor of Electrical Engineering and Computer Science at MIT and a professor in the Institute for Data, Systems, and Society, and a principal investigator at CSAIL. “This early work shows how we can address such challenges by moving generative models into full simulation runs.”
Researchers in the field of bioinformatics have praised the system’s ability to simulate molecular transitions. “MDGen models molecular dynamics simulations as joint distributions of structure embeddings, capturing the motion of molecules between discrete time steps,” said Simon Olson, an associate professor at Chalmers University of Technology, who was not involved in the research. “By leveraging hidden learning objectives, MDGen enables innovative use cases such as sampling transition paths and redrawing trajectories connecting metastable phases by drawing analogies.”
The researchers’ MDGen work was supported in part by the National Institute of General Medical Sciences, the U.S. Department of Energy, the National Science Foundation, the Machine Learning Consortium for Drug Discovery and Synthesis, the Abdul Latif Jameel Clinic for Machine Learning in Healthcare, the Defense Threat Reduction Agency, and the Defense Advanced Research Projects Agency.
