DOE OSTI · 3375422
A multimodal large language model for materials science
Abstract
Understanding and predicting the properties of inorganic materials is crucial for accelerating advancements in materials science and driving applications in energy, electronics and beyond. Integrating material structure data with language-based information through multimodal large language models (LLMs) offers great potential to support these efforts by enhancing human–artificial intelligence interaction. However, a key challenge lies in integrating atomic structures at full resolution into LLMs. In this work, we introduce MatterChat, a versatile structure-aware multimodal LLM that unifies material structural data and textual inputs into a single cohesive model. MatterChat uses a bridging module to effectively align a pretrained universal machine learning interatomic potential with a pretrained LLM, reducing training costs and enhancing flexibility. Our results demonstrate that MatterChat greatly improves performance in material property prediction and human–artificial intelligence interaction, surpassing general-purpose LLMs such as GPT-4. We also demonstrate its usefulness in applications such as more advanced scientific reasoning and step-by-step material synthesis.
Keep this discovery
Explore connections, maps & timelines
Tang, Yingheng [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)] (ORCID:0009000153622546), Xu, Wenbin [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)] (ORCID:0009000257090142), Cao, Jie [University of Colorado, Boulder, CO (United States). NSF AI Institute for Student-AI Teaming (iSAT)] (ORCID:000000018268954X), Gao, Weilu [University of Utah, Salt Lake City, UT (United States)] (ORCID:000000033139034X), Farrell, Steven [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)], Erichson, Benjamin [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); University of California, Berkeley, CA (United States). International Computer Science Institute (ICSI)] (ORCID:0000000306673516), Mahoney, Michael W. [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); University of California, Berkeley, CA (United States)], Nonaka, Andy [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)], Yao, Zhi Jackie [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)] (ORCID:0000000158638275). 2026-04-24. A multimodal large language model for materials science. https://doi.org/10.1038/s42256-026-01214-y
Cite the original work for its findings. Save a collection to share your selection of sources.