SEARCH · Engineering Papers

Results for “Large Language Models (LLMs)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

Li, Chaojian [ORNL] (ORCID:0000000340309777)↗

DOE OSTI2024

eGridGPT: Trustworthy AI in the Control Room

This report outlines the synergy between human decision making and generative artificial intelligence (GenAI), where GenAI supports power system operators by analyzing procedures, suggesting actions, simulating scenarios with physics-based digital twins, and recommending optimal decisions. This report is the first research effort to apply large language models (LLMs), a type of GenAI, in the power grid control room. The authors describe the Electric Grid Generative Pretrained Transformer (eGridGPT), an LLM that virtually assists system operators. Developed with cybersecurity and regulatory requirements in mind, eGridGPT represents an opportunity to responsibly evolve control room technologies to meet the needs of a rapidly changing grid. As an innovative concept, eGridGPT seeks to spearhead productive discussions about the advanced technologies in the control room of the future amid the transition to clean energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Developing an Interactive OpenMP Book with Large Language Models

Abstract. This paper presents an approach to authoring a textbook titled Interactive OpenMP Programming with the assistance of Large Language Models (LLMs). The writing process utilized state-of-the-art LLMs, including Gemini Pro 1.5, Claude 3, and ChatGPT-4, to generate the initial structure and outline of the book, as well as the initial content for specific chapters. This content included detailed descriptions of individual OpenMP constructs and practical programming examples. The outline and content have then undergone extensive manual revisions to meet our book goals. In this paper, we report our findings about the capabilities and limitations of these LLMs. We address critical questions concerning the necessity of textbook resources and the effectiveness of LLMs in creating fundamental and practical programming content. Our findings suggest that while LLMs offer significant advantages in generating textbook content, they require careful integration with traditional educational methodologies to ensure depth, accuracy, and pedagogical effectiveness. The Interactive OpenMP Programming book is developed with the framework of Jupyter Book, enabling the execution of code within the book from the web browser, providing instant feedback and a dynamic learning experience that stands in contrast to traditional educational resources. The book represents a significant step towards modernizing programming education, offering insights into practical strategies for generating the textbook through advanced AI tools.

Large Language Model · OpenMP · Interactive Book ·↗

DOE OSTI2025

Connecting Minds: AI Use Cases to Bridge Power Systems and Large Language Models for Practical Applications

Recent advances in artificial intelligence (AI) and development of large language models (LLMs) present the opportunity to develop a new generation of power systems applications. In contrast with early power system AI applications based on structured numerical data, LLMs offer unique capabilities to perform logical reasoning using text documents, unstructured data, and application programming interface (API) calls to computational software. This paper seeks to bridge the knowledge gap between power systems engineers and LLM developers through a crosscutting explanation of use cases, characteristics, requirements, practical considerations from the perspectives of both LLM capabilities and industry needs. Specific focus is given to applications that can be realistically deployed by electric utilities. After introducing the architecture of LLMs and unique challenges of the power systems domain, this paper proposes twenty representative LLM applications grouped into categories of 1) power system operations, 2) asset management, 3) system planning and analytics, and 4) energy management and protection systems. Five use cases are presented within each category with descriptions of the motivation, objectives, approaches, example inputs / outputs, and benefits of each use case.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Operating advanced scientific instruments with AI agents that learn on the job

Advanced scientific user facilities, such as next generation X-ray light sources and self-driving laboratories, are revolutionizing scientific discovery by automating routine tasks and enabling rapid experimentation and characterizations. However, these facilities must continuously evolve to support new experimental workflows, adapt to diverse user projects, and meet growing demands for more intricate instruments and experiments. This continuous development introduces significant operational complexity, necessitating a focus on usability, reproducibility, and intuitive human-instrument interaction. In this work, we explore the integration of agentic AI, powered by Large Language Models (LLMs), as a transformative tool to achieve this goal. We present our approach to developing a human-in-the-loop pipeline for operating advanced instruments including an X-ray nanoprobe beamline and an autonomous robotic station dedicated to the design and characterization of materials. Specifically, we evaluate the potential of various LLMs as trainable scientific assistants for orchestrating complex, multi-task workflows, which also include multimodal data, optimizing their performance through optional human input and iterative learning. We demonstrate the ability of AI agents to bridge the gap between advanced automation and user-friendly operation, paving the way for more adaptable and intelligent scientific facilities.

Large Language Models↗

DOE OSTI2026

Data Driven Approach to Public Opinion Mining on Autonomous Vehicles: Sentiment Analysis of Social Media Comments Using Large Language Models

In the realm of online identity, social media has emerged as a rich and dynamic source of user-generated content, making it an invaluable resource for understanding public sentiment on a wide range of topics. Individuals often share their raw emotions and candid opinions on these platforms without fear of judgment or backlash. In this study, we conduct a sentiment analysis on user comments collected from various online platforms, with a specific focus on discussions surrounding autonomous vehicles. Leveraging the capabilities of large language models (LLMs), we classify each comment into one of five sentiment categories: Very Negative, Negative, Neutral, Positive, and Very Positive. Our approach demonstrates the effectiveness of LLMs in capturing nuanced contextual sentiment, offering a scalable and state-of-the-art alternative to traditional manual annotation methods. The results reveal key trends and insights into public perception, enabling a deeper understanding of how autonomous vehicle technologies are received by the online community. Our findings underscore the dynamic nature of public sentiment, which is shaped not only by advances in autonomous vehicle technology but also by contextual events such as regulatory developments, political adjustment and safety incidents.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗

DOE OSTI2025

Leveraging large language models to automate the identification of healthcare access barriers for veterans

Objective: To develop and evaluate an automated system for identifying healthcare barriers focusing on transportation issues in veterans’ clinical notes using large language models (LLMs) and to assess the impact of different prompting strategies on classification performance and explanation consistency. Methods: We developed a hybrid system combining pattern matching for templated notes with LLM analysis for free-text notes. Using 2000 manually annotated clinical notes, we compared four prompting strategies (dual-role short, dual-role long, analysis-first, analysis-only) across Mistral-7B and Llama-3.1 models. We evaluated classification performance using standard metrics and assessed explanation consistency through embedding similarity analysis. Results: The analysis-first strategy achieved superior performance, with Mistral-7B reaching an F1 score of 0.914, outperforming traditional machine learning approaches (GBM: 0.786, BERT: 0.811). LLMs demonstrated higher explanation consistency within models (mean cosine similarity 0.887–0.908) compared to cross-model similarities (0.767–0.872). Pattern matching successfully handled 6.7% of templated notes deterministically. Mistral-7B showed greater internal consistency but higher abstention rates compared to Llama-3.1. Conclusion: Requiring LLMs to analyze evidence before classification improves both accuracy and explanation consistency for identifying transportation barriers in clinical notes. This approach enables automated barrier detection at scale while providing clinically relevant explanations, supporting both population-level healthcare planning and individual patient care decisions.

Healthcare access barriers↗

DOE OSTI2026

Cross-domain digital twin architecture for predictive maintenance via machine learning and Large Language Models

This research introduces a comprehensive framework for creating and deploying a digital twin platform for continuous monitoring and predictive maintenance within industrial settings. Through utilizing advanced technologies, including Unreal Engine 5, Unity 3D, the Message Queue Telemetry Transport protocol, Random Forest machine learning algorithms, and Large Language Models (LLMs), we establish a platform that digitally reproduces physical equipment and translates digital controls into real-world actions. This facilitates preventive maintenance approaches and improves operational effectiveness. The digital twin platform gathers sensor data from operational equipment, analyzes it using machine learning, and delivers practical insights to prevent potential malfunctions and enhance equipment performance. Furthermore, the incorporation of a web portal enables efficient monitoring and access to historical data, educational materials, and equipment status information. Preliminary findings indicate that digital twins can transform industrial equipment management and maintenance methodologies.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Confidence-weighted integration of human and machine judgments for superior decision-making

Large language models (LLMs) can surpass humans in certain forecasting tasks. What role does this leave for humans in the overall decision process? One possibility is that humans, despite performing worse than LLMs, can still add value when teamed with them. A human and machine team can surpass each individual teammate when team members’ confidence is well calibrated and team members diverge in which tasks they find difficult (i.e., calibration and diversity are needed). We simplified and extended a Bayesian approach to combining judgments using a logistic regression framework that integrates confidence-weighted judgments for any number of team members. Using this straightforward method, we demonstrated its effectiveness in both image classification and neuroscience forecasting tasks. Combining human judgments with one or more machines consistently improved overall team performance. Our hope is that this simple and effective strategy for integrating the judgments of humans and machines will lead to productive collaborations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Sentiment analysis of the United States public support of nuclear power on social media using large language models

This study utilized large language models (LLMs) to analyze public sentiment in the United States (US) regarding nuclear power on social media, focusing on X/Twitter, considering climate change challenges and advancements in nuclear power technology. Approximately, 1.26 million nuclear tweets from 2008–2023 were examined to fine-tune LLMs for sentiment classification. We found the crucial role of accurate data labeling for model performance, with potential implications for a 15% improvement, achieved through high-confidence labels. LLMs demonstrated better performance compared to traditional machine learning classifiers, with reduced susceptibility to overfitting and up to 96% classification accuracy. LLMs are used to segment the US public tweets into policy and energy-related categories, revealing that 68% are politically themed. Policy tweets tended to convey negative sentiment, often reflecting opposing political perspectives and focusing on nuclear deals and international relations. Energy-related tweets covered diverse topics with predominantly neutral to positive sentiment, indicating broad support for nuclear power in 48 out of 50 US states. The US public positive sentiments toward nuclear power stemmed from its high power density, reliability regardless of weather conditions, environmental benefits, application versatility, and recent innovations and advancements in both fission and fusion technologies. Negative sentiments primarily focused on waste management, high capital costs, and safety concerns. The neutral campaign highlighted global nuclear facts and advancements, with varying tones leaning towards positivity or negativity. An interesting neutral theme was the advocacy for the combined use of renewable and nuclear energy to attain net-zero goals.

Energy & Fuels↗

DOE OSTI2025

Role of Uncertainty Quantification in the Explainability of Large Language Models for the Nuclear Industry

The meteoric rise of generative artificial intelligence (AI) large language models (LLMs) has created an opportunity to utilize them to increase efficiencies in a multitude of industries. While LLMs carry great potential to revolutionize the manner in which work is performed, numerous known deficiencies limit their utility, including the black box nature of the models, the stochastic nature of the response (i.e., presenting the same prompt multiple times results in different responses), and the potential for hallucination. Widespread adoption of LLMs in safety-critical industries such as nuclear will require some form of explainability to assure end users that the LLM’s response to a given query is valid. Model uncertainty is inherently linked to the concepts of trust and explainability, and can be used to identify situations in which the model is insufficiently certain about its answer. Although uncertainty is not enough in and of itself to determine the suitability of an answer—a model can be very certain of an inaccurate answer—it still provides valuable supporting information. Practical methodologies for gauging or quantifying the uncertainty in LLM outputs are presented herein, along with examples based on nuclear-specific prompts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities

Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Virtual Scientific Companion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Large language models for batteries

Large Language Models (LLMs) are advanced artificial intelligence systems capable of solving diverse tasks using language, reasoning, and external tools. Despite their growing deployment in academia and industry, their potential remains underexplored in battery research. This review presents a comprehensive overview of existing and emerging applications of LLMs in batterie field, addressing two critical questions: What can LLMs offer to support battery-related tasks, and how to develop more effective models for this purpose. We begin by outlining the principles of LLMs and criteria for selecting appropriate models and tools for battery research and development. We then explore their roles in text-mining, data interpretation, and the development of intelligent battery systems. In parallel, we discuss technical challenges, such as data standardizing and sharing, model evaluation, and tool integration. Lastly, we propose future research directions with short-, medium-, and long-term goals and highlight more broad perspectives for connecting experts and cross-disciplinary collaborations.

SoC↗

DOE OSTI2026

Agentic traffic intelligence: Augmented human-in-the-loop scenario generation for microscopic traffic simulation

Traditional microscopic traffic simulation generation often relies on static datasets and manual design, limiting its ability to simulate complex conditions easily. This paper presents a novel framework, Agentic Traffic Intelligence, which combines human approval large language models (LLMs), the Real-Twin tool, and multi-agent systems to perform realistic microscopic traffic simulation scenario generation. The proposed framework incorporates human-in-the-loop (HIL) control, retrieval-augmented generation (RAG), and multi-agent control mechanisms. HIL mechanisms are used to guide multiple LLMs focused on attributes for microscopic simulation generation and to improve the interpretability and transparency of LLM execution for users. RAG enhances context extraction by dynamically integrating external knowledge sources for traffic scenario generation foundations. A multi-agent architecture with supervisory control coordinates the interaction of simulation components, including traffic simulators, control logic, and calibration tools. This enables the synthesis of simulation-ready scenarios that reflect dynamic demand profiles and behavior controls. Furthermore, the framework fuses multisource traffic data with unstructured context and supports iterative refinement through interactive user feedback. Validated through microscopic simulation using Simulation of Urban Mobility, the generated scenarios demonstrate high-fidelity network generation with inflow and turn movement and behavioral calibration, offering a robust and efficient tool for stress-testing and optimizing urban mobility systems.

Hierarchical multi-agent control↗

DOE OSTI2024

Improving Text Classification with Large Language Model-Based Data Augmentation

Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Automatic building energy model development and debugging using large language models agentic workflow

Building energy modeling (BEM) is a complex process that demands significant time and expertise, limiting its broader application in building design and operations. While Large Language Models (LLMs) agentic workflow have facilitated complex engineering processes, their application in BEM has not been specifically explored. This paper investigates the feasibility of automating BEM using LLM agentic workflow. Here, we developed a generic LLM-planning-based workflow that takes a building description as input and generates an error-free EnergyPlus building energy model. Our robust workflow includes four core agents: 1) Building Description Pre-Processing, 2) IDF Object Information Extraction, 3) Single IDF Object Generator Suite, and 4) IDF Debugging Agent. These agents divide the complex tasks into manageable sub-steps, enabling LLMs to generate accurate and reliable results at each stage. The case study demonstrates the successful translation of a building description into an error-free EnergyPlus model for the iUnit modular building at the National Renewable Energy Laboratory. The effectiveness of our workflow surpasses: 1) naive prompt engineering, 2) other LLM-based workflows, and 3) manual modeling, in terms of accuracy, reliability, and time efficiency. The paper concludes with a discussion on the interplay between foundational models and LLM agent planning design, advocating for the use of fine-tuned, specialized models to advance this field.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics

We seek to transform how new and emergent variants of pandemic-causing viruses, specifically SARS-CoV-2, are identified and classified. By adapting large language models (LLMs) for genomic data, we build genome-scale language models (GenSLMs) which can learn the evolutionary landscape of SARS-CoV-2 genomes. By pre-training on over 110 million prokaryotic gene sequences and fine-tuning a SARS-CoV-2-specific model on 1.5 million genomes, we show that GenSLMs can accurately and rapidly identify variants of concern. Thus, to our knowledge, GenSLMs represents one of the first whole-genome scale foundation models which can generalize to other prediction tasks. We demonstrate scaling of GenSLMs on GPU-based supercomputers and AI-hardware accelerators utilizing 1.63 Zettaflops in training runs with a sustained performance of 121 PFLOPS in mixed precision and peak of 850 PFLOPS. We present initial scientific insights from examining GenSLMs in tracking evolutionary dynamics of SARS-CoV-2, paving the path to realizing this on large biological data.

Zvyagin, Maxim↗