Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Large Language Models (LLM)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

CAFE AU LAIT: Compute-Aware Federated Augmented Low-Rank AI Training

Federated finetuning is crucial for unlocking the knowledge embedded in pretrained Large Language Models (LLMs) when data are geographically distributed across clients. Unlike finetuning with data from a single institution, federated finetuning allows collaboration across multiple institutions, enabling the utilization of diverse and decentralized datasets while preserving data privacy. Given the high computing costs of LLM training and the emphasis on energy efficiency in Federated Learning (FL), Low-Rank Adaptation (LoRA) has emerged as a widely adopted algorithm due to its significantly reduced number of trainable parameters. However, this assumes that all data silos have the necessary computing resources to compute local updates of LLMs. Nevertheless, in practice, the computing resources across clients are highly heterogeneous: while some may have access to hundreds of GPUs, others might have limited or no GPU access. Recently, federated finetuning using synthetic data has been proposed, allowing clients to participate in a collaborative training run without training LLMs locally. However, our experimental results reveal a performance gap between models trained using synthetic data and those trained using local updates. Motivated by the observed heterogeneity in computing resources and the performance gap, we propose a novel two-stage algorithm that leverages the storage and computing capabilities of a strong server. In the first stage, under the coordination of the strong server, clients with limited computing resources collaborate to generate synthetic data, which is transferred to and stored on the strong server. In the second stage, the strong server uses this synthetic data on behalf of the resource-constrained clients to perform federated LoRA finetuning alongside clients with sufficient computing resources. This approach ensures that all clients can participate in the finetuning process. Experimental results demonstrate that incorporating local updates from even a small fraction of clients improves performance compared to using synthetic data for all clients. Furthermore, we incorporate the Gaussian mechanism in both stages to guarantee client-level differential privacy.

Wang, Jiayi [ORNL]↗
DOE OSTI2026

Agentic traffic intelligence: Augmented human-in-the-loop scenario generation for microscopic traffic simulation

Traditional microscopic traffic simulation generation often relies on static datasets and manual design, limiting its ability to simulate complex conditions easily. This paper presents a novel framework, Agentic Traffic Intelligence, which combines human approval large language models (LLMs), the Real-Twin tool, and multi-agent systems to perform realistic microscopic traffic simulation scenario generation. The proposed framework incorporates human-in-the-loop (HIL) control, retrieval-augmented generation (RAG), and multi-agent control mechanisms. HIL mechanisms are used to guide multiple LLMs focused on attributes for microscopic simulation generation and to improve the interpretability and transparency of LLM execution for users. RAG enhances context extraction by dynamically integrating external knowledge sources for traffic scenario generation foundations. A multi-agent architecture with supervisory control coordinates the interaction of simulation components, including traffic simulators, control logic, and calibration tools. This enables the synthesis of simulation-ready scenarios that reflect dynamic demand profiles and behavior controls. Furthermore, the framework fuses multisource traffic data with unstructured context and supports iterative refinement through interactive user feedback. Validated through microscopic simulation using Simulation of Urban Mobility, the generated scenarios demonstrate high-fidelity network generation with inflow and turn movement and behavioral calibration, offering a robust and efficient tool for stress-testing and optimizing urban mobility systems.

Hierarchical multi-agent control↗
DOE OSTI2024

Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning

Abstract Motivation Creating knowledge bases and ontologies is a time consuming task that relies on manual curation. AI/NLP approaches can assist expert curators in populating these knowledge bases, but current approaches rely on extensive training data, and are not able to populate arbitrarily complex nested knowledge schemas. Results Here we present Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES), a Knowledge Extraction approach that relies on the ability of Large Language Models (LLMs) to perform zero-shot learning and general-purpose query answering from flexible prompts and return information conforming to a specified schema. Given a detailed, user-defined knowledge schema and an input text, SPIRES recursively performs prompt interrogation against an LLM to obtain a set of responses matching the provided schema. SPIRES uses existing ontologies and vocabularies to provide identifiers for matched elements. We present examples of applying SPIRES in different domains, including extraction of food recipes, multi-species cellular signaling pathways, disease treatments, multi-step drug mechanisms, and chemical to disease relationships. Current SPIRES accuracy is comparable to the mid-range of existing Relation Extraction methods, but greatly surpasses an LLM’s native capability of grounding entities with unique identifiers. SPIRES has the advantage of easy customization, flexibility, and, crucially, the ability to perform new tasks in the absence of any new training data. This method supports a general strategy of leveraging the language interpreting capabilities of LLMs to assemble knowledge bases, assisting manual knowledge curation and acquisition while supporting validation with publicly-available databases and ontologies external to the LLM. Availability and implementation SPIRES is available as part of the open source OntoGPT package: https://github.com/monarch-initiative/ontogpt.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Intelligent Manufacturing Support: Specialized LLMs for Composite Material Processing and Equipment Operation

Engineering educational curriculum and standards cover many material and manufacturing options. However, engineers and designers are often unfamiliar with certain composite materials or manufacturing techniques. Large language models (LLMs) could potentially bridge the gap. Their capacity to store and retrieve data from large databases provides them with a breadth of knowledge across disciplines. However, their generalized knowledge base can lack targeted, industry-specific knowledge. To this end, we present two LLM-based applications based on the GPT-4 architecture: (1) The Composites Guide: a system that provides expert knowledge on composites material and connects users with research and industry professionals who can provide additional support and (2) The Equipment Assistant: a system that provides guidance for manufacturing tool operation and material characterization. By combining the knowledge of general AI models with industry-specific knowledge, both applications are intended to provide more meaningful information for engineers. In this paper, we discuss the development of the applications and evaluate it through a benchmark and two informal user studies. The benchmark analysis uses the Rouge and Bertscore metrics to evaluate our models’ performance against GPT-4o. The results show that GPT-4o and the proposed models perform similarly or better on the ROUGE and BERTScore metrics. The two user studies supplement this quantitative evaluation by asking experts to provide qualitative and open-ended feedback about our model’s performance on a set of domain-specific questions. The results of both studies highlight a potential for more detailed and specific responses with the Composites Guide and the Equipment Assistant.

Kapoor, Gunnika [Oak Ridge National Laboratory (OR↗
DOE OSTI2025

Enhancing ChatPORT with CUDA-to-SYCL Kernel Translation Capability

Large Language Models (LLMs) have shown strong capabilities in general code translation. However, code translation involving parallel programming models remains largely unexplored. This work enhances the capabilities of code LLMs in CUDA-to-SYCL kernel translation with parameter-efficient fine-tuning. The resultant fine-tuned LLM, called ChatPORT, is an effort to provide high-fidelity translations from one programming model to another. We describe the preparation of datasets from heterogeneous computing benchmarks for model fine-tuning and testing, the parameter-efficient fine-tuning of 19 open-source code models ranging in size from 0.5 to 34 billion parameters and evaluate the correctness rates of the SYCL kernels by the fine-tuned models. The experimental results show that most code models fail to translate CUDA codes to SYCL correctly. However, fine-tuning these models using a small set of CUDA and SYCL kernels can enhance the capabilities of these models in kernel translation. Depending on the sizes of the models, the correctness rate ranges from 19.9% to 81.7% for a test dataset of 62 CUDA kernels.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2024

Predictive Indicators of the Performance of Large Language Models

In several mission contexts, it is desirable to estimate the performance of large language models (LLMs) on tasks that we cannot run directly. In light of published “scaling laws” our hypothesis is that some tasks should be consistently more challenging than others based on characteristics of the task. The goal of this project was to begin quantifying how much information about LLM performance can be gained from the features of a model and a task. Two of our statistical models struggled to converge. Pass/fail test results may provide limited information for inference beyond model quality and task difficulty, but we see no evidence at this time for significant feature interaction effect sizes, arguing for simple models. Future work extending the models to capitalize on perplexity of ground truth answers is suggested. This project also introduces “Depth of Knowledge Variant Testing” as a strategy for more finely assessing language models on open domain question and answer tasks. We developed sets of questions that ask a language model to produce similar information while demonstrating increasing depth of knowledge, and also relabeled existing Q&A test questions with their depth of knowledge. Our results suggest further consideration of Bloom’s taxonomy and further refinement of prompts to properly elicit information at varying depths. In the course of this work, we set up a basic infrastructure for standardizing tasks and testing many language models on these tasks. In addition to testing the predictive quality of model features and performance across test suites, with this project we have introduced two new task features to contextualize each test question: the Dewey Classification main category of information covered, and the Bloom’s taxonomy level that corresponds to the depth of knowledge probed by the question. Splits across these and other features produced over five hundred task subtypes with distinct feature vectors, which we tested on half a dozen models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Privacy-Aware RAG-Enabled LLMs for Collaborative AI in Organizations

Recent advancements in Large Language Models (LLMs) based on Transformer architectures have significantly improved capabilities in natural language processing and generation. However, deploying LLMs for inter-organizational communication poses challenges, in ensuring privacy and facilitating effective collaboration. This paper introduces a novel decentralized inference meta-agent chatbot that leverages privacy-aware Retrieval-Augmented Generation (RAG)-enabled LLMs for collaborative AI communication across organizations. Built on Microsoft’s Autogen, the platform enables LLMs to autonomously refine responses, enhancing accuracy and relevance. It incorporates advanced hallucination mitigation techniques using Uptrain and a privacy-focused RAG framework that employs synthetic document generation to protect sensitive information. Comprehensive evaluations demonstrate the platform’s effectiveness in maintaining contextual relevance and stringent privacy standards, effectively addressing critical challenges in LLM-enhanced collaborative AI communication. This work represents a significant step toward secure and efficient inter-organizational collaboration using advanced generative AI technologies.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

ATTNChecker: Highly-Optimized Fault Tolerant Attention for Large Language Model Training

Large Language Models (LLMs) have demonstrated remarkable performance in various natural language processing tasks. However, the training of these models is computationally intensive and susceptible to faults, particularly in the attention mechanism, which is a critical component of transformer-based LLMs. In this paper, we investigate the impact of faults on LLM training, focusing on INF, NaN, and near-INF values in the computation results with systematic fault injection experiments. We observe the propagation patterns of these errors, which can trigger non-trainable states in the model and disrupt training, forcing the procedure to load from checkpoints. To mitigate the impact of these faults, we propose ATTNChecker, the first Algorithm-Based Fault Tolerance (ABFT) technique tailored for the attention mechanism in LLMs. ATTNChecker is designed based on fault propagation patterns of LLM and incorporates performance optimization to adapt to both system reliability and model vulnerability while providing lightweight protection for fast LLM training. Evaluations on four LLMs show that ATTNChecker on average incurs on average 7% overhead on training while detecting and correcting all extreme errors. Compared with the state-of-the-art checkpoint/restore approach, ATTNChecker reduces recovery overhead by up to 49×.

Liang, Yuhang [University of Alabama - Birmingham]↗
DOE OSTI2026

Generative large language models for predictive maintenance planning

Maintenance planning and the generation of necessary components for tasks can prove time-consuming and complex. Automating the creation of recurring or similar tasks by leveraging previous planning packages and data, while uncovering insights to automate planning package generation, presents an opportunity to conserve valuable time and resources. This work aims to harness the textual and probabilistic capabilities of large language models (LLMs) to automate the generation of planning packages. Utilizing diverse data sources ranging from raw data to handwritten text, both singular and collaborative LLMs are trained and tested. Results demonstrate their capability to generate essential planning package components, effectively replicating the statistical patterns in the data. This demonstrates the use of these tools inside a digital asset for automated planning. This work outlines a methodology for constructing datasets, a training suite, and evaluation methods for LLM-based textual and conversational planning tools utilized in an asset digital twin. Results indicate that the fine-tuned models generate estimated planning information within the statistical ranges observed in real maintenance data. The models achieve high accuracy (>90%) in document question-answering and instruction generation tasks. Furthermore, the conversational retrieval-augmented generation (RAG) assistant system achieves 100% document retrieval accuracy, while conversational information capture exceeds 98% across the majority of work-package assistant modules.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Optimizing Geospatial Assessments for Nuclear Safeguards Applications with Large Language Models

A multidisciplinary team at Argonne National Laboratory evaluated the ability of large language models (LLMs) to identify geographic locations from open-source text and assessed post-processing measures to strengthen the reliability of those extractions in support of international nuclear safeguards. The study focused on addressing challenges such as toponym ambiguity, imprecise descriptions, and misinformation, which often undermine the accuracy of LLM-derived geospatial assessments. By integrating authoritative geospatial datasets, employing rigorous validation techniques, and leveraging human-in-the-loop processes, the project aimed to enhance the precision, transparency, and reproducibility of geospatial localization workflows. The findings demonstrate that while LLMs exhibit significant potential for accelerating geospatial analysis, their outputs require systematic grounding and verification to ensure reliability in high-stakes applications. This work contributes to the broader field of geospatial intelligence and supports strategic objectives of international organizations such as the International Atomic Energy Agency (IAEA) and the U.S. Department of Energy (DOE).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Large language models for transportation research: Methodologies, state of the art, and future opportunities

The rapid rise of large language models (LLMs) is transforming transportation research, with significant advancements emerging between 2023 and 2025, a period marked by the inception and swift growth of adopting and adapting LLMs for various transportation applications. Despite these significant advancements, however, a systematic review and synthesis of the existing literature remains lacking. This paper aims to fill this gap by providing a comprehensive review of the methodologies and applications of LLMs in transportation. We explore key applications, including autonomous driving, travel behavior prediction, and general transportation-related queries, alongside LLM methodologies such as zero- or few-shot learning, prompt engineering, and fine-tuning. From the review, critical research gaps are identified. From the methodological perspective, many of the research limitations can be addressed by integrating LLMs with existing tools and refining LLM architectures. From the application perspective, research opportunities for LLMs to address various transportation challenges are also explored. By synthesizing these findings, this review not only presents the state-of-the-art LLM adoption and adaptation in transportation, but also proposes future research directions as well as insights and recommendations for policymakers and practitioners, paving the way for greater LLM-driven research innovations in transportation in the future.

42 ENGINEERING↗
DOE OSTI2025

GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

From Text to Maps: LLM-Driven Extraction and Geotagging of Epidemiological Data

Epidemiological datasets are essential for public health analysis and decision-making, yet they remain scarce and often difficult to compile due to inconsistent data formats, language barriers, and evolving political boundaries. Traditional methods of creating such datasets involve extensive manual effort and are prone to errors in accurate location extraction. To address these challenges, we propose utilizing large language models (LLMs) to automate the extraction and geotagging of epidemiological data from textual documents. Our approach significantly reduces the manual effort required, limiting human intervention to validating a subset of records against text snippets and verifying the geotagging reasoning, as opposed to reviewing multiple entire documents manually to extract, clean, and geotag. Additionally, the LLMs identify information often overlooked by human annotators, further enhancing the dataset’s completeness. Our findings demonstrate that LLMs can be effectively used to semi-automate the extraction and geotagging of epidemiological data, offering several key advantages: (1) comprehensive information extraction with minimal risk of missing critical details; (2) minimal human intervention; (3) higher-resolution data with more precise geotagging; and (4) significantly reduced resource demands compared to traditional methods.

Harrod, Karly↗
DOE OSTI2026

Scaling open-weight large language models for hydropower regulatory information extraction: A systematic analysis

Information extraction from regulatory and technical documents using large language models (LLMs) involves practical trade-offs between extraction quality and computational cost. We evaluate eight open-weight LLMs spanning 0.6B–70B parameters on hydropower licensing documents and report deployment-oriented evidence under a unified extraction schema and evaluation protocol. Across the model set, we observe clear scale-dependent trends in both baseline extraction quality and the effectiveness of reflective reasoning (self-checking) under our fixed-prompt, no-augmentation setting. Mid-scale models often provide a favorable balance of accuracy and efficiency, whereas the smallest models show limited or inconsistent gains from the reasoning variants tested. Larger models achieve the highest overall F1 scores but incur substantially greater compute and infrastructure requirements. We further find that reliability failure modes can distort conventional metrics in this domain: in particular, high recall can coincide with systematic extraction errors when models fabricate values for fields that are absent from the source text, underscoring the importance of conservative null handling and evidence-grounded evaluation. Overall, our study provides a reproducible resource–performance comparison for open-weight LLM-based extraction in hydropower regulatory documentation and offers practical guidance for model selection under different deployment constraints.

Evaluation protocol↗
DOE OSTI2026

Leveraging Large Language Models for Understanding Fundamental Principles of Catalysis

Heterogeneous catalysis presents a distinct challenge for artificial intelligence (AI). Data sets are often small and inconsistently reported, catalyst representations are not standardized, and extracting fundamental knowledge requires integrating performance data, spectroscopic characterizations, and mechanistic models across multiple scales. Language offers a unifying representation across these modalities, making catalysis well suited for leveraging large language models (LLMs). By standardizing how catalytic data is represented, LLMs make dispersed experimental results more accessible to downstream statistical modeling. In this perspective, we focus our discussion around three opportunities where LLMs can significantly contribute to catalysis: (1) text to properties; (2) text to structure; and (3) text to mechanistic models. The discussion is followed by a perspective section on LLM-readiness of data, aligning LLM outputs with scientific correctness, and bridging lab-scale discovery to industrial deployment. Across each area, the most productive applications couple dispersed chemical knowledge with physics-grounded validation to produce verifiable hypotheses and actionable representations.

Catalysts↗
DOE OSTI2025

LLMs for Mfg.—On the State of Large Language Models and Applications to Manufacturing

Additive Manufacturing (AM), referred to as 3D printing, has emerged as a key pillar of Industry 4.0 enabling layer-by-layer fabrication of intricate geometries from CAD models. In parallel, Large Language Models (LLMs), deep learning models for natural language generation trained on vast text corpora, have demonstrated unprecedented capabilities in understanding and generating human-like text. The convergence of these trends opens new opportunities at the intersection of AM and AI/ML, where LLMs can assist engineers and researchers in design, manufacture planning, and knowledge discovery. Recent academic work has begun to explore LLM applications in AM and adjacent fields, such as material science, mechanical engineering, and design for additive manufacturing. This exploration ranges from intelligent process planning to domain-specific knowledge retrieval. This survey provides a comprehensive review of current developments, focusing on peer-reviewed literature contributions that apply, adapt, and advance LLMs in general and domain-specific domains. We analyze state-of-the-art (SOTA) techniques, such as fine-tuning foundational models for specific domains, retrieval-augmented generation (RAG) pipelines, knowledge graph integration, and delve into the architectures and evaluation methods employed. The goal of this survey is to inform researchers and practitioners of the current capabilities and limitations of LLMs in general and in domain-specific applications, and to outline how these models are being tailored to meet the requirements of these applications.

36 MATERIALS SCIENCE↗