SEARCH · Engineering Papers

Results for “Large Language Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Natural Language Processing-Enhanced Nuclear Industry Operating Experience Data Analysis: Aggregation and Interpretation of Multi-Report Analysis Results

Industry-wide operating experience is a critical source of raw data for reliability and risk model parameter estimations for nuclear power plants. A large portion of operating experience data are failure events stored as reports that contain unstructured data, such as narratives. In current practice, a failure report is usually reviewed and manually coded by analysts. The coding is based on extracting several event characteristics such as system name, component type, sub-part type, failure mode, and failure cause. Event narratives are mostly used to help understand events and extract their characteristics. In this line of research, we aim to maximize the usage of event narratives by leveraging natural language processing (NLP) methods to automatically convert an event narrative to a causal graph. This research has promise to improve physical understanding of failure initiation and propagation and to facilitate use of non-failure data (e.g., near-misses and degradations) to complement the limited data pool of failures. In our previous work, we developed an NLP tool and applied it to analyze a number of licensee event reports submitted by U.S. nuclear power plants to the Nuclear Regulatory Commission. In this paper, we will report our recent research progress in aggregating the results of multiple reports, developing network model(s), and drawing statistical insights.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2023

An autonomous laboratory for the accelerated synthesis of novel materials

To close the gap between the rates of computational screening and experimental realization of novel materials, we introduce the A-Lab, an autonomous laboratory for the solid-state synthesis of inorganic powders. This platform uses computations, historical data from the literature, machine learning (ML) and active learning to plan and interpret the outcomes of experiments performed using robotics. Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets including a variety of oxides and phosphates that were identified using large-scale ab initio phase-stability data from the Materials Project and Google DeepMind. Synthesis recipes were proposed by natural-language models trained on the literature and optimized using an active-learning approach grounded in thermodynamics. Analysis of the failed syntheses provides direct and actionable suggestions to improve current techniques for materials screening and synthesis design. The high success rate demonstrates the effectiveness of artificial-intelligence-driven platforms for autonomous materials discovery and motivates further integration of computations, historical knowledge and robotics.

36 MATERIALS SCIENCE↗

DOE OSTI2019

A Semantic Search Capability for a Grid Model Repository

The operation of electric grids is becoming more challenging due to continued increases in wind and solar generation, the proliferation of decentralization generation resources, and cyber security threats. To support the needed research, new large scale realistic public grid models have been developed and placed in a repository for access by researchers. To assist researchers in easily finding grid models that are suitable for their specific research needs, an approach to searching numerous large grid models was evaluated, prototyped and developed. Here, the research identified multiple possible approaches to using graph database techniques and a natural language processing for the search algorithm. Results of the prototype, the final approach for production repository and the search algorithm formulation are described.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2014

OpenSoC Fabric

Recent advancements in technology scaling have shown a trend towards greater integration with large-scale chips containing thousands of processors connected to memories and other I/O devices using non-trivial network topologies. Software simulation proves insufficient to study the tradeoffs in such complex systems due to slow execution time, whereas hardware RTL development is too time-consuming. We present OpenSoC Fabric, an on-chip network generation infrastructure which aims to provide a parameterizable and powerful on-chip network generator for evaluating future high performance computing architectures based on SoC technology. OpenSoC Fabric leverages a new hardware DSL, Chisel, which contains powerful abstractions provided by its base language, Scala, and generates both software (C++) and hardware (Verilog) models from a single code base. The OpenSoC Fabric2 infrastructure is modeled after existing state-of-the-art simulators, offers large and powerful collections of configuration options, and follows object-oriented design and functional programming to make functionality extension as easy as possible.

Source record↗

DOE OSTI2014

OpenSoC Fabric

Source record↗

DOE OSTI2018

Generation of Reversible C++ Code for Optimistic Parallel Discrete Event Simulation

The reversible execution of C/C++ code has been a target of research and engineering for more than a decade as reversible computation has become a central notion in large-scale parallel discrete event simulation (PDES). The simulation models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper, we focus on parallel simulation models that are written with user-defined C++ abstractions and abstractions of the C++ Standard Library. We present an approach based on incremental state saving for establishing reversibility of C++ and an evaluation for a kinetic Monte-Carlo simulation implemented in C++. Although a significant runtime overhead is introduced with our technique, it is an enormous win that it allows using the entire C++ language, and has that code automatically transformed into reversible code to enable parallel execution with the Rensselaer’s optimistic simulation system (ROSS).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Models and Processes to Extract Drug-like Molecules From Natural Language Text

Researchers worldwide are seeking to repurpose existing drugs or discover new drugs to counter the disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A promising source of candidates for such studies is molecules that have been reported in the scientific literature to be drug-like in the context of viral research. However, this literature is too large for human review and features unusual vocabularies for which existing named entity recognition (NER) models are ineffective. We report here on a project that leverages both human and artificial intelligence to detect references to such molecules in free text. We present 1) a iterative model-in-the-loop method that makes judicious use of scarce human expertise in generating training data for a NER model, and 2) the application and evaluation of this method to the problem of identifying drug-like molecules in the COVID-19 Open Research Dataset Challenge (CORD-19) corpus of 198,875 papers. We show that by repeatedly presenting human labelers only with samples for which an evolving NER model is uncertain, our human-machine hybrid pipeline requires only modest amounts of non-expert human labeling time (tens of hours to label 1778 samples) to generate an NER model with an F-1 score of 80.5%—on par with that of non-expert humans—and when applied to CORD’19, identifies 10,912 putative drug-like molecules. This enriched the computational screening team’s targets by 3,591 molecules, of which 18 ranked in the top 0.1% of all 6.6 million molecules screened for docking against the 3CLPro protein.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2025

pyTCR: A tropical cyclone rainfall model for python

pyTCR is a climatology software package developed in the Python programming language. It integrates the capabilities of several legacy physical models and increases computational efficiency to allow rapid estimation of tropical cyclone (TC) rainfall consistent with the large-scale environment. Specifically, pyTCR implements a horizontally distributed and vertically integrated model [Zhu et al., 2013] for simulating rainfall driven by TCs. Along storm tracks, rainfall is estimated by computing the cross-boundary-layer, upward water vapor transport caused by different mechanisms including frictional convergence, vortex stretching, large-scale baroclinic effect (i.e., wind shear), topographic forcing, and radiative cooling [Lu et al., 2018]. The package provides essential functionalities for modeling and interpreting spatio-temporal TC rainfall data. pyTCR requires a limited number of model input parameters, making it a convenient and useful tool for analyzing rainfall mechanisms driven by TCs. To sample rare (most intense) rainfall events that are often of great societal interest, pyTCR adapts and leverages outputs from a statistical-dynamical TC downscaling model [Lin et al., 2023] capable of rapidly generating a large number of synthetic TCs given a certain climate. As a result, pyTCR significantly reduces computational effort and improves the efficiency in capturing extreme TC rainfall events at the tail of the distributions from limited datasets. Furthermore, the TC downscaling model is forced entirely by large-scale environmental conditions from reanalysis data or coupled General Circulation Models (GCMs), simplifying the projection of TC-induced rainfall and wind speed under future climate using pyTCR. Finally, pyTCR can be coupled with hydrological and wind models to assess risks associated with independent and compound events (e.g., storm surges and freshwater flooding).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2013

Neurons to algorithms LDRD final report.

Over the last three years the Neurons to Algorithms (N2A) LDRD project teams has built infrastructure to discover computational structures in the brain. This consists of a modeling language, a tool that enables model development and simulation in that language, and initial connections with the Neuroinformatics community, a group working toward similar goals. The approach of N2A is to express large complex systems like the brain as populations of a discrete part types that have specific structural relationships with each other, along with internal and structural dynamics. Such an evolving mathematical system may be able to capture the essence of neural processing, and ultimately of thought itself. This final report is a cover for the actual products of the project: the N2A Language Specification, the N2A Application, and a journal paper summarizing our methods.

Rothganger, Fredrick H.↗

DOE OSTI2022

A fast and accurate modeling approach for water and steam thermodynamics with practical applications in district heating system simulation

In U.S. district heating (DH) systems, steam is the most common heat transport medium. Industry demand for new advanced modeling capabilities of complete steam DH systems is increasing; however, the existing models for water/steam thermodynamics are too slow for large system simulations because of computationally expensive algebraic loops that require the solution to nonlinear systems of equations. For practical applications, this work presents a novel split-medium approach that implements numerically efficient liquid water models alongside various water/steam models, breaking costly algebraic loops by decoupling mass and energy balance equations. New component models for steam DH systems are also presented. We implemented the models in the equation based Modelica language and evaluated accuracy and computing speed across multiple scales: from fundamental thermodynamic properties to complete districts featuring 10 to 200 buildings. Compared to district models with the IF97 water/steam model and equipment models from the Modelica Standard Library, the new implementation improves the scaling rate for large districts from cubic to quadratic with negligible compromise to accuracy. Additionally, for an annual simulation with 180 buildings, this translates to a computing time reduction from 33 to 1-1.5 h. These results are critically important for industry practitioners to simulate steam DH systems at large scales.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2018

Predicting Foreign Language Usage from English-Only Social Media Posts

Social media is known for its multicultural and multilingual interactions, a natural product of which is code-mixing. Multilingual speakers mix languages they tweet to address a different audience, express certain feelings, or attract attention. This paper presents a large-scale analysis of 6 million tweets produced by 23 thousand multilingual users speaking 11 other languages besides English. We rely on this multilingual corpus to build predictive models for a novel task – inferring non- English languages that users speak exclusively from their English tweets. We contrast the predictive power of different linguistic signals and report that lexical content and syntactic structure of English tweets are the most predictive of non-English languages that users speak on Twitter. By analyzing cross-lingual transfer – the influence of non-English languages on various levels of linguistic performance in English, we present novel findings on stylistic and syntactic variations across speakers of 11 languages.

code-swicthing↗

DOE OSTI2014

SU-E-T-323: The FLUKA Monte Carlo Code in Ion Beam Therapy

Purpose: Monte Carlo (MC) codes are increasingly used in the ion beam therapy community due to their detailed description of radiation transport and interaction with matter. The suitability of a MC code demands accurate and reliable physical models for the transport and the interaction of all components of the mixed radiation field. This contribution will address an overview of the recent developments in the FLUKA code oriented to its application in ion beam therapy. Methods: FLUKA is a general purpose MC code which allows the calculations of particle transport and interactions with matter, covering an extended range of applications. The user can manage the code through a graphic interface (FLAIR) developed using the Python programming language. Results: This contribution will present recent refinements in the description of the ionization processes and comparisons between FLUKA results and experimental data of ion beam therapy facilities. Moreover, several validations of the largely improved FLUKA nuclear models for imaging application to treatment monitoring will be shown. The complex calculation of prompt gamma ray emission compares favorably with experimental data and can be considered adequate for the intended applications. New features in the modeling of proton induced nuclear interactions also provide reliable cross section predictions for the production of radionuclides. Of great interest for the community are the developments introduced in FLAIR. The most recent efforts concern the capability of importing computed-tomography images in order to build automatically patient geometries and the implementation of different types of existing positron-emission-tomography scanner devices for imaging applications. Conclusion: The FLUA code has been already chosen as reference MC code in many ion beam therapy centers, and is being continuously improved in order to match the needs of ion beam therapy applications. Parts of this work have been supported by the European FP7 project ENVISION (grant agreement no. 241851)

07 ISOTOPE AND RADIATION SOURCES↗

DOE OSTI2021

NukeLM: Pre-Trained and Fine-Tuned Language Models for the Nuclear and Energy Domains

Natural language processing (NLP) tasks (text classification, named entity recognition, etc.) have seen amazing improvements over the last few years. This is due to models such as BERT that achieve deep knowledge transfer by using a large pre-trained model, then fine-tuning the model on specific tasks. The BERT architecture has shown even better performance on domain-specific tasks when the model is pre-trained using domain-relevant texts. Here, inspired by these recent advancements, we have developed NukeLM, a nuclear-domain BERT model pre-trained on 1.5 million abstracts from the DOE Office of Scientific and Technical Information (OSTI) database. This NukeLM model is then fine-tuned for the classification of research articles into either binary classes (related to the nuclear fuel cycle (NFC) or not) or multiple categories related to the subject of the article. We show that continued pre-training of a BERT-style architecture prior to fine-tuning results in greater performance in both article classification tasks. This information is critical for properly triaging manuscripts, a necessary task for better understanding citation networks that publish in the nuclear space and uncovering new areas of research in the nuclear (or nuclear relevant) domain.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2024

Assessing the evolution of research topics in a biological field using plant science as an example

Scientific advances due to conceptual or technological innovations can be revealed by examining how research topics have evolved. But such topical evolution is difficult to uncover and quantify because of the large body of literature and the need for expert knowledge in a wide range of areas in a field. Using plant biology as an example, we used machine learning and language models to classify plant science citations into topics representing interconnected, evolving subfields. The changes in prevalence of topical records over the last 50 years reflect shifts in major research trends and recent radiation of new topics, as well as turnover of model species and vastly different plant science research trajectories among countries. Our approaches readily summarize the topical diversity and evolution of a scientific field with hundreds of thousands of relevant papers, and they can be applied broadly to other fields.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2014

UPC++: A PGAS extension for C++

Partitioned Global Address Space (PGAS) languages are convenient for expressing algorithms with large, random-access data, and they have proven to provide high performance and scalability through lightweight one-sided communication and locality control. While very convenient for moving data around the system, PGAS languages have taken different views on the model of computation, with the static Single Program Multiple Data (SPMD) model providing the best scalability. In this paper we present UPC++, a PGAS extension for C++ that has three main objectives: 1) to provide an object-oriented PGAS programming model in the context of the popular C++ language, 2) to add useful parallel programming idioms unavailable in UPC, such as asynchronous remote function invocation and multidimensional arrays, to support complex scientific applications, 3) to offer an easy on-ramp to PGAS programming through interoperability with other existing parallel programming systems (e.g., MPI, OpenMP, CUDA). We implement UPC++ with a 'compiler-free' approach using C++ templates and runtime libraries. We borrow heavily from previous PGAS languages and describe the design decisions that led to this particular set of language features, providing significantly more expressiveness than UPC with very similar performance characteristics. We evaluate the programmability and performance of UPC++ using five benchmarks on two representative supercomputers, demonstrating that UPC++ can deliver excellent performance at large scale up to 32K cores while offering PGAS productivity features to C++ applications.

Zheng, Y↗

DOE OSTI2013

DBI Galileon in the effective field theory of inflation: orthogonal non-Gaussianities and constraints from the Trispectrum

Very few explicit inflationary scenarios are known to generate a large bispectrum of orthogonal shape. Dirac-Born-Infeld Galileon inflation, in which an induced gravity term is added to the DBI action, is one such model. We formulate it in the language of the effective field theory of inflation by identifying the unitary gauge operators that govern the behavior of its cosmological fluctuations. We show how to recover rather easily from this its power spectrum and bispectrum, which we calculated previously using standard cosmological perturbation theory. We push our calculations up to the determination of the fourth-order action and of the trispectrum, in which shapes absent in k-inflation arise due to the presence of higher-order derivative operators. We finally discuss the combined constraints set on this model by current observational bounds on the bispectrum and trispectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Performance Analysis of Deep Learning Workloads on Leading-edge Systems

This work examines the performance of leading-edge systems designed for machine learning computing, including the NVIDIA DGX-2, Amazon Web Services (AWS) P3, IBM Power System Accelerated Compute Server AC922, and a consumer-grade Exxact TensorEX TS4 GPU server. Representative deep learning workloads from the fields of computer vision and natural language processing are the focus of the analysis. Performance analysis is performed along with a number of important dimensions. Performance of the communication interconnects and large and high-throughput deep learning models are considered. Different potential use models for the systems as standalone and in the cloud also are examined. The effect of various optimization of the deep learning models and system configurations is included in the analysis.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Data Centers and Digital Assurance Workshop 3 – Mitigations for Digital Assurance Risks

The third session of the TADA (Technical Assistance for Digital Assurance) Data Centers Cohort, held on November 18, 2025, focused on developing mitigation strategies for digital assurance risks identified in previous workshops. Hosted by Idaho National Laboratory (INL) and ScottMadden, the session emphasized the application of Cyber-Informed Engineering (CIE) to data center infrastructure, particularly at the utility–data center interface. Participants revisited and ranked key digital assurance risks, including architecture and interface weaknesses, governance gaps, and AI-enabled threats. The workshop introduced the 12 principles of CIE, advocating for consequence-focused design, engineered controls, and secure information architecture to proactively reduce cyber-physical vulnerabilities. These principles were applied to critical data center systems such as power distribution, UPS, cooling, SCADA/BMS, and grid-forming batteries. The session also addressed governance challenges at the interconnection boundary, highlighting the need for clear roles in telemetry sharing, firmware management, and trip settings. Special attention was given to emerging risks from behind-the-meter (BTM) generation, including reverse-power flow and the integration of small modular reactors (SMRs), which shift data centers from large loads to complex generation nodes. Participants explored how interconnection agreements can serve as enforceable instruments for digital assurance, and reviewed gaps in current standards such as NERC CIP, IEC 62443, and IEEE 1547. The workshop concluded with pathways to standardization, including model agreement language, state-level programs, and expanded NERC guidance. INL also presented tools and frameworks for secure procurement and supplier risk management, reinforcing the need for integrated engineering and policy solutions to secure the evolving data center–grid ecosystem. Session 3 of 3.

24 - POWER TRANSMISSION AND DISTRIBUTION↗