Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Methods for evaluation and treatment of data shift

This is a code repository for a set of tools for detecting and mitigating data shifts in machine learning. The goal of the tools is to provide capabilities for determining when new data sets differ from training data sets and for adapting existing models to new data or correcting data shifts (via domain adaptation). The components will be written in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning and signal processing.

Parikh, Nidhi↗
DOE OSTI2024

Understanding Generative AI Content with Embedding Models

The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an implicit form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and those generated from AI models.

Vargas, Max↗
DOE OSTI2024

MOOSE-Workbench integration and MOOSE meshing capability enhancements to facilitate inputs and outputs for multiphysics modeling

The Multiphysics Object-Oriented Simulation Environment (MOOSE) is an open-source framework that supports many of the US Department of Energy’s (DOE’s) Nuclear Energy Advanced Modeling and Simulation (NEAMS) technical areas (TA). These TAs develop and use NEAMS physics and coupling modules in multiple ways to enable the research and development of complex physics models. In addition to the MOOSE framework, the NEAMS Workbench user interface provides a common analysis environment with user-interaction accelerators that streamline the tasks of model creation, review, execution, and out put inspection. In FY 2024, objectives were realized in the MOOSE framework application development support and user-oriented improvements. Application development improvements support both developers and users with an expanded Reactor Module and Mesh System, stateful material property support for mortar contact, and customizable convergence criteria. Additionally, new user-oriented features were implemented in the MOOSE framework language server, including autocompletion snippets, definition from source and find reference navigations, and syntax overrides. Lastly, improvements were made to the input interpreter necessary to support the MOOSE language server and the NEAMS Workbench so that they can interact with syntactically incomplete user inputs. These improvements and more were intended to address stakeholder feedback and improve developer and user ability to conduct advanced nuclear energy modeling and simulation in support of DOE and industry needs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Demonstrating solarpilot ’s Python Application Programmable Interface Through Heliostat Optimal Aimpoint Strategy Use Case

solarpilot is a software package that generates solar field layouts and characterizes the optical performance of concentrating solar power (CSP) tower systems. solarpilot was developed by the National Renewable Energy Laboratory (NREL) as a stand-alone desktop application but has also been incorporated into NREL’s System Advisor Model (SAM) in a simplified format. Prior means for user interaction with solarpilot have included the application’s graphical interface, the SAM routines with limited configurability, and through a built-in scripting language called “LK.” This article presents a new, full-featured, python-based application programmable interface (API) for solarpilot, which we hereafter refer to as CoPylot. CoPylot enables python users to perform detailed CSP tower analysis utilizing either the Hermite expansion technique (analytical) or the SolTrace ray-tracing engine. CoPylot’s enables CSP researchers to perform analysis that was previously not possible through solarpilot’s existing interfaces. This article discusses the capabilities of CoPylot and presents a use case wherein we populate a model that obtains optimal solar field aiming strategies.

14 SOLAR ENERGY↗
DOE OSTI2023

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Considerations regarding the Use of Computer Vision Machine Learning in Safety-Related or Risk-Significant Applications in Nuclear Power Plants

With the advancements made to date in the field of artificial intelligence (AI), significant potential exists to utilize AI capabilities for nuclear power plant (NPP) applications. AI can replicate human decision making and it is usually faster and more accurate than humans. For implementations that impact critical NPP applications (e.g., safety-related or non-safety systems that potentially affect overall plant risk), a deeper safety analysis of the AI methods is necessary. AI applied to NPP operations could resemble the use of digital I&C (DI&C) because such applications involve digital computer hardware and custom-designed software that input plant data, execute complex software algorithms, and output the results to a system or licensed human operator to potentially provoke an action. For AI methods to be compliant with current safety requirements for DI&C, AI compatibility must be evaluated, and AI-related gaps may exist that prevent the prompt deployment of AI in NPPs. This effort aims to evaluate how example AI technologies align with the DI&C safety framework, and discusses how they could be analyzed, modeled, tested, and validated in a manner similar to typical DI&C technologies. Because AI is a broad field that encompasses areas such as machine learning (ML), natural language processing, and computer vision, this research focused on a subset of methods categorized as the computer vision ML (CVML) methods. This report explores two CVML use cases, gauge reading and fire watch, considered relevant to the DI&C standards, as they could play a safety-critical role. For the gauge reading use case, a CVML-enabled technology that can read gauges at oblique angles is utilized. For the fire watch use case, a CVML-enabled technology is utilized that migrates fire watch from a manual (human) approach to automated fire detection. These use cases are mainly intended to give context to the CVML system discussion. This effort assumes the worst-case scenario, with the CVML system being used to replace a safety-related or risk-significant system, thus requiring evaluation. Evaluating CVML against most of the relevant safety requirements for DI&C yielded several CVML-specific considerations due to the uniqueness of its characteristics in comparison with typical DI&C systems. For example, CVML models often employ commonly used (open-source) datasets, and it is not always possible to determine the level of overlap among open-source datasets. Therefore, the independence of the developed CVML models when demonstrating diversity is questionable, therefore creating vulnerability to common cause failure (CCF). The design verification process is also impacted since the data overlap could result in overestimation of the software validation and verification (V&V) performance results. Section 2 of this report evaluates a list of the identified CVML-specific characteristics and discusses the resulting considerations and potential solutions in the context of each referenced requirement. A summation is provided in Section 3. This report is not to be used as a guideline. It was developed to identify and consider issues in the implementation of ML technologies used to augment activities that may have a bearing on plant operation. The report draws parallels to the use of DI&C technologies, for which many standards are available to guide their use in nuclear plant operation. It considers the technologies and some of the potential implications of their use in safety-related applications but is not intended to address regulatory or licensing related issues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Trustworthy Physics-Informed Deep Learning for Predictive Scientific Computing

This project has developed powerful trustworthy physics-informed deep learning (TPiDL) models and methods to fundamentally enhance the scale and power of computational modeling in the scientific and engineering domains. Deep learning (DL) has radically advanced the state-of-the-art in machine learning, computer vision, natural language processing, and also scientific computing. Nevertheless, progress has been driven almost entirely by empirical observations, hacks, and tricks. Under the support of this project, the graph operator learning tools and advanced trustworthy physical informed neural networks have been developed. In addition, stochastic gradient replica-exchange Markov Chain Monte Carlo (MCMC) sampling algorithms have been designed to quantify the uncertainties and speed up the training of large-scale neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Designing interagency responses to wicked problems: A viable system model board game

Government agencies struggle to address wicked problems because they are open-ended, highly interdependent issues that cross agency, stakeholder, jurisdictional, and geopolitical boundaries. While both quantitative modelling and qualitative problem structuring methodologies have been used to support interagency decision making in the past, co-designing an effective interagency organization to collaboratively tackle wicked problems is more challenging. Few approaches have been developed to enable such efforts. This paper explains how the viable system model (VSM) was implemented through a board game, which was employed to co-design an interagency meta-organization that would be capable of more effectively collaborating to jointly address a wicked problem: international organized drug crime and its interface with local gangs in Chicago, USA. The board game was developed to make the VSM easier for the participants to learn, given that the cybernetic language and engineering-influenced diagrams in the original literature can be off-putting to leaders and managers. The board game was used as the final stage of a multi-method, systemic approach, which involved boundary critique and problem structuring as well as deployment of the VSM. The research findings indicate that the VSM board game, used as part of a larger mixed-methods systemic intervention, contributes to building trust in the value of systems thinking amongst the participants, and sets up a rich context for collaboration on multi-agency co-design. The game therefore offers significant promise as part of the co-design of interagency responses to wicked problems because it creates an embodied process for stakeholders to learn about the VSM. It also reduces the work involved in this learning. Thus, the game enables an effective appropriation of the VSM language and criteria.

Business & Economics↗
DOE OSTI2025

Enabling Efficient Sparse Computations using Linear Algebra Aware Compilers

This project developed the LAPIS compiler framework, built on the Multilevel Intermediate Representation (MLIR), to optimize sparse linear algebra operations and support performance portability across diverse architectures. The main innovation of LAPIS is the Kokkos dialect, which allows for lowering codes from a high productivity language to different architectures in an elegant way. The dialect also allows the conversion of lower-level MLIR code to C++ Kokkos code, facilitating the integration of scientific machine learning (SciML) models into applications. To extend LAPIS for distributed memory architectures, a new partition dialect was created to manage the distribution of sparse tensors and express communication patterns for sparse linear algebra operations. This dialect also supports the distributed execution of operators and includes algorithmic optimizations to minimize communication to improve performance. The project also demonstrates that MLIR can enable effective linear algebra-level optimizations, improving performance on different GPUs for both sparse and dense linear algebra kernels. Key applications of LAPIS include sparse linear algebra and graph kernels, TenSQL, a relational database management solution built on GraphBLAS, and the development of subgraph isomorphism and monomorphism kernels, showcasing performance portability. In summary, the LAPIS framework supports productivity, performance, portability, and distributed memory execution, while also enabling linear algebra-level optimizations that are challenging in traditional programming languages, with successful applications ranging from simple sparse linear algebra to complex graph kernels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

AdvEP

AdvEP is a code repository which contains PyTorch implementations of various adversarial attacks on a deep neural network trained with Equilibrium Propagation (EP), which is a neuromorphic learning framework. AdvEP allows for the training, testing, and conducting white/black-box attacks of EP models on a wide variety of applications and datasets. AdvEP is based on the open-source code https://github.com/Laborieux-Axel/Equilibrium-Propagation which was developed to train energy models. AdvEP was created by modifying the original code to perform and test against adversarial attacks. AdvEP was developed in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning. AdvEP interfaces heavily with the open-source PyTorch Python package as well as the open-source Adversarial Robustness Toolbox (ART) package.

Mansingh, Siddarth↗
DOE OSTI2024

Towards Unlocking Insights from Logbooks Using AI

Electronic logbooks contain valuable information about activities and events concerning their associated particle accelerator facilities. However, the highly technical nature of logbook entries can hinder their usability and automation. As natural language processing (NLP) continues advancing, it offers opportunities to address various challenges that logbooks present. This work explores jointly testing a tailored Retrieval Augmented Generation (RAG) model for enhancing the usability of particle accelerator logbooks at institutes like DESY, BESSY, Fermilab, BNL, SLAC, LBNL and CERN. The RAG model uses a corpus built on logbook contributions and aims to unlock insights from these logbooks by leveraging retrieval over facility datasets, including discussion about potential multimodal sources. Our goals are to increase the FAIR-ness (findability, accessibility, interoperability, and reusability) of logbooks by exploiting their information content to streamline everyday use, to enable macro-analysis for root cause analysis, and to facilitate problem-solving automation.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Towards Unlocking Insights from Logbooks Using AI

Electronic logbooks contain valuable information about activities and events concerning their associated particle accelerator facilities. However, the highly technical nature of logbook entries can hinder their usability and automation. As natural language processing (NLP) continues advancing, it offers opportunities to address various challenges that logbooks present. This work explores jointly testing a tailored Retrieval Augmented Generation (RAG) model for enhancing the usability of particle accelerator logbooks at institutes like DESY, BESSY, Fermilab, BNL, SLAC, LBNL, and CERN. The RAG model uses a corpus built on logbook contributions and aims to unlock insights from these logbooks by leveraging retrieval over facility datasets, including discussion about potential multimodal sources. Our goals are to increase the FAIR-ness (findability, accessibility, interoperability, and reusability) of logbooks by exploiting their information content to streamline everyday use, enable macro-analysis for root cause analysis, and facilitate problem-solving automation.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Interactive Corpus Analysis Tool

The Interactive Corpus Analysis Tool (ICAT) is an interactive machine learning dashboard for unlabeled text/natural language processing datasets that allows a user to iteratively and visually define features, explore and label instances of their dataset, and simultaneously train a logistic regression model. ICAT was created to allow subject matter experts in a specific domain to directly train their own models for unlabeled datasets visually, without needing to be a machine learning expert or needing to know how to code the models themselves. This approach allows users to directly leverage the power of machine learning, but critically, also involves the user in the development of the machine learning model.

Martindale, Nathan [Oak Ridge National Lab. (ORNL)↗
DOE OSTI2025

Arctic shrub expansion generates regional variation in litter decomposition by altering litter quality and the decomposition environment

Abstract The expansion of deciduous shrubs into the graminoid‐dominated arctic tundra is expected to alter litter decomposition by changing litter quality and local abiotic and biotic conditions. However, it remains unclear how shrub expansion will affect litter decomposition at regional scales, where macroclimate is expected to be the dominant regulator of decomposition. To determine the relative influence of macroclimate and local controls on regional patterns of litter mass loss and nitrogen release, we conducted two hierarchical litter decomposition experiments across spatial scales. We decomposed leaf and root litter from a prominent graminoid ( Eriophorum vaginatum ) and three genera of deciduous shrubs ( Betula , Alnus and Salix ) for 1 year within replicated plots at five sites spanning a 160 km latitudinal gradient in northern Alaska. Using Eriophorum litter as a substrate, we found that macroclimate was the primary regulator of mass loss but had opposing effects on leaf and root litter. As summer temperature increased along the latitudinal gradient (11.9 to 13.9°C), leaf litter mass loss increased by 20% whereas root litter mass loss decreased by 33%. Leaf nitrogen release also increased with summer temperature. Conversely, root nitrogen release was controlled by the vegetation type of the decomposition environment. Using different shrub litters as substrates, we found that litter quality and its interaction with soil microclimate and macroclimate controlled decomposition. Overall, shrub root litters decomposed faster than Eriophorum root litter, losing 53% more mass and 190% more nitrogen across all sites and decomposition environments. For leaf litter, however, patterns varied by litter genus, with Salix losing more mass and Betula and Alnus losing less mass than Eriophorum . Our findings demonstrate that shrub expansion in the Arctic can regulate leaf and root litter decomposition at the regional scale through its effects on local controls, primarily litter quality. Ongoing increases in shrub cover are likely to accelerate the turnover of root litter carbon and nitrogen pools in tundra ecosystems. Therefore, including shrub‐related processes in Earth system models will improve our ability to predict regional‐scale litter decomposition and its effects on carbon and nutrient cycling in a warming Arctic. Read the free Plain Language Summary for this article on the Journal blog.

Vozzo, Justin T. [Department of Natural Resources ↗
DOE OSTI2022

SplitML

SplitML (Signal Processing Library for Interference rejecTion by Machine Learning) is a code repository for a set of tools for interference rejection in complex time-domain signals. The goal of the tools is to provide machine learning modeling capabilities for rejecting interference. Recent related machine learning algorithms for signal processing have focused mostly on speech enhancement or multi-speaker speech separation; the tools in SplitML will extend these innovations to generic time-domain signals of interest. SplitML includes tools for generating synthetic noisy signals, code for customized machine learning models applicable to interference rejection, and tools for evaluating such algorithms against standard signal processing techniques. These components are written in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning and signal processing.

Klein, Natalie↗
DOE OSTI2021

Feature Extraction: Improving Remote Sensor Classification of Non-Proliferation

This research focuses on developing algorithms for nuclear non-proliferation detection using remote sensor modeling. To improve the performance of classification models, we implemented a data pipeline with feature extraction. This pipeline takes raw data and transforms it into smaller data points called features that still describe the model. Improving this classification works towards the departments of energy’s missions of ensuring American’s security and prosperity by creating technology that addresses nuclear challenges. To conduct this analysis, we used the Python programming language and some key packages, including tsfresh and TSFEL. Originally tsfresh was selected because it has the most statistical features out of all the packages. Later TSFEL was incorporated due to the additional features it can extract from data, such as temporal and spectral. However, feature extraction becomes challenging in the presence of missing values. In this case, two additional Python packages were added to our workflow, NumPy and pandas, allowing for the feature extraction process to handle unknown values. Our data pipeline was tested on data collected from a simulation that describes the process state of a physical example. The results show the pipeline’s capability to consume and extract a total 17 features from tabular data. Future work includes producing classifications using decision tree-based models such as XGBoost and improving data collection by analyzing feature importance.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

From Machine Learning to Machine Reasoning: A Model-based Approach to Analyze Equipment Reliability Data

In current nuclear power plants (NPPs) a large amount of condition-based data which can be used to assess and monitor component health and performance. Assessing component health from such data can be performed with a large variety of methods. While the analysis of numeric data can be performed with several methods, the extraction of information from textual data remains a challenge. Currently employed natural language processing (NLP) methods do not really provide quantitative information that might be contained in IRs. In addition, the integration of numeric and textual data to identify possible causal relationships between data elements is still an unresolved challenge. This paper presents an approach to extract information from textual (e.g., incident or maintenance reports) and numeric data that relies on model based system engineer (MBSE) models. MBSE are diagrams designed to represent system and component dependencies (from both a form and functional point of view). In our approach, MBSE models emulate system engineer knowledge about component/system architecture. NLP methods are employed to perform syntactic and semantic analyses. Syntactic analysis analyzes the grammatical structure of a sentence while semantic analysis is designed to analyze the logic structure of a sentence. An innovative element of our approach is that semantic analysis uses MBSE models to identify links between textual elements. Similarly, numeric data is directly linked to elements of the MBSE models in order to map which functions are being monitored.

97 - MATHEMATICS AND COMPUTING↗