Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Catalyst: Fast and flexible modeling of reaction networks

We introduce Catalyst.jl, a flexible and feature-filled Julia library for modeling and high-performance simulation of chemical reaction networks (CRNs). Catalyst supports simulating stochastic chemical kinetics (jump process), chemical Langevin equation (stochastic differential equation), and reaction rate equation (ordinary differential equation) representations for CRNs. Through comprehensive benchmarks, we demonstrate that Catalyst simulation runtimes are often one to two orders of magnitude faster than other popular tools. More broadly, Catalyst acts as both a domain-specific language and an intermediate representation for symbolically encoding CRN models as Julia-native objects. This enables a pipeline of symbolically specifying, analyzing, and modifying CRNs; converting Catalyst models to symbolic representations of concrete mathematical models; and generating compiled code for numerical solvers. Leveraging ModelingToolkit.jl and Symbolics.jl, Catalyst models can be analyzed, simplified, and compiled into optimized representations for use in numerical solvers. Finally, we demonstrate Catalyst’s broad extensibility and composability by highlighting how it can compose with a variety of Julia libraries, and how existing open-source biological modeling projects have extended its intermediate representation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Numerical simulation projects in micromagnetics with Jupyter

We report a case study where an existing materials science course was modified to include numerical simulation projects on the micromagnetic behavior of materials. The Ubermag micromagnetic simulation software package is used in order to solve problems computationally. The simulation software is controlled through the Python code in Jupyter notebooks. Our experience is that the self-paced problem-solving nature of the project work can facilitate a better in-depth exploration of the course contents. We discuss which aspects of the Ubermag and the project Jupyter ecosystem have been beneficial for the students' learning experience and which could be transferred to similar teaching activities in other subject areas.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Application Experiences on a GPU-Accelerated Arm-based HPC Testbed

This paper assesses and reports the experience of ten teams working to port, validate, and benchmark several High Performance Computing applications on a novel GPU-accelerated Arm testbed system. The testbed consists of eight NVIDIA Arm HPC Developer Kit systems, each one equipped with a server-class Arm CPU from Ampere Computing and two data center GPUs from NVIDIA Corp. The systems are connected together using InfiniBand interconnect. The selected applications and mini-apps are written using several programming languages and use multiple accelerator-based programming models for GPUs such as CUDA, OpenACC, and OpenMP offloading. Working on application porting requires a robust and easy-to-access programming environment, including a variety of compilers and optimized scientific libraries. The goal of this work is to evaluate platform readiness and assess the effort required from developers to deploy well-established scientific workloads on current and future generation Arm-based GPU-accelerated HPC systems. The reported case studies demonstrate that the current level of maturity and diversity of software and tools is already adequate for large-scale production deployments.

Elwasif, Wael↗
DOE OSTI2023

A2E2G (Atmosphere to Electrons to the Grid platform) [SWR-23-22]

A2E2G is a platform that integrates 1) forecasting tools to account for weather uncertainty, with 2) aerodynamic wind plant models to account for wake dynamics and wind plant operation, and 3) economic models to advise on operation for a wind power plant that offers grid services in addition to energy. The A2E2G platform can be used as a high-level controller for a wind plant for market participation and real-time wind plant control. The A2E2g platform is a holistic Python tool with modules that can be run to 1) advise on market participation and 2) control and operate a wind power plant in real time. The A2E2g framework assumes two stages: the first stage is in day-ahead and the second stage is in real-time. Managing uncertainty is key in the first stage and managing variability is key in the second stage. The different components have models written and developed in the Python programming language. The code is assembled into a Python package and can be easily downloaded and installed from the A2E2g repository (https://github.com/NREL/a2e2g).

Sinner, Michael↗
DOE OSTI2024

Summer 2024 INL Intern Poster Session Submission - Brian Schumitz

This LRS submission is my poster for the INL Intern Poster Session, Summer 2024. Abstract: The Software Engineering and Cybersecurity Lab (SECL) at Montana State University has developed PIQUE, a system for evaluating software quality. PIQUE's adaptability allows for language-specific static-analysis operations, including a model for assessing cloud microservice ecosystems. These ecosystems often rely on Docker for efficient deployment and management of containerized services. Our research focuses on evaluating the network quality within these microservice ecosystems. To automate this process, we're utilizing Snort, an open-source intrusion detection system renowned for its ability to detect and log network traffic. By leveraging Snort's customizable rules, we aim to construct comprehensive testing methods for measuring and quantifying the network quality based on traffic between Docker containers. This research aims to enhance the overall security and reliability of cloud microservice ecosystems by providing automated and robust quality evaluation mechanisms, ultimately contributing to the advancement of software engineering practices in these environments

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Machine Learning and Economic Models to Enable Risk-Informed Condition Based Maintenance of a Nuclear Plant Asset

The primary objective of this research is to address challenges in the implementation of risk-informed, condition-based predictive maintenance (PdM), which reduces operating costs while still maintaining the safety and reliability of commercial nuclear power plants (NPPs). To achieve the objective, risk models are being developed by taking advantage of advancements in data analytics, deep learning, machine learning (ML), and artificial intelligence (AI). The notable outcomes presented in the report include ? Development of a ML models using heterogeneous plant process and vibration data collected at different spatial and temporal resolutions from the Salem?s CWS to diagnose a circulating water pump (CWP) failure based on salient fault signatures. The developed diagnostic models are extendable to other faults associated with CWPs and CWP motors given associated fault signatures. ? Development of a natural language processing (NLP) technique to automatically classify the WO data into different categories. The developed NLP technique was validated on independent WO data. This automates the tedious and time-consuming activity of mining and classifying WOs by subject matter experts. ? Estimation of mean time between downtime (i.e., time duration between time instances when 1 or more CWPs are not available) and developed an approach to establish reliability of CWS components using unstructured WO data along with CWS plant process data. ? Formulation of economic model based on Markov chain models. The parameters of associated with the transition rate between different states of Markov chain models were estimated using WO data. The economic model formulation and discussion captures both time-independent and time-dependent parameter variation, leading to risk-informed decision-making.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Fidelity-preserving enhancement of ptychography with foundational text-to-image models

Ptychographic phase retrieval enables high-resolution imaging of complex samples but often suffers from artifacts such as grid pathology and multislice crosstalk, which degrade reconstructed images. We propose a plug-and-play (PnP) framework that integrates physics model-based phase retrieval with text-guided image editing using foundational diffusion models. By employing the alternating direction method of multipliers, our approach ensures consensus between data fidelity and artifact removal subproblems, maintaining physical consistency while enhancing image quality. Artifact removal is achieved using a text-guided diffusion image editing method (LEDITS++) with a pre-trained foundational diffusion model, allowing users to specify artifacts for removal in natural language. Demonstrations on simulated and experimental datasets show significant improvements in artifact suppression and structural fidelity, validated by metrics such as peak signal-to-noise ratio and diffraction pattern consistency. This work highlights the combination of text-guided generative models and model-based phase retrieval algorithms as a transferable and fidelity-preserving method for high-quality diffraction imaging.

image editing↗
DOE OSTI2023

A rule-free workflow for the automated generation of databases from scientific literature

Abstract In recent times, transformer networks have achieved state-of-the-art performance in a wide range of natural language processing tasks. Here we present a workflow based on the fine-tuning of BERT models for different downstream tasks, which results in the automated extraction of structured information from unstructured natural language in scientific literature. Contrary to existing methods for the automated extraction of structured compound-property relations from similar sources, our workflow does not rely on the definition of intricate grammar rules. Hence, it can be adapted to a new task without requiring extensive implementation efforts and knowledge. We test our data-extraction workflow by automatically generating a database for Curie temperatures and one for band gaps. These are then compared with manually curated datasets and with those obtained with a state-of-the-art rule-based method. Furthermore, in order to showcase the practical utility of the automatically extracted data in a material-design workflow, we employ them to construct machine-learning models to predict Curie temperatures and band gaps. In general, we find that, although more noisy, automatically extracted datasets can grow fast in volume and that such volume partially compensates for the inaccuracy in downstream tasks.

36 MATERIALS SCIENCE↗
DOE OSTI2021

A pre-training and self-training approach for biomedical named entity recognition

Named entity recognition (NER) is a key component of many scientific literature mining tasks, such as information retrieval, information extraction, and question answering; however, many modern approaches require large amounts of labeled training data in order to be effective. This severely limits the effectiveness of NER models in applications where expert annotations are difficult and expensive to obtain. In this work, we explore the effectiveness of transfer learning and semi-supervised self-training to improve the performance of NER models in biomedical settings with very limited labeled data (250-2000 labeled samples). We first pre-train a BiLSTM-CRF and a BERT model on a very large general biomedical NER corpus such as MedMentions or Semantic Medline, and then we fine-tune the model on a more specific target NER task that has very limited training data; finally, we apply semi-supervised self-training using unlabeled data to further boost model performance. We show that in NER tasks that focus on common biomedical entity types such as those in the Unified Medical Language System (UMLS), combining transfer learning with self-training enables a NER model such as a BiLSTM-CRF or BERT to obtain similar performance with the same model trained on 3x-8x the amount of labeled data. We further show that our approach can also boost performance in a low-resource application where entities types are more rare and not specifically covered in UMLS.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Thermodynamic modeling with uncertainty quantification using the modified quasichemical model in quadruplet approximation: Implementation into PyCalphad and ESPEI

The modified quasichemical model in the quadruplet approximation (MQMQA) considers the first- and the second-nearest-neighbor coordination and interactions, particularly useful in describing short-range ordering (SRO) in complex liquids such as molten salts, slag in metal processing, and electrolytic solutions. Here, the present work implements the MQMQA into the Python based open-source software PyCalphad for thermodynamic calculations. This endeavor facilitates the development of MQMQA-based thermodynamic database with uncertainty quantification (UQ) and propagation (UP) using the open-source software ESPEI. A new database structure based on Extensible Markup Language (XML) is proposed for ESPEI evaluation of MQMQA model parameters. Using the KF-NiF 2 , KCl-NaCl-MgCl 2 , and CaCl 2 -CaF 2 -LiCl-LiF salt systems as examples, we demonstrate the successful implementation of MQMQA in PyCalphad through thermodynamic calculations of Gibbs energy, equilibrium quadruplet fractions, and phase diagram, as well as database development with UQ and UP using ESPEI. Furthermore, as an application of the present implementation, both the LiF–TbF 3 and LiF-HoF 3 systems have been modeled by MQMQA for the first time, which are in good agreement with experiments. The present implementation hence offers an open-source capability for performing CALPHAD modeling for complex liquids with SRO using MQMQA plus a new XML database structure.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Foundation Models of Scientific Knowledge for Chemistry: Opportunities, Challenges and Lessons Learned

Foundation models pre-trained on large corpora demonstrate significant gains across many natural language processing tasks and domains e.g., law, healthcare, education, etc. However, only limited efforts have investigated the opportunities and limitations of applying these powerful models to science and security applications. In this work we develop foundation models of scientific knowledge for chemistry to augment scientists with the advanced ability to perceive and reason at scale previously unimagined. Specifically, we build large-scale (1.47B parameter) general-purpose models for chemistry that can be effectively used to perform a wide range of in-domain and out-of-domain tasks. Evaluating these models in a zero-shot setting, we analyze the effect of model and data scaling, knowledge depth, and temporality on model performance in context of model training efficiency. Our novel findings demonstrate that (1) model size significantly contributes to the task performance when evaluated in a zero-shot setting; (2) data quality (aka diversity) affects model performance more than data quantity; (3) similarly, unlike previous work (Luu et al., 2021) temporal order of the documents in the corpus boosts model performance only for specific tasks, e.g., SciQ; and (4) models pre-trained from scratch perform better on in-domain tasks than those tuned from general-purpose models like Open AI’s GPT-2.

Foundation Models, Chemistry↗
DOE OSTI2024

Optimal Electrification Using Renewable Energies: Microgrid Installation Model with Combined Mixture k-Means Clustering Algorithm, Mixed Integer Linear Programming, and Onsset Method

Optimal planning and design of microgrids are priorities in the electrification of off-grid areas. Indeed, in one of the Sustainable Development Goals (SDG 7), the UN recommends universal access to electricity for all at the lowest cost. Several optimization methods with different strategies have been proposed in the literature as ways to achieve this goal. This paper proposes a microgrid installation and planning model based on a combination of several techniques. The programming language Python 3.10 was used in conjunction with machine learning techniques such as unsupervised learning based on K-means clustering and deterministic optimization methods based on mixed linear programming. These methods were complemented by the open-source spatial method for optimal electrification planning: onsset. Four levels of study were carried out. The first level consisted of simulating the model obtained with a cluster, which is considered based on the elbow and k-means clustering method as a case study. The second level involved sizing the microgrid with a capacity of 40 kW and optimizing all the resources available on site. The example of the different resources in the Togo case was considered. At the third level, the work consisted of proposing an optimal connection model for the microgrid based on voltage stability constraints and considering, above all, the capacity limit of the source substation. Finally, the fourth level involved a planning study of electrification strategies based mainly on microgrids according to the study scenario. The results of the first level of study enabled us to obtain an optimal location for the centroid of the cluster under consideration, according to the different load positions of this cluster. Then, the results of the second level of study were used to highlight the optimal resources obtained and proposed by the optimization model formulated based on the various technology costs, such as investment, maintenance, and operating costs, which were based on the technical limits of the various technologies. In these results, solar systems account for 80% of the maximum load considered, compared to 7.5% for wind systems and 12.5% for battery systems. Next, an optimal microgrid connection model was proposed based on the constraints of a voltage stability limit estimated to be 10% of the maximum voltage drop. The results obtained for the third level of study enabled us to present selective results for load nodes in relation to the source station node. Finally, the last results made it possible to plan electrification using different network technologies and systems in the short and long term. The case study of Togo was taken into account. The various results obtained from the different techniques provide the necessary leads for a feasibility study for optimal electrification of off-grid areas using microgrid systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Impact of heating and cooling loads on battery energy storage system sizing in extreme cold climates

Efficient operation of battery energy storage systems requires that battery temperature remains within a specific range. Current techno-economic models neglect the parasitic loads heating and cooling operations have on these devices, assuming they operate at constant temperature. In this work, these effects are investigated considering the optimal sizing of battery energy storage systems when deployed in cold environments. Here, a peak shaving application is presented as a linear programming problem which is then formulated in the PYOMO optimization programming language. The building energy simulation software EnergyPlus is used to model the heating, ventilation, and air conditioning load of the battery energy storage system enclosure. Case studies are conducted for eight locations in the United States considering a nickel manganese cobalt oxide lithium ion battery type and whether the power conversion system is inside or outside the enclosure. The results show an increase of 42% to 300% in energy capacity size, 43% to 217% in power rating, and 43% to 296% increase in capital cost dependent on location. This analysis shows that the heating, ventilation, and air conditioning load can have a large impact on the optimal sizes and cost of a battery energy storage system and merit consideration in techno-economic studies.

25 ENERGY STORAGE↗
DOE OSTI2023

Equation-based and data-driven modeling: Open-source software current state and future directions

Here, a review of current trends in scientific computing reveals a broad shift to open-source and higher-level programming languages such as Python and growing career opportunities over the next decade. Open-source modeling tools accelerate innovation in equation-based and data-driven applications. Significant resources have been deployed to develop data-driven tools (PyTorch, TensorFlow, Scikit-learn) from tech companies that rely on machine learning services to meet business needs while keeping the foundational tools open. Open-source equation-based tools such as Pyomo, CasADi, Gekko, and JuMP are also gaining momentum according to user community and development pace metrics. Integration of data-driven and principles-based tools is emerging. New compute hardware, productivity software, and training resources have the potential to radically accelerate progress. However, long-term support mechanisms are still necessary to sustain the momentum and maintenance of critical foundational packages.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

BLAST-Lite (Battery Lifetime Analysis and Simulation Tool - Lite) [SWR-22-69] Related to: BLAST aka: BLAST-Py

Battery Lifetime Analysis and Simulation Toolsuite (BLAST) provides a library of battery lifetime and degradation models for various commercial lithium-ion batteries from recent years. Degradation models are identified from publicly available lab-based aging data using NREL's battery life model identification toolkit. The battery life models predicted the expected lifetime of batteries used in mobile or stationary applications as functions of their temperature and use (state-of-charge, depth-of-discharge, and charge/discharge rates). Model implementation is in both Python and MATLAB programming languages. The MATLAB code also provides example applications (stationary storage and EV), climate data, and simple thermal management options. For more information on battery health diagnostics, prediction, and optimization, see NREL's Battery Lifespan webpage.

Smith, Kandler↗
DOE OSTI2020

Understanding and Explicitly Measuring Linguistic and Stylistic Properties of Deception via Generation and Translation

Massive digital disinformation is one of the main risks of modern society. Hundreds of models and linguistic analyses have been done to compare and contrast misleading and credible content online. However, most models do not remove the confounding factor of a topic or narrative when training, so the resulting models learn a clear topical separation for misleading versus credible content. We study the feasibility of using two strategies to disentangle the topic bias from the models to understand and explicitly measure linguistic and stylistic properties of content from misleading versus credible content. First, we develop conditional generative models to create news content that is characteristic of different credibility levels. We perform multi-dimensional evaluation of model performance on mimicking both the style and linguistic differences that distinguish news of different credibility using machine translation metrics and classification models. We show that even though generative models are able to imitate both the style and language of the original content, additional conditioning on both the news category and the topic leads to reduced performance. In a second approach, we perform deception style ``transfer" by translating deceptive content into the style of credible content and vice versa. Extending earlier studies, we demonstrate that, when conditioned on a topic, deceptive content is shorter, less readable, more biased, and more subjective than credible content, and transferring the style from deceptive to credible content is more challenging than the opposite direction.

Saldanha, Emily G.↗
DOE OSTI2020

Development of neural network force fields for corrosion studies

To fully understand the chemistry and physics of corrosion, novel methods of simulation must be developed. One approach is designing machine learning (ML) algorithms integrated with density functional theory to develop adaptive force fields to gain insight into corrosion behavior namely at the surface of metal oxides. Current methods of modeling corrosion are slow due to the computational cost of resolving both reaction mechanics and mass transport processes. Machine learning methods can be implemented to obtain structure-activity relationships at both the molecular and bulk scale while still retaining the accuracy of density functional theory (DFT) and significantly decreasing the time needed for simulations of complex chemical processes in the various environments of corrosion. Multiscale models are needed for corrosion studies to fully understand its processes not only at the atomic length scale (chemical bonding, energies, and forces), but also at the nano and meso length scales (solid-state physics and material science processes). Current methods of study include DFT, molecular dynamics, and Monte Carlo. The limitation of DFT is that only a small number of atoms or molecules can be simulated at that level of theory. Density functional theory is used to study the electronic structure of atoms and molecules, and calculate the force component of each atom. However, these calculations are limited to about 1000 atoms. Custom periodic boundary conditions (PBC) can be used to describe the various environments and defects that affect the atomic forces to produce a large data set from which a training set can be derived. Machine learning can be utilized to overcome the barrier of modeling macroscopic and multi-scale processes from ab initio calculations through the development of adaptive force fields. Local environments determine the atomic forces of a given system, therefore adaptive force fields must be created to produce reliable quantum mechanical calculations. This can be achieved by developing a learning algorithm that uses the mapped atomic forces or fingerprint as an input to produce energies and magnetic moments as output. A systematic approach was used to begin to build a data set in order to accurately describe the atomic forces in various environments. In Figure 4 below, a simple PBC cell of Fe{sub 2}O{sub 3} was first optimized. A surface optimization was performed next, followed by a hydroxylated surface optimization. Once this calculation has converged, the adsorption of halide species to the hydroxylated surface will be investigated. TensorFlow is an open source platform for machine learning developed by Google. Using a high level application program interface (API) such as Keras allows for building and training ML models easily in a number of different environments and languages. For this project, a neural network was developed within Anaconda in Python. Future Work: Further development of reference data set; Refining neural network and learning algorithm; Fingerprinting atomic environment to enable mapping of atomic force components; Choosing appropriate training set from reference data; Learning from training set and enabling non-linear mapping of training set fingerprints and the atomic forces; Estimation of uncertainty to identify ranges of outside applicability; Testing and analysis of molecular dynamic simulations.

36 MATERIALS SCIENCE↗