Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “language models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Interpreting and Accelerating Transformers for Jet Tagging

Attention-based transformers are ubiquitous in machine learning applications from natural language processing to computer vision. In high energy physics, one central application is to classify collimated particle showers in colliders based on the particle of origin, known as jet tagging. In this work, we study the interpretatbility and prospects for acceleration of Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging performance. We analyzing ParT's attention maps and particle-pair correlations in the eta-phi plane, revealing intriguing features, such as a binary attention pattern that identifies critical substructure in jets. These insights enhance our understanding of the model's internal workings and learning process and hint at ways to improve its efficiency. Along these lines, we also explore low-rank attention, attention alternatives, and dynamic quantization to accelerate transformers for jet tagging. With quantization, we achieve a 50% reduction in model size and a 10% increase in inference speed without compromising accuracy. These combined efforts enhance both the performance and the interpretability of transformers in high-energy physics, opening avenues for more efficient and physics-driven model designs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Leaf area index (LAI), Teller site, Seward Peninsula, Alaska, 2022

Leaf area index (LAI) sampled from locations on the Teller MM27 NGEE-Arctic site, Seward Peninsula, Alaska. These data were collected in support of the ongoing NGEE-Arctic and NASA ABoVE data synthesis work. Samples were collected in July 2022 at 100 locations that cover the down slope half of Teller MM27 hillslope. This data package includes sample information and LAI data (.csv), an ESRI shape file (.shp) and Keyhole Markup Language (.kml) that define the location and area of each LAI measurement. Metadata files include data descriptions (_dd.csv) for tabular data, file level metadata (.csv), and methods and the instrument manual (.pdf). The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Advanced Manufacturing Basic Entity Relationships Ontology

A data ontology defined using the W3C Web Ontology Language (OWL) format. It defines classes and attributes for objects directly implicated in manufacturing such as materials, preform geometries, and manufacturing processes and settings. It also includes classes to describe entities that are instrumental to making digital twins and performing predictive activities on manufacturing data such as designs of experiment and predictive models.

Harris, BrennanKay↗
DOE OSTI2025

Complex Dependence of Calcite Crack Kinetics on Salinity: The Role of DLVO and Hydration Forces

Abstract Subcritical crack growth (SCG) plays an important role in many geological processes such as delayed earth rupture and rock weathering. The complex dependency of SCG on the in‐crack fluid chemistry, however, is still poorly understood. In this study, we utilize the newly developed surface force‐based fracture theory (SFFT) to elucidate the relative contributions of surface forces and solute transport to the crack growth kinetics of calcite in NaCl solutions. Expanding on Barenblatt's cohesive crack model, SFFT introduces an effective stress intensity at the crack tip that encompasses all the relevant intermolecular forces across the crack in addition to the external far‐field stresses. The nonlinear system of equations portraying the crack opening profile, the solute distribution in a propagating crack, and the crack growth velocity are numerically solved via an implicit scheme. After carefully calibrating the model for calcite‐water systems, the SFFT is used to predict the SCG response of calcite at different NaCl concentrations, based on various hypotheses. These predictions are then compared to existing SCG data from the literature. We demonstrate that the experimentally observed variation of SCG rate with NaCl concentration cannot be explained solely by DLVO forces (electrostatic and Van der Waals interactions). This can be remediated by introducing an exponentially decaying hydration force with a nonlinear, nonmonotonic dependence on NaCl concentration. Furthermore, we demonstrate that accounting for both diffusive and advective transport of ions is important in explaining the absence of a stage‐II SCG response for calcite in electrolyte solutions. Plain Language Summary Subcritical crack growth (SCG) refers to the slow propagation of cracks in materials under a stress below the threshold for catastrophic failure. SCG is a key process in many geological events, for example, delayed earth ruptures and rock weathering. New initiatives such as underground CO 2 and H 2 storage in carbonate reservoirs further call for better understanding of SCG in carbonate minerals subjected to varying fluid chemistry. This study examines the SCG of calcite, a key mineral found in carbonate rocks, intergranular cement in sandstones, and filling material in mineral veins and faults, determining their deformation and strength. A mathematical model is developed to describe how the crack opens and propagates, how solutes (like salts) distribute within the crack, and how the crack surfaces interact with each other. We used the model to predict calcite SCG in water at different salt concentrations and compared it with experimental data. Our results revealed that the hydration force is the dominating factor in determining the complex, non‐linear dependency of SCG on salinity. We also found that both the movement of ions by diffusion and by bulk water flow are crucial for explaining the SCG rates, especially when the cracks grow quickly. Key Points Surface Force‐Based Fracture Theory predicts the complex subcritical crack growth patterns of calcite crystals immersed in NaCl solutions Results highlight the dominant role of hydration forces in altering the fracture behavior of calcite compared to VdW and electric double‐layer forces Advective solute transport explains the absence of stages‐II and ‐III subcritical crack growth responses in solid‐liquid systems

DLVO↗
DOE OSTI2021

Building Toward the Future in Chemical and Materials Simulation with Accessible and Intelligently Designed Web Applications

Over the last few decades, significant progress has been made in the development and use of electronic structure and other molecular simulation methods. As these methods become more mature and are able to simulate larger and more complex chemical simulations, the need for improvement in scientific visualization, molecular builders, simplified input to simulation methods, and the development of new approaches and languages to describe simulations, along with workflows to carry them out, becomes more apparent. In this chapter, we describe our recent efforts in developing a prototype open-source computational tool called Arrows that combines NWChem, SQL and NoSQL databases, email, web APIs, and web applications in a way that make molecular and materials modeling accessible to all scientists and engineers. At the same time, because of its simplified input, it provides a framework for expert users to carry out large numbers of calculations and run complex workflows.

2↗
DOE OSTI2023

Modeling and Rapid Prototyping of Integrated Transmission-Distribution OPF Formulations with PowerModelsITD.jl

Conventional electric power systems are composed of different unidirectional power flow stages of generation, transmission, and distribution, managed independently by transmission system and distribution system operators. However, as distribution systems increase in complexity due to the integration of distributed energy resources, coordination between transmission and distribution networks will be imperative for the optimal operation of the power grid. However, coupling models and formulations between transmission and distribution is non-trivial, in particular due to the common practice of modeling transmission systems as single-phase, and distribution systems as multi-conductor phase-unbalanced. To enable the rapid prototyping of power flow formulations, in particular in the modeling of the boundary conditions between these two seemingly incompatible data models, we introduce PowerModelsITD.jl, a free, open-source toolkit written in Julia for integrated transmission-distribution (ITD) optimization that leverages mature optimization libraries from the InfrastructureModels.jl-ecosystem. The primary objective of the proposed framework is to provide baseline implementations of steady-state ITD optimization problems, while providing a common platform for the evaluation of emerging formulations and optimization problems. In this work, we introduce the nonlinear formulations currently supported in PowerModelsITD.jl, which include AC-polar, AC-rectangular, current-voltage, and a linear network transportation model. Results are validated using combinations of IEEE transmission and distribution networks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Fiats: Functional inference and training for surrogates

Fiats provides a platform for research on the training and deployment of neural-network surrogate models for computational science. Fiats also supports exploring, advancing, and combining functional, object-oriented, and parallel programming patterns in Fortran 2023. As such, the Fiats name has dual expansions: “Functional Inference And Training for Surrogates” or “Fortran Inference And Training for Science.” Fiats inference and training procedures are pure and therefore satisfy a language constraint imposed on procedure invocations inside Fortran’s parallel loop construct: do concurrent. Furthermore, the Fiats training procedures are built around a do concurrent parallel reduction. Several compilers can automatically parallelize do concurrent on Central Processing Units (CPUs) or Graphics Processing Units (GPUs). Fiats thus aims to achieve performance portability through standard language mechanisms.

Rouson, Damian [Lawrence Berkeley National Laborat↗
DOE OSTI2025

High-Fidelity Modeling of a Type-5 Wind Turbine Gearbox

Type-5 wind turbines are characterized by their use of a hydraulic torque converter and permanent synchronous generator. This combination promotes steady and grid-ready energy without the use of a power converter. Thus, researchers were prompted to study the potential impact on grid reliability, stability, and resilience using a Real Time Digital Simulator (RTDS) model of the type-5 turbine, including a high-fidelity model of its gear box.

13 - HYDRO ENERGY↗
DOE OSTI2021

Simulating low-energy neutrino interactions with MARLEY

Monte Carlo event generators are a critical tool for the interpretation of data obtained by neutrino experiments. Several modern event generators are available which are well-suited to the GeV energy scale used in studies of accelerator neutrinos. However, theoretical modeling differences make their immediate application to lower energies difficult. In this paper, I present a new event generator, MARLEY, which is designed to better address the simulation needs of the low-energy (tens of MeV and below) neutrino community. The code is written in C++14 with an optional interface to the popular ROOT data analysis framework. The current release of MARLEY (version 1.2.0) emphasizes simulations of the reaction 40 Ar ( ν e , e − ) 40 K ⁎ but is extensible to other channels with suitable user input. This paper provides detailed documentation of MARLEY's implementation and usage, including guidance on how generated events may be analyzed and how MARLEY may be interfaced with external codes such as Geant4. Further information about MARLEY is available on the official website at http://www.marleygen.org. Program title:MARLEY 1.2.0 CPC Library link to program files:https://doi.org/10.17632/4v7zxnc8j3.1 Developer's respository link:http://github.com/MARLEY-MC/marley Code Ocean capsule:https://codeocean.com/capsule/9868179 Licensing provisions: GNU General Public License 3.0 Programming language: C++14 External routines/libraries used: GNU Scientific Library [1,2] (required), ROOT [3,4] (optional) Nature of problem: Simulation of neutrino-nucleus scattering events at energies of tens-of-MeV and below Solution method: Initial two-to-two scattering kinematics are sampled using the allowed approximation differential cross section and tables of precomputed nuclear matrix elements. Subsequent de-excitations of the remnant nucleus are simulated using a Monte Carlo implementation of the Hauser-Feshbach statistical model and tabulated γ-ray decay schemes for discrete nuclear levels. Additional comments including restrictions and unusual features: Input data are provided with the code that are suitable for producing simulations of the charged-current reaction 40 Ar ( ν e , e − ) 40 K ⁎ , coherent elastic neutrino-nucleus scattering on spin-zero target nuclei, and neutrino-electron elastic scattering on any atomic target. Preparation of new reaction input files (whose format is documented in Appendix B) would enable other reaction channels and nuclear targets to be handled by the existing code framework. Although there is no maximum neutrino energy enforced by the code itself, realistic neutrino-nucleus scattering events may be generated up to roughly 50 MeV. Above this energy, the effects of forbidden nuclear transitions, which are neglected in the current treatment of the cross sections (see section 2.1), become increasingly important.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Biclique: an R package for maximal biclique enumeration in bipartite graphs

Bipartite graphs are widely used to model relationships between pairs of heterogeneous data types. Maximal bicliques are foundational structures in such graphs, and their enumeration is an important task in systems biology, epidemiology and many other problem domains. Thus, there is a need for an efficient, general purpose, publicly available tool to enumerate maximal bicliques in bipartite graphs. The statistical programming language R is a logical choice for such a tool, but until now no R package has existed for this purpose. Our objective is to provide such a package, so that the research community can more easily perform this computationally demanding task. Biclique is an R package that takes as input a bipartite graph and produces a listing of all maximal bicliques in this graph. Input and output formats are straightforward, with examples provided both in this paper and in the package documentation. Biclique employs a state-of-the-art algorithm previously developed for basic research in functional genomics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Exploring Decarbonization and Clean Energy Pathways for Disadvantaged Communities in California

California has a state-wide goal of carbon neutrality by 2045. Decarbonization for disadvantaged communities (DACs) poses extra challenges due to financial, informational, language, and other barriers. This paper presents the methodology, results, and analysis of energy efficiency measures (EEMs) to save energy, reduce CO 2 emission, and promote clean energy access at the district scale for two DACs in Fresno, California. The methods are broadly applicable to other neighborhoods across the U.S. 22 EEMs were identified and modelled for all residential buildings in the two DACs both individually and as packages. Results show that for energy and CO 2 reduction purposes, the top performing EEM package can decrease energy use and CO 2 emissions by an average of 60%. For electrification, heat pump water heaters are a viable solution, coupled with air source or mini-split heat pumps. Replacing gasoline vehicles with electric vehicles is another important measure in electrification and reducing GHG emissions.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

MVP-CTL Fellowship for Suicide Intervention (CRADA Final Report)

Research conducted with Crisis Text Line (CTL) served as an important stepping stone in the group’s ongoing work of developing natural language processing (NLP) methodologies to discover indicators of suicide factors in the U.S. veteran population. With rising cases of suicide, detecting risk factors among veterans is of growing concern. Socio-economic factors are often under-reported along structured medical variables, and extracting information through NLP methods may improve the sensitivity of suicide prediction models. The research explored through the CTL collaboration provided insights on suicide crisis detection and de-escalation, experience in applying scalable data analysis on real world datasets, contributions to study designs for detecting rare-event and produced transferable methods applicable to the domain of veteran suicide mitigation. Tools developed during this research directly benefits ongoing work on the Million Veteran Project (MVP) collaboration with the Department of Veteran Affairs (VA).

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Accelerating Hydraulic Fracture Imaging by Deep Transfer Learning

Deep transfer learning has a great success story in computer vision (CV), natural language processing, and many other fields. In this communication, we are going to push forward the deep transfer learning to the hydraulic fracture imaging problem by proposing a two-step approach: 1) train a convolutional neural network (CNN) to reconstruct target geometries by a relatively large amount of approximated field patterns generated from a simplified model and 2) fine-tune the top layers of transferred CNN by a small amount of true field patterns generated through a full model. The advantages include the rapid generation of large amount of data through the simplified model and the high reconstruction accuracy through a careful design of the deep transfer learning. Here, the CNN trained through the deep transfer learning can accurately reconstruct the lateral extent and direction of fractures with unseen conductivity and white Gaussian noise, showing a notable acceleration/accuracy enhancement over the previous CNN trained by mixing a small number of true data into the approximated data as data augmentation.

02 PETROLEUM↗
DOE OSTI2026

TRACE Input Modernization

This work presents a Tom’s Obvious Minimal Language (TOML)-based representation of input for the US Nuclear Regulatory Commission’s TRAC/RELAP Advanced Computational Engine (TRACE) thermal hydraulics code. Implemented using the Workbench Analysis Sequence Processor (WASP), the approach maps traditional TRACE input structures to a hierarchical format composed of named parameters, typed values, and native data collections. The resulting representation preserves TRACE’s existing modeling capabilities while providing a modern, structured interface for model development and management. WASP further extends TOML through a file import directive that supports modular model composition and reusable input organization. In addition, WASP provides extended array data entry convenience with various data repeat and interpolation capabilities. Examples of the new TOML syntax are provided for major TRACE input categories, including hydraulic components, heat structures, control systems, and trip logic. The TOML representation establishes a foundation for improved validation, tooling, automation, and model maintainability while remaining compatible with existing TRACE workflows. To facilitate migration to the TOML-based input format, the TRACE executable now supports conversion of native TRACE input into an intermediate JSON representation. A Python utility subsequently transforms the JSON data into an equivalent TOML model. Lastly, the TRACE executable now supports execution using TOML-formatted input.

Lefebvre, Robert A. [Oak Ridge National Laboratory↗
DOE OSTI2020

CCAMP: An Integrated Translation and Optimization Framework for OpenACC and OpenMP

Heterogeneous computing and exploration into specialized accelerators are inevitable in current and future supercomputers. Although this diversity of devices is promising for performance, the array of architectures presents programming challenges. High-level programming strategies have emerged to face these challenges, such as the OpenMP offloading model and OpenACC. The varying levels of support for these standards, however, within vendor-specific and open-source tools, as well as the lack of performance portability across devices, have prevented the standards from achieving their goals. To address these shortcomings, we present CCAMP, an OpenMP and OpenACC interoperable framework. CCAMP provides two primary facilities: language translation between the two standards and device-specific directive optimization within each standard. We show that by using the CCAMP framework, programmers can easily transplant non-portable code into new ecosystems for new architectures. Additionally, by using CCAMP device-specific directive optimizations, users can achieve optimized performance across architectures using a single source code.

Lambert, Jacob↗
DOE OSTI2022

A Semi-Automated Approach for Curating a Glossary of Key Terms for Open-Source Data Queries

In FY20, the Savannah River National Laboratory (SRNL) was funded by the National Nuclear Security Administration’s Office of Defense Nuclear Non-Proliferation Research and Development (NA-22) to build a machine learning based modeling pipeline that could extract proliferation events of interest from open text-based data sources. As a test case, the research team targeted the identification/fusion of events and indicators that fissile core fabrication would be executed at the Savannah River Site prior to its official announcement in May of 2018. The demonstration prototype proved successful by applying natural language processing and graph theoretical techniques to identify contextual shifts in key words and phrases that acted as indicators that pit production would be carried out at the Savannah River Site up to two years prior to the official announcement.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Design patterns of biological cells

Design patterns are generalized solutions to frequently recurring problems. They were initially developed by architects and computer scientists to create a higher level of abstraction for their designs. Here, we extend these concepts to cell biology to lend a new perspective on the evolved designs of cells' underlying reaction networks. We present a catalog of 21 design patterns divided into three categories: creational patterns describe processes that build the cell, structural patterns describe the layouts of reaction networks, and behavioral patterns describe reaction network function. Applying this pattern language to the E. coli central metabolic reaction network, the yeast pheromone response signaling network, and other examples lends new insights into these systems.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Spatio-temporal multivariate cluster evolution analysis for detecting and tracking climate impacts

Recent years have seen a growing concern about climate change and its impacts. While Earth System Models (ESMs) can be invaluable tools for studying the impacts of climate change, the complex coupling processes encoded in ESMs and the large amounts of data produced by these models, together with the high internal variability of the Earth system, can obscure important source-to-impact relationships. Here, this paper presents a novel and efficient unsupervised data-driven approach for detecting statistically-significant impacts and tracing spatio-temporal source-impact pathways in the climate through a unique combination of ideas from anomaly detection, clustering and Natural Language Processing (NLP). Using as an exemplar the 1991 eruption of Mount Pinatubo in the Philippines, we demonstrate that the proposed approach is capable of detecting known post-eruption impacts/events. We additionally describe a methodology for extracting meaningful sequences of post-eruption impacts/events by using NLP to efficiently mine frequent multivariate cluster evolutions, which can be used to confirm or discover the chain of physical processes between a climate source and its impact(s).

Anomaly detection↗