Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learning rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Towards Online Machine Learning in DUNE Data Acquisition

Processing the large volumes of data produced by liquid argon time projection chamber (LArTPC) experiments presents a significant challenge, especially those at the scale of DUNE. This is a particular challenge when aiming to trigger on low-energy neutrinos from core-collapse supernovae, which are typically buried in a high-rate radiological background. To enable real-time event selection suitable for such rare signals, we are developing machine learning based data filtering methods. In order to demonstrate the feasibility of this approach, we implemented such pipeline using the ICEBERG detector at Fermilab as a small-scale LArTPC, with a focus on identifying Michel electrons as a proxy for low-energy neutrino interactions. This poster will present the current status of integrating these machine learning models into the data acquisition (DAQ) system of this detector.

Dalager, Olivia [Fermilab]↗

Learning viscoelasticity models from indirect data using deep neural networks

In this study, we propose a novel approach to model viscoelasticity materials, where rate-dependent and non-linear constitutive relationships are approximated with deep neural networks. We assume that inputs and outputs of the neural networks are not directly observable, and therefore common training techniques with input–output pairs for the neural networks are inapplicable. To that end, we develop a novel computational approach to both calibrate parametric and learn neural-network-based constitutive relations of viscoelasticity materials from indirect displacement data in the context of multiple-physics systems. We show that limited displacement data holds sufficient information to quantify the viscoelasticity behavior. We formulate the inverse computation – modeling viscoelasticity properties from observed displacement data – as a PDE-constrained optimization problem and minimize the error functional using a gradient-based optimization method. The gradients are computed by a combination of automatic differentiation and implicit function differentiation rules. The effectiveness of our method is demonstrated through numerous benchmark problems in geomechanics and porous media transport.

97 MATHEMATICS AND COMPUTING↗

Combining artificial intelligence and physics-based modeling to directly assess atomic site stabilities: from sub-nanometer clusters to extended surfaces

The performance of functional materials is dictated by chemical and structural properties of individual atomic sites. In catalysts, for instance, the thermodynamic stability of constituting atomic sites is a key descriptor from which more complex properties, such as molecular adsorption energies and reaction rates, can be derived. In this study, we present a widely applicable machine learning (ML) approach to instantaneously compute the stability of individual atomic sites in structurally and electronically complex nano-materials. Conventionally, we determine such site stabilities using computationally intensive first-principles calculations. With our approach, we predict the stability of atomic sites in sub-nanometer metal clusters of 3–55 atoms with mean absolute errors in the range of 0.11–0.14 eV. To extract physical insights from the ML model, we introduce a genetic algorithm (GA) for feature selection. This algorithm distills the key structural and chemical properties governing the stability of atomic sites in size-selected nanoparticles, allowing for physical interpretability of the models and revealing structure–property relationships. The results of the GA are generally model and materials specific. In the limit of large nanoparticles, the GA identifies features consistent with physics-based models for metal–metal interactions. By combining the ML model with the physics-based model, we predict atomic site stabilities in real time for structures ranging from sub-nanometer metal clusters (3–55 atom) to larger nanoparticles (147 to 309 atoms) to extended surfaces using a physically interpretable framework. Finally, we present a proof of principle showcasing how our approach can determine stable and active nanocatalysts across a generic materials space of structure and composition.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Structure–activity relationship-based chemical classification of highly imbalanced Tox21 datasets

Abstract The specificity of toxicant-target biomolecule interactions lends to the very imbalanced nature of many toxicity datasets, causing poor performance in Structure–Activity Relationship (SAR)-based chemical classification. Undersampling and oversampling are representative techniques for handling such an imbalance challenge. However, removing inactive chemical compound instances from the majority class using an undersampling technique can result in information loss, whereas increasing active toxicant instances in the minority class by interpolation tends to introduce artificial minority instances that often cross into the majority class space, giving rise to class overlapping and a higher false prediction rate. In this study, in order to improve the prediction accuracy of imbalanced learning, we employed SMOTEENN, a combination of Synthetic Minority Over-sampling Technique (SMOTE) and Edited Nearest Neighbor (ENN) algorithms, to oversample the minority class by creating synthetic samples, followed by cleaning the mislabeled instances. We chose the highly imbalanced Tox21 dataset, which consisted of 12 in vitro bioassays for > 10,000 chemicals that were distributed unevenly between binary classes. With Random Forest (RF) as the base classifier and bagging as the ensemble strategy, we applied four hybrid learning methods, i.e., RF without imbalance handling (RF), RF with Random Undersampling (RUS), RF with SMOTE (SMO), and RF with SMOTEENN (SMN). The performance of the four learning methods was compared using nine evaluation metrics, among which F 1 score, Matthews correlation coefficient and Brier score provided a more consistent assessment of the overall performance across the 12 datasets. The Friedman’s aligned ranks test and the subsequent Bergmann-Hommel post hoc test showed that SMN significantly outperformed the other three methods. We also found that a strong negative correlation existed between the prediction accuracy and the imbalance ratio (IR), which is defined as the number of inactive compounds divided by the number of active compounds. SMN became less effective when IR exceeded a certain threshold (e.g., > 28). The ability to separate the few active compounds from the vast amounts of inactive ones is of great importance in computational toxicology. This work demonstrates that the performance of SAR-based, imbalanced chemical toxicity classification can be significantly improved through the use of data rebalancing.

Idakwo, Gabriel↗

Advanced Algal Biofoundries for the Production of Polyurethane Precursors

The primary goal of the BEEPs project was to develop a process that could accelerate the development of algae as bioproduction platforms, from initial chemical product concept to an economically viable market supply. Under this program we elected to develop strains of algae that could generate polyurethane precursors, while simultaneously developing basic genetic tools to enable improved algal production systems. This program was specifically designed to incorporate National Laboratories as a means to utilize the expertise and facilities for new bio-production platforms. To that end, we designed a program to collaborate with the Agile BioFoundry at Lawrence Berkeley National Laboratory (LBNL) and computational platforms at Pacific Northwest National Laboratory (PNNL). In addition to these National Laboratory partners, we also had academic partners from UC Davis and Georgia Tech, as well as commercial partners Algenesis Materials and BASF. To achieve these goals, we initially focused on developing the genetic tools and high throughput screening technologies necessary to generate and assess production of polymer precursors (succinic acid) in algae and cyanobacteria, including advanced promoters and biosensors. In parallel, we computationally identified potential production bottlenecks and then used the developed genetic tools to increase production rates and yields. Constant feedback of data was used in conjunction with machine learning, high-throughput cell sorting, and synthetic biology, for additional targeted metabolic engineering. Multiple rounds of tool design, building, testing, and learning were supplied to partners at PNNL and LBNL to develop new models and tools that could expedite bioproduction platform development and increase yield performance. We targeted, and achieved, the FOA requirement yield metric of 20 g/L, as a milestone and deliverable from at least one of our engineered strains for the production of succinic acid.

09 BIOMASS FUELS↗

An Analysis of Published DAS Studies for Application to SPE Phase III

Distributed Acoustic Sensing (DAS) is an emerging technology capable of recording the acoustic wavefield at unprecedented spatial resolution. However, this new tool requires significant refinements before it becomes operational for explosion monitoring objectives. Recent studies have shown significant development of array processing with DAS data. In this report we explore three such array processing methods including DAS strain-rate data versus geophone measured ground motion, beamforming for event parameters, and machine learning based denoising. Prior to applying these algorithms to the Source Physics Experiment Phase II and Phase III data, we validate these methods through replication analysis.

47 OTHER INSTRUMENTATION↗

Million Worker Study

Radiation health issues have been an important aspect of DOE's Worker Safety and Health programs. The goal is to ensure that workers are adequately protected from the various radiological hazards associated with DOE sites and operations. Since the early 1940’s DOE has supported the conduct of epidemiologic studies of practically every DOE (AEC) facility and collected these data, now managed by the Oak Ridge Associated Universities (ORAU), in remarkable detail. In the early 2000s, the Office of Health, Safety and Security authorized access to specific DOE worker datasets. Shortly thereafter, the DOE Office of Science provided funds for a pilot study that confirmed the feasibility of the Million Worker Study (MWS), and then additional resources were provided (with other agencies) to extend the follow-up of many populations, not just DOE workers, but also atomic veterans, industrial radiographers, nuclear power plant workers and medical radiation workers (the Million Persons Study (MPS)). This proposal was a continuation of support provided by DOE, specifically to extend the follow-up of the worker populations at the Mallinckrodt Chemical Works (MCW) and Los Alamos National Laboratory (LANL). The work addresses DOE’s interest in clarifying the health risks of their workers as well as contributing knowledge on radiation risks that is relevant today with regard to compensation schemes. The findings are also important for the US public in light of the increased population exposure to medical imaging, environmental circumstances such as hydraulic fracturing, increased exposures during high altitude flights, and with regard to nuclear accidents such as Fukushima and possible terrorist events. Furthermore, it is important to consider reducing the uncertainty in current risk estimates by developing risk coefficients based on healthy American workers who are more representative of U.S. workers and the general public than 1945 Japanese survivors of the atomic bombs living in a war-torn country which experienced deprivation, malnourishment, and increased rates of infections and other diseases. But more importantly, it is important to learn whether radiation exposures received gradually over time (e.g., years) are more or less effective in causing health effects, cancer in particular, than if the radiation dose is received all at once in a fraction of a second as experienced by the Japanese atomic bomb survivors. Finally, the ability to evaluate and combine large populations with intakes of radionuclides such as uranium, radium, plutonium, americium and polonium will provide new quantitative knowledge on human health effects that hitherto has not been possible. This cost-efficient study has built on the investments made and foundations laid by investigators and government agencies, including DOE, over the past 30-40 years, which have established early worker cohorts that can now provide answers to questions on the lifetime human health risks associated with low-level radiation exposures. Collaborating institutions included: International Epidemiology Institute, Oak Ridge Associated Universities, Oak Ridge National Laboratory, Los Alamos National Laboratory, Landauer, Inc., and Vanderbilt University. Follow-up of these two populations and the integration of them with the many other cohorts (now a total of 31) in the MPS continues under a separate DOE grant (DE-AU0000046).

61 RADIATION PROTECTION AND DOSIMETRY↗

Domestic and international consensus standards for nuclear criticality safety – overview & status [Abstract]

The domestic and international consensus standards for nuclear criticality safety (NCS) were developed as the lessons-learned from process criticality accidents. These consensus standards were developed to reduce the rate of process criticality accidents in facilities that process, store, handle or transport fissionable materials by hand. After a significant increase in criticality accidents through the mid-1960’s, the rate of criticality accidents decreased due to these standards and the criticality accident rate is extremely low from an industrial safety perspective.

ANS↗

Domestic and International Consensus Standards for Nuclear Criticality Safety - Overview & Status

The domestic and international consensus standards for nuclear criticality safety (NCS) were developed based on the lessons-learned from process criticality accidents. These consensus standards were developed to reduce the rate of process criticality accidents in facilities that process, store, handle, or transport fissionable materials by hand. After a significant increase in criticality accidents through the mid-1960s, the rate of criticality accidents decreased as a result of these standards, and the criticality accident rate is extremely low from an industrial safety perspective. The last known criticality accident inside the United States was in 1978 (nearly 47 years ago) at the Idaho Chemical Processing Plant, and outside the United States, an accident occurred at Tokai-mura, Japan, in 1999 (almost 26 years ago). The domestic consensus standards for NCS include the American National Standards (ANS) that are prepared and published by the American Nuclear Society and approved by the American National Standards Institute (ANSI). The ANS Standards Board, the NCS Consensus Committee, and the ANS-8 Subcommittee oversee the development and maintenance of these standards. There are currently 18 standards in the ANS-8 series. Nine ANS-8 standards are either in revision mode or planned for revision. A new standard for the use of nondestructive assay measurements (ANSI/ANS-8.28-2024) for NCS was approved in March of 2024. The international consensus standards for NCS calculations, procedures, and practices are maintained and developed within the International Organization for Standardization, Technical Committee 85 “Nuclear Energy,” Subcommittee 5, “Nuclear Fuel Technology,” and Working Group 8, “Nuclear Criticality Safety.” Eleven standards are currently available, four standards are proposed for revision, and four standards are at various stages of development. This paper provides the NCS community with a high-level overview and status report of domestic and international NCS consensus standards to stimulate interest and to support their continued development.

Bowen, Douglas G [ORNL] (ORCID:0000000212460026)↗

Development of lean, efficient, and fast physics-framed deep-learning-based proxy models for subsurface carbon storage

In this work, we present deep-learning-based surrogate models for CCUS developed with four different algorithms and a physics-framed two-phase flow problem involving displacement of water by CO 2 . The deep-learning models were trained using 3D datasets describing the pressure plume, CO 2 saturation plume, and water extraction rate generated by numerical simulation. The hyperparameters defining the architecture of the neural networks were optimized to determine the slimmest network size and training parameters that give the most efficient performance at the least training cost. To develop a robust model that closely mimics the governing physical laws, the discretized form of the two-phase fluid transport equation was used to formulate the supervised deep-learning task. The algorithms investigated in this study predicted the data to above 95% accuracy, with the multi-layer perceptron model demonstrating the best performance by balancing training speed, prediction time, and prediction accuracy with lean network capacity. Furthermore, the surrogate models simultaneously predict reservoir pressure and CO 2 saturation in every grid block, including the surface well extraction rate and bottomhole pressure, at all simulation times for a given static model realization in just a few seconds on a standard desktop computer. A key outcome of this study is that limits can be placed on network design parameters to avoid over designing neural networks, with associated efficiencies in training and prediction times. This is very useful because large volumes of data may be generated in CCUS projects and over-design of neural network architectures imposes penalties that are antithetical to the goal of near-real time forecasting.

58 GEOSCIENCES↗

Multiprincipal Component P2-Na 0.6 (Ti 0.2 Mn 0.2 Co 0.2 Ni 0.2 Ru 0.2 )O 2 as a High-Rate Cathode for Sodium-Ion Batteries

Mixing transition metal cations in nearly equiatomic proportions in layered oxide cathode materials is a new strategy for improving the performances of Na-ion batteries. The mixing of cations not only offers entropic stabilization of the crystal structure but also benefits the diffusion of Na ions with tuned diffusion activation energy barriers. In light of this strategy, a high-rate Na 0.6 (Ti 0.2 Mn 0.2 Co 0.2 Ni 0.2 Ru 0.2 )O 2 cathode was designed, synthesized, and investigated, combining graph-based deep learning calculations and complementary experimental characterizations. This new cathode material delivers high discharge capacities of 164 mA g –1 at 0.1 C and 68 mAh g –1 at a very high rate of 86 C, demonstrating an outstanding high rate capability. Ex situ and operando synchrotron X-ray diffraction were used to reveal the detailed structural evolution of the cathode upon cycling. Using the climbing-image nudged elastic-band calculation and Ab initio molecular dynamics simulations, we show that the optimal transition metal composition enables a percolating network of low barrier pathways for fast, macroscopic Na diffusion, resulting in the observed high rate performance.

25 ENERGY STORAGE↗

Proxy quality control of biomass particles using thermogravimetric analysis and Gaussian process regression models

Abstract The temperature experienced by reactants during preparation in a reactor is a key component in determining the yield and homogeneity of usable chemical products such as biomass particles. Thermocouples with sensors can be used to monitor spatial temperature gradients within reactors but these sensors are often too expensive and/or invasive. The present work proposes a strategy to identify optimal machine learning models to infer the maximum effective temperature experienced by particles during oxidative biomass torrefaction using key thermochemical combustion parameters. The maximum rate of weight loss, the corresponding temperature, and fixed carbon content on a dry‐ash‐free basis are used as literature‐based predictor variables obtained from thermogravimetric analysis. The evaluation of 24 machine‐learning models using the standard tenfold cross‐validation method suggests that the exponential Gaussian process regression (GPR) model is the most effective, followed by other GPR models. These high‐performing GPR models were also utilized to predict the effective preparation temperature distribution of reactor‐produced biomass particles under eight conditions of varying residence time and air‐to‐biomass ratio. The effective preparation temperature and residence time of individual biomass particles were then encoded into the torrefaction severity factor and used to estimate the energy yield of the reactor output as a novel quality control method. © 2023 The Authors. Biofuels, Bioproducts and Biorefining published by Society of Industrial Chemistry and John Wiley & Sons Ltd.

09 BIOMASS FUELS↗

Prediction of Specificity of α-Conotoxins to Subtypes of Human Nicotinic Acetylcholine Receptors with Semi-supervised Machine Learning

Conotoxins are a family of highly toxic neurotoxins composed of cysteine-rich peptides produced by marine cone snails. The most lethal cone snail species to humans is Conus geographus, with fatality rates of up to ∼65% from a single sting, which is caused mostly by the activity of α-conotoxins against human nicotinic acetylcholine receptors (nAChRs). While sequence-based machine learning (ML) classifiers have been trained to identify targets of conotoxins binding voltage-gated ion channels, no ML model has been built to predict the subtype-specific nAChR targets of α-conotoxins. Here, we trained an ML model in a semi-supervised manner to predict the specificity of α-conotoxin binding toward different human nAChR subtypes to overcome the challenge of limited data in subtype-specific nAChR targets of α-conotoxins and the issue that one α-conotoxin can bind multiple nAChR subtypes with high selectivity. We considered additional features of sequences of α-conotoxins in training our ML model, including the secondary structure propensities and electrostatic properties, which resulted in better prediction capability for the ML model. Notably, we identify that most α-conotoxins bind to α3β2, α1γδ, and α7 subtypes of human nAChRs. Our findings from this study provide a framework for predicting targets of various kinds of toxins.

59 BASIC BIOLOGICAL SCIENCES↗

A Multi-Objective Bayesian Optimized Human Assessed Multi-Target Generated Spectral Recommender System for Rapid Pareto Discoveries of Material Properties

Optimization for different tasks like material characterization, synthesis, and functional properties for desired applications over multi-dimensional control parameter and function spaces need a rapid strategic search through active learning. However, in all cases prior to optimization, the target material properties are assumed known and fixed, which mostly deviates from real-world scenarios in material synthesis. This can be critical for running expensive experiments on new materials, when the experimental results are fuzzy for any scientific outcomes due to improper target setting, ultimately wasting time and cost. The failure rate and cost are even higher over exploring on multi-target space, where we want to learn the pareto among multiple properties, to jointly optimize during material synthesis for desired applications. To address the challenge, here we introduce the human-operator attempt flexibility in the active learning based automated experiment framework, with generating multiple human assessed targets through a voting-based recommender system during real-time microscope measurements over the large material image space, sequentially learn/update multiple desired targets through a weighting system, and adaptively search in multiple material properties functional space for non-dominated pareto discoveries to maximize the custom structural similarity based acquisition function. We term this a multi-objective Bayesian optimized human assessed multi-target generated spectral recommender systems (MOBO-HAM-SRS). The approach has been demonstrated to peizoresponse force spectroscopy of a ferroelectric thin film, exploring with different kernels and acquisition functions. This work shows an advancement towards human-AI collaborated automated experiments, steering optimization trajectories through human overpowering AI at the early stage when uncertainty is high and AI overpowering human at the later stage with rapid exploration towards optimal goal, following human-assessed multiple targets properties.

Biswas, Arpan↗

Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states

Maintenance policy optimization usually is faced with challenges that arise from an imperfect knowledge of system degradation models and from the partial observability of system degradation states. Here, this paper proposes a reinforcement learning method to address these two challenges for a class of maintenance problems with Markov degradation processes. The reinforcement learning approach consists of a learning component and a planning component. Using sequentially collected observations, at each step of decision-making the learning component improves the knowledge of system degradation in terms of the probability distributions of the transition rates based on sequential Bayesian inference. Using the updated transition rates, at each step of decision-making the maintenance policy optimization problem is then formulated as a partially observable Markov decision problem, and the planning component computes the optimal maintenance policy that maximizes the expected cumulative reward. The proposed method is illustrated using a numerical example with repair and inspection maintenance actions. The result shows that as more observations are collected, the learning component progressively learns the true system degradation process, and the planning component adjusts the optimal maintenance policy accordingly as well, which leads to increased reward.

42 ENGINEERING↗

Computational investigation of water glasses using machine-learning potentials

The molecular origins of water’s anomalous properties have long been a subject of scientific inquiry. The liquid–liquid phase transition hypothesis, which posits the existence of distinct low-density and high-density liquid states separated by a first-order phase transition terminating at a critical point, has gained increasing experimental and computational support and offers a thermodynamically consistent framework for many of water’s anomalies. However, experimental challenges in avoiding crystallization near the postulated liquid–liquid critical point have focused attention to water’s canonical glassy states: low-density and high-density amorphous ice. Here, we use two Deep Potential machine-learning models, trained on the Strongly Constrained and Appropriately Normed density functional and the highly accurate Many-Body Polarizable potential, to conduct an investigation of water’s glassy phenomenology based on quantum mechanical calculations. Despite not being explicitly trained on amorphous ices, both models accurately capture the structure and transformation of the water glasses, including their interconversion along different thermodynamic paths. Isobaric quenching of liquid water at various pressures generates a continuum of intermediate amorphous ices and density fluctuations increase near the liquid–liquid critical pressure. The glass transition temperatures of the amorphous ices produced at different pressures exhibit two distinct branches, corresponding to low-density and high-density amorphous ice behaviors, consistent with experiment and the liquid–liquid transition hypothesis. Extrapolating transformation pressures from isothermal compressions to experimental compression rates brings our simulations into excellent agreement with data. Our findings demonstrate that machine-learning potentials trained on equilibrium phases can effectively model nonequilibrium glassy behavior and pave the way for studying long-timescale, out-of-equilibrium processes with quantum mechanical accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Using Machine Learning to Predict Future Temperature Outputs in Geothermal Systems

Optimizing the power output, and economic value, of geothermal power plants over decades of operation is a major challenge in renewable energy. Optimizing the output requires the ability to predict the mass flow rates and the output temperatures of production wells based on the inputs of injection wells, as well as the time history of the system. Machine Learning (ML) that incorporates the known physics of geothermal systems is one possible solution to this challenge. In this work, we explore the ability of ML algorithms to predict future temperature outputs based on historical data. Considering the challenges with obtaining an empirical dataset from field data that is large enough to enable reliable ML, we propose an alternate approach: developing a high-fidelity reservoir model and using computational resources to build a dataset that enables ML. As a first step towards achieving this goal, we present preliminary results from applying ML to predict the temperature timeseries of simple modeled geothermal systems. We describe the application of relevant state-of-the-art ML approaches, such as the Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN), to extract temporal structures in the model data. We assess the accuracy of the forecasts we obtain, compare the selected approaches, and share the lessons learned that would inform the process of training and utilizing ML algorithms for larger and more complex geothermal systems.

GEOTHERMAL ENERGY↗

Evaluating a Commercial Dynamic Line Rating Software with the National PMU Dataset

To accelerate the development of data-driven applications for power systems, the Department of Energy (DOE) supported the collection and curation of a synchrophasor dataset spanning two years of observations from transmission utilities across the US. This National PMU Dataset (NPDS) was anonymized and distributed to awardees of a DOE research grant under nondisclosure agreements (NDAs) but has also been retained at PNNL to enable further research. Agreements with data contributors prevent the data from being shared outside the organization. However, establishing a blind research validation methodology is envisioned to maximize the value proposition of the NPDS. In this validation strategy, researchers may share algorithms/software (potentially as executables to protect intellectual property) with PNNL, and PNNL will share feedback about the software’s performance on subsets of the NPDS. Such a blind methodology ensures that sensitive information about critical infrastructure remains protected, but the value of the NPDS can be extended to research beyond PNNL. Through iterative feedback, the algorithms may be tweaked to address real-world artifacts. As the NPDS data is temporally and geographically diverse, it may capture features absent in smaller datasets used during the development of the algorithm under test. This report presents lessons learned from applying the blind validation methodology to LineID™, a synchrophasor-based dynamic line rating software developed by Topolonet Corporation. Improvements made to the software through iterative feedback, limitations of the validation methodology, as well as how the limitations of the NPDS affected the evaluation process are discussed. Observations indicate that the proposed validation methodology can be valuable for evaluating other tools in the future.

97 MATHEMATICS AND COMPUTING↗