Results for “learning rate” — Page 7

DOE OSTI2022

Supervised learning and the finite-temperature string method for computing committor functions and reaction rates

A central object in the computational studies of rare events is the committor function. Though costly to compute, the committor function encodes complete mechanistic information of the processes involving rare events, including reaction rates and transition-state ensembles. Under the framework of transition path theory, Rotskoff et al. [Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Proceedings of Machine Learning Research (PLMR, 2022), Vol. 145, pp. 757–780] proposes an algorithm where a feedback loop couples a neural network that models the committor function with importance sampling, mainly umbrella sampling, which collects data needed for adaptive training. Here, in this work, we show additional modifications are needed to improve the accuracy of the algorithm. The first modification adds elements of supervised learning, which allows the neural network to improve its prediction by fitting to sample-mean estimates of committor values obtained from short molecular dynamics trajectories. The second modification replaces the committor-based umbrella sampling with the finite-temperature string (FTS) method, which enables homogeneous sampling in regions where transition pathways are located. We test our modifications on low-dimensional systems with non-convex potential energy where reference solutions can be found via analytical or finite element methods, and show how combining supervised learning and the FTS method yields accurate computation of committor functions and reaction rates. We also provide an error analysis for algorithms that use the FTS method, using which reaction rates can be accurately estimated during training with a small number of samples. The methods are then applied to a molecular system in which no reference solution is known, where accurate computations of committor functions and reaction rates can still be obtained.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

1D Modeling for the Development of a Digital Twin for Hydropower Systems with Applications for a Testing Rig

Modeling and simulation constitute two important parts in constructing a sensible digital twin to mimic the dynamics of hydropower systems. Because of the physical nature of a hydropower system, the basic modeling should cover the water flow systems from the reservoir to the penstock (inlet water pipes), penstock to hydro turbine, and hydro turbine to generator and from the linkage of the hydropower systems to the grid. This report describes initial attempts to model these dynamic components, including the formulation of the linearized state space model for hydro turbine systems using the well-known six-coefficients linearization method and the formulation of the voltage and power dynamical models of a synchronous generator. To validate the accuracy of the proposed model, data collected from Norwegian University of Science and Technology were used to obtain relevant parameters for the model, and the desired simulation results were obtained. Additionally, neural network modeling and learning were also developed and applied to model the generation torque and water flow rate subsystems to demonstrate a potential learning scheme, which can be used in the development of a digital twin.

13 HYDRO ENERGY↗

DOE OSTI2024

Anticipating Optical Availability in Hybrid RF/FSO Links Using RF Beacons and Deep Learning

Radiofrequency (RF) communications offer reliable but low data rates and energy-inefficient satellite links, while free-space optical (FSO) promises high bandwidth but struggles with disturbances imposed by atmospheric effects. A hybrid RF/FSO architecture aims to achieve optimal reliability along with high data rates for space communications. Accurate prediction of dynamic ground-to-satellite FSO link availability is critical for routing decisions in low-earth orbit constellations. In this paper, we propose a system leveraging ubiquitous RF links to proactively forecast FSO link degradation prior to signal drops below threshold levels. This enables pre-calculation of rerouting to maximally maintain high data rate FSO links throughout the duration of weather effects. We implement a supervised learning model to anticipate FSO attenuation based on the analysis of RF patterns. Through the simulation of a dense lower earth orbit (LEO) satellite constellation, we demonstrate the efficacy of our approach in a simulated satellite network, highlighting the balance between predictive accuracy and prediction duration. An emulated cloud attenuation model is proposed to provide insight into the temporal profiles of RF signals and their correlation to FSO channel dynamics. Our investigation sheds light on the trade-offs between prediction horizon and accuracy arising from RF beacon numbers and proximity.

FSO availability↗

DOE OSTI2025

Unsupervised Clustering and Supervised Regression Learning to Select High Temperature Oxidation-Resistant Materials

High temperature oxidation and corrosion degradation mechanisms dictate the lifetime of materials critical to energy production. The combination of modeling and experimental approaches such as machine learning (ML) and data analytics, with sufficient experimental data, can accelerate the development of new materials while limiting its cost. In the present work, ML will be applied to two high temperature oxidation data libraries (Oak Ridge National Laboratory and National Air and Space Administration) that comprised of about 5000 mass change sample datasheets for a variety of materials and temperatures in dry air and air + 10 % H2O. A python code was developed to prepare the data for machine learning by collecting and formatting oxidation rate constants, alloy compositions and environment of exposure into a single data frame. Scikit-learn library and Statistics and Machine Learning Toolbox within MathWorks were then used to perform unsupervised clustering and supervised regression learning. The impact of dataset distribution on the performance of the developed ML models was evaluated. Potential strategies to improve the predictions and enhance extrapolative capability of the previously trained model were investigated.

Romedenne, Marie [ORNL] (ORCID:0000000317936561)↗

DOE OSTI2023

Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate

In complex physical process characterization, such as the measurement of the regression rate for solid hybrid rocket fuels, where both the observation data and the model used have uncertainties originating from multiple sources, combining these in a systematic way for quantities of interest (QoI) remains a challenge. In this paper, we present a forward propagation uncertainty quantification (UQ) process to produce a probabilistic distribution for the observed regression rate r. We characterized two input data uncertainty sources from the experiment (the distortion from the camera U c and the non-zero-angle fuel placement U Y ), the prediction and model form uncertainty from the deep neural network (U m ), as well as the variability from the manually segmented images used for training it (U s ). Here, we conducted seven case studies on combinations of these uncertainty sources with the model form uncertainty. The main contribution of this paper is the investigation and inclusion of the experimental image data uncertainties involved, and how to include them in a workflow when the QoI is the result of multiple sequential processes.

42 ENGINEERING↗

DOE OSTI2025

Real-time neutron multiplicity and source localization for criticality safety during fuel debris removal

Advancing neutron detection and analysis techniques for complex radiation environments is an ongoing focus in nuclear instrumentation and monitoring. This proposal presents research and development of a generalized real-time neutron monitoring and analysis system, applicable to any detector capable of producing time-tagged neutron count data. While the work is demonstrated using the Neutron Multiplication Analysis Detector (NoMAD), a modular 15-tube helium-3 (He-3) array, due to its availability, spatial resolution, and flexible deployment, the methods developed are extensible to other systems, including organic scintillators and fast digital detectors. This research investigates two complementary analytical techniques for real-time characterization of neutron emitting sources: neutron multiplicity estimation based on the Hage-Cifarelli formalism and spatial localization using supervised machine learning applied to spatial count rate patterns. These methods are designed to operate under dynamic, evolving conditions such as fuel debris retrieval or reactor startup, where neutron-emitting material geometries may be partially unknown or changing over time. By integrating statistical neutron emission data with spatial localization, this research aims to develop and evaluate methods for real time neutron monitoring, source characterization, and material verification. Key contributions include implementation of a low-latency data pipeline for continuous neutron multiplicity analysis, development and validation of machine learning models for spatial inference, and experimental evaluation of system performance under variable measurement conditions. The outcomes are intended to support applications in nuclear safeguards, verification, emergency response, and reactor startup.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Evaluating county-level lung cancer incidence from environmental radiation exposure, PM 2.5 , and other exposures with regression and machine learning models

Characterizing the interplay between exposures shaping the human exposome is vital for uncovering the etiology of complex diseases. For example, cancer risk is modified by a range of multifactorial external environmental exposures. Environmental, socioeconomic, and lifestyle factors all shape lung cancer risk. However, epidemiological studies of radon aimed at identifying populations at high risk for lung cancer often fail to consider multiple exposures simultaneously. For example, moderating factors, such as PM 2.5 , may affect the transport of radon progeny to lung tissue. This ecological analysis leveraged a population-level dataset from the National Cancer Institute’s Surveillance, Epidemiology, and End-Results data (2013–17) to simultaneously investigate the effect of multiple sources of low-dose radiation (gross γ activity and indoor radon) and PM 2.5 on lung cancer incidence rates in the USA. County-level factors (environmental, sociodemographic, lifestyle) were controlled for, and Poisson regression and random forest models were used to assess the association between radon exposure and lung and bronchus cancer incidence rates. Tree-based machine learning (ML) method perform better than traditional regression: Poisson regression: 6.29/7.13 (mean absolute percentage error, MAPE), 12.70/12.77 (root mean square error, RMSE); Poisson random forest regression: 1.22/1.16 (MAPE), 8.01/8.15 (RMSE). The effect of PM 2.5 increased with the concentration of environmental radon, thereby confirming findings from previous studies that investigated the possible synergistic effect of radon and PM 2.5 on health outcomes. In summary, the results demonstrated (1) a need to consider multiple environmental exposures when assessing radon exposure’s association with lung cancer risk, thereby highlighting (1) the importance of an exposomics framework and (2) that employing ML models may capture the complex interplay between environmental exposures and health, as in the case of indoor radon exposure and lung cancer incidence.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

DOE OSTI2026

Differentiable multiphase flow model for physics-informed machine learning in reservoir pressure management

Accurate subsurface reservoir pressure control is extremely challenging due to geological heterogeneity and multiphase fluid-flow dynamics. Predicting behavior in this setting relies on high-fidelity physics-based simulations that are computationally expensive. Yet, the uncertain, heterogeneous properties that control these flows make it necessary to perform many of these expensive simulations, which is often prohibitive. To address these challenges, we introduce a physics-informed machine learning workflow that couples a fully differentiable multiphase flow simulator, which is implemented in the DPFEHM framework with a convolutional neural network (CNN). The CNN learns to predict fluid extraction rates from heterogeneous permeability fields to enforce pressure limits at critical reservoir locations. By incorporating transient multiphase flow physics into the training process, our method enables more practical and accurate predictions for realistic injection-extraction scenarios compared to previous works. To speed up training, we pretrain the model on single-phase, steady-state simulations and then finetune it on full multiphase scenarios, which dramatically reduces the computational cost. We demonstrate that high-accuracy training can be achieved with fewer than three thousand full-physics multiphase flow simulations – compared to previous estimates requiring up to ten million. This drastic reduction in the number of simulations is achieved by leveraging transfer learning from much less expensive single phase simulations.

25 ENERGY STORAGE↗

DOE OSTI2024

Reliable edge machine learning hardware for scientific applications

Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing. This leads to unique validation challenges for VLSI implementations of ML algorithms: enabling bit-accurate functional simulations for performance validation in experimental software frameworks, verifying those ML models are robust under extreme quantization and pruning, and enabling ultra-fine-grained model inspection for efficient fault tolerance. We discuss approaches to developing and validating reliable algorithms at the scientific edge under such strict latency, resource, power, and area requirements in extreme experimental environments. We study metrics for developing robust algorithms, present preliminary results and mitigation strategies, and conclude with an outlook of these and future directions of research towards the longer-term goal of developing autonomous scientific experimentation methods for accelerated scientific discovery.

Baldi, Tommaso↗

DOE OSTI2022

Digitally manufactured air plasma-on-water reactor for nitrate production

The sustainable production of food to support the increasing world population is one of humanity’s most pressing challenges. Plasma activated water, produced using renewable energy, can help fulfill plants’ needs in sustainable agriculture approaches. The design, implementation, and characterization of a digitally manufactured air plasma-on-water reactor (POWR) for the synthesis of nitrate as green nitrogen fertilizer is presented. The interaction of air plasma-generated reactive oxygen and nitrogen species with water produces nitrate (NO 3 - ) and related species, which are the main nitrogen-containing nutrients for plants. The mild conditions of the operation of the POWR opens the possibility to use plastics, particularly through digital manufacturing strategies such as 3D-printing, for its fabrication. A pin-to-plate reactor configuration powered by high-voltage alternating power is chosen due to its simplicity and efficacy. A computational thermal-fluid model is used to evaluate the design and attain expected operational characteristics. The experimental characterization of the POWR encompassed design and operation parameters, namely electrode-water spacing, air flow rate, and voltage level. A machine learning approach is implemented to extract and quantify characteristic features of the plasma–water interaction, such plasma volume and plasma–water interface area. Experimental results revealed that the nitrate production rate varies linearly with dimensionless plasma volume. The design, fabrication, and characterization methods presented can be adapted to other POWRs and help enable on-demand nitrogen fertilizer production at low environmental and economic cost.

Physics↗

DOE OSTI2020

A Machine Learning-based Approach to Predict the Aggregate Flexibility of HVAC Systems

Use of renewable energy resources can play a significant role in mitigation of climate change since they reduce the dependency on fossil fuels. However, high penetration of renewable energy resources into the power grid may cause some reliability problems. Ancillary services are key to realize the potential of renewable resources without disturbing the grid reliability. In recent years, many control approaches focused on exploiting the aggregate flexible heating, ventilation, and air-conditioning (HVAC) loads of buildings. However, there are still research gaps in assessing in advance the resulting indoor temperature responses for given total power and weather profiles before implementing the controls in the buildings. Such an assessment is critical to guarantee satisfying both the grid related objective (reliability) and the building related objective (occupants' comfort). Towards addressing this research gap, this paper presents a machine-learning approach for predicting the ratings of indoor temperature responses. In this approach, a prediction model was developed using a dataset generated by a set of model-free control simulations. The developed model achieved high prediction accuracy and showed the potential of the proposed approach.

Amasyali, Kadir↗

DOE OSTI2022

Demand Response in Industrial Facilities: Peak Electric Demand

The US Department of Energy’s (DOE’s) Better Buildings, Better Plants Program (Better Plants) is a voluntary energy efficiency leadership initiative for US manufacturers and water/wastewater entities. The program encourages organizations to commit to reducing the energy intensity of their US operations over a 10-year period, typically by 25%. Companies joining Better Plants are recognized by DOE for their leadership in implementing energy efficiency practices and for reducing their energy intensity. Better Plants Partners are assigned to a Technical Account Manager, who can help companies establish energy intensity baselines, develop energy management plans, and identify key resources and incentives from DOE, other federal agencies, states, utilities, and other organizations that can enable them to reach their goals. Better Plants Partners are expected to report their progress to DOE once a year. This involves establishing an energy intensity baseline upon joining the program and then tracking their progress over time. Demand Response in Industrial Facilities: Peak Electrical Demand is intended to help companies understand peak demand response programs offering by their local utility. Manufacturing industries can learn about time-varying rates and smart technologies they can use to help them reduce their energy bills. This guidance document is applicable to companies participating at either the program or challenge level. Although this guide is intended primarily to assist companies participating in Better Plants, the methodologies and guidance within the document are applicable to any organization interested in understanding peak demand response programs.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Demand Response in Industrial Facilities: Peak Electric Demand

The US Department of Energy’s (DOE’s) Better Buildings, Better Plants Program (Better Plants) is a voluntary energy efficiency leadership initiative for US manufacturers and water/wastewater entities. The program encourages organizations to commit to reducing the energy intensity of their US operations over a 10-year period, typically by 25%. Companies joining Better Plants are recognized by DOE for their leadership in implementing energy efficiency practices and for reducing their energy intensity. Better Plants Partners are assigned to a Technical Account Manager, who can help companies establish energy intensity baselines, develop energy management plans, and identify key resources and incentives from DOE, other federal agencies, states, utilities, and other organizations that can enable them to reach their goals. Better Plants Partners are expected to report their progress to DOE once a year. This involves establishing an energy intensity baseline upon joining the program and then tracking their progress over time. Demand Response in Industrial Facilities: Peak Electrical Demand is intended to help companies understand peak demand response programs offering by their local utility. Manufacturing industries can learn about time-varying rates and smart technologies they can use to help them reduce their energy bills. This guidance document is applicable to companies participating at either the program or challenge level. Although this guide is intended primarily to assist companies participating in Better Plants, the methodologies and guidance within the document are applicable to any organization interested in understanding peak demand response programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2024

Selective consolidation of learning and memory via recall-gated plasticity

In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve synaptic plasticity rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.

Lindsey, Jack W. (ORCID:0000000309307327)↗

DOE OSTI2023

Batch Active Learning for Multispectral and Hyperspectral Image Segmentation Using Similarity Graphs

Abstract Graph learning, when used as a semi-supervised learning (SSL) method, performs well for classification tasks with a low label rate. We provide a graph-based batch active learning pipeline for pixel/patch neighborhood multi- or hyperspectral image segmentation. Our batch active learning approach selects a collection of unlabeled pixels that satisfy a graph local maximum constraint for the active learning acquisition function that determines the relative importance of each pixel to the classification. This work builds on recent advances in the design of novel active learning acquisition functions (e.g., the Model Change approach in arXiv:2110.07739) while adding important further developments including patch-neighborhood image analysis and batch active learning methods to further increase the accuracy and greatly increase the computational efficiency of these methods. In addition to improvements in the accuracy, our approach can greatly reduce the number of labeled pixels needed to achieve the same level of the accuracy based on randomly selected labeled pixels.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

A neural network for beam background decomposition in Belle II at SuperKEKB

Here, we describe a neural network for predicting the background hit rate in the Belle II detector produced by the SuperKEKB electron-positron collider. The neural network, BGNet, learns to predict the individual contributions of different physical background sources, such as beam-gas scattering or continuous top-up injections into the collider, to Belle II sub-detector rates. The samples for learning are archived 1 Hz time series of diagnostic variables from the SuperKEKB collider subsystems and measured hit rates of Belle II used as regression targets. We test the learned model by predicting detector hit rates on archived data from different run periods not used during training. We show that a feature attribution method can help interpret the source of changes in the background level over time.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Process prediction and detection of faults using probabilistic bidirectional recurrent neural networks on real plant data

Attaining Industry 4.0 for manufacturing operations requires advanced monitoring systems and real-time data analytics of plant data, among other topics. We propose a Probabilistic Bidirectional Recurrent Network (PBRN) for industrial process monitoring for the early detection of faults. The model is based on a Gated Recurrent Unit (GRU) neural network that allows the model to retain long-term dependencies between sensor data along a time horizon, hence learning the dynamic behavior of the process. To reduce the false-positive detection rate of the model, we compel the model to learn from a highly noisy sensor reading while outputting noise-free sensor outputs. The performance of the proposed model is compared to other data-driven statistical process monitoring schemes using real plant data from an industrial Air Separations Unit (ASU) containing noisy sensor readings. We show that the model can learn from noisy data without reducing its performance. Using two different fault cases, we demonstrate the model’s ability to carry out early fault detection with average false-positive rates of 2.9% and 4.9% for both fault cases. The missed detection rates are 0.1% and 0.2%, respectively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗