Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Active Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

PERFORMANCE ASSESSMENT ASSISTANCE ACTIVITIES IN THE DOE COMPLEX

The United States Department of Energy Office of Environmental Management (DOE-EM) has established a Performance Assessment Community of Practice (PA CoP) to foster the sharing of information among performance assessment (PA) and risk assessment practitioners, regulators and oversight personnel. The general intent is to contribute to continuous improvement in the consistency, technical adequacy and quality of implementation of PAs and risk assessments around the DOE Complex. The PA CoP activities have involved commercial disposal facilities and international participants to provide a global perspective. The PA CoP has also sponsored annual technical exchanges as a means to foster improved communication and to share lessons learned from on-going modelling activities. The PA CoP encourages activities to provide programmatic and technical assistance in the form of sharing experience and lessons learned with practitioners during the development of PAs and risk assessments. This assistance complements DOE-EM reviews through the Low-Level Waste Disposal Facility Federal Review Group (LFRG) that are conducted after modelling efforts are completed. Such up-front assistance is providing additional value in terms of improving consistency and sharing of information. There has been a substantial increase in the amount of assistance being provided. The assistance has been well received by practitioners and regulators that have been involved. The paper highlights assistance and sharing of information that has been conducted in the last two years to support activities underway in support of proposed disposal facilities at Paducah, Portsmouth, and the Idaho National Laboratory and tank closure at Hanford.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak

Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. Here, in this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Elucidating and predicting the dynamic evolution of water and land systems due to natural and energy-related forcings

Focal Area(s): 3. Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI; & 1. Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization, unsupervised learning (including deep learning), and hardware-related efforts involving AI (e.g., edge computing). Science Challenge: Interactions between water, land, and energy systems are complex and occur on a variety of scales, ranging from local to basinal to regional. Accurately predicting the behavior of ground water and surface water systems for 5-10 years and beyond requires an understanding of the current system and the ability to model both the natural system at scale and human-induced forcings related to energy and other activities. Artificial intelligence and machine learning (AI/ML) combined with modern compilation and integration efforts for U.S. groundwater and surface water systems present potential solutions to bolstering detailed physics-based models of these systems. Big data tied with ML and physics-based modeling can drive breakthroughs in understanding the earth system, but research is often impeded by data access (e.g., privacy issues), quality, formats, gaps, multi-source, multi-scale, integration, and spatiotemporal challenges. Effective integration of real data and simulated (synthetic) data that fill gaps is critical. Overcoming these complex data and model integration challenges will enable a transformational approach to acquiring enhanced understanding of environmental systems.

54 ENVIRONMENTAL SCIENCES↗

Domain Aware Deep-learning Algorithms Integrated with Scientific-computing Technologies (DADAIST)

This technical report summarized the contribution of the DADAIST project funded by the Data Model Convergence Initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). Specifically, we report the development of the NeuroMANCER (Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations), a new open-source Scientific Machine Learning library for formulating and solving parametric constrained optimization problems, physics-informed system identification, and parametric optimal control problems. NeuroMANCER is using differentiable programming to combine modern data-driven models and optimization modeling language into a coherent algorithmic and software framework. NeuroMANCER is a Pytorch-based framework and adopts much of its philosophy focused on research and development, rapid prototyping, and streamlined deployment. Strong emphasis is given to extensibility, interoperability with the PyTorch ecosystem, and quick adaptability to custom domain problems. Neuromancer repository contains a comprehensive library of differentiable modules, including custom activation functions, matrix factorizations, deep learning architectures, neural differential equations, differential equation solvers, implicit layers such as iterative solvers, high-level API for symbolic expressions, API for modeling and control of dynamical systems, and extensive set of tutorial code examples in the form of python scripts and jupyter notebooks.

97 MATHEMATICS AND COMPUTING↗

Progress of Gas Injection EOR Surveillance in the Bakken Unconventional Play—Technical Review and Machine Learning Study

Although considerable laboratory and modeling activities were performed to investigate the enhanced oil recovery (EOR) mechanisms and potential in unconventional reservoirs, only limited research has been reported to investigate actual EOR implementations and their surveillance in fields. Eleven EOR pilot tests that used CO2, rich gas, surfactant, water, etc., have been conducted in the Bakken unconventional play since 2008. Gas injection was involved in eight of these pilots with huff ‘n’ puff, flooding, and injectivity operations. Surveillance data, including daily production/injection rates, bottomhole injection pressure, gas composition, well logs, and tracer testing, were collected from these tests to generate time-series plots or analytics that can inform operators of downhole conditions. A technical review showed that pressure buildup, conformance issues, and timely gas breakthrough detection were some of the main challenges because of the interconnected fractures between injection and offset wells. The latest operation of co-injecting gas, water, and surfactant through the same injection well showed that these challenges could be mitigated by careful EOR design and continuous reservoir monitoring. Reservoir simulation and machine learning were then conducted for operators to rapidly predict EOR performance and take control actions to improve EOR outcomes in unconventional reservoirs.

Energy & Fuels↗

Machine Learning‐Augmented Molecular Dynamics Simulations (MD) Reveal Insights Into the Disconnect Between Affinity and Activation of ZTP Riboswitch Ligands

Abstract The challenge of targeting RNA with small molecules necessitates a better understanding of RNA–ligand interaction mechanisms. However, the dynamic nature of nucleic acids, their ligand‐induced stabilization, and how conformational changes influence gene expression pose significant difficulties for experimental investigation. This work employs a combination of computational and experimental methods to address these challenges. By integrating structure‐informed design, crystallography, and machine learning‐augmented all‐atom molecular dynamics simulations (MD), we synthesized, biophysically and biochemically characterized, and studied the dissociation of a library of small molecule activators of the 5‐aminoimidazole–4–carboxamide ribonucleotide triphosphate (ZTP) riboswitch, a ligand‐binding RNA motif that regulates bacterial gene expression. We uncovered key interaction mechanisms, revealing valuable insights into the role of ligand binding kinetics on riboswitch activation. Further, we established that ligand on‐rates determine activation potency as opposed to binding affinity and elucidated RNA structural differences, which provide mechanistic insights into the interplay of RNA structure on riboswitch activation.

Chemistry↗

Discussing spent nuclear fuel in high school classrooms: addressing public fears through early education

The Inreach program combines the Deep River Science Academy (DRSA) 'learning through research' approach with state of the art communication technology to bring scientific research to high school classrooms. The Inreach program follows the DRSA teaching model where a university student tutor works on a research project with scientific staff at AECL's Chalk River Laboratories. Participating high school classes are located across Canada. The high school students learn about the ongoing research activities via weekly web conferences. In order to engage the students and encourage participation in the conferences, themed exercises linked to the research project are provided to the students. The DRSA's Inreach program uses a cost-effective internet technology to reach a wide audience, in an interactive setting, without anyone leaving their desks or offices. An example Inreach research project is presented here: an investigation of the potential of the Canadian supercritical water cooled reactor (SCWR) concept to burn transuranic elements (Np, Pu, Am, Cm) to reduce the impact of used nuclear fuel. During this project a university student worked with AECL (Atomic Energy of Canada Limited) researchers on technical aspects of the project, and high school students followed their progress and learned about the composition, hazards, and disposition options for used nuclear fuel. Previous projects included the effects of tritium on cellular viability and neutron diffraction measurement of residual stresses in automobile engines.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Understanding the Inner-Workings of Language Models Through Representation Dissimilarity

We use model stitching to understand the internal representations of language models. Similar to vision models, we find that "more is better," and representations learned with more data and larger width can improve the performance of weaker models via stitching. We likewise find that certain architecture choices, using GeLU vs SoLU activation functions, influence the quality of learned representations. Finally, model stitching (as opposed to other model diagnostic methods, like mode connectivity) can localize the different generalization strategies of text classifiers under domain shift to certain hidden layers.

Brown, Davis R.↗

Analysis of MINERvA tunings of GENIE based on Hadron Multiplicity using Machine Learning

Scientists with the MINERvA neutrino interaction experiment have long studied distributions of hadrons (protons and pions) to improve the models for these reactions. Until now, most work has relied on easy to find, energetic, and well tracked hadrons. But in order to take full advantage of MINERνA data it is necessary to develop an efficient enough tool to recognize hadron tracks with lower energy, more transverse angle and more challenging patterns of activity. In this analysis, a Machine Learning tool based on Convolutional Neural Network and Transfer Learning has been developed to tackle that issue. It is found to be extremely capable of recognizing hadron activities especially when there are few and not very energetic hadrons, the accuracy is 76%. But it is also found that this network is very reluctant to classify too many hadrons. The classification is also affected by the presence of other particles in the event such photons, hadrons below a threshold or neutrons. This network is used to compare hadron multiplicity of data to the MC models and distinguish between different MC tunes. Some MINERνA tunes are explored in this analysis, but none of them described the data very well, especially not for all different hadrons multiplicities at once. Four combinations of those tunes were more successful, but none of them could be recognized as the best one. Nevertheless, this analysis develops an ML network that sets the basis of further exploration of hadron multiplicity as a tool to analyze MC tunes at MINERνA and beyond.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Identification of putative estrogen receptor-mediated endocrine disrupting chemicals using QSAR- and structure-based virtual screening approaches

Identification of endocrine disrupting chemicals is one of the important goals of environmental chemical hazard screening. We report on the development of validated in silico predictors of chemicals likely to cause estrogen receptor (ER)-mediated endocrine disruption to facilitate their prioritization for future screening. A database of relative binding affinity of a large number of ERα and/or ERβ ligands was assembled (546 for ERα and 137 for ERβ). Both single-task learning (STL) and multi-task learning (MTL) continuous quantitative structure–activity relationship (QSAR) models were developed for predicting ligand binding affinity to ERα or ERβ. High predictive accuracy was achieved for ERα binding affinity (MTL R{sup 2} = 0.71, STL R{sup 2} = 0.73). For ERβ binding affinity, MTL models were significantly more predictive (R{sup 2} = 0.53, p < 0.05) than STL models. In addition, docking studies were performed on a set of ER agonists/antagonists (67 agonists and 39 antagonists for ERα, 48 agonists and 32 antagonists for ERβ, supplemented by putative decoys/non-binders) using the following ER structures (in complexes with respective ligands) retrieved from the Protein Data Bank: ERα agonist (PDB ID: 1L2I), ERα antagonist (PDB ID: 3DT3), ERβ agonist (PDB ID: 2NV7), and ERβ antagonist (PDB ID: 1L2J). We found that all four ER conformations discriminated their corresponding ligands from presumed non-binders. Finally, both QSAR models and ER structures were employed in parallel to virtually screen several large libraries of environmental chemicals to derive a ligand- and structure-based prioritized list of putative estrogenic compounds to be used for in vitro and in vivo experimental validation. - Highlights: • This is the largest curated dataset inclusive of ERα and β (the latter is unique). • New methodology that for the first time affords acceptable ERβ models. • A combination of QSAR and docking enables prediction of affinity and function. • The results have potential applications to green chemistry. • Models are publicly available for virtual screening via a web portal.

60 APPLIED LIFE SCIENCES↗

FAIR AI models in high energy physics

Abstract The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning models—algorithms that have been trained on data without being explicitly programmed—and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template’s use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗

DFSynthesizer: Dataflow-based Synthesis of Spiking Neural Networks to Neuromorphic Hardware

Spiking Neural Networks (SNNs) are an emerging computation model that uses event-driven activation and bio-inspired learning algorithms. SNN-based machine learning programs are typically executed on tile-based neuromorphic hardware platforms, where each tile consists of a computation unit called a crossbar, which maps neurons and synapses of the program. However, synthesizing such programs on an off-the-shelf neuromorphic hardware is challenging. This is because of the inherent resource and latency limitations of the hardware, which impact both model performance, e.g., accuracy, and hardware performance, e.g., throughput. We propose DFSynthesizer, an end-to-end framework for synthesizing SNN-based machine learning programs to neuromorphic hardware. The proposed framework works in four steps. First, it analyzes a machine learning program and generates SNN workload using representative data. Second, it partitions the SNN workload and generates clusters that fit on crossbars of the target neuromorphic hardware. Third, it exploits the rich semantics of the Synchronous Dataflow Graph (SDFG) to represent a clustered SNN program, allowing for performance analysis in terms of key hardware constraints such as number of crossbars, dimension of each crossbar, buffer space on tiles, and tile communication bandwidth. Finally, it uses a novel scheduling algorithm to execute clusters on crossbars of the hardware, guaranteeing hardware performance. We evaluate DFSynthesizer with 10 commonly used machine learning programs. Our results demonstrate that DFSynthesizer provides a much tighter performance guarantee compared to current mapping approaches.

Computer Science↗

A novel procedure on next generation sequencing data analysis using text mining algorithm

Background: Next-generation sequencing (NGS) technologies have provided researchers with vast possibilities in various biological and biomedical research areas. Efficient data mining strategies are in high demand for large scale comparative and evolutional studies to be performed on the large amounts of data derived from NGS projects. Topic modeling is an active research field in machine learning and has been mainly used as an analytical tool to structure large textual corpora for data mining. Methods: We report a novel procedure to analyse NGS data using topic modeling. It consists of four major procedures: NGS data retrieval, preprocessing, topic modeling, and data mining using Latent Dirichlet Allocation (LDA) topic outputs. The NGS data set of the Salmonella enterica strains were used as a case study to show the workflow of this procedure. The perplexity measurement of the topic numbers and the convergence efficiencies of Gibbs sampling were calculated and discussed for achieving the best result from the proposed procedure. Results: The output topics by LDA algorithms could be treated as features of Salmonella strains to accurately describe the genetic diversity of fliC gene in various serotypes. The results of a two-way hierarchical clustering and data matrix analysis on LDA-derived matrices successfully classified Salmonella serotypes based on the NGS data. The implementation of topic modeling in NGS data analysis procedure provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data. Conclusion: The implementation of topic modeling in NGS data analysis provides a new way to elucidate genetic information from NGS data, and identify the gene-phenotype relationships and biomarkers, especially in the era of biological and medical big data.

59 BASIC BIOLOGICAL SCIENCES↗