Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Continual learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Reservoir Computing as a Tool for Climate Predictability Studies

Reduced-order dynamical models play a central role in developing our understanding of predictability of climate irrespective of whether we are dealing with the actual climate system or surrogate climate models. In this context, the linear inverse modeling (LIM) approach, by capturing a few essential interactions between dynamical components of the full system, has proven valuable in providing insights into predictability of the full system. We demonstrate that reservoir computing (RC), a form of learning suitable for systems with chaotic dynamics, provides an alternative nonlinear approach that improves on the predictive skill of the LIM approach. We do this in the example setting of predicting sea surface temperature in the North Atlantic in the preindustrial control simulation of a popular earth system model, the Community Earth System Model so that we can compare the performance of the new RC-based approach with the traditional LIM approach both when learning data are plentiful and when such data are more limited. The improved predictive skill of the RC approach over a wide range of conditions—larger number of retained EOF coefficients, extending well into the limited data regime, etc.—suggests that this machine-learning technique may have a use in climate predictability studies. While the possibility of developing a climate emulator—the ability to continue the evolution of the system on the attractor long after failing to be able to track the reference trajectory—is demonstrated in the Lorenz-63 system, it is suggested that further development of the RC approach may permit such uses of the new approach in more realistic predictability studies.

54 ENVIRONMENTAL SCIENCES↗

Machine Learning-Based Atmospheric Phenomena Detection Platform

As the number of Earth pointing satellites has increased over the last several decades, the data volume retrieved from instruments onboard these satellites has also increased. It is expected that this trend will continue as more data intensive missions and small satellite constellations are launched. Currently, feature detection - namely atmospheric phenomena - in these datasets is performed manually and is thus not scalable with the growing data archives. Recent advancements in computational efficiency allow for the Earth science community to leverage machine learning to identify interesting atmospheric phenomena. Given the wide range of distinctive features in various atmospheric phenomena, a specialized machine learning model is required for accurate detection of these phenomena independently. The Phenomena Portal, developed at NASA IMPACT, is designed to provide visualization for the output from these machine learning models. In addition, detected events for each atmospheric phenomena are stored in a database that can be used to more easily use/subset larger spatiotemporal datasets. The user interface also incorporates additional features to enhance the user experience including spatiotemporal analysis, multiple base layer images, and a slider to filter events with lower probabilities of positive detection. Each detection supports user feedback on whether the detection is true or false that can then be stored and used to improve the machine learning model performance.

Gurung, Iksha↗

Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning

Finding the most likely path to a set of failure states is important to the analysis of safety-critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates.As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.

Verification and Validation↗

SSTDR and FDR Detection of Un-Energized and Energized Cable Anomalies Including Thermal Degradation Using Machine Learning

Historically, cables are initially qualified for nuclear power plant use for 40 years. As plants extend their operating license to 60 and 80 years, continued use of these cables must shift to a performance-based approach since it is cost prohibitive to completely replace cables that are likely still capable of performing their design function. A variety of cable tests are available and are commonly applied during outages when the cables can be taken out of service. Frequency domain reflectometry (FDR) is one of these test methods that is being more broadly accepted and used because it not only detects anomalies along the cable with a low-voltage signal that does not stress the cable insulation, but the technique also locates the anomalies. This supports follow-up local inspection and local repair or partial replacement of a damaged cable segment. Currently, FDR testing is only applied to cables that are taken out of service since the test instrument would be damaged by operational voltages.

36 MATERIALS SCIENCE↗

Lessons Learned during Thermal Hardware Integration on the Global Precipitation Measurement Satellite

The Global Precipitation Measurement mission is a joint NASA/JAXA mission scheduled for launch in late 2013. The integration of thermal hardware onto the satellite began in the Fall of 2010 and will continue through the Summer of 2012. The thermal hardware on the mission included several constant conductance heat pipes, heaters, thermostats, thermocouples radiator coatings and blankets. During integration several problems arose and insights were gained that would help future satellite integrations. Also lessons learned from previous missions were implemented with varying degrees of success. These insights can be arranged into three categories. 1) the specification of flight hardware using analysis results and the available mechanical resources. 2) The integration of thermal flight hardware onto the spacecraft, 3) The preparation and implementation of testing the thermal flight via touch tests, resistance measurements and thermal vacuum testing.

Cottingham, Christine↗

Brief communication: Monitoring snow depth using small, cheap, and easy-to-deploy snow–ground interface temperature sensors

Abstract. Temporally continuous snow depth estimates are vital for understanding changing snow patterns and impacts on permafrost in the Arctic. We trained a random forest machine learning model to predict snow depth from variability in snow–ground interface temperature. The model performed well on Alaska's Seward Peninsula where it was trained and at Arctic evaluation sites (RMSE ≤ 0.15 m). It performed poorly at temperate sites with deeper snowpacks, partially due to training data limitations. Small temperature sensors are cheap and easy to deploy, so this technique enables spatially distributed and temporally continuous snowpack monitoring at high latitudes to an extent previously infeasible.

54 ENVIRONMENTAL SCIENCES↗

Multiple Kernel Learning for Heterogeneous Anomaly Detection: Algorithm and Aviation Safety Case Study

The world-wide aviation system is one of the most complex dynamical systems ever developed and is generating data at an extremely rapid rate. Most modern commercial aircraft record several hundred flight parameters including information from the guidance, navigation, and control systems, the avionics and propulsion systems, and the pilot inputs into the aircraft. These parameters may be continuous measurements or binary or categorical measurements recorded in one second intervals for the duration of the flight. Currently, most approaches to aviation safety are reactive, meaning that they are designed to react to an aviation safety incident or accident. In this paper, we discuss a novel approach based on the theory of multiple kernel learning to detect potential safety anomalies in very large data bases of discrete and continuous data from world-wide operations of commercial fleets. We pose a general anomaly detection problem which includes both discrete and continuous data streams, where we assume that the discrete streams have a causal influence on the continuous streams. We also assume that atypical sequence of events in the discrete streams can lead to off-nominal system performance. We discuss the application domain, novel algorithms, and also discuss results on real-world data sets. Our algorithm uncovers operationally significant events in high dimensional data streams in the aviation industry which are not detectable using state of the art methods

Das, Santanu↗

Toward a Deep Learning System for Making Sense of Unlabeled Multimodal Data

Analysts and decision makers are constantly charged with making sense of large amounts of disparate data. With the proliferation of smartphones and the ever-growing array of new sensor technologies, the old adage of swimming in sensors but drowning in data remains especially relevant. As collections of images, video, audio, and text continue to grow, the need for computer-assisted analysis likewise increases. The “data deluge” presents both daunting challenges as well as remarkable opportunities. In this article, we describe Deep Learning approaches for learning representations of data from vast amounts of unlabeled or weakly labeled training sets. These feature representations enable the computer-assisted retrieval and tagging that can help our analysts find patterns in the vast ocean of collected data.

97 MATHEMATICS AND COMPUTING↗

Accelerating Low-Income Financing and Transactions (LIFT) for Solar Access Everywhere (Final Technical Report)

The Accelerating Low-Income Financing and Transactions (LIFT) for Solar Access Everywhere project’s goal was to expand Low-to-Moderate Income (LMI) solar access for homeowners and renters. The LIFT project researched and gathered data on 453 LMI community solar project across the country. Following three years of research, the project delivered three groundbreaking research papers in June 2022, focused on 1) customer experience, 2) the growth of community solar programs, and 3) project-level financial best practices for serving LMI communities. These were followed by a user-friendly web-based Toolkit allowing users to interact with project data and key findings in November 2022. The customer experience research examined community solar subscribers’ primary motivations to join and remain satisfied with projects. Our research identified 453 projects across the country that dedicated some portion of the system capacity to LMI households. Seventeen of these projects participated in the LIFT customer experience research, allowing the project team to survey their customers and gain insight into how LMI subscribers feel about community solar and the programs that serve them. Subscribers in our sample indicated that the most critical issue that motivated them to participate in their program, however, was not savings but helping the environment. This was true for both LMI and non-LMI subscribers. Helping the environment was also the most important issue for LMI subscribers to measure how well their program was working for them. LIFT also explored how rapidly community solar has grown since its inception in 2006, publishing results in the Growth of U.S. Community Solar Serving LMI Households report. The results showed that community solar projects serving LMI households are one of the fastest growing segments of the solar industry. The report identifies and recommends ways developers should overcome real or perceived risks to LMI customer acquisition and subscriber management. Through the analysis of community solar project finance research, LIFT showed that most community solar projects serving LMI households are financed in the same ways mainstream community solar projects are financed. The value stacks and financial returns are no different, although LMI inclusion and participation rate varied across programs in our sample, ranging from between 10% and 100%. Based on the findings from the LIFT research, the team built a web-based user-friendly Toolkit, consisting of case studies, project finance best practices, and several tools built around the national dataset of 453 community solar projects that serve LMI households. These allow users to engage with the dataset in multiple ways; to explore the landscape of LMI community solar in the U.S., and to design community solar projects to optimize LMI inclusion, equity, and savings levels. The Toolkit also includes a library of LIFT-generated and LIFT-curated resources for users to learn more about how to best serve LMI communities through community solar. LIFT officially published the Toolkit on October 31, 2022, followed by a launch event (public webinar) on November 17, 2022. The core LIFT partners continue to engage in outreach and dissemination efforts to promote the LIFT Toolkit and research publications. Our driving motivation is to continue enabling solar developers to leverage the findings of this three-year research effort. By implication, the LIFT Toolkit is designed for use by utilities, energy service providers, and financiers or investors as a learning and decision-making tool to rapidly scale project models that optimize LMI inclusion and maximize real household savings.

14 SOLAR ENERGY↗

Rapid discovery of high hardness multi-principal-element alloys using a generative adversarial network model

Multi-principal element alloys (MPEAs) continue to gain research prominence due to their promising high-temperature microstructural and mechanical properties. Recently, machine learning (ML) and materials informatics have been used extensively for screening MPEAs, however, most of these efforts were focused on constructing classification and regression models for predicting phase stability and mechanical properties of known compositions. These approaches may accelerate the screening process but optimizing new compositions with desirable properties within a practical time frame from an infinitely large design space of MPEA systems remains a grand challenge. To tackle this composition optimization challenge, a generative adversarial network coupled with a neural-network ML model was utilized to design MPEAs by filtering compositions that have high hardness. Even in a high-dimensional space with 18 elements as descriptors, the ML model was able to generate optimized compositions from which one composition was found to have 10% higher hardness (941 HV) than the maximum in the training data (857 HV). Density-functional theory was used to provide thermodynamic and electronic insights to higher hardness of the new MPEA found. The present work can optimize compositions from a wide design space of 18 elements (including W, Ta and Nb) that presents an opportunity to synthesize new compositions for applications ranging from corrosion-resistant alloys to nuclear materials. Here the findings suggest that generative ML can greatly accelerate materials discovery by identifying novel compositions, which can serve as a data-informed tool to guide experiments.

36 MATERIALS SCIENCE↗

Partition dataset according to amino acid type improves the prediction of deleterious non-synonymous SNPs

Highlights: Black-Right-Pointing-Pointer Proper dataset partition can improve the prediction of deleterious nsSNPs. Black-Right-Pointing-Pointer Partition according to original residue type at nsSNP is a good criterion. Black-Right-Pointing-Pointer Similar strategy is supposed promising in other machine learning problems. -- Abstract: Many non-synonymous SNPs (nsSNPs) are associated with diseases, and numerous machine learning methods have been applied to train classifiers for sorting disease-associated nsSNPs from neutral ones. The continuously accumulated nsSNP data allows us to further explore better prediction approaches. In this work, we partitioned the training data into 20 subsets according to either original or substituted amino acid type at the nsSNP site. Using support vector machine (SVM), training classification models on each subset resulted in an overall accuracy of 76.3% or 74.9% depending on the two different partition criteria, while training on the whole dataset obtained an accuracy of only 72.6%. Moreover, the dataset was also randomly divided into 20 subsets, but the corresponding accuracy was only 73.2%. Our results demonstrated that partitioning the whole training dataset into subsets properly, i.e., according to the residue type at the nsSNP site, will improve the performance of the trained classifiers significantly, which should be valuable in developing better tools for predicting the disease-association of nsSNPs.

60 APPLIED LIFE SCIENCES↗

Roadmap and Benchmarking: Privacy in Federated Load Forecasting

Data-driven techniques for energy demand forecasting continue to emerge with promising impacts on distribution grid planning. However, the development of robust and generalizable machine learning models requires that representative high quality training data are available. Distributed energy resources have begun to embed intelligence, gathering large amounts of data on customer demand, behavior, and household devices that are connected to the grid. Though utilities aggregate meter-level demand data for load shaping, demand response, outage management, reliability planning, and billing applications, there lies an inherent privacy concern in sharing consumption data that may identify individual consumer behavioral patterns. Hence, while sharing the data is crucial, the private sensitive customer data must be safeguarded from being exposed or manipulated. In this study, we propose a roadmap for implementing a based privacy preserving framework to support the advancement of data-driven analytics in data-sensitive distributed energy resources environments. The roadmap incorporates federated learning–a distributed training framework, differential privacy–a statistical framework that provides guarantees to safeguard the leakage of sensitive data, secure multiparty computation and homomorphic encryption– techniques for encrypting model gradients and applying secure aggregation on the server. Moreover, we perform baseline experiments on the federated short-term load forecasting (STLF) task using open-source residential load profile datasets, offering insights into the challenges of integrating differential privacy into federated learning.

Abebe, Waqwoya [Oak Ridge National Laboratory (ORN↗

Evaluating the Limits of QAOA Parameter Transfer at High-Rounds on Sparse Ising Models With Geometrically Local Cubic Terms

The emergent practical applicability of the Quantum Approximate Optimization Algorithm (QAOA) for approximate combinatorial optimization is a subject of considerable interest. One of the primary limitations of QAOA is the task of finding a set of good parameters, which is usually done using a variational optimization loop. Parameter transfer, or parameter concentration, is a phenomenon where QAOA angles trained on problem instances that are self-similar tend to perform well for other problem instances from that similar class. This suggests a potentially highly efficient and scalable non-variational learning method for QAOA angle finding. In this work, we systematically study QAOA parameter transferability from small problem sizes (16 and 27 decision variables) onto large problem instances (up to 156 qubits) for heavy-hex graph Ising models with geometrically local higher order terms using the Julia based QAOA simulation tool \texttt{JuliQAOA} to perform classical angle finding for up to $49$ QAOA layers ($p$). Parameter transfer of the fixed angles is validated using a combination of full statevector, Projected Entangled Pair States (PEPS), Matrix Product State (MPS), and LOWESA numerical simulations. We find that the QAOA parameter transfer from single instances applied to other (unseen) problem instances does not in general provide monotonically improving performance as a function of $p$ - there are many cases where the performance temporarily decreases as a function of $p$ - but despite this the transferred angles have a general trend of improved expectation value as the QAOA depth increases, in many cases converging close to the true ground-state energy of the $100+$ qubit instances. We also sample the hardware-compatible Ising models using the ensemble of transfer-learned QAOA parameters on several superconducting qubit IBM Quantum processors with 127, 133, and 156 qubits. We find continuous solution quality improvement of the hardware-compatible QAOA circuits run on the IBM NISQ processors up to $p=5$ on \texttt{ibm\_fez}, up to $p=9$ on \texttt{ibm\_torino}, and up to $p=10$ on \texttt{ibm\_pittsburgh}.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

U.S. stock market interaction network as learned by the Boltzmann machine

Here, we study historical dynamics of joint equilibrium distribution of stock returns in the U.S. stock market using the Boltzmann distribution model being parametrized by external fields and pairwise couplings. Within Boltzmann learning framework for statistical inference, we analyze historical behavior of the parameters inferred using exact and approximate learning algorithms. Since the model and inference methods require use of binary variables, effect of this mapping of continuous returns to the discrete domain is studied. The presented results show that binarization preserves the correlation structure of the market. Properties of distributions of external fields and couplings as well as the market interaction network and industry sector clustering structure are studied for different historical dates and moving window sizes. We demonstrate that the observed positive heavy tail in distribution of couplings is related to the sparse clustering structure of the market. We also show that discrepancies between the model’s parameters might be used as a precursor of financial instabilities.

97 MATHEMATICS AND COMPUTING↗

Code Description for "Brief Communication: Monitoring snow depth using small, cheap, and easy-to-deploy ground surface temperature sensors"

Temporally continuous snow depth estimates are vital for understanding changing snow patterns and impacts on permafrost in the Arctic. We train a random forest machine learning model to predict snow depth from variability in ground surface temperature. To our knowledge, this is the first time that small ground surface temperature sensors have been used to estimate snow depth. The model performs well at sites where the model was trained and at pan-arctic evaluation sites (RMSE <= 0.15 m). Small temperature sensors are cheap and easy-to-deploy, so this technique enables spatially distributed and temporally continuous snowpack monitoring to an extent previously infeasible. The model is flexible and can be applied to datasets retroactively to retrieve snow depth estimates at additional sites. This code package includes a *.joblib file of the trained random forest model and a *.ipynb file showing how to clean input data, train the random forest model, and apply the model.

Bachand, Claire↗

Evaluation and Analysis of a Few International Periodic Safety Review Summary Reports (Technical Letter Report)

At the request of the United States (U.S.) government, the International Atomic Energy Agency (IAEA) assembled a team of 20 senior safety experts to review the regulatory framework for the safety of operating nuclear power plants in the United States. This review focused on the effectiveness of the regulatory functions implemented by the NRC and on its commitment to nuclear safety and continuous improvement. One suggestion resulting from that review was that the U.S. Nuclear Regulatory Commission (NRC) incorporate lessons learned from periodic safety reviews (PSRs) performed in other countries as an input to the NRC’s assessment processes.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Fundamental studies of energy-and hole/electron- transfer in hydroporphyrin architectures

The long-term objective of the Bocian/Holten/Lindsey research program is to design, synthesize, and characterize tetrapyrrole-based molecular architectures that absorb sunlight, funnel energy, and separate charge with high efficiency and in a manner compatible with current and future solar-energy conversion schemes. The synthetic tetrapyrroles include porphyrins and hydroporphyrins; the latter classes of molecules encompass analogues of the naturally occurring chlorophylls and bacteriochlorophylls (e.g., chlorins, bacteriochlorins, and their derivatives). The attainment of the goals of the research program requires the close interplay of molecular design and synthesis (Lindsey group), static and time-resolved optical spectroscopic measurements (Holten group), and electrochemical, electron paramagnetic resonance, and resonance Raman studies, as well as density functional theory calculations (Bocian Group). The proposed research encompasses four interrelated themes: (1) Determination of the rates of ground-state hole/electron transfer between (hydro)porphyrins in multipigment arrays as a function of array size, distance between components, linker type, site of linker connection, and frontier molecular orbital composition. (2) Examination of excited-state energy transfer among hydroporphyrins in multipigment arrrays, including both pairwise and non-adjacent transfer, with a chief aim to identify the relative contributions of through-space (Förster) and through-bond (Dexter) mechanisms of energy transfer, including the roles of site of linker connection and frontier molecular orbital composition. (3) Elucidation of the role of substituents in tuning the spectral and electronic properties of bacteriochlorins, with a primary aim of learning how to shift the long-wavelength absorption band deeper into the near-infrared region. (4) Continued development of the software package PhotochemCAD for spectral manipulations and calculations through the compilation of a database of spectra for naturally occurring and synthetic hydroporphyrins. The availability of such data should augment efforts in the design of light-harvesting systems where spectral coverage in the red and near-infrared regions is desired. Collectively, the proposed studies will provide fundamental insights into molecular properties, interactions, and processes relevant to the design of molecular architectures for solar-energy conversion. The accomplishment of these goals is only possible through a highly synergistic program that encompasses molecular design, synthesis, and characterization.

14 SOLAR ENERGY↗

Controls-structures-electromagnetics interaction program

A technology development program is described involving Controls Structures Electromagnetics Interaction (CSEI) for large space structures. The CSEI program was developed as part of the continuing effort following the successful kinematic deployment and RF tests of the 15 meter Hoop/Column antenna. One lesson learned was the importance of making reflector surface adjustment after fabrication and deployment. Given are program objectives, ground based test configuration, Intelsat adaptive feed, reflector shape prediction model, control experiment concepts, master schedule, and Control Of Flexible Structures-II (COFS-II) baseline configuration.

Grantham, William L.↗