Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Active Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

Machine-Learning Methods Enable Exhaustive Searches for Active Bimetallic Facets and Reveal Active Site Motifs for CO 2 Reduction

Bimetallic catalysts are promising for the most difficult thermal and electrochemical reactions, but modeling the many diverse active sites on polycrystalline samples is an open challenge. Here, we present a general framework for addressing this complexity in a systematic and predictive fashion. Active sites for every stable low-index facet of a bimetallic crystal are enumerated and cataloged, yielding hundreds of possible active sites. The activity of these sites is explored in parallel using a neural-network-based surrogate model to share information between the many density functional theory (DFT) relaxations, resulting in activity estimates with an order of magnitude fewer explicit DFT calculations. Sites with interesting activity were found and provide targets for follow-up calculations. This process was applied to the electrochemical reduction of CO 2 on nickel gallium bimetallics and indicated that most facets had similar activity to Ni surfaces, but a few exposed Ni sites with a very favorable on-top CO configuration. This motif emerged naturally from the predictive modeling and represents a class of intermetallic CO 2 reduction catalysts. These sites rationalize recent experimental reports of nickel gallium activity and why previous materials screens missed this exciting material. Most importantly these methods suggest that bimetallic catalysts will be discovered by studying facet reactivity and diversity of active sites more systematically.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Defending Against Adversarial Examples

Adversarial machine learning is an active field of research that seeks to investigate the security of machine learning methods against cyber-attacks. An important branch of this field is adversarial examples, which seek to trick machine learning models into misclassifying inputs by maliciously tampering with input data. As a result of the pervasiveness of machine learning models in diverse areas such as computer vision, health care, and national security, this vulnerability is a rapidly growing threat. With the increasing use of AI solutions, threats against AI must be considered before deploying systems in a contested space. Adversarial machine learning is a problem strongly tied to software security, and just like other more common software vulnerabilities, it exploits a weakness in software, like components of machine learning models. During this project, we attempted to survey and replicate several adversarial machine learning techniques with the goal of developing capabilities for Sandia to advise and defend against these threats. To accomplish this, we scanned state of the art research for robust defenses against adversarial examples and applied them to a machine learning problem.

97 MATHEMATICS AND COMPUTING↗

Automated algorithms to build active galactic nucleus classifiers

ABSTRACT We present a machine learning model to classify active galactic nuclei (AGNs) and galaxies (AGN-galaxy classifier) and a model to identify type 1 (optically unabsorbed) and type 2 (optically absorbed) AGN (type 1/2 classifier). We test tree-based algorithms, using training samples built from the X-ray Multi-Mirror Mission–Newton (XMM–Newton) catalogue and the Sloan Digital Sky Survey (SDSS), with labels derived from the SDSS survey. The performance was tested making use of simulations and of cross-validation techniques. With a set of features including spectroscopic redshifts and X-ray parameters connected to source properties (e.g. fluxes and extension), as well as features related to X-ray instrumental conditions, the precision and recall for AGN identification are 94 and 93 per cent, while the type 1/2 classifier has a precision of 74 per cent and a recall of 80 per cent for type 2 AGNs. The performance obtained with photometric redshifts is very similar to that achieved with spectroscopic redshifts in both test cases, while there is a decrease in performance when excluding redshifts. Our machine learning model trained on X-ray features can accurately identify AGN in extragalactic surveys. The type 1/2 classifier has a valuable performance for type 2 AGNs, but its ability to generalize without redshifts is hampered by the limited census of absorbed AGN at high redshift.

Falocco, S. (ORCID:0000000299841103)↗

Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks

We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima.

machine leaning, Bad minima, Inverse problems, Phy↗

MO-E-18C-06: Enriching Medical Physics Education By Visualizing The Invisible

Purpose: To enhance the understanding of medical physics concepts and develop higher levels of learning relating to invisible physics phenomena such as radiation. To provide medical physics educators in all countries of the world with understanding of knowledge structures in the human brain, the different levels of learning, and the types of knowledge required for higher level functions such as problem solving, creative innovations, and applied clinical applications. To provide medical physics educators with an open access resource (tool) that they can use in their teaching activities to enrich and elevate the level of learning for their students, residents, etc. with respect to the invisible realm of medical physics. Methods: An experienced clinical medical physicist and educator has created and provided with open access three complementary web-based resources to achieve the purposes described above. One is a module focusing on the medical physics learning process with respect to mental knowledge structures, how they relate to outcomes and applications, and learning activities that are required to develop the required knowledge structures. The second is an extensive set of visuals that educators can use in their activities (classes, small group discussions, etc.) to visualize the invisible. The third is an interactive online simulation where learners can adjust factors and visually observe changes in x-radiation.These resources are available online at www.BLINDED FOR REVIEW . Results: Medical physics education, especially for non-physicists, is becoming much more interesting and useful especially with respect to invisible radiation. The global impact is that medical imaging professionals can be more effective in optimizing x-ray imaging procedures and risk management when they have knowledge levels that enhance problem solving, innovation, and creativity. Conclusion: Medical physics educators in all institutions can be much more effective and efficient in the sharing of their knowledge and experience when enhanced with high-quality visuals.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Large-scale genomic analyses with machine learning uncover predictive patterns associated with fungal phytopathogenic lifestyles and traits

Abstract Invasive plant pathogenic fungi have a global impact, with devastating economic and environmental effects on crops and forests. Biosurveillance, a critical component of threat mitigation, requires risk prediction based on fungal lifestyles and traits. Recent studies have revealed distinct genomic patterns associated with specific groups of plant pathogenic fungi. We sought to establish whether these phytopathogenic genomic patterns hold across diverse taxonomic and ecological groups from the Ascomycota and Basidiomycota, and furthermore, if those patterns can be used in a predictive capacity for biosurveillance. Using a supervised machine learning approach that integrates phylogenetic and genomic data, we analyzed 387 fungal genomes to test a proof-of-concept for the use of genomic signatures in predicting fungal phytopathogenic lifestyles and traits during biosurveillance activities. Our machine learning feature sets were derived from genome annotation data of carbohydrate-active enzymes (CAZymes), peptidases, secondary metabolite clusters (SMCs), transporters, and transcription factors. We found that machine learning could successfully predict fungal lifestyles and traits across taxonomic groups, with the best predictive performance coming from feature sets comprising CAZyme, peptidase, and SMC data. While phylogeny was an important component in most predictions, the inclusion of genomic data improved prediction performance for every lifestyle and trait tested. Plant pathogenicity was one of the best-predicted traits, showing the promise of predictive genomics for biosurveillance applications. Furthermore, our machine learning approach revealed expansions in the number of genes from specific CAZyme and peptidase families in the genomes of plant pathogens compared to non-phytopathogenic genomes (saprotrophs, endo- and ectomycorrhizal fungi). Such genomic feature profiles give insight into the evolution of fungal phytopathogenicity and could be useful to predict the risks of unknown fungi in future biosurveillance activities.

59 BASIC BIOLOGICAL SCIENCES↗

Automation of Active Space Selection for Multireference Methods via Machine Learning on Chemical Bond Dissociation

Predicting and understanding the chemical bond is one of the major challenges of computational quantum chemistry. Kohn–Sham density functional theory (KS-DFT) is the most common method, but approximate density functionals may not be able to describe systems where multiple electronic configurations are equally important. Multiconfigurational wave functions, on the other hand, can provide a detailed understanding of the electronic structures and chemical bonds of such systems. In the complete active space self-consistent field (CASSCF) method, one performs a full configuration interaction calculation in an active space consisting of active electrons and active orbitals. However, CASSCF and its variants require the selection of these active spaces. This choice is not black box; it requires significant experience and testing by the user, and thus active space methods are not considered particularly user-friendly and are employed only by a minority of quantum chemists. Our goal is to popularize these methods by making it easier to make good active space choices. Here, we present a machine learning protocol that performs an automated selection of active spaces for chemical bond dissociation calculations of main group diatomic molecules. The protocol shows high prediction performance for a given target system as long as a properly correlated system is chosen for training. Good active spaces are correctly predicted with a considerably better success rate than random guess (larger than 80% precision for most systems studied). Our automated machine learning protocol shows that a “black-box” mode is possible for facilitating and accelerating the large-scale calculations on multireference systems where single-reference methods such as KS-DFT cannot be applied.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Active anomaly detection for time-domain discoveries

Aims. We present the first evidence that adaptive learning techniques can boost the discovery of unusual objects within astronomical light curve data sets. Methods. Our method follows an active learning strategy where the learning algorithm chooses objects which can potentially improve the learner if additional information about them is provided. This new information is subsequently used to update the machine learning model, allowing its accuracy to evolve with each new information. For the case of anomaly detection, the algorithm aims to maximize the number of scientifically interesting anomalies presented to the expert by slightly modifying the weights of a traditional Isolation Forest (IF) at each iteration. In order to demonstrate the potential of such techniques, we apply the Active Anomaly Discovery (AAD) algorithm to 2 data sets: simulated light curves from the Photometric LSST Astronomical Time-Series Classification Challenge (PLAsTiCC) and real light curves from the Open Supernova Catalog. We compare the AAD results to those of a static IF. For both methods, we performed a detailed analysis for all objects with the ~2% highest anomaly scores. Results. We show that, in the real data scenario , AAD was able to identify ~80% more true anomalies than the IF. This result is the first evidence that AAD algorithms can play a central role in the search for new physics in the era of large scale sky surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

A core lighting curriculum for university students and lighting professionals

Here, in collaboration with a group of lighting professionals, learning outcomes were defined, prioritized, organized and mapped to a three-course sequence of lighting courses within a Bachelor of Science in Architectural Engineering degree programme. Syllabi and educational exercises were developed to support the learning outcomes – including classroom activities, homework assignments and design projects. The learning exercises balance the technical foundations of applied illuminating engineering with the artistic aspects of applied lighting design and are intended to promote significant and lasting learning by providing students with education that is useful and relevant to current lighting practice. The process for identifying and prioritizing lighting content is described, a process that could be adapted by other lighting educators to other pedagogical contexts. The syllabi and learning exercises are disseminated for reuse or adaptation, or for self-study by independent learners.

42 ENGINEERING↗

Machine-learning-assisted material discovery of oxygen-rich highly porous carbon active materials for aqueous supercapacitors

Abstract Porous carbons are the active materials of choice for supercapacitor applications because of their power capability, long-term cycle stability, and wide operating temperatures. However, the development of carbon active materials with improved physicochemical and electrochemical properties is generally carried out via time-consuming and cost-ineffective experimental processes. In this regard, machine-learning technology provides a data-driven approach to examine previously reported research works to find the critical features for developing ideal carbon materials for supercapacitors. Here, we report the design of a machine-learning-derived activation strategy that uses sodium amide and cross-linked polymer precursors to synthesize highly porous carbons (i.e., with specific surface areas > 4000 m 2 /g). Tuning the pore size and oxygen content of the carbonaceous materials, we report a highly porous carbon-base electrode with 0.7 mg/cm 2 of electrode mass loading that exhibits a high specific capacitance of 610 F/g in 1 M H 2 SO 4 . This result approaches the specific capacitance of a porous carbon electrode predicted by the machine learning approach. We also investigate the charge storage mechanism and electrolyte transport properties via step potential electrochemical spectroscopy and quasielastic neutron scattering measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Probe microscopy is all you need *

We pose that microscopy offers an ideal real-world experimental environment for the development and deployment of active Bayesian and reinforcement learning methods. Indeed, the tremendous progress achieved by machine learning (ML) and artificial intelligence over the last decade has been largely achieved via the utilization of static data sets, from the paradigmatic MNIST to the bespoke corpora of text and image data used to train large models such as GPT3, DALL·E and others. However, it is now recognized that continuous, minute improvements to state-of-the-art do not necessarily translate to advances in real-world applications. We argue that a promising pathway for the development of ML methods is via the route of domain-specific deployable algorithms in areas such as electron and scanning probe microscopy and chemical imaging. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing. Favorable environment characteristics of scanning and electron microscopy include low risk, extensive availability of domain-specific priors and rewards, relatively small effects of exogenous variables, and often the presence of both upstream first principles as well as downstream learnable physical models for both statics and dynamics. Recent developments in programmable interfaces, edge computing, and access to application programming interfaces (APIs) facilitating microscope control, all render the deployment of ML codes on operational microscopes straightforward. We discuss these considerations and hope that these arguments will lead to create novel set of development targets for the ML community by accelerating both real world ML applications and scientific progress.

47 OTHER INSTRUMENTATION↗

Social Learning and Solar Photovoltaic Adoption

Growing literature points to the effectiveness of leveraging social interactions and nudges to spur adoption of prosocial behaviors. This study investigates a large-scale behavioral intervention designed to actively leverage social learning and peer interactions to encourage adoption of residential solar photovoltaic systems. Municipalities choose a solar installer offering group pricing and undertake an informational campaign driven by volunteer ambassadors. We find a causal treatment effect of 37 installations per municipality from the campaigns and no evidence of harvesting or persistence. The intervention also lowers installation prices. Randomized controlled trials based on the intervention show that selection into the program is important, whereas group pricing is not. Our results suggest that the program provided economies of scale and lowered consumer acquisition costs, leading to low-cost emission reductions. This paper was accepted by Matthew Shum, marketing.

Business & Economics↗

Data-Driven Day-Ahead PV Estimation Using Hybrid Deep Learning

Ongoing smart grid activities and associated automation resulted in rich set of data. Here, these data can be utilized for monitoring and estimation of real time photovoltaic (PV) generation. Inherent variability in PV and related impact on power systems is a challenging problem. Improving the accuracy of PV generation estimation is beneficial for both the PV owners and the grid operators. Recently, deep learning algorithms possible by the availability of data have shown its advantages for time series estimation; however, its application on PV generation estimation is still in the early stage. In this paper, a hybrid estimation model with a combination of long-short-term-memory network (LSTM) and persistence model (PM) is developed to provide day-ahead PV estimation at 15-minute time interval with high accuracy and robustness. Simulation results show the superior performance of the proposed method over existing methods for most of the test cases.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Machine Learning Predicts the Timing and Shear Stress Evolution of Lab Earthquakes Using Active Seismic Monitoring of Fault Zone Processes

Abstract Machine learning (ML) techniques have become increasingly important in seismology and earthquake science. Lab‐based studies have used acoustic emission data to predict time‐to‐failure and stress state, and in a few cases, the same approach has been used for field data. However, the underlying physical mechanisms that allow lab earthquake prediction and seismic forecasting remain poorly resolved. Here, we address this knowledge gap by coupling active‐source seismic data, which probe asperity‐scale processes, with ML methods. We show that elastic waves passing through the lab fault zone contain information that can predict the full spectrum of labquakes from slow slip instabilities to highly aperiodic events. The ML methods utilize systematic changes in P‐wave amplitude and velocity to accurately predict the timing and shear stress during labquakes. The ML predictions improve in accuracy closer to fault failure, demonstrating that the predictive power of the ultrasonic signals improves as the fault approaches failure. Our results demonstrate that the relationship between the ultrasonic parameters and fault slip rate, and in turn, the systematically evolving real area of contact and asperity stiffness allow the gradient boosting algorithm to “learn” about the state of the fault and its proximity to failure. Broadly, our results demonstrate the utility of physics‐informed ML in forecasting the imminence of fault slip at the laboratory scale, which may have important implications for earthquake mechanics in nature.

58 GEOSCIENCES↗

Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning (DL) research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high-dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace–Beltrami equation on a high-dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs with the softmax function as output activation, the new framework demonstrates the following major advantages: First, it is better applicable to data-efficient learning in which we train high capacity DNNs without using a large number of training data. Second, it remarkably improves both natural accuracy on the clean images and robust accuracy on the adversarial images crafted by both white-box and black-box adversarial attacks. Third, it is a natural choice for semi-supervised learning. This paper is a significant extension of our earlier work published in NeurIPS, 2018. For reproducibility, the code is available at https://github.com/BaoWangMath/DNN-DataDependentActivation .

Mathematics↗

Ensuring Workforce Readiness for the Energy Industry Through 3D Simulations

In February 2009, during his joint address to Congress, President Obama challenged every American to commit to at least one year of college education and set a goal that the United States should once again have the highest proportion of college graduates in the world by the year 2020. This push by the Obama administration reflected the understanding that while a college degree was becoming a prerequisite to obtain many of the jobs in the 21. century economy; the United States still lagged behind other countries with respect to higher education attainment. Because of this, the U.S. is facing a skills gap between the credentials required for the jobs of today and tomorrow, and the number of skilled workers eligible to fill those jobs. Within the energy industry, the skills gap is compounded by two factors: an expanding domestic industry in energy generation and transmission, and an aging workforce set to retire in large numbers. In fact, a 2013 survey administered by Center for Energy Workforce Development (CEWD) estimates that 36% of the workforce may need to be replaced in the next five years, and nearly 50% will need to be replaced by the year 2023. As a 2014 report by CEWD notes, 'The message for workforce development is clear: qualified and skilled utility technicians and engineers will continue to be in high demand in all types of energy generation, transmission, and distribution jobs'. Simulations provide students the ability to practice proper procedure in a safe environment, test out multiple scenarios to determine an optimal outcome, and repeat practice to solidify learning. Industry currently applies simulation (i.e., 3D Electrical Substation Simulation) as it affords an opportunity for technicians to train in a safe, virtual environment, thus eliminating risks associated with arc flash hazards. The intent of 3D simulation is to enhance the students' understanding of the concepts learned in the course and better prepare students for field work in a related technical discipline. The 3D simulation will reinforce the concepts learned during a specific course and are intended to add real world context to the curriculum. The 3D simulation will be designed such that a student will perform specific job activities incorporating concepts learned during the course. This method of instruction ensures students have an understanding of the concepts and are able to apply the concepts in a real world context. Different scenarios will be developed to ensure a thorough understanding of the concepts with real time feedback when there is a misunderstanding of the concepts. (authors)

97 MATHEMATICS AND COMPUTING↗

Predicting the propensity for thermally activated β events in metallic glasses via interpretable machine learning

Abstract The elementary excitations in metallic glasses (MGs), i.e., β processes that involve hopping between nearby sub-basins, underlie many unusual properties of the amorphous alloys. A high-efficacy prediction of the propensity for those activated processes from solely the atomic positions, however, has remained a daunting challenge. Recently, employing well-designed site environment descriptors and machine learning (ML), notable progress has been made in predicting the propensity for stress-activated β processes (i.e., shear transformations) from the static structure. However, the complex tensorial stress field and direction-dependent activation could induce non-trivial noises in the data, limiting the accuracy of the structure-property mapping learned. Here, we focus on the thermally activated elementary excitations and generate high-quality data in several Cu-Zr MGs, allowing quantitative mapping of the potential energy landscape. After fingerprinting the atomic environment with short- and medium-range interstice distribution, ML can identify the atoms with strong resistance or high compliance to thermal activation, at a high accuracy over ML models for stress-driven activation events. Interestingly, a quantitative “between-task” transferring test reveals that our learnt model can also generalize to predict the propensity of shear transformation. Our dataset is potentially useful for benchmarking future ML models on structure-property relationships in MGs.

36 MATERIALS SCIENCE↗