Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Active Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Weighted active space protocol for multireference machine-learned potentials

Multireference methods such as multiconfiguration pair-density functional theory accurately capture electronic correlation in systems with strong multiconfigurational character, but their cost precludes direct use in molecular dynamics. Combining these methods with machine-learned interatomic potentials (MLPs) can extend their reach. However, the sensitivity of multireference calculations to the choice of the active space complicates the consistent evaluation of energies and gradients across structurally diverse nuclear configurations. To overcome this limitation, we introduce the weighted active space protocol (WASP), a systematic approach to assign a consistent active space for a given system across uncorrelated configurations. By integrating WASP with MLPs and enhanced sampling techniques, we propose a data-efficient active learning cycle that enables the training of an MLP on multireference data. We demonstrated the approach on the TiC + -catalyzed C–H activation of methane, a reaction that poses challenges for Kohn–Sham density functional theory due to its significant multireference character. This framework enables accurate and efficient modeling of catalytic dynamics, establishing a paradigm for simulating complex reactive processes beyond the limits of conventional electronic-structure methods.

enhanced sampling↗

Improving BC Mixing State and CCN Activity Representation With Machine Learning in the Community Atmosphere Model Version 6 (CAM6)

Abstract Representing mixing state of black carbon (BC) is challenging for global climate models (GCMs). The Community Atmosphere Model version 6 (CAM6) with the four‐mode version of the Modal Aerosol Module (MAM4) represents aerosols as fully internal mixtures with uniform composition within each aerosol mode, resulting in high degree of internal mixing of BC with non‐BC species and large mass ratio of coating to BC ( R BC , the mass ratio of non‐BC species to BC in BC‐containing particles). To improve BC mixing state representation, we coupled a machine learning (ML) model of BC mixing state index trained on particle‐resolved simulations to the CAM6 with MAM4 (MAM4‐ML). In MAM4‐ML, we use R BC to partition accumulation mode particles into two new modes, BC‐free particles and BC‐containing particles. We adjust R BC to make the modeled BC mixing state index ( χ mode ) match the one predicted by the ML model ( χ ML ). On a global average, the mass fraction of BC‐containing particles in accumulation mode decreases from 100% (MAM4‐default) to 48% (MAM4‐ML). The globally averaged χ mode decreases from 78% (MAM4‐default) to 63% (MAM4‐ML, 19% reduction) and agrees well with χ ML (66%). The R BC decreases by 52% for accumulation mode and better agrees with observations. The hygroscopicity drops by 9% for BC‐containing particles in accumulation mode, leading to a 20% reduction in the BC activation fraction. The surface BC concentration increases most (6.9%) in the Arctic, and the BC burden increases by 4%, globally. Our study highlights the application of the ML model for improving key aerosol processes in GCMs.

Meteorology & Atmospheric Sciences↗

“Inverting” X-ray Absorption Spectra of Catalysts by Machine Learning in Search for Activity Descriptors

The rapid growth of methods emerging in the past decade for synthesis of “designer” catalysts—ranging from the size and shape-selected nanoparticles to mass-selected clusters, to precisely engineered bimetallic surfaces, to single site and pair site catalysts—has opened opportunities for tailoring the catalyst structure for the desired activity and selectivity. It has also sharpened the need for developing approaches to the operando characterization, ones that identify the catalytic active sites and follow their evolutions in reaction conditions. Commonly used methods for determination of the activity descriptors in the nanocatalysts, based on the correlation between the changes in catalyst performance and evolution of its structural and electronic properties, are hampered by the paucity of experimental techniques that can detect such properties with high accuracy and in reaction conditions. Out of many such techniques, X-ray absorption spectroscopy (XAS) stands out as an element-specific method that is very sensitive to the local geometric and electronic properties of the metal atoms and their surroundings and, therefore, is able to track catalyst structure modifications in operando conditions. Despite the vast amount of structure-specific information (such as, e.g., the charge states and radial distribution function of neighbors of selected atomic species) stored in the XAS data of catalysts, extracting it from the spectra is challenging, especially in the conditions of low metal weight loading, nanoscale dimensions, heterogeneous size and composition distributions, and harsh reaction environment. In this Perspective, we discuss the recent developments in XAS data analysis achieved by employing supervised and unsupervised machine learning (ML) methods for structural characterization of catalysts. By benefiting from the sensitivity of ML methods to subtle variations in experimental data, a previously “hidden” relationship between the X-ray absorption spectrum and descriptors of material’s structure and/or composition can be found, as illustrated on representative examples of mono-, hetero-, and nonmetallic catalysts. In the case of supervised ML, the experimental spectra can be rapidly “inverted”, and the structure of the catalyst can be tracked in real time and in reaction conditions. Finally, emerging opportunities for catalysis research that the ML methods enable, such as high-throughput data analysis, and their applications to other experimental probes of catalyst structure are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Novel Active Optimization Approach for Rapid and Efficient Design Space Exploration Using Ensemble Machine Learning

In this work, a novel design optimization technique based on active learning, which involves dynamic exploration and exploitation of the design space of interest using an ensemble of machine learning algorithms, is presented. In this approach, a hybrid methodology incorporating an explorative weak learner (regularized basis function model) that fits high-level information about the response surface and an exploitative strong learner (based on committee machine) that fits finer details around promising regions identified by the weak learner is employed. For each design iteration, an aristocratic approach is used to select a set of nominees, where points that meet a threshold merit value as predicted by the weak learner are selected for evaluation. In addition to these points, the global optimum as predicted by the strong learner is also evaluated to enable rapid convergence to the actual global optimum once the most promising region has been identified by the optimizer. Additionally, this methodology is first tested by applying it to the optimization of a two-dimensional multi-modal surface and, subsequently, to a complex internal combustion (IC) engine combustion optimization case with nine control parameters related to fuel injection, initial thermodynamic conditions, and in-cylinder flow. It is found that the new approach significantly lowers the number of function evaluations that are needed to reach the optimum design configuration (by up to 80%) when compared to conventional optimization techniques, such as particle swarm and genetic algorithm-based optimization techniques.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Forecasting influenza activity using machine-learned mobility map

Human mobility is a primary driver of infectious disease spread. However, existing data is limited in availability, coverage, granularity, and timeliness. Data-driven forecasts of disease dynamics are crucial for decision-making by health officials and private citizens alike. In this work, we focus on a machine-learned anonymized mobility map (hereon referred to as AMM) aggregated over hundreds of millions of smartphones and evaluate its utility in forecasting epidemics. We factor AMM into a metapopulation model to retrospectively forecast influenza in the USA and Australia. We show that the AMM model performs on-par with those based on commuter surveys, which are sparsely available and expensive. We also compare it with gravity and radiation based models of mobility, and find that the radiation model’s performance is quite similar to AMM and commuter flows. Additionally, we demonstrate our model’s ability to predict disease spread even across state boundaries. Our work contributes towards developing timely infectious disease forecasting at a global scale using human mobility datasets expanding their applications in the area of infectious disease epidemiology.

60 APPLIED LIFE SCIENCES↗

Active Optical Control with Machine Learning: A Proof of Concept for the Vera C. Rubin Observatory

Abstract The Active Optics System of the Vera C. Rubin Observatory (Rubin) uses information provided by four wave front sensors to determine deviations between the reconstructed wave front and the ideal wave front. The observed deviations are used to adjust the control parameters of the optical system to maintain image quality across the 3.°5 field of view. The baseline approach from the project is to obtain amplitudes of the Zernike polynomials describing the distorted wave front from out-of-focus images collected by the wave front sensors. These Zernike amplitudes are related via an “influence matrix” to the control parameters necessary to correct the wave front. In this paper, we use deep-learning methods to extract the control parameters directly from the images captured by the wave front sensors. Our neural net model uses anti-aliasing pooling to boost performance, and a domain-specific loss function to aid learning and generalization. The accuracy of the control parameters derived from our model exceeds Rubin requirements even in the presence of full-moon background levels and mis-centering of reference stars. Although the training process is time consuming, model evaluation requires only a few milliseconds. This low latency should allow for the correction of the optical configuration during the readout and slew interval between successive exposures.

Astronomy & Astrophysics↗

Findings on Subtask 3.1 - Bakken Rich Gas Enhanced Oil Recovery Project

Total in-place oil for the Bakken petroleum system (BPS) (which includes the Bakken and Three Forks Formations) has been estimated to be 600 billion barrels (bbl). However, BPS wells have decline rates as high as 85% over the first 3 years of their lives, and primary recovery factors typically range from 3% to 10% of original oil in place. Given the low initial recovery rates, even small incremental productivity improvements could dramatically increase technically recoverable oil in the BPS. One potential solution is enhanced oil recovery (EOR) using gas injection, such as carbon dioxide (CO2) or hydrocarbon (HC) gases. While commonly used in conventional reservoirs, CO2 EOR in unconventional tight oil reservoirs has been limited to pilot tests. EOR using rich gas (mixture of methane, ethane, and propane) has also been employed in numerous pilots in several unconventional plays and has recently been successfully applied in the Eagle Ford play. If successful, large-scale gas-based EOR in the BPS could dramatically increase oil productivity and recovery factors and extend the life of the play for decades. While CO2 may be a technically suitable working fluid for EOR in the BPS, supplies are limited and costs for using CO2 in EOR pilots are prohibitively high. Meanwhile, produced gas flaring has presented challenges for BPS operators in North Dakota. Analysis conducted by the North Dakota Pipeline Authority indicates that the current gas-gathering infrastructure in North Dakota is insufficient to accommodate all of the associated gas that is produced from the BPS. The geographically isolated location of North Dakota relative to large natural gas markets, combined with suppressed natural gas prices, has made it economically challenging for industry to invest capital in expanding gas-gathering infrastructure in the state. These circumstances led to a research program conducted by the Energy & Environmental Research Center (EERC) in partnership with Liberty Resources Management Company LLC (LR) to examine the potential to use rich gas injection for EOR and mitigate flaring. A rich gas EOR pilot test was designed and executed by LR at its Stomping Horse development area in Williams County, North Dakota. From July 2018 through May 2019, a total of 160 million standard cubic feet (MMscf) of rich produced gas was injected into the BPS using five different wells in a sequential injection strategy. LR’s Leon–Gohrick drill spacing unit (DSU) was used as the test site. Regulatory oversight was provided by the North Dakota Industrial Commission (NDIC). Technical support was provided by the EERC through a series of laboratory, modeling, and field-based activities, and additional post-pilot research activities incorporated learnings from the test, developed new laboratory data, improved fracture modeling methods, and developed machine learning and big data analytics. The results from the Stomping Horse rich gas EOR pilot activities indicate that developing an effective, economical EOR approach for the BPS will require more field tests. Another key lesson learned from the Stomping Horse tests is that detailed pre- and posttest data on reservoir conditions and fluids production are essential. Robust reservoir characterization provides information that is crucial to creating realistic geomodels and conducting valid dynamic simulations of potential EOR scenarios. A detailed understanding of the completions and production history of offset wells is also necessary for valid test result interpretations. This knowledge is essential to designing the operational parameters of injectivity tests and interpreting the results. A conformance control strategy is also essential to success. Laboratory-based examinations of rich gas interactions with reservoir fluids and rocks were conducted, with an emphasis on determining the ability to mobilize oil in the tight reservoir rocks and shales of the BPS. Injection fluid composition was shown to have a positive impact on reducing reservoir oil minimum miscibility pressure (MMP), reducing interfacial tension (IFT), and altering wettability. IFT and contact angle measurements demonstrated that wettability can be altered in the presence of rich gas, suggesting the potential to improve oil recovery. Iterative modeling of surface infrastructure and reservoir performance using data generated by the various project activities was conducted. A geologic model of the Stomping Horse area was built; history-matched oil, gas, and water production was used in simulations of various EOR scenarios. Early programmatic modeling results were used to support LR’s design and operation of the EOR pilot and to provide insight regarding optimization of future commercial-scale BPS EOR design and operations. Post-pilot modeling focused on alternative methods of understanding complex fracture networks and accelerating simulation time. These led to improved simulation run times and provide excellent history-matching results. Several of these iterative models were used as the bases for developing algorithms into machine learning and big data analytics. History matching in reservoir simulation is time-consuming and computer processing-intensive. Machine learning algorithms were created, and an automated history-matching tool was developed. A large set of synthetic reservoir simulations were created to generate well responses (oil, gas, and water production, well bottomhole pressure [BHP], and tracer or propane breakthrough) for a set of EOR operating parameters that included offset well status (open or closed), injectate (rich gas or propane), injection rate, and injection well BHP. A user interface was developed to provide real-time visualization. Machine learning-based models were developed to provide rapid forecasting of well performance given a set of user-defined EOR operating parameters. These predictive models allow the user to modify the offset well status, injection rate, and injection well BHP and rapidly forecast future production performance. The combination of real-time visualization tools with real-time forecasting tools provides a framework for real-time control—operational changes that the EOR site operator can enact (e.g., changing gas injection rates) to affect the observed performance and potentially improve the EOR outcome. There is great reason to be optimistic about the future of EOR in the Bakken. The results of the laboratory studies suggest significant potential for high rates of oil mobilization using produced field gas injection under the right conditions. The results of the lab studies, combined with rigorous statistical analysis of well production data and associated modeling efforts, confirm the notion that fluid mobility within the reservoir is controlled by fractures. As more knowledge is gained about the nature and distribution of fracture networks in the Bakken, the industry will be in a better position to predict and, ultimately, influence fluid mobility. New field tests are necessary to develop a more complete understanding of those conditions. Thoughtful and creatively engineered field tests within a well-characterized geologic setting will yield the fundamental knowledge needed to take Bakken oil production to the next level. This subtask was cofunded through the EERC–U.S. Department of Energy Joint Program on Research and Development for Fossil Energy-Related Resources Cooperative Agreement No. DE-FE0024233. Nonfederal funding was provided by the North Dakota Industrial Commission’s Oil and Gas Research Program and Computer Modelling Group.

04 OIL SHALES AND TAR SANDS↗

Visualization System for Evolutionary Neural Networks for Deep Learning

Deep learning is actively used in a wide range of fields for scientific discovery. To effectively apply deep learning to a particular problem, it is important to select an appropriate network architecture and other hyper-parameters (at each layer). Evolving architectures and hyper-parameters using a genetic algorithm is one current approach to search the huge space of all possible configurations to find those more optimal for the problem. However, examining an evolutionary process and tuning the genetic algorithm are challenging, pushing most users to treat the process as a black box. To address this challenge, we propose a visualization system for evolutionary neural networks for deep learning. The key feature of our visualization system is to provide a visual analytics environment for evaluating a genetic algorithm in order to improve the underlying operations to reduce time to find good solutions. Our system is able to not only visualize how a genetic algorithm traverses its search space but also allows users to examine evolving networks in-depth to get insights to improve performance through interactive visualization components.

Chae, Junghoon↗

Hampshire College Center for Science Education. Final Report on Activities Supported by the Department of Energy Grant No. DE-FG02-06ER64256

Hampshire College's Center for Science Education (Center) focuses on teacher professional development, curriculum development, and student enrichment programs. The Center also maintains research programs on teacher change, student learning and instructional effectiveness. The Center's work promotes learning that persists over time and transfers to new situations in and out of school. The projects develop the implications of the increasing agreement among teachers and researchers that effective learning involves active concept mastery and consistent practice with inquiry and critical thinking. The Center's objective is to help strengthen the pipeline of U.S. students pursuing postsecondary study in STEM fields. The Center achieves this by fostering an educational environment in which science is taught as an active, directly experienced endeavor across the K-16 continuum. Too often, young people are dissuaded from pursuing science because they do not see its relevance, instead experiencing it as dry, rote, technical. In contrast, when science is taught as a hands-on, inquiry-driven process, students are encouraged to ask questions grounded in their own curiosity and seek experimental solutions accordingly. In this way, they quickly discover both the profound relevance of science to their daily lives and its accessibility to them. Essentially, they learn to think and act like real scientists. The Center’s approach is multi-faceted: it includes direct inquiry-based science instruction to secondary and postsecondary students, educating the next generation of teachers, and providing new educational opportunities for teachers already working in the schools. Funding from the Department of Energy focused on the last population, enabling in-service teachers to explore and experience the pedagogy of inquiry-based science for themselves, and to take it back to their classrooms and students. The Center has demonstrated that the inquiry-based approach to science learning is compatible with existing state curriculum frameworks and produces students who understand and are positively inclined toward science. Funds from this Department of Energy grant supported three projects that involved K-16 science outreach: 1. Teaching Issues and Experiments in Ecology (TIEE). TIEE a peer-reviewed online journal and curriculum resource for postsecondary science teachers. 2. The Collaboration for Excellence in Science Education (CESE). CESE is a partnership with the Amherst, Massachusetts school system to foster the professional development of science teachers, and to perform research on student learning in the sciences and on teacher change. The project draws on Hampshire's long experience with inquiry-oriented and interdisciplinary education, as well as on its unique strengths in cognitive science. The project is run as design research, working with teachers to improve their practices and studying student and/or teacher outcomes. 3. Day in the Lab. Grant funds partially supported the expansion of the ongoing science outreach activities of the School of Natural Science. These activities are focused on local districts with large minority enrollments, including the Amherst, Holyoke and Springfield Public School Districts, and the Pioneer Valley Performing Arts Charter School (PVPA). Each of the three projects supported by the grant met or exceeded its goals. In part, the successes we met were due to continuity and communication among the staff of the programs. At the beginning of the CESE project, a science outreach coordinator was recruited. He worked throughout the grant period along with a senior researcher and the project's curriculum director. Additionally, the director and an undergraduate student conducted research on teacher change. The science outreach coordinator acted as a liaison among Hampshire College, the school districts, and a number of local businesses and agencies, providing organizational support, discussion facilitation, classroom support for teachers, and materials purchase. His presence in the schools kept teachers engaged and supported. He also brought the PVPA Charter School into the project. He worked closely with the educational outreach coordinator at Hampshire who oversaw the Day in the Lab program. Together, they have ensured the continuity of support to the schools through the use and placement of student interns. Finally, the director and coordinators worked with the Hitchock Center for the Environment to bring the two science professional development efforts in Amherst together. The joint development of workshops for elementary teachers was extremely successful. A major reason for the successes of the CESE program was the strength of the teacher outreach team and the sheer number of hours spent building relationships, talking about teaching and learning, planning projects, developing curriculum, and working with experts throughout the Pioneer Valley.

99 GENERAL AND MISCELLANEOUS↗

Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II

Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000$\times$. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.

Coletti, Mark↗

Machine learning to accelerate screening for Marcus reorganization energies

Understanding and predicting the charge transport properties of π-conjugated materials is an important challenge for designing new organic electronic devices, such as solar cells, plastic transistors, light-emitting devices, and chemical sensors. A key component of the hopping mechanism of charge transfer in these materials is the Marcus reorganization energy which serves as an activation barrier to hole or electron transfer. While modern density functional methods have proven to accurately predict trends in intramolecular reorganization energy, such calculations are computationally expensive. In this work, we outline active machine learning methods to predict computed intramolecular reorganization energies of a wide range of polythiophenes and their use toward screening new compounds with low internal reorganization energies. Our models have an overall root mean square error (RMSE) of ±0.113 eV, but a much smaller RMSE of only ±0.036 eV on the new screening set. Since the larger error derives from high-reorganization energy compounds, the new method is highly effective to screen for compounds with potentially efficient charge transport parameters.

14 SOLAR ENERGY↗

Benchmarking the performance of uncertainty quantification methods for neural network-based interatomic potentials

Machine-learned interatomic potentials (ML-IAPs) continue to gain popularity as accurate, computationally efficient replacements for traditional, physics-based interatomic potentials and expensive ab initio methods. Uncertainty quantification (UQ) of ML-IAPs is a growing area of research as UQ is critical in many applications of IAPs, such as developing curated datasets, active learning-based data augmentation, self-improving models, and estimating the uncertainty of molecular dynamics simulations. In this paper, we construct and benchmark a series of different neural network potentials (NNPs) with varying network architectures to determine the performance of these models with respect to both the mean and uncertainty calibration error. Each NNP method is specifically designed to predict either epistemic or aleatoric uncertainty with particular focus on the differences in behavior between the epistemic and aleatoric uncertainty estimates. We benchmark these methods using multiple datasets common in the ML-IAP literature. The results show that the aleatoric uncertainty from single-shot model architectures is a competitive alternative to ensemble-based epistemic uncertainty predictions in regions of sufficient data-density. However, in regions where the representative data is sparse, aleatoric uncertainty models tend to overpredict and epistemic methods tend to underpredict the actual model error. We conclude that the type of UQ is crucial when discussing performance of probabilistic model results as different methods have different performance characteristics depending on the regime in which they are evaluated. Therefore, the type of UQ method should be carefully evaluated against both the data characteristics and requirements for the intended application.

97 MATHEMATICS AND COMPUTING↗