Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “generative machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

Machine learning based rate optimization under geologic uncertainty

We propose a novel approach for rate optimization during a waterflood under geologic uncertainty in reservoir properties such as permeability and porosity. The traditional approach typically involves several runs of the forward simulator. This may not scale well when the optimization is to be performed at the full field-level and over multiple geologic realizations. A machine-learning (ML) based approach which is quick and scalable for rate optimization over multiple geologic realizations is proposed instead. The training data for the model is generated by running the forward simulator with randomly assigned well rates using multiple geologic realizations. A reduced order representation of the permeability heterogeneity in each of the realizations is derived using a grid connectivity transformation (GCT). This step involves finding basis functions corresponding to the different modal frequencies of the grid connectivity represented by the grid Laplacian. The projection of the heterogeneous property field along these basis functions gives the basis coefficients that form the reduced order representation. Subsequently, for each training datapoint, streamlines are traced and the minimum time of flight (TOF) representing the tracer breakthrough time at each producer is recorded. The basis coefficients and well rates are fed to a machine learning model as input and the minimum TOF at the producers forms the output of the model. This trained model can then be used along with an optimizer for computing the optimal injection rates to maximize the injection sweep efficiency. This corresponds to minimizing the variance in the minimum TOF within each well group. Different architectures of neural network are tested using 5-fold cross validation to decide the best ML model to compute the streamline time of flight. The trained model is used to perform well rate optimization over multiple realizations of geology by using a risk tolerance penalty. The optimal well rates thus obtained are compared with two cases: a) equal well rates assigned to all injectors and producers and b) well rates obtained by optimizing over a single realization without considering the uncertainty in geology. The optimal well rates are seen to offer better oil recovery and sweep efficiency than both cases.

02 PETROLEUM↗

Bio-inspired gas sensing: boosting performance with sensor optimization guided by “machine learning”

The performance of existing gas sensors often degrades in field conditions because of the loss of measurement accuracy in the presence of interferences. Thus, new sensing approaches are required with improved sensor selectivity. We are developing a new generation of gas sensors, known as multivariable sensors, that have several independent responses for multi-gas detection with a single sensor. In this study, we analyze the capabilities of natural and fabricated photonic three-dimensional (3-D) nanostructures as sensors for the detection of different gaseous species, such as vapors and non-condensable gases. We employed bare Morpho butterfly wing scales to control their gas selectivity with different illumination angles. Next, we chemically functionalized Morpho butterfly wing scales with a fluorinated silane to boost the response of these nanostructures to the vapors of interest and to suppress the response to ambient humidity. Further, we followed our previously developed design rules for sensing nanostructures and fabricated bioinspired inorganic 3-D nanostructures to achieve functionality beyond natural Morpho scales. These fabricated nanostructures have embedded catalytically active gold nanoparticles to operate at high temperatures of ≈300 °C for the detection of gases for solid oxide fuel cell (SOFC) applications. Our performance advances in the detection of multiple gaseous species with specific nanostructure designs were achieved by coupling the spectral responses of these nanostructures with machine learning (a.k.a. multivariate analysis, chemometrics) tools. Our newly acquired knowledge from studies of these natural and fabricated inorganic nanostructures coupled with machine learning data analytics allowed us to advance our design rules for sensing nanostructures toward the required gas selectivity for numerous gas monitoring scenarios at room and high temperatures for industrial, environmental, and other applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Machine learning enabled discovery of application dependent design principles for two-dimensional materials

Abstract The unique electronic and mechanical properties of two-dimensional (2D) materials make them promising next-generation candidates for a variety of applications. Large-scale searches for high-performing 2D materials are limited to calculating descriptors with computationally demanding first-principles density functional theory. In this work, we alleviate this issue by extending and generalizing crystal graph convolutional neural networks to systems with planar periodicity and train an ensemble of models to predict thermodynamic, mechanical and electronic properties. We carry out a screening of nearly 45,000 structures for two separate applications: mechanical strength and photovoltaics. By collecting statistics of the screened candidates, we investigate structural and compositional design principles that impact the properties of the structures surveyed. Our approach recovers some well-accepted design rules: hybrid organic-inorganic perovskites with lead and tin tend to be good candidates for solar cell applications and titanium based MXenes usually have high stiffness coefficients. Interestingly, other members of the group 4 elements also contribute to increasing the mechanical strength of MXenes. For all-inorganic perovskites, we discover some compositions that have not been deeply studied in the field of photovoltaics and thus open up paths for further investigation. We open-source the code-base to spur further development in this space.

Venturi, Victor (ORCID:0000000156727549)↗

Explicit physics-informed neural networks for nonlinear closure: The case of transport in tissues

In upscaling methods, closures for nonlinear problems present a well-known challenge. While a number of theoretical methods have been proposed for handling such closures, nonlinearities still remain a significant obstacle for many problems. In this work, we use a combination of formal upscaling and data-driven machine learning for explicitly closing a nonlinear transport and reaction process in multiscale tissues. The classical effectiveness factor model is used to formulate the macroscale reaction kinetics. We train a multilayer perceptron network using training data generated by direct numerical simulations over microscale examples. Once trained, the network is used in an algorithm for numerically solving the upscaled (coarse-grained) differential equation describing mass transport and reaction in two example tissues. The network is described as being explicit in the sense that the network is trained using macroscale concentrations and gradients of concentration as components of the feature space rather than incorporating them as part of a constraint in the optimization process. Network training and solutions to the macroscale transport equations were computed for two different tissues. The two tissue types (brain and liver) exhibit markedly different geometrical complexity and spatial scale (cell size and sample size). The upscaled solutions for the average concentration are compared with numerical solutions derived from the microscale concentration fields by a posteriori averaging. There are three outcomes of this work of particular note. 1) Our overall approach results in an upscaled nonlinear PDE. The PDE is closed using a neural network, and our approach results in the definition of the classical effectiveness factor for effecting closure. 2) We identify particular source terms for the closure problem that are important for representing the structure of the closure. These source terms involve macroscale concentrations and their gradients. We adopt these source terms to use as explicit features in the learning algorithm. We find the trained networks that include the macroscale source terms generate models that are able to predict the correction factor with increased fidelity over those that do not. 3) We find that the trained network exhibits good generalizability, and it is able to predict the effectiveness factor with high fidelity for realistically-structured tissues despite the significantly different scale and geometrical complexity of the two example tissue types. This latter result emphasizes our purposeful connection between conventional averaging methods with the use of machine learning for closure; this contrasts with some machine learning methods for upscaling where the exact form of the macroscale equation remains unknown.

97 MATHEMATICS AND COMPUTING↗

Categorization of Radioxenon

This report summarizes a study into some false positive issues in the use of radioxenon as a method to verify a clandestine nuclear weapons explosion. False positives arise due to similarities between the radioxenon signature generated in medical isotope production and that generated in a nuclear weapon explosion. This report also discusses how to categorize the radioxenon by levels of urgency for manual analysis and interpretation and recommends applying machine learning and time series analysis techniques in the automation of radioxenon characterization. The literature indicates that medical isotope production is a major contributor to atmospheric radioxenon and is the main source of confusion in determining the source of radioxenon. While radioxenon emissions from nuclear power plants can be distinguished from that from nuclear weapon explosions, emissions from medical isotope production generate signatures similar to certain nuclide ratios found in nuclear weapons explosions. Different techniques for analyzing nuclide concentrations and ratios as well as including other sensing modalities via sensor fusion are discussed.

CTBTO↗

Data-driven modeling of power generation for a coal power plant under cycling

Increased penetration of renewables for power generation has negatively impacted the dynamics of conventional fossil fuel-based power plants. The power plants operating on the base load are forced to cycle, to adjust to the fluctuating power demands. This results in an inefficient operation of the coal power plants, which leads up to higher operating losses. To overcome such operational challenge associated with cycling and to develop an optimal process control, this work analyzes a set of models for predicting power generation. Moreover, the power generation is intrinsically affected by the state of the power plant components, and therefore our model development also incorporates additional power plant process variables while forecasting the power generation. We present and compare multiple state-of-the-art forecasting data-driven methods for power generation to determine the most adequate and accurate model. We also develop an interpretable attention-based transformer model to explain the importance of process variables during training and forecasting. The trained deep neural network (DNN) LSTM model has good accuracy in predicting gross power generation under various prediction horizons with/without cycling events and outperforms the other models for long-term forecasting. The DNN memory-based models show significant superiority over other state-of-the-art machine learning models for short, medium and long range predictions. The transformer-based model with attention enhances the selection of historical data for multi-horizon forecasting, and also allows to interpret the significance of internal power plant components on the power generation. This newly gained insights can be used by operation engineers to anticipate and monitor the health of power plant equipment during high cycling periods.

01 COAL, LIGNITE, AND PEAT↗

Learning Heterogeneous Features Jointly: A Deep End-to-End Framework for Multi-Step Short-Term Wind Power Prediction

Leveraging multiple heterogeneous measurements to predict wind power has long been a challenging task in the electrical community. In this paper, a deep architecture incorporated with multitask learning and multimodal learning for wind power prediction, termed predictive stacked autoencoder (PSAE), is presented. PSAE is a unified framework integrating multiple stacked autoencoders (SAEs), one feature fusion layer, and one prediction terminal layer, which expands the architecture from two spatial dimensions, including the depth and width, compared to conventional prediction models. Initially, the SAEs at the bottom of PSAE extracted features from multiple kinds of measurements respectively. Following, the feature fusion layer encodes the high-order features extracted by different SAEs into a unified feature that is more informative and representative for wind power prediction. Finally, the prediction terminal layer functions as a regression machine which generates the predicted targets based on the fusion features. Trained in an end-to-end (E2E) manner, PSAE is capable of learning heterogeneous features jointly and achieving the prediction task of sequence-to-sequence (S2S). Experiments for multi-step short-term predictions are conducted on real-world data, and the results demonstrate the superiority of PSAE to prior methods.

17 WIND ENERGY↗

Reinforced double-threaded slide-ring networks for accelerated hydrogel discovery and 3D printing

Traditionally, slide-ring gels are stretchable but soft as a result of an elasticity-stretchability trade-off. Herein, we introduce a new approach to breaking this trade-off and creating reinforced slide-ring networks with mobile crosslinkers. Our approach involves the construction of a polyethylene glycol double-threaded γ-cyclodextrin-based pro-slide-ring crosslinker that serves as a modular component for 3D printing and copolymerization. The resulting crystalline-domain-reinforced slide-ring hydrogels, or CrysDoS-gels, exhibit both high elasticity and high stretchability. The modular synthesis allows for high-throughput synthesis of CrysDoS-gels, generating a large amount of data for structure-property analysis. Here, by employing data science techniques, such as machine learning and linear regression, not only were we able to identify which chemical components influence the mechanical properties of CrysDoS-gels, but this analysis also aided in the discovery of better-performing CrysDoS-gels. Finally, we demonstrate the potential application of the newly discovered CrysDoS-gels as sensing devices by 3D printing them as stress sensors with high sensitivity and a broad detection range.

3D-printing↗

Skopi: a simulation package for diffractive imaging of noncrystalline biomolecules

X-ray free-electron lasers (XFELs) have the ability to produce ultra-bright femtosecond X-ray pulses for coherent diffraction imaging of biomolecules. While the development of methods and algorithms for macromolecular crystallography is now mature, XFEL experiments involving aerosolized or solvated biomolecular samples offer new challenges in terms of both experimental design and data processing. Skopi is a simulation package that can generate single-hit diffraction images for reconstruction algorithms, multi-hit diffraction images of aggregated particles for training machine learning classifiers using labeled data, diffraction images of randomly distributed particles for fluctuation X-ray scattering algorithms, and diffraction images of reference and target particles for holographic reconstruction algorithms. Skopi is a resource to aid feasibility studies and advance the development of algorithms for noncrystalline experiments at XFEL facilities.

59 BASIC BIOLOGICAL SCIENCES↗

Assurance of Reasoning Enabled Systems (ARES)

ARES was in part motivated by the determination of President’s Council of Advisors on Science and Technology (PCAST) on May 13th, 2023 that published a set of inquiries: In an era in which convincing images, audio, and text can be generated with ease on a massive scale, how can we ensure reliable access to verifiable, trustworthy information? How can we be certain that a particular piece of media is genuinely from the claimed source? What technologies, policies, and infrastructure can be developed to detect and counter AI-generated disinformation? In an effort to automatically analyze and patch/optimize code the work in this report describes various neural Machine Learning (ML) analysis engine implementations to assist in situations where source code is deficient or completely lacking to decompile (lift) binary code to ’C’. The goal is to gradually reduce human intervention. To this end, two Large Language Model (LLM) variants (Code LLama 2, LLama 3.1 and Starcoder1, Starcoder 2) where finetuned with ’before/after’ code pairs on the OpenBLAS library. LLama trained on the lowering process, Starcoder trained on the lifting process with National Security Agency’s (NSA) open-source Ghidra decompiler assist. The inferencing test results indicate correctness for only very short sequences for Starcoder 2. Moving forward, the experiments conclude with a set of recommendations of required resources and technologies

97 MATHEMATICS AND COMPUTING↗

A global DSCOVR/EPIC-based hourly/daily shortwave radiation/PAR dataset

Accurate characterization of the global downward shortwave (SW) and photosynthetically active radiation (PAR) is fundamental for Earth system modeling and global change research. Combined with a machine-learning method, we used the Earth Polychromatic Imaging Camera (EPIC) data onboard the Deep Space Climate Observatory (DSCOVR) to generate the global SW/PAR dataset at both hourly and daily temporal scales and 0.1degree×0.1degree spatial resolution. Covering a period from June, 2015 to June (about 4 years), 2019, the dataset consists of total SW/PAR and their direct and diffuse components. Our dataset will be valuable for understanding the diurnal cycles and seasonal patterns of ecosystem processes, as well as the solar energy utilizations

14 SOLAR ENERGY↗

Data reduction through optimized scalar quantization for more compact neural networks

Raw data generation for several existing and planned large physics experiments now exceeds TB/s rates, generating untenable data sets in very little time. Those data often demonstrate high dimensionality while containing limited information. Meanwhile, Machine Learning algorithms are now becoming an essential part of data processing and data analysis. Those algorithms can be used offline for post processing and post data analysis, or they can be used online for real time processing providing ultra low latency experiment monitoring. Both use cases would benefit from data throughput reduction while preserving relevant information: one by reducing the offline storage requirements by several orders of magnitude and the other by allowing ultra fast online inferencing with low complexity Machine Learning models. Moreover, reducing the data source throughput also reduces material cost, power and data management requirements. In this work we demonstrate optimized nonuniform scalar quantization for data source reduction. This data reduction allows lower dimensional representations while preserving the relevant information of the data, thus enabling high accuracy Tiny Machine Learning classifier models for online fast inferences. We demonstrate this approach with an initial proof of concept targeting the CookieBox, an array of electron spectrometers used for angular streaking, that was developed for LCLS-II as an online beam diagnostic tool. We used the Lloyd-Max algorithm with the CookieBox dataset to design an optimized nonuniform scalar quantizer. Optimized quantization lets us reduce input data volume by 69% with no significant impact on inference accuracy. When we tolerate a 2% loss on inference accuracy, we achieved 81% of input data reduction. Finally, the change from a 7-bit to a 3-bit input data quantization reduces our neural network size by 38%.

97 MATHEMATICS AND COMPUTING↗

A Fast Framework for Generating Radioactive Mixture Spectra and Its Application to Remote High-Performance Mixture Identification

Remote detection of radioactive materials in mixtures using handheld or portal detectors remains a challenge because of factors such as low concentration, environmental interference, sensor noise, and other complications. This work introduces a fast framework for generating realistic mixture spectra. Moreover, we present mixture isotope identification using data generated by the fast framework. Researchers have examined a range of conventional and recent algorithms within the fields of machine learning and deep learning. An application to uranium enrichment-level prediction has been included. Extensive simulation experiments validated the efficacy of the proposed framework.

GADRAS↗

Enhancing Air Quality Applications in the Hindu Kush-Himalayan Region Using Satellite, Model, and Machine Learning Techniques

Air pollution in the Hindu Kush Himalayan (HKH) region of South Asia is a severe issue, as increases in emissions over the past two decades have degraded air quality (AQ) across the region, which poses major threats to human health, the ecosystem, climate, and agriculture. A diversity of anthropogenic and natural emission sources including transportation, power plants, industries, open biomass burning of crop residue, forest fires, cooking and heating fires, and dust storms contribute to unhealthy AQ and transboundary pollution issues in the region. Further complicating matters is the importance of meteorology and terrain on AQ, especially in the Kathmandu Valley where extreme haze episodes frequently develop from the atmospherically stable weather conditions during the winter monsoon. This study uses state-of-the-art satellite observations and modeling capabilities in conjunction with machine learning techniques to develop a comprehensive toolkit for enhancing AQ monitoring and forecasting in HKH. The toolkit incorporates new generation satellite observations from the TROPOspheric Monitoring Instrument (TROPOMI), Geostationary Environment Monitoring Spectrometer (GEMS), and Advanced Meteorological Imager (AMI), which provide unprecedented resolution on aerosols and trace gases, including nitrogen dioxide (NO 2 ), formaldehyde (CH2O), sulfur dioxide (SO 2 ), carbon monoxide (CO), and ozone (O 3 ), and aerosol optical depth (AOD). Value-added products [e.g., Particulate matter with diameters less than 2.5 micrometers (PM2.5)] are developed from the suite of satellite observations to further improve AQ monitoring capabilities in the region. The satellite products are also used to assimilate a high-resolution chemical transport model tailored for the HKH region, which is providing daily, 54-hour AQ forecasts with horizontal grid spacings of 12- and 4-km. This presentation will provide an overview of the suite of satellite- and model-based products in the AQ toolkit and application and performance of the toolkit for AQ monitoring and forecasting in HKH.

Air Quality↗

Snow Depth from AMSR-2 Using Multispectral Satellite Data in an Artificial Neural Network

By using diffusion theory and Monte Carlo lidar radiative transfer simulations, Hu et al. (2022b) has derived snow depth from the first-, second- and third-order moments of the lidar backscattering pathlength distribution. Lu et al. (2022) calculated the snow depth by applying the methods to the satellite ICESat-2 lidar measurements over the Arctic sea ice, as well as land surfaces of Northern Hemisphere. In this paper, an artificial neural network (ANN) algorithm, employing several channels from Advanced Microwave Scanning Radiometer 2 (AMSR-2) and the humidity vertical profiles from Global Modeling and Assimilation Office (GMAO) Goddard Earth Observing System for Instrument Teams (GEOS-IT) product, is trained to determine snow depth identified by time and geolocation matched 2019 ICESat-2 snow-depth data during winter months over the Arctic sea ice. The trained ANN snow-depth was applied to 2018 AMSR-2 clear pixel data, although the algorithms perform reasonably well in thinner clouds. The validation data (different from the training set) of ANN snow depth from AMSR-2 showed a good agreement with time matched and co-located snow-depth values from ICESat-2. The bias was near zero, with mean absolute error (MAE) 0.05 cm and a root-mean-square-error (RMSE) 0.08 cm. Prior applying the trained ANN snow depth to AMSR-2 data, a cloud screening algorithm was developed with a similar approach. A separate ANN cloud mask was trained to determine an AMSR-2 pixel is clear or cloudy with time and geolocation matched 2015 CALIOP Vertical Feature Mask (VFM) over Arctic sea ice. The ANN cloud mask from AMSR-2 under-estimated cloud fraction by 3-6% compared to CALIOP . The additional research is needed to conclusively evaluate the ANN cloud mask accuracy. Finally, this paper will lay the foundation for a sustained long-term snowfall and snow-storm monitoring system. The future Cloud Aerosol LIdar for Global scale Observations of the ocean-Land Atmosphere system (CALIGOLA) mission will provide a means to calculate snow depth from the lidar backscattering pathlength distribution, benefiting from the UV, visible and infrared pulses. With the calculated snow depth as the truth one could develop a machine learning algorithm, as it was done in this paper, using a passive microwave instrument available at that time to generate a wide range of snow depth data, covering extensive spatial areas in the cross-orbit direction.

Neural Network↗

Enhancing Air Quality Applications in the Hindu Kush-Himalayan Region Using Satellite, Model, and Machine Learning Techniques

Air pollution in the Hindu Kush Himalayan (HKH) region of South Asia is a severe issue, as increases in emissions over the past two decades have degraded air quality (AQ) across the region, which poses major threats to human health, the ecosystem, climate, and agriculture. A diversity of anthropogenic and natural emission sources including transportation, power plants, industries, open biomass burning of crop residue, forest fires, cooking and heating fires, and dust storms contribute to unhealthy AQ and transboundary pollution issues in the region. Further complicating matters is the importance of meteorology and terrain on AQ, especially in the Kathmandu Valley where extreme haze episodes frequently develop from the atmospherically stable weather conditions during the winter monsoon. This study uses state-of-the-art satellite observations and modeling capabilities in conjunction with machine learning techniques to develop a comprehensive toolkit for enhancing AQ monitoring and forecasting in HKH. The toolkit incorporates new generation satellite observations from the TROPOspheric Monitoring Instrument (TROPOMI), Geostationary Environment Monitoring Spectrometer (GEMS), and Advanced Meteorological Imager (AMI), which provide unprecedented resolution on aerosols and trace gases, including nitrogen dioxide, formaldehyde, sulfur dioxide, carbon monoxide, and ozone, and aerosol optical depth. Value-added products, such as level 4 PM2.5 products, are developed from the suite of satellite observations to further improve AQ monitoring capabilities in the region. The satellite products are also used to assimilate a high-resolution chemical transport model tailored for the HKH region, which is providing daily, 54-hour AQ forecasts with horizontal grid spacings of 12- and 4-km. This presentation will provide an overview of the suite of satellite- and model-based products in the AQ toolkit and application and performance of the toolkit for AQ monitoring and forecasting in HKH.

Forecasting↗

Perspective on Solar Energy Materials, Design and Discovery, Defects, Disorder, and Interfaces

The energy transition still faces a daunting math. Currently, more than three quarters of final energy consumption occurs in form of fuels vs less than one quarter in electricity. On the other hand, renewable energy additions come almost exclusively in the form of electricity (dominantly photovoltaics and wind). Thus, meeting the terawatt challenge requires enormous growth in renewables, sufficient to convert excess electricity into fuels, as well as the development of non-electricity based solar fuel technologies, for example via solar thermochemical routes. In this presentation, we will review successes and challenges in photovoltaic materials from silicon to CdTe to halide perovskites and potential emerging materials, with a particular perspective on the role of first principles calculations and predictions. Materials design and discovery is evolving to incorporate defects, disorder, and interfaces. These phenomena play also an important role in the thermochemical generation of hydrogen, allowing to utilize synergies between renewable electricity and fuels in computational research. At the same time, machine learning approaches are increasingly geared toward properties of non-ideal materials, for example defect formation, allowing the screening for more complex material behavior as needed for the respective applications.

fuels↗

Surrogate-driven Variance-based Sensitivity Analysis of Thermal Storage Tanks in Integrated Energy Systems

Sensitivity analysis and uncertainty quantification are essential steps for enhancing the accuracy of computational models by identifying and mitigating uncertainties. This study focuses on these steps for the Thermal Energy Delivery System at Idaho National Laboratory, specifically targeting the thermocline tank. Using a Modelica/Dymola simulation model, the study perturbed various design parameters and boundary conditions, including shape factor, porosity, outlet temperature, inlet mass flow rate, and system pressure, to predict and quantify uncertainty in the tank’s ax- ial temperature. A dataset of over 1,000 simulations was generated, and surrogate models were developed using the pyMAISE (Michigan Artificial Intelligence Standard Environment) library, which is an Automatic Machine Learning library for nuclear engineering applications. The optimal model, a feedforward neural network with two hidden layers, achieved an R2 score above 0.99 and a mean absolute error below 1 Kelvin. Sensitivity analyses using Sobol indices and Fourier amplitude sensitivity testing methods on this surrogate model revealed that the inlet mass flow rate at initial timestamps and porosity significantly impacts predicted temperatures across all sensors and time steps.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗