Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Hierarchical deep reinforcement learning reveals a modular mechanism of cell movement

Time-lapse images of cells and tissues contain rich information about dynamic cell behaviours, which reflect the underlying processes of proliferation, differentiation and morphogenesis. However, we lack computational tools for effective inference. Here we exploit deep reinforcement learning (DRL) to infer cell–cell interactions and collective cell behaviours in tissue morphogenesis from three-dimensional (3D) time-lapse images. We use hierarchical DRL (HDRL), known for multiscale learning and data efficiency, to examine cell migrations based on images with a ubiquitous nuclear label and simple rules formulated from empirical statistics of the images. When applied to Caenorhabditis elegans embryogenesis, HDRL reveals a multiphase, modular organization of cell movement. Imaging with additional cellular markers confirms the modular organization as a novel migration mechanism, which we term sequential rosettes. Furthermore, HDRL forms a transferable model that successfully differentiates sequential rosettes-based migration from others. Our study demonstrates a powerful approach to infer the underlying biology from time-lapse imaging without prior knowledge.

59 BASIC BIOLOGICAL SCIENCES↗

The case for digital twins in metal additive manufacturing

The digital twin (DT) is a relatively new concept that is finding increased acceptance in industry. A DT is generally considered as comprising a physical entity, its virtual replica, and two-way digital data communications in-between. Its primary purpose is to leverage the process intelligence captured within digital models—or usually their faster-solving surrogates—towards generating increased value from the physical entities. The surrogate models are created using machine learning based on data obtained from the field, experiments and digital models, which may be physics-based or statistics-based. Anomaly detection and correction, and diagnostic closed-loop process control are examples of how a process DT can be deployed. In the manufacturing industry, its use can achieve improvements in product quality and process productivity. Metal additive manufacturing (AM) stands to gain tremendously from the use of DTs. This is because the AM process is inherently chaotic, resulting in poor repeatability. However, a DT acting in a supervisory role can inject certainty into the process by actively keeping it within bounds through real-time control commands. Closed-loop feedforward control is achieved by observing the process through sensors that monitor critical parameters and, if there are any deviations from their respective optimal ranges, suitable corrective actions are triggered. The type of corrective action (e.g. a change in laser power or a modification to the scanning speed) and its magnitude are determined by interrogating the surrogate models. Because of their artificial intelligence (AI)-endowed predictive capabilities, which allow them to foresee a future state of the physical twin (e.g. the AM process), DTs proactively take context-sensitive preventative steps, whereas traditional closed-loop feedback control is usually reactive. Apart from assisting a build process in real-time, a DT can help with planning the build of a part by pinpointing the optimum processing window relevant to the desired outcome. Again, the surrogate models are consulted to obtain the required information. In this article, we explain how the application of DTs to the metal AM process can significantly widen its application space by making the process more repeatable (through quality assurance) and cheaper (by getting builds right the first time).

36 MATERIALS SCIENCE↗

Prediction and compression of lattice QCD data using machine learning algorithms on quantum annealer

We present regression and compression algorithms for lattice QCD data utilizing the efficient binary optimization ability of quantum annealers. In the regression algorithm, we encode the correlation between the input and output variables into a sparse coding machine learning algorithm. The trained correlation pattern is used to predict lattice QCD observables of unseen lattice configurations from other observables measured on the lattice. In the compression algorithm, we define a mapping from lattice QCD data of floating-point numbers to the binary coefficients that closely reconstruct the input data from a set of basis vectors. Since the reconstruction is not exact, the mapping defines a lossy compression, but, a reasonably small number of binary coefficients are able to reconstruct the input vector of lattice QCD data with the reconstruction error much smaller than the statistical fluctuation. In both applications, we use D-Wave quantum annealers to solve the NP-hard binary optimization problems of the machine learning algorithms.

79 ASTRONOMY AND ASTROPHYSICS↗

Predicting weather impacts on corn production in a data-limited region using a transfer learning approach

The stability of food supply and prices may depend more on annual changes in yields from year-to-year variability in weather than on longer-term average changes from changing climatic conditions. However, the absence of high-quality data on crop yields at fine spatial resolutions in many regions of the world makes it challenging to statistically model their response to interannual variability in weather patterns. Therefore, there is a need for empirical methods that can project annual crop yield changes even in limited data regions. Here, we propose a transfer learning algorithm that uses high spatial resolution data from one region to project yields in another region with more limited data. The goal of our work is to understand what data types can be beneficial for transferring learning from a source region to a very different target region with more limited data. We utilize Long Short-Term Memory to develop a transfer learning model that is trained on historical county-level corn yield in the United States and predicts district-level corn yield variations in India. Even using smaller amounts of data in India, simulating a data-scarce region, we achieve an average root mean square error of 0.48 bu acre−1 in predicting interannual yield variations. Using Shapley values to interpret results, we explore the contribution of the different weather parameters to interannual yield variability and find a larger influence of precipitation-related variables. Our study demonstrates the usefulness of this method for transferring models of weather impacts on crop yields trained on a data-rich country to one with more limited data. It suggests the potential of applying the transfer learning model to mitigate the need for extensive raw data globally.

Vishwakarma, Srishti [ORNL] (ORCID:000000031674419↗

Automated Image Segmentation and Processing Pipeline Applied to X–Ray Computed Tomography Studies of Pitting Corrosion in Aluminum Wires

Understanding pitting corrosion is critical, yet its kinetics and morphology remain challenging to study from X-ray computed tomography (XCT) due to manual segmentation barriers. To address this, an automated pipeline leveraging deep learning for efficient large-scale XCT analysis is developed, revealing new corrosion insights. The pipeline enables pit segmentation, 3D reconstruction, statistical characterization, and a topological transformation for visualization. Here, the pipeline is applied to 87 648 XCT images capturing commercial purity aluminum (1100 Al) wire exposed to sodium chloride (NaCl) salt particles over a period of 122 h. The pipeline achieves complete feature extraction and statistical quantification across the entire XCT dataset, leveraging distributed computing environment for high efficiency. Global growth kinetics such as high-level stepwise sigmoidal volume loss patterns and granular individual pit developments are both captured for 36 detected pits. By combining automation, computer vision, and extensive XCT datasets, this research accelerates precise corrosion assessment to enable materials science discoveries at scale.

36 MATERIALS SCIENCE↗

Lithium-Ion Battery Diagnostics Using Electrochemical Impedance via Machine-Learning

Diagnosing battery states such as health, state-of-charge, or temperature is crucial for ensuring the safety and reliability of electrochemical energy storage systems. While some states, such as temperature, may be measured using cheap sensors, accurate diagnosis of battery health metrics usually requires time-consuming performance measurements, making them infeasible for use in real-world operation. These health metrics can be measured during lab-testing and then estimated on-line using predictive life models or via state observer algorithms such as Kalman filters, but these predictive methods should be supplemented by actual measurement of battery health whenever possible to ensure reliability. Rapid measurement of battery health may be done by various types of fast diagnostic techniques such as electrochemical impedance spectroscopy (EIS), which can be performed in only a few minutes and require only a fraction of the energy and power needed for a full charge and discharge measurement. But there is a substantial challenge for estimating battery health using EIS data, as EIS is sensitive to cell temperature, state-of-charge, current, and resting time in addition to health. Thus, utilizing EIS data to predict battery capacity requires correcting for all these additional variables, a task that is extremely difficult to handle analytically. This talk utilizes machine-learning methods to estimate the effectiveness of battery capacity prediction from EIS data, leveraging a data set of hundreds of EIS measurements recorded at varying temperature and state-of-charge throughout a 500-day aging study of 32 commercial, large-format NMC-Graphite lithium-ion batteries. Using EIS as input to machine-learning models is complicated by the nonlinear response of impedance to battery health, temperature, and state-of-charge, as well as the collinearity between the impedance response at neighboring frequencies, which can easily lead to overfit models. To train robust models, features from EIS data need to be extracted from the data or some subset of critical frequencies selected. Many approaches for extracting and selecting features from EIS data from electrochemical analysis and machine-learning fields were identified for analysis: using the entire raw spectra; selection of one, two, or many frequencies from the entire spectra; selecting interesting points from the EIS measurement using domain knowledge; fitting EIS with an equivalent-circuit model; calculating statistics on the raw impedance values; and reducing the dimensionality of the data using unsupervised linear (principal component analysis) and non-linear (uniform manifold approximation and projection) methods. These approaches were rigorously compared using a machine-learning pipeline approach, training linear, Gaussian process, and random forest regression models and quantifying performance using cross-validation as well as a held-out test set. An artificial neural network model trained on the raw spectra was also tested. Promising pipelines were fine-tuned via Bayesian hyperparameter optimization using cross-validation loss and training with class-specific weights to counter data set imbalance. The most reliable method for utilizing impedance in this work was the selection of two optimal frequencies through an exhaustive search, resulting in about 2% mean absolute error on test data for both Gaussian process and random forest model architectures. Interrogation of a variety of models reveals critical frequencies of 100 Hz and 103 Hz for this data set, though the optimal set of frequencies is not necessarily intuitive, i.e., the best performing models are not simply those that use impedance at frequencies that have the highest correlation to the relative discharge capacity. The best performing model is an ensemble model, which is able to predict battery capacity with 1.9% mean absolute error for unseen cells using impedance recorded at a variety of temperatures and states-of-charge.

battery↗

Attribution of heterogeneous stress distributions in low-grain polycrystals under conditions leading to damage

In high-purity polycrystalline metallic materials, voids tend to favor grain boundaries as nucleation sites due to the elevated stress states produced by granular interactions and the weakened grain boundary from the relative atomic disorder. To quantify the key factors of this elevated stress state, simple compression of a small multi-grain cylinder of body-centered cubic tantalum was simulated using a single crystal plasticity model that incorporates non-Schmid effects. Four increasingly complex synthetic microstructures were created to tractably incorporate grain boundary interactions, and a statistically significant number of combinations were performed by varying the initial crystallographic orientations of the microstructure. Most of these simulations produce the maximum von Mises stress on a grain boundary and less frequently at the multi-grain junctions. To build a statistical model for the maximum von Mises stress at the grain boundary, physically based features that could contribute to the elevated stress state were selected. Then, a learning algorithm based on information theory was used to identify which of these features contributed the most information to the data set. The identified features include a grain’s propensity to accommodate both elastic and plastic deformations and their directional components. The misalignment of the direction of each grain’s mechanical response was found to be strongly correlated to the magnitude of the stress near the grain boundary. For all of the synthetic microstructures, the statistical models produce a residual distribution that is nearly Gaussian with a variance of, at most, 10% of the prior distribution. The successful performance of the statistical model implies the correct identification of the physical features that cause severe stress localization in polycrystalline materials. The statistical models constructed here can be used to formulate a physically motivated void nucleation model which is sensitive to a microstructure’s propensity to produce elevated stress states. As a result, these statistical models also enable the design of material microstructures, in which the crystallographic orientation is chosen to resist void nucleation.

36 MATERIALS SCIENCE↗

Computer Model Calibration with Time Series Data Using Deep Learning and Quantile Regression

Computer models play a key role in many scientific and engineering problems. One major source of uncertainty in computer model experiments is input parameter uncertainty. Computer model calibration is a formal statistical procedure to infer input parameters by combining information from model runs and observational data. The existing standard calibration framework suffers from inferential issues when the model output and observational data are high-dimensional dependent data, such as large time series, due to the difficulty in building an emulator and the nonidentifiability between effects from input parameters and data-model discrepancy. To overcome these challenges, we, in this work, propose a new calibration framework based on a deep neural network (DNN) with long short-term memory layers that directly emulates the inverse relationship between the model output and input parameters. Adopting the “learning with noise” idea, we train our DNN model to filter out the effects from data-model discrepancy on input parameter inference. We also formulate a new way to construct interval predictions for DNN using quantile regression to quantify the uncertainty in input parameter estimates. Through a simulation study and real data application with the Weather Research and Forecasting Model Hydrological modeling system (WRF-Hydro), we show our approach can yield accurate point estimates and well-calibrated interval estimates for input parameters.

97 MATHEMATICS AND COMPUTING↗

Quantitative Insight to Fission Gas Pores Distribution in Irradiated Annular U-10Zr Metallic Fuel Using Machine Learning

Metallic fuels, particularly U-10Zr and its performance in reactor irradiation conditions, have been thoroughly investigated and are a promising candidate for next-generation sodium-cooled fast spectrum nuclear reactors. Irradiation in reactors can lead to the formation of fission gas and increased pore formation which can significantly impact fuel performance. Due to the large number of pores and various phases formed in metallic fuel during irradiation, a quantitative description of fission gas pores as a function of irradiation conditions is not yet available, undermining the fidelity of fuel performance modeling to support fuel qualification. It has been difficult to clearly detect pore boundaries and distinguish matrix phases from fission gas pores using optical microscopy by using simple threshold methods working with low magnification images. The pre-trained deep learning model for fission gas pore detection was applied to ~10,260 high magnification scanning electron microscopy images. The model increased the accuracy of fission gas pore segmentation to obtain statistical features, which cannot be processed manually. A pre-trained decision tree model was used to classify pores as isolated or connected pores, providing new insight into the correlation between the movement of lanthanides, solid fission products, and the radial temperature gradient developed in fuel irradiation conditions. This paper emphasizes the potential that artificial intelligence-based machine learning models have to accelerate qualification and support nuclear fuel development.

36 MATERIALS SCIENCE↗

Atomic cluster expansion potential for large scale simulations of hydrocarbons under shock compression

We present an Atomic Cluster Expansion (ACE) machine learned potential developed for high-fidelity atomistic simulations of hydrocarbons, targeting pressures and temperatures near and above supercritical fluid regimes for molecular fluids. A diverse set of stoichiometries were covered in training, including 1:0 (pure carbon), 1:4 (methane), and 1:1 (benzene), and rich bonding environments sampled at supercritical temperatures, hydrogen rich, reactive mixtures where metastable stoichiometries arise, including 1:2 (ethylene) and 1:3 (ethane). A high-fidelity training database was constructed by performing large-scale quantum molecular dynamic simulations [density functional theory (DFT) MD] of diamond, graphite, methane, and benzene. A novel approach to selecting structures from DFT MD is also presented, which allows for the rapid selection of unique DFT MD frames from complex trajectories. Comparisons to DFT and experimental data demonstrate that the presented ACE potential accurately reproduces isotherms, carbon melting curves, radial distribution functions, and shock Hugoniots for carbon and hydrocarbon systems for pressures up to 100 GPa and temperatures up to 6000 K for hydrocarbon systems and up to 9000 K for pure carbon systems. This work delivers a potential that can be used for accurate, large-scale simulations of shocked hydrocarbons and demonstrates a methodology for fitting and validating machine learning interatomic potentials to complex molecular environments, which can be applied to energetic materials in future works.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deep learning-driven super-resolution in Raman hyperspectral imaging: Efficient high-resolution reconstruction from low-resolution data

Deep learning (DL) has become an indispensable tool in hyperspectral data analysis, automatically extracting valuable features from complex, high-dimensional datasets. Super-resolution reconstruction, an essential aspect of hyperspectral data, involves enhancing spatial resolution, particularly relevant to low-resolution hyperspectral data. Yet, the pursuit of super-resolution in hyperspectral analysis is fraught with challenges, including acquiring ground truth high-resolution data for training, generalization, and scalability. The pressing issue of extended spectral acquisition times, notably for high-resolution scans, is a significant roadblock in hyperspectral imaging. Super-resolution methods offer a promising solution by providing higher spatial resolution data to expedite data collection and yield more efficient outcomes. This paper delves into a practical application of these concepts using Raman imaging, where spectral acquisition times can be prohibitively long. In this context, DL-based super-resolution models demonstrate their efficacy by predicting and reconstructing high-resolution Raman data from low-resolution input, eliminating the need for resource-intensive high-resolution scans. While previous work often relied on substantial high-resolution datasets, this study showcases the ability to achieve similar outcomes even with limited data, presenting a more practical and cost-effective approach. In conclusion, the results offer a glimpse into the transformative potential of this technology to streamline hyperspectral imaging applications by saving valuable time and resources through the successful generation of high-resolution data from low-resolution inputs.

42 ENGINEERING↗

clif

SAND2022-12903 O clif contains code to perform climate fingerprinting. It includes capabilities for principal component analysis of climate data, Fourier analysis, a collection of pre-processing scripts, and time series statistical tools. It uses a templated object-oriented programming approach to integrate all classes with the operator in Scikit Learn. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Nichol, Jeffrey↗

A framework for inferring and analyzing pharmacotherapy treatment patterns

Background: To discover pharmacotherapy prescription patterns and their statistical associations with outcomes through a clinical pathway inference framework applied to real-world data. Methods: We apply machine learning steps in our framework using a 2006 to 2020 cohort of veterans with major depressive disorder (MDD). Outpatient antidepressant pharmacy fills, dispensed inpatient antidepressant medications, emergency department visits, self-harm, and all-cause mortality data were extracted from the Department of Veterans Affairs Corporate Data Warehouse. Results: Our MDD cohort consisted of 252,179 individuals. During the study period there were 98,417 emergency department visits, 1,016 cases of self-harm, and 1,507 deaths from all causes. The top ten prescription patterns accounted for 69.3% of the data for individuals starting antidepressants at the fluoxetine equivalent of 20-39 mg. Additionally, we found associations between outcomes and dosage change. Conclusions: For 252,179 Veterans who served in Iraq and Afghanistan with subsequent MDD noted in their electronic medical records, we documented and described the major pharmacotherapy prescription patterns implemented by Veterans Health Administration providers. Ten patterns accounted for almost 70% of the data. Associations between antidepressant usage and outcomes in observational data may be confounded. The low numbers of adverse events, especially those associated with all-cause mortality, make our calculations imprecise. Furthermore, our outcomes are also indications for both disease and treatment. Despite these limitations, we demonstrate the usefulness of our framework in providing operational insight into clinical practice, and our results underscore the need for increased monitoring during critical points of treatment.

60 APPLIED LIFE SCIENCES↗

Automated, high-accuracy classification of textured microstructures using a convolutional neural network

Crystallographic texture is an important descriptor of material properties but requires time-intensive electron backscatter diffraction (EBSD) for identifying grain orientations. While some metrics such as grain size or grain aspect ratio can distinguish textured microstructures from untextured microstructures after significant grain growth, such morphological differences are not always visually observable. This paper explores the use of deep learning to classify experimentally measured textured microstructures without knowledge of crystallographic orientation. A deep convolutional neural network is used to extract high-order morphological features from binary images to distinguish textured microstructures from untextured microstructures. The convolutional neural network results are compared with a statistical Kolmogorov–Smirnov tests with traditional morphological metrics for describing microstructures. Results show that the convolutional neural network achieves a significantly improved classification accuracy, particularly at early stages of grain growth, highlighting the capability of deep learning to identify the subtle morphological patterns resulting from texture. The results demonstrate the potential of a convolutional neural network as a tool for reliable and automated microstructure classification with minimal preprocessing.

36 MATERIALS SCIENCE↗

A comparative analysis of YOLOv8 and U-Net image segmentation approaches for transmission electron micrographs of polycrystalline thin films

Metallic thin films offer a platform to experimentally study the dynamics of microstructural evolution, but the required transmission electron microscopy (TEM)-based imaging generates complex images that are challenging to segment and quantify. This work provides a comparative analysis of a new YOLOv8 model and an established U-Net model for bright-field TEM images of polycrystals, employing a framework leveraging physical observables to evaluate performance against two hand-traced benchmark datasets. This methodology obviates the comparison of large, diversely structured, and manually labeled datasets that are required to assess performance on a per-image/per-pixel basis. It is found that the YOLOv8 model, adapted for real-time instance segmentation, has up to 43× faster inferencing (NVIDIA GeForce RTX 4090) compared to U-Net and reconstructs hand-traced grain size distributions (GSDs) with excellent fidelity, finding mean diameter within 3% for grains near an optimal magnification; for grains that deviate from the optimal pixel-diameter, the size of small- (large)-diameter grains is systematically over- (under)-estimated. This is partially mitigated by including scale-aware augmentations during training. Moreover, when the bias is corrected post-inference by a rigid shift in distribution, the YOLOv8 model reproduces ground truth GSDs with exceptional fidelity, with statistical tests indicating <5% probability that the distributions are distinct. Based on ground truth data, calibration curves pertaining to this shift can be constructed for a given model. This issue is not present in the U-Net model’s results, indicating that for quantitative measurements where the true size of objects is of interest, special procedures must be implemented for YOLO-based models.

36 MATERIALS SCIENCE↗

Machine Learning Algorithms for Automated NIF Capsule Mandrel Selection

Small shells, approximately 2 mm in diameter, made from Poly(α-methylstyrene) (PAMS) are used as mandrels in the production of glow discharge polymer capsules located at the center of inertial confinement fusion experiments. The visual inspection process of microscope images of these shell mandrels, including detection of micron-sized defects on the shell surface as well as the handling and sorting, is a very labor-intensive, repetitive, and highly subjective process that stands to benefit greatly from automation. As part of an effort to decrease the number of labor hours spent in capsule handling, inspection, and metrology, the development of robotic systems was presented in a paper by Carlson et al., “Automation in Target Fabrication” [Fusion Sci. Technol., Vol. 70, p. 274 (2016)]. The current work expands the automated image acquisition systems developed previously and adds the use of convolutional neural networks to select capsules best suited for use in the downstream production process. Through the use of these machine learning algorithms, the selection process becomes robust, repeatable, and operator independent. Finally, as an added benefit the system developed as part of this work is able to provide defect statistics on entire shell batches and feed this information upstream to the production team.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Estimating Fine-Resolution Shortwave Broadband Albedo of Croplands from Harmonized Landsat and Sentinel-2 Data

Altered surface albedo due to land-cover conversions and management is a significant driver of global climate change. Albedo can be directly measured at ground stations, and remote sensing data can be used to scale-up albedo values to regional and global levels. Some previous studies have retrieved fine-resolution (10–30 m) instantaneous albedo and coarse-resolution (500–1000 m) daily mean albedo from remote sensing data, but they all required the input of Moderate Resolution Imaging Spectroradiometer (MODIS) albedo information at 500-m resolution, and none have assembled both instantaneous and daily albedo based exclusively on fine-resolution satellite data. Here, to address this issue, we compiled 387 instantaneous and 346 daily albedo records using field net radiometer measurements from the bioenergy croplands at the W. K. Kellogg Biological Station in southwest Michigan. We then connected these albedo records with a suite of variables derived from harmonized Landsat and Sentinel-2 data through two machine learning algorithms (random forest regression and extreme gradient boosting) to retrieve clear-sky instantaneous and daily shortwave broadband albedo. The performance statistics indicate reasonable accuracy of model results [root-mean-square error (RMSE)] around or below 0.03 except for snow-covered surfaces), suggesting that the retrieval of both instantaneous and daily albedo based exclusively on fine-resolution satellite data is promising. To facilitate the use of fine-resolution albedo products at the global level, future efforts need to include more albedo records of diverse surface cover types, as well as to accurately model daily albedo for cloudy days to address the “clear-sky bias.”

Harmonized Landsat and Sentinel-2↗

Deep Learning-Based Weather-Related Power Outage Prediction with Socio-Economic and Power Infrastructure Data

This paper presents a deep learning-based approach for hourly power outage probability prediction within census tracts encompassing a utility company's service territory. Two distinct deep learning models, conditional Multi-Layer Perceptron (MLP) and unconditional MLP, were developed to forecast power outage probabilities, leveraging a rich array of input features gathered from publicly available sources including weather data, weather station locations, power infrastructure maps, socio-economic and demographic statistics, and power outage records. Given a one-hour-ahead weather forecast, the models predict the power outage probability for each census tract, taking into account both the weather prediction and the location's characteristics. The deep learning models employed different loss functions to optimize prediction performance. Our experimental results underscore the significance of socio-economic factors in enhancing the accuracy of power outage predictions at the census tract level.

24 POWER TRANSMISSION AND DISTRIBUTION↗