Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed computing, machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

A nonlinear data-driven reduced order model for computational homogenization with physics/pattern-guided sampling

Developing an accurate nonlinear reduced order model from simulation data has been an outstanding research topic for many years. For many physical systems, data collection is very expensive and the optimal data distribution is not known in advance. Thus, maximizing the information gain remains a grand challenge. In a recent paper, Bhattacharjee and Matouš (2016) proposed a manifold-based nonlinear reduced order model for multiscale problems in mechanics of materials. Expanding this work here, we develop a novel sampling strategy based on the physics/pattern-guided data distribution. Our adaptive sampling strategy relies on enrichment of sub-manifolds based on the principal stretches and rotational sensitivity analysis. This novel sampling strategy substantially decreases the number of snapshots needed for accurate reduced order model construction (i.e., ~5 x reduction of snapshots over Bhattacharjee and Matouš (2016)). Moreover, we build the nonlinear manifold using the displacement rather than deformation gradient data. We provide rigorous verification and error assessment. We demonstrate both localization and homogenization of the multiscale solution on a large particulate composite unit cell.

97 MATHEMATICS AND COMPUTING↗

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Machine Learning Reveals the State of Intermittent Frictional Dynamics in a Sheared Granular Fault

Seismogenic plate boundaries are posited to behave in a similar manner to a densely packed granular medium, where fault and block systems rapidly rearrange the distribution of forces within themselves, as particles do in slowly sheared granular systems. Machine learning is used to show that statistical features of velocity signals from individual particles in a simulated sheared granular fault contain information regarding the instantaneous global state of intermittent frictional stick-slip dynamics. We demonstrate that combining features built from the signals of more particles can improve the accuracy of the global model and discuss the physical basis behind the decrease in error. We show that the statistical features such as median and higher moments of the signals that represent the particle displacement in the direction of shearing are among the best predictive features. We demonstrate here novel insights into the applications of machine learning in studying frictional processes occurring in geophysical systems.

58 GEOSCIENCES↗

Designing accurate emulators for scientific processes using calibration-driven deep models

Abstract Predictive models that accurately emulate complex scientific processes can achieve speed-ups over numerical simulators or experiments and at the same time provide surrogates for improving the subsequent analysis. Consequently, there is a recent surge in utilizing modern machine learning methods to build data-driven emulators. In this work, we study an often overlooked, yet important, problem of choosing loss functions while designing such emulators. Popular choices such as the mean squared error or the mean absolute error are based on a symmetric noise assumption and can be unsuitable for heterogeneous data or asymmetric noise distributions. We propose Learn-by-Calibrating, a novel deep learning approach based on interval calibration for designing emulators that can effectively recover the inherent noise structure without any explicit priors. Using a large suite of use-cases, we demonstrate the efficacy of our approach in providing high-quality emulators, when compared to widely-adopted loss function choices, even in small-data regimes.

97 MATHEMATICS AND COMPUTING↗

Quantifying Streambed Grain Size, Uncertainty, and Hydrobiogeochemical Parameters Using Machine Learning Model YOLO

Abstract Streambed grain sizes control river hydro‐biogeochemical (HBGC) processes and functions. However, measuring their quantities, distributions, and uncertainties is challenging due to the diversity and heterogeneity of natural streams. This work presents a photo‐driven, artificial intelligence (AI)‐enabled, and theory‐based workflow for extracting the quantities, distributions, and uncertainties of streambed grain sizes from photos. Specifically, we first trained You Only Look Once, an object detection AI, using 11,977 grain labels from 36 photos collected from nine different stream environments. We demonstrated its accuracy with a coefficient of determination of 0.98, a Nash–Sutcliffe efficiency of 0.98, and a mean absolute relative error of 6.65% in predicting the median grain size of 20 ground‐truth photos representing nine typical stream environments. The AI is then used to extract the grain size distributions and determine their characteristic grain sizes, including the 10th, 50th, 60th, and 84th percentiles, for 1,999 photos taken at 66 sites within a watershed in the Northwest US. The results indicate that the 10th, median, 60th, and 84th percentiles of the grain sizes follow log‐normal distributions, with most likely values of 2.49, 6.62, 7.68, and 10.78 cm, respectively. The average uncertainties associated with these values are 9.70%, 7.33%, 9.27%, and 11.11%, respectively. These data allow for the computation of the quantities, distributions, and uncertainties of streambed HBGC parameters, including Manning's coefficient, Darcy‐Weisbach friction factor, top layer interstitial velocity magnitude, and nitrate uptake velocity. Additionally, major sources of uncertainty in grain sizes and their impact on HBGC parameters are examined.

58 GEOSCIENCES↗

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗

Comparing storm resolving models and climates via unsupervised machine learning

Global storm-resolving models (GSRMs) have gained widespread interest because of the unprecedented detail with which they resolve the global climate. However, it remains difficult to quantify objective differences in how GSRMs resolve complex atmospheric formations. This lack of comprehensive tools for comparing model similarities is a problem in many disparate fields that involve simulation tools for complex data. To address this challenge we develop methods to estimate distributional distances based on both nonlinear dimensionality reduction and vector quantization. Our approach automatically learns physically meaningful notions of similarity from low-dimensional latent data representations that the different models produce. This enables an intercomparison of nine GSRMs based on their high-dimensional simulation data (2D vertical velocity snapshots) and reveals that only six are similar in their representation of atmospheric dynamics. Furthermore, we uncover signatures of the convective response to global warming in a fully unsupervised way. Our study provides a path toward evaluating future high-resolution simulation data more objectively.

54 ENVIRONMENTAL SCIENCES↗

Data-driven based coordinated smart inverter control for distributed energy resources

Smart inverters (SI) for distributed energy resources (DER) are becoming popular since they have the ability to stabilize as well as restore the voltage and frequency of power systems. Aiming at establishing the mathematical models combined with SI control methods, multiple optimization methods are developed. However, the computational complexity of solving such a mathematical model with various uncertainties limits the real-time application of the SI control. To conquer this challenge, a data-driven-based SI control approach is developed to achieve coordinated control in the high penetration DER system. First, an optimization problem for maximizing the active power generation and minimizing the power loss is designed using the Volt/VAR control. To reduce the time consumption, the recurrent neural network (RNN) is proposed to model the relationship between the uncertainties and control actions during the offline site. The RNN with different sub-structures such as the long short-term memory cell and gated recurrent unit cell are included to enrich the diversity of features. In the last stage, different experiment comparisons, including multiple uncertainties maps and stateof- art machine learning methods, are conducted to verify the effectiveness of the proposed method based on the IEEE 123 bus power system. The results demonstrate that the proposed method can effectively achieve a rapid and coordinated control with a lower error rate.

Qiu, Wei↗

Method and system for analyzing gas leak based on machine learning

Embodiments of the present invention provide a system for estimating a location of a gas leak, based on machine learning from forward gas concentration data provided by an analog or scale model including a gas source. The system improves significantly over previous systems by providing high quality, physically accurate forward modeling data inexpensively. During operation, the system configures an aerosol source at a first location to emit a gaseous aerosol. The system then configures a laser source to illuminate the aerosol with a laser sheet. The system may then obtain an image of a reflection of the laser sheet from the aerosol. The system may then analyze the image to quantify a three-dimensional concentration distribution of the aerosol. The system may then estimate, based on solving an inverse problem and an observed second gas concentration, a second location of a second gas source.

03 NATURAL GAS↗

A Grand Challenge "Uncertainty Project" to Accelerate Advances in Earth System Predictability: AI-Enabled Concepts and Applications

This proposal is emerging from GISS ModelE3 ESM development in the area of cloud physics, so we begin with an example of research needs/gaps from that work. Here, some of our greatest development concerns arise where we lack fundamental process-level understanding, as in ice formation. Namely, it is currently unclear what is the main process that is forming the majority of ice crystals in commonly occurring convection, apparently via secondary ice production at warm temperatures. We are keenly awaiting laboratory data for candidate mechanisms, which is not yet in hand to crucially establish their efficiency. Our progress is also hampered by a lack of uncertainty characterization in currently available measurements of ice crystal number size distributions. Furthermore, the same multiplication process may be responsible for a majority of ice crystals in many extratropical mixed-phase clouds, whose variable representation in CMIP6 ESMs may be a leading cause of differences in cloud phase feedback and ECS. Yet we have been required to deliver an ESM with the cloud physics knowledge at hand. The proposed grand challenge project is AI-enabled via application of machine learning (ML) to climate model and observational data streams (focal area 3), and applications include AI-guided observing system design and model/component/parameterization selection (areas 1 and 2). The project is structurally agnostic as to whether model or observing system components use AI approaches or not, but uncertainties must be estimated and propagatable in both.

58 GEOSCIENCES↗

Estimation of RTT and Loss Rate of Wide-Area Connections Using MPI Measurements

Scientific computations are expected to be increasingly distributed across wide-area networks, and the Message Passing Interface (MPI) has been shown to scale to support their communications over long distances. These computations should account for certain network parameters to ensure an effective execution, for example, by avoiding highly congested and long connections. The execution times of MPI basic operations reflect the connection parameters, including the Round Trip Time (RTT) and loss rate. We describe five machine leaning methods to estimate the connection RTT and loss rate using execution times of MPI basic operations. We utilize execution time measurements of MPI Sendrecv operations collected over emulated 10 Gbps connections with 0-366 ms round-trip times, wherein the longest connection spans the globe, under up to 20% periodic losses. These methods provide disparate, namely, linear and non-linear, and smooth and non-smooth, estimates of RTT and loss rate. Our results show that accurate estimates can be generated at low loss rates but they become inaccurate at loss rates 10% and higher. Overall, these results constitute a case study of the strengths and limitations of machine learning methods in inferring network-level parameters using application-level measurements.

Rao, Nageswara S.↗

Out-of-distribution generalization for learning quantum dynamics

Generalization bounds are a critical tool to assess the training data requirements of Quantum Machine Learning (QML). Recent work has established guarantees for in-distribution generalization of quantum neural networks (QNNs), where training and testing data are drawn from the same data distribution. However, there are currently no results on out-of-distribution generalization in QML, where we require a trained model to perform well even on data drawn from a different distribution to the training distribution. Here, we prove out-of-distribution generalization for the task of learning an unknown unitary. In particular, we show that one can learn the action of a unitary on entangled states having trained only product states. Since product states can be prepared using only single-qubit gates, this advances the prospects of learning quantum dynamics on near term quantum hardware, and further opens up new methods for both the classical and quantum compilation of quantum circuits.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

How Can Probabilistic Solar Power Forecasts Be Used to Lower Costs and Improve Reliability in Power Spot Markets? A Review and Application to Flexiramp Requirements

Net load uncertainty in electricity spot markets is rapidly growing. There are five general approaches by which system operators and market participants can use probabilistic forecasts of wind, solar, and load to help manage this uncertainty. These include operator situation awareness, resource risk hedging, reserves procurement, definition of contingencies, and explicit stochastic optimization. We review these approaches, and then provide a case study in which a method for using probabilistic solar forecasts to define needs for reserves is developed and evaluated. The case study has three parts. First, we describe building blocks for enhancing the Watt-Sun solar forecasting system to produce probabilistic irradiance and power forecasts. Second, relationships between Watt-Sun forecasts for multiple sites in California and the system's need for flexible ramp capability (flexiramp) are defined by machine learning and statistical methods. Third, the performance of present methods to defining flexiramp requirements, which are not conditioned on weather and renewables forecasts, is compared with that of probabilistic solar forecast-based requirements, using a multi-timescale production costing model with an 1820-bus representation of the WECC power system. Significant potential savings in fuel and flexiramp procurement costs from using solar-informed reserve requirements are found.

14 SOLAR ENERGY↗

Deep learning interfacial momentum closures in coarse-mesh CFD two-phase flow simulation using validation data

Multiphase flow phenomena have been widely observed in the industrial applications while it remains a challenging yet unsolved problems. Three-dimensional computational fluid dynamics (CFD) approaches resolve the flow fields on a finer special and temporal scales which can complement the dedicated experimental study. However, closures have to be introduced to reflect the underlying physics in multiphase flow. Among them, the interfacial forces, including drag, lift, turbulent dispersion and wall lubrication forces, play in important role on the bubble’s distribution and migration in liquid-vapor two-phase flow. Development of those closures traditionally rely on the experimental data and analytical derivation with simplified assumptions which usually cannot deliver a universal solution across wide range of flow conditions. In this paper, a data-driven approach, named as Feature Similarity Measurement (FSM), is developed and applied to improve the simulation capability of two-phase flow with coarse-mesh CFD approach. Interfacial momentum transfer in adiabatic bubbly flow serves as the focus of the present study. Both a mature and a simplified set of interfacial closures are taken as the low fidelity data. Experimental data and fine mesh CFD simulations results are adopted as high-fidelity data. Qualitative and quantitative analysis are performed in this paper which reveals that FSM can substantially improve the prediction of coarse mesh CFD model regardless of the choice of interfacial closures and it provides scalability and consistency across discontinuous flow regimes. Furthermore, it demonstrates that data-driven method can aid the multiphase flow modeling by exploring the connections between local physical features and simulation errors.

97 MATHEMATICS AND COMPUTING↗

Bayesian Adaptive Polynomial Chaos Expansions

Polynomial chaos expansions (PCEs) are widely used for uncertainty quantification (UQ) tasks, particularly in the applied mathematics community. However, PCE has received comparatively less attention in the statistics literature, and fully Bayesian formulations remain rare—especially with implementations in R. Motivated by the success of adaptive Bayesian machine learning models such as BART, BASS and BPPR, we develop a new fully Bayesian adaptive PCE method with an efficient and accessible R implementation: khaos. Our approach includes a novel proposal distribution that enables data-driven interaction selection and supports a modified g-prior tailored to PCE structure. Through simulation studies and real-world UQ applications, we demonstrate that the Bayesian adaptive PCE provides competitive performance for surrogate modeling, global sensitivity analysis and ordinal regression tasks.

97 MATHEMATICS AND COMPUTING↗

Database development and exploration of process–microstructure relationships using variational autoencoders

The paper demonstrates graphical representation of a large database containing process–microstructure relationships using an unsupervised machine learning algorithm. Correlating microstructural features to processing is an essential first step to answer the difficult problem of process sequence design. Here, a large database of 346,200 orientation distribution functions resulting from a variety of process sequences is constructed, where each sequence comprises up to four stages of tension, compression and rolling along different directions in various permutations. This open-source database is constructed for collaborative development of process design algorithms. The paper demonstrates a novel application of the large database: graphical representation of texture–process relationships. A variational autoencoder is used to reduce the entire database to a two dimensional latent space where variations in processes and properties can be visualized. Using proximity analysis in this latent space, we can quickly unearth multiple process solutions to the problem of texture or property design.

36 MATERIALS SCIENCE↗

DS-GL: Advancing Graph Learning via Harnessing the Power of Nature within Dynamic Systems

With the rapid digitization of the world, an increasing number of real-world applications are turning to nonEuclidean data, modeled as graphs. Due to their intrinsic high complexity and irregularity, learning from graph data demands tremendous computational power. Recently, CMOS-compatible Ising machines, i.e., dynamic systems composed of CMOS components, have emerged as a new approach that harnesses the inherent power of natural annealing within dynamic systems to efficiently resolve binary optimization problems and have been adopted for traditional graph computation, such as max-cut. However, when performing complex Graph Learning (GL) tasks, Ising machines face significant hurdles: (i) they are inherently binary and thus ill-suited for real-valued problems; (ii) their expensive all-to-all coupling network that guarantees effective natural annealing poses daunting scalability concerns. To address these challenges, this paper proposes a nature-powered graph learning framework dubbed DS-GL, which is the first effort to transform the process of solving graph learning problems into the natural annealing process within a parameterized dynamic system embodied as a CMOS chip. To tackle the two major hurdles, DS-GL first augments the Ising machine architecture to modify the self-reaction term of its Hamiltonian function from linear to quadratic, effectively serving as an energy regulator. This adjustment maintains the system’s original physical interpretation while enabling it to process continuous, real-valued data. Second, to address the scaling issue, DS-GL further upgrades the real-valued dense Ising machine by decomposing it into a mesh-based multi-PE dynamic system that supports efficient distributed spatial-temporal co-annealing across different PEs through sparse interconnects. By exploiting the inherent sparsity and component structures in real-world graphs, DS-GL is able to map complex graph learning tasks onto the scalable dynamic system while maintaining high accuracy. Evaluations with three diverse GL applications across six real-world datasets, including traffic flow and COVID-19 prediction, show that DS-GL can deliver from 102× to 106× speedups and 500× energy reduction over Graph Neural Networks on GPUs, with 5% - 20% accuracy enhancement.

Song, Ruibing↗

Machine learnt bond order potential to model metal–organic (Co–C) heterostructures

A fundamental understanding of the inter-relationships between structure, morphology, atomic scale dynamics, chemistry, and physical properties of mixed metallic-covalent systems is essential to design novel functional materials for applications in flexible nano-electronics, energy storage and catalysis. To achieve such knowledge, it is imperative to develop robust and computationally efficient atomistic models that describe atomic interactions accurately within a single framework. Here, we present a unified Tersoff–Brenner type bond order potential (BOP) for a Co–C system, trained against lattice parameters, cohesive energies, equation of state, and elastic constants of different crystalline phases of cobalt as well as orthorhombic Co2C derived from density functional theory (DFT) calculations. The independent BOP parameters are determined using a combination of supervised machine learning (genetic algorithms) and local minimization via the simplex method. Our newly developed BOP accurately describes the structural, thermodynamic, mechanical, and surface properties of both the elemental components as well as the carbide phases, in excellent accordance with DFT calculations and experiments. Using our machine-learnt BOP potential, we performed large-scale molecular dynamics simulations to investigate the effect of metal/carbon concentration on the structure and mechanical properties of porous architectures obtained via self-assembly of cobalt nanoparticles and fullerene molecules. Such porous structures have implications in flexible electronics, where materials with high electrical conductivity and low elastic stiffness are desired. Using unsupervised machine learning (clustering), we identify the pore structure, pore-distribution, and metallic conduction pathways in self-assembled structures at different C/Co ratios. We find that as the C/Co ratio increases, the connectivity between the Co nanoparticles becomes limited, likely resulting in low electrical conductivity; on the other hand, such C-rich hybrid structures are highly flexible (i.e., low stiffness). The BOP model developed in this work is a valuable tool to investigate atomic scale processes, structure–property relationships, and temperature/pressure response of Co–C systems, as well as design organic–inorganic hybrid structures with a desired set of properties.

Narayanan, Badri↗