Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “high-dimensional optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗

Hyperplane decision trees as piecewise linear surrogate models for chemical process design

Recent trends in chemical engineering research point towards an increasing reliance on data-driven modeling approaches. Neural networks, for instance, have proven to be accurate when data is plentiful and high-dimensional, but in many cases, they require computationally-intensive training procedures. Here, in this work, we describe hyperplane decision trees (HT) as a highly expressive and low-compute machine learning model architecture. These models are locally linear and have linear decision boundaries, resulting in a piecewise linear model of the data. This property allows them to be converted into mixed-integer linear constraints which can be globally optimized. Our open-source PyTorch implementation of this method is a fast, flexible, and accessible way to build accurate piecewise linear models of data.

Decision trees↗

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗

Bayesian tomography of high-dimensional on-chip biphoton frequency combs with randomized measurements

Abstract Owing in large part to the advent of integrated biphoton frequency combs, recent years have witnessed increased attention to quantum information processing in the frequency domain for its inherent high dimensionality and entanglement compatible with fiber-optic networks. Quantum state tomography of such states, however, has required complex and precise engineering of active frequency mixing operations, which are difficult to scale. To address these limitations, we propose a solution that employs a pulse shaper and electro-optic phase modulator to perform random operations instead of mixing in a prescribed manner. We successfully verify the entanglement and reconstruct the full density matrix of biphoton frequency combs generated from an on-chip Si 3 N 4 microring resonator in up to an 8 × 8-dimensional two-qudit Hilbert space, the highest dimension to date for frequency bins. More generally, our employed Bayesian statistical model can be tailored to a variety of quantum systems with restricted measurement capabilities, forming an opportunistic tomographic framework that utilizes all available data in an optimal way.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Cost function for low-dimensional manifold topology assessment

Abstract In reduced-order modeling, complex systems that exhibit high state-space dimensionality are described and evolved using a small number of parameters. These parameters can be obtained in a data-driven way, where a high-dimensional dataset is projected onto a lower-dimensional basis. A complex system is then restricted to states on a low-dimensional manifold where it can be efficiently modeled. While this approach brings computational benefits, obtaining a good quality of the manifold topology becomes a crucial aspect when models, such as nonlinear regression, are built on top of the manifold. Here, we present a quantitative metric for characterizing manifold topologies. Our metric pays attention to non-uniqueness and spatial gradients in physical quantities of interest, and can be applied to manifolds of arbitrary dimensionality. Using the metric as a cost function in optimization algorithms, we show that optimized low-dimensional projections can be found. We delineate a few applications of the cost function to datasets representing argon plasma, reacting flows and atmospheric pollutant dispersion. We demonstrate how the cost function can assess various dimensionality reduction and manifold learning techniques as well as data preprocessing strategies in their capacity to yield quality low-dimensional projections. We show that improved manifold topologies can facilitate building nonlinear regression models.

42 ENGINEERING↗

An implementation of neural simulation-based inference for parameter estimation in ATLAS

Neural simulation-based inference (NSBI) is a powerful class of machine-learning-based methods for statistical inference that naturally handles high-dimensional parameter estimation without the need to bin data into low-dimensional summary histograms. Such methods are promising for a range of measurements, including at the Large Hadron Collider, where no single observable may be optimal to scan over the entire theoretical phase space under consideration, or where binning data into histograms could result in a loss of sensitivity. This work develops a NSBI framework for statistical inference, using neural networks to estimate probability density ratios, which enables the application to a full-scale analysis. It incorporates a large number of systematic uncertainties, quantifies the uncertainty due to the finite number of events in training samples, develops a method to construct confidence intervals, and demonstrates a series of intermediate diagnostic checks that can be performed to validate the robustness of the method. As an example, the power and feasibility of the method are assessed on simulated data for a simplified version of an off-shell Higgs boson couplings measurement in the four-lepton final states. This approach represents an extension to the standard statistical methodology used by the experiments at the Large Hadron Collider, and can benefit many physics analyses.

frequentist statistics↗

Efficient calculation of self magnetic field, self-force, and self-inductance for electromagnetic coils with rectangular cross-section

Abstract For designing high-field electromagnets, the Lorentz force on coils needs to be computed in order to design suitable support structures, and the inductance should be computed to evaluate the stored energy and dynamics. Also, the magnetic field and its variation inside the conductor is of interest for computing stress and strain, and due to superconducting quench limits. For these force, inductance, energy, and internal field calculations, the coils cannot be naively approximated as infinitesimally thin filaments due to divergences when the source and evaluation points coincide, so more computationally demanding calculations are usually required, resolving the finite cross-section of the conductors. Here, we present a new alternative method that enables the internal magnetic field vector, self-force, and self-inductance to be computed rapidly and accurately within a 1D filament model. The method is applicable to coils for which the curve center-line can have general noncircular shape, as long as the conductor width is small compared to the radius of curvature. This paper extends a previous calculation for circular-cross-section conductors (Hurwitz et al 2024 IEEE Trans. Magn. ) to consider the case of rectangular cross-section. The reduced model is derived by rigorous analysis of the singularity, regularizing the filament integrals such that they match the true high-dimensional integrals at high coil aspect ratio. The new filament model exactly recovers analytic results for a circular coil, and is shown to accurately reproduce full finite-cross-section calculations for a non-planar coil of a stellarator magnetic fusion device. Due to the efficiency of the model here, it is well suited for use inside design optimization.

Landreman, Matt (ORCID:000000027233577X)↗

High-dimensional maximum-entropy phase space tomography using normalizing flows

Particle accelerators generate charged-particle beams with tailored distributions in six-dimensional position-momentum space (phase space). Knowledge of the phase space distribution enables model-based beam optimization and control. In the absence of direct measurements, the distribution must be tomographically reconstructed from its projections. In this paper, we highlight that such problems can be severely underdetermined and that entropy maximization is the most conservative solution strategy. We leverage —invertible generative models—to extend maximum-entropy tomography to six-dimensional phase space and perform numerical experiments to validate the model's performance. Our numerical experiments demonstrate consistency with exact two-dimensional maximum-entropy solutions and the ability to fit complicated six-dimensional distributions to large measurement sets in reasonable time. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗

Efficient graph representation framework for chemical molecule similarity tasks

Graph data has emerged in numerous scientific domains and machine learning techniques have been widely used for analysis and learning of diverse data for prediction and decision. Machine learning techniques can readily address complex problems by leveraging their structural information. But graphs cannot be directly used for existing machine learning algorithms unless encoded as vectors. The problem of efficient representation of graphs is a substantial challenge in graph machine learning. In this paper, we propose a novel two-stage framework for the representation of chemical molecule graphs based on the strengths of Graph Isomorphism Networks (GINs) and Siamese autoencoders. In the first stage, the GIN model is constructed and trained using the structural information of chemical molecule graphs. Node attributes, edge attributes, and edge indices are used as input data, while graph attributes are used as labels. The GIN model effectively captures the structural characteristics of graphs and can accurately predict graph attributes, i.e., molecular properties. It also generates Graph Embeddings, represented as vectors that encode the structural information of graphs. In the second stage, Graph Embedding vectors are further optimized for downstream similarity tasks while preserving the graph structural information. The Siamese autoencoder is constructed and trained, which reduces the dimensionality of the Graph Embedding vectors, while maximizing the preservation of structural information in the original high-dimensional vectors. The resulting low-dimensional Graph Embeddings can be effectively utilized for tasks such as approximate nearest neighbor search. The experimental results demonstrate the effectiveness of our proposed framework in accurately predicting graph similarity.

Ma, Jiaji↗

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

Surrogate-Based Autotuning for Randomized Sketching Algorithms in Regression Problems

Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be effective in handling high-dimensional computational problems, providing high-quality empirical performance as well as strong probabilistic guarantees. However, their practical application is complicated by the fact that the user needs to set various algorithm-specific tuning parameters which are different from those used in traditional NLA. This paper demonstrates how a surrogate-based autotuning approach can be used to address fundamental problems of parameter selection in RandNLA algorithms. In particular, we provide a detailed investigation of surrogate-based autotuning for sketch-and-precondition (SAP)-based randomized least squares methods, which have been one of the great success stories in modern RandNLA. Empirical results show that our surrogate-based autotuning approach can achieve near-optimal performance with much less tuning cost than a random search (up to about 7.6x fewer trials of different parameter configurations). Moreover, while our experiments focus on least squares, our results demonstrate a general-purpose autotuning pipeline applicable to any kind of RandNLA algorithm.

Cho, Younghyun↗

Divertor Plasma Detachment Control Neural Network

DivControlNN is a state-of-the-art software tool that leverages advanced machine learning techniques to predict and control divertor plasma behavior in fusion reactors. Plasma, a highly energetic and electrically charged gas, requires meticulous management to protect reactor components and maintain optimal energy production. Conventional simulation methods, although extremely detailed, typically demand extensive computational time-making them unsuitable for real-time control scenarios. DivControlNN addresses this challenge by learning from tens of thousands of high-fidelity simulations, thereby creating a rapid surrogate model that can deliver near-instantaneous predictions. At the core of its functionality is a sophisticated technique known as latent space mapping, which condenses complex, high-dimensional plasma data into a compact, lower-dimensional representation. This streamlined representation enables the system to quickly forecast essential plasma properties and determine the precise conditions required for effective detachment. Detachment is a crucial process in which the plasma is cooled before reaching the divertor plates, thereby reducing heat loads and mitigating material erosion. In recent experiments conducted on the KSTAR tokamak in South Korea, DivControlNN successfully guided the detachment process without any fine-tuning-even when applied to a new tungsten divertor configuration. By achieving a computational speed-up of over one hundred million times compared to traditional simulation methods while maintaining low prediction errors, DivControlNN stands to significantly enhance real-time control and diagnostic capabilities in future fusion reactors. This breakthrough paves the way for safer, more reliable reactor operation and represents a major advancement toward realizing fusion energy as a practical, sustainable, and clean power source.

Xu, Xueqiao [Lawrence Livermore National Laborator↗

Ultracoherent SRF Cavity-Based Multi-Qudit Platform with Error-Resilient Control

Superconducting radio-frequency (SRF) cavities offer a promising platform for quantum computing due to their long coherence times, yet integrating nonlinear elements like transmons for control often introduces additional loss. We report a multimode quantum system based on a 2-cell elliptical shaped SRF cavity, comprising two cavity modes weakly coupled to an ancillary transmon circuit, designed to preserve coherence while enabling efficient control of the cavity modes. We mitigate the detrimental effects of the transmon decoherence through careful design optimization that reduces transmon-cavity couplings and participation in the dielectric substrate and lossy interfaces, to achieve single-photon lifetimes of 20.6\,ms and 15.6\,ms for the two modes, and a pure dephasing time exceeding 40\,ms. This marks an order-of-magnitude improvement over prior 3D multimode memories. Leveraging sideband interactions and novel error-resilient protocols, including measurement-based correction and post-selection, we achieve high-fidelity control over quantum states. This enables the preparation of Fock states up to $N = 20$ with fidelities exceeding 95\%, the highest reported to date to the authors' knowledge, as well as two-mode entanglement with an estimated coherence-limited fidelities of 99.9\% after post-selection. These results establish our platform as a robust foundation for quantum information processing, allowing for future extensions to high-dimensional qudit encodings.

Kim, T. [Northwestern U.]↗

Ultracoherent superconducting cavity-based multiqudit platform with error-resilient control

Superconducting radio-frequency (SRF) cavities offer a promising platform for quantum computing due to their long coherence times, yet integrating nonlinear elements like transmons for control often introduces additional loss. We report a multimode quantum system based on a 2-cell elliptical-shaped SRF cavity, comprising two cavity modes weakly coupled to an ancillary transmon circuit, designed to preserve coherence while enabling efficient control of the cavity modes. We mitigate the detrimental effects of the transmon decoherence through careful design optimization that reduces transmon-cavity couplings and participation in the dielectric substrate and lossy interfaces, to achieve single-photon lifetimes of 20.6 ms and 15.6 ms for the two modes, and a pure dephasing time exceeding 40 ms. This marks an order-of-magnitude improvement over prior 3D multimode memories. Leveraging sideband interactions and novel error-resilient protocols, including measurement-based correction and post-selection, we achieve high-fidelity control over quantum states. This enables the preparation of Fock states up to N = 20 with fidelities exceeding 95%, the highest reported to date to the authors' knowledge, as well as two-mode entanglement with an estimated coherence-limited fidelities of 99.9% after post-selection. These results establish our platform as a robust foundation for quantum information processing, allowing for future extensions to high-dimensional qudit encodings.

Lu, Yao [Fermilab] (ORCID:000000020413698X)↗

Ultracoherent superconducting cavity-based multiqudit platform with error-resilient control

Superconducting radio-frequency (SRF) cavities offer a promising platform for quantum computing due to their long coherence times, yet integrating nonlinear elements like transmons for control often introduces additional loss. We report a multimode quantum system based on a 2-cell elliptical-shaped SRF cavity, comprising two cavity modes weakly coupled to an ancillary transmon circuit, designed to preserve coherence while enabling efficient control of the cavity modes. We mitigate the detrimental effects of the transmon decoherence through careful design optimization that reduces transmon-cavity couplings and participation in the dielectric substrate and lossy interfaces, to achieve single-photon lifetimes of 20.6 ms and 15.6 ms for the two modes, and a pure dephasing time exceeding 40 ms. This marks an order-of-magnitude improvement over prior 3D multimode memories. Leveraging sideband interactions and novel error-resilient protocols, including measurement-based correction and post-selection, we achieve high-fidelity control over quantum states. This enables the preparation of Fock states up to N = 20 with fidelities exceeding 95%, the highest reported to date to the authors' knowledge, as well as two-mode entanglement with an estimated coherence-limited fidelities of 99.9% after post-selection. These results establish our platform as a robust foundation for quantum information processing, allowing for future extensions to high-dimensional qudit encodings.

Lu, Yao [Fermilab] (ORCID:000000020413698X)↗

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗

Ultracoherent superconducting cavity-based multiqudit platform with error-resilient control

Superconducting radio-frequency (SRF) cavities offer a promising platform for quantum computing due to their long coherence times, yet integrating nonlinear elements like transmons for control often introduces additional loss. We report a multimode quantum system based on a 2-cell elliptical shaped SRF cavity, comprising two cavity modes weakly coupled to an ancillary transmon circuit, designed to preserve coherence while enabling efficient control of the cavity modes. We mitigate the detrimental effects of the transmon decoherence through careful design optimization that reduces transmon-cavity couplings and participation in the dielectric substrate and lossy interfaces, to achieve single-photon lifetimes of 20.6 ms and 15.6 ms for the two modes, and a pure dephasing time exceeding 40 ms. This marks an order-of-magnitude improvement over prior 3D multimode memories. Leveraging sideband interactions and novel error-resilient protocols, including measurement-based correction and post-selection, we achieve high-fidelity control over quantum states. This enables the preparation of Fock states up to $N = 20$ with fidelities exceeding 95%, the highest reported to date to the authors' knowledge, as well as two-mode entanglement with an estimated coherence-limited fidelities of 99.9% after post-selection. These results establish our platform as a robust foundation for quantum information processing, allowing for future extensions to high-dimensional qudit encodings.

Kim, Taeyoon [Fermilab; Northwestern U.]↗

Landscaper v1

Understanding the inner workings of machine learning models through their loss landscapes offers crucial insights into model properties, optimization dynamics, and generalizability. However, accessing these insights has traditionally required specialized mathematical expertise, limiting broader adoption. Landscaper is an open-source Python package designed to bridge this gap. Landscaper seamlessly integrates a suite of multi-dimensional loss landscape analyses with cutting-edge topological data analysis (TDA) methods. This powerful combination makes both fundamental loss landscape analysis and advanced TDA techniques accessible to the broader scientific ML community, without requiring deep pre-existing mathematical knowledge. Landscaper offers three key functionalities: * Construction: Builds detailed loss landscape representations through versatile low and high-dimensional sampling techniques. * Quantification: Applies advanced metrics, including a novel topological data analysis (TDA) based smoothness metric, enabling new perspectives on model behavior. * Visualization: Offers intuitive tools to visualize and interpret loss landscapes, providing actionable insights beyond traditional performance metrics.

Weber, Gunther [Lawrence Berkeley National Laborat↗