Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network embeddings”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The Portals 4.3 Network Programming Interface

This report presents a specification for the Portals 4 network programming interface. Portals 4 is intended to allow scalable, high-performance network communication between nodes of a parallel computing system. Portals 4 is well suited to massively parallel processing and embedded systems. Portals 4 represents an adaption of the data movement layer developed for massively parallel processing platforms, such as the 4500-node Intel TeraFLOPS machine. Sandia's Cplant cluster project motivated the development of Version 3.0, which was later extended to Version 3.3 as part of the Cray Red Storm machine and XT line. Version 4 is targeted to the next generation of machines employing advanced network interface architectures that support enhanced offload capabilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Raspberry colloid-templated approach for the synthesis of palladium-based oxidation catalysts with enhanced hydrothermal stability and low-temperature activity

It is becoming increasingly urgent to develop and utilize novel, more efficient and stable materials for mobile and stationary emission control applications as the deleterious consequences of anthropogenic air pollution are becoming more evident and pressing. Tightening regulations, particularly related to automotive exhaust treatment, together with continued improvements in engine design, that result in lowering the engine operating temperatures and inadvertently lead to the release of an overwhelming proportion of pollutants during the cold start, present new challenges for materials design, specifically for oxidation catalysts. In particular, improvements in the low-temperature activity while maintaining catalyst stability at high temperatures are required from the next-generation catalyst. Typical catalysts for removal of pollutants from automotive exhaust streams incorporate platinum group metals (PGMs). They tend to be inefficient at low temperatures (below 250 °C), thus accounting for the cold start problem, yet sinter and lose their activity at high temperatures that are frequently encountered during catalyst operation. High PGM loadings are often employed to compensate for catalyst inefficiencies and fast degradation, ultimately resulting in high-cost catalytic converters. Here, we have developed a new approach for the design and formation of catalytic materials that allows for both significantly more efficient PGM incorporation and improved overall catalyst performance at reduced PGM loadings. The method provides control over the composition and geometry of the support through self-assembly of sacrificial composite template — “raspberry” polymeric colloids decorated with catalytic particles — accompanied by infiltration with metal-oxide precursor and subsequent removal of the colloids. This method simultaneously structures the porous network and organizes the catalytic particles within it. Uniquely, the resulting catalytic particles are partially embedded in the support matrix and partially exposed to the pore interior, producing catalytic sites that are both stable and accessible. Herein, the feasibility of this novel and versatile approach for automotive catalytic conversion is demonstrated: the studies include testing alumina-based raspberry-colloid-templated (RCT) catalysts containing Pd nanoparticles (RCT Pd/Al2O3) for oxidation of propane and carbon monoxide under simulated diesel exhaust conditions and hydrothermal aging at 800 °C for 50 h in the simulated stream. The RCT Pd/Al2O3 catalysts exhibit exceptional activity toward CO oxidation, reduced reaction onset temperature, and high stability to elevated temperatures (demonstrated through prolonged exposure to temperatures up to 950 °C) and reactive gas streams, without migration, sintering or loss of the precious metal NPs. Notably, the novel catalyst shows the same or slightly better performance than the commercial catalysts even when the PGM load is reduced by ~80 % compared to the commercial counterparts. These results provide confidence for the utilization of the RCT approach for the fabrication of robust nanostructured catalysts for next-generation, energy-efficient catalytic converters with improved performance at low and high temperatures and reduced costs. The RCT methodology is, in addition, highly generalizable, and can thus be applied for the design of a wide range of catalytic systems in the automotive sector and beyond.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Methods for microwave characterization of electro-optic crystals for quantum transduction

Microwave-optic quantum transducers are essential devices to develop distributed quantum networks and implement related quantum communication protocols. Three dimensional high-coherence time microwave cavities embedded with electro-optic nonlinear dielectric materials provide promising platforms to boost the efficiency of the microwave-optic frequency conversion. However, so far, the properties for such dielectric crystals operating at milli-Kelvin cryogenic temperatures have not been well understood. Here, we propose a scheme to precisely measure and benchmark the dielectric constant and analyze the dissipation mechanisms of electro-optic materials, such as Lithium Niobate, at the quantum threshold. We will use Fermilab’s three dimensional superconducting cavities with long coherence time. The proposed method of microwave characterization lays the foundations for engineering quantum transduction devices and quantum sensors with high conversion efficiency and fidelity.

Zorzetti, Silvia↗
DOE OSTI2020

Interactive Visual Study of Multiple Attributes Learning Model of X-Ray Scattering Images

Existing interactive visualization tools for deep learning are mostly applied to the training, debugging, and refinement of neural network models working on natural images. However, visual analytics tools are lacking for the specific application of x-ray image classification with multiple structural attributes. In this paper, we present an interactive system for domain scientists to visually study the multiple attributes learning models applied to x-ray scattering images. It allows domain scientists to interactively explore this important type of scientific images in embedded spaces that are defined on the model prediction output, the actual labels, and the discovered feature space of neural networks. Users are allowed to flexibly select instance images, their clusters, and compare them regarding the specified visual representation of attributes. The exploration is guided by the manifestation of model performance related to mutual relationships among attributes, which often affect the learning accuracy and effectiveness. The system thus supports domain scientists to improve the training dataset and model, find questionable attributes labels, and identify outlier images or spurious data clusters. Case studies and scientists feedback demonstrate its functionalities and usefulness.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Reinforcement Learning via Gaussian Processes with Neural Network Dual Kernels

While deep neural networks (DNNs) and Gaussian Processes (GPs) are both popularly utilized to solve problems in reinforcement learning, both approaches feature undesirable drawbacks for challenging problems. DNNs learn complex non-linear embeddings, but do not naturally quantify uncertainty and are often data-inefficient to train. GPs infer posterior distributions over functions, but popular kernels exhibit limited expressivity on complex and high-dimensional data. Fortunately, recently discovered conjugate and neural tangent kernel functions encode the behavior of overparameterized neural networks in the kernel domain. We demonstrate that these kernels can be efficiently applied to regression and reinforcement learning problems by analyzing a baseline case study.We apply GPs with neural network dual kernels to solve reinforcement learning tasks for the first time. We demonstrate, using the well understood mountain-car problem, that GPs empowered with dual kernels perform at least as well as those using the conventional radial basis function kernel. Finally, we conjecture that by inheriting the probabilistic rigor of GPs and the powerful embedding properties of DNNs, GPs using NN dual kernels will empower future reinforcement learning models on difficult domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

The Third Dimension of Ferroelectric Domain Walls

Abstract Ferroelectric domain walls are quasi‐2D systems that show great promise for the development of nonvolatile memory, memristor technology, and electronic components with ultrasmall feature size. Electric fields, for example, can change the domain wall orientation relative to the spontaneous polarization and switch between resistive and conductive states, controlling the electrical current. Being embedded in a 3D material, however, the domain walls are not perfectly flat and can form networks, which leads to complex physical structures. In this work, the importance of the nanoscale structure for the emergent transport properties is demonstrated, studying electronic conduction in the 3D network of neutral and charged domain walls in ErMnO 3 . By combining tomographic microscopy techniques and finite element modeling, the contribution of domain walls within the bulk is clarified and the significance of curvature effects for the local conduction is shown down to the nanoscale. The findings provide insights into the propagation of electrical currents in domain wall networks, reveal additional degrees of freedom for their control, and provide quantitative guidelines for the design of domain‐wall‐based technology.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Distributed Acoustic Sensing as a Distributed Hydraulic Sensor in Fractured Bedrock

Distributed acoustic sensing (DAS) was originally intended to measure oscillatory strain at frequencies of 1 Hz or more on a fiber optic cable. Recently, measurements at much lower frequencies have opened the possibility of using DAS as a dynamic strain sensor in boreholes. A fiber optic cable mechanically coupled to a geologic formation will strain in response to hydraulic stresses in pores and fractures. A DAS interrogator can measure dynamic strain in the borehole, which can be related to fluid pressure through the mechanical compliance properties of the formation. Because DAS makes distributed measurements, it is capable of both locating hydraulically active features and quantifying the fluid pressure in the formation. We present field experiments in which a fiber optic cable was mechanically coupled to two crystalline rock boreholes. The formation was stressed hydraulically at another well using alternating injection and pumping. The DAS instrument measured oscillating strain at the location of a fracture zone known to be hydraulically active. Rock displacements of less than 1 nm were measured. Laboratory experiments confirm that displacement is measured correctly. These results suggest that fiber optic cable embedded in geologic formations may be used to map hydraulic connections in three-dimensional fracture networks. A great advantage of this approach is that strain, an indirect measure of hydraulic stress, can be measured without beforehand knowledge of flowing fractures that intersect boreholes. The technology has obvious applications in water resources, geothermal energy, CO 2 sequestration, and remediation of groundwater in fractured bedrock.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Netload Range Cost Curves for Coordinated Transmission-Distribution Planning Under DER Growth Uncertainty

The increasing penetration of distributed energy resources (DERs) requires better coordination between transmission and distribution (T&D) planning to ensure system security and cost efficiency. However, misaligned planning horizons, computational burdens, and privacy concerns hinder effective coordination, leading to either underutilized resources caused by overinvestments or reliability risks due to underinvestment. To address this challenge, we introduce netload range cost curves (NRCCs), a novel approach for managing long-term DER growth uncertainty through T&D coordination, while preserving existing data-sharing and regulatory structures. NRCCs provide pairs of (i) peak substation netload guarantees and (ii) corresponding distribution upgrade options and costs, enabling their seamless integration into transmission planning workflows. To compute NRCCs efficiently, we develop a transmission-aware distribution network planning (TADNP), which is subsequently integrated to an iterative computation procedure. These NRCCs are then embedded into an NRCC-informed transmission planning model to enable resource-efficient coordination. We illustrate our proposed approach with a case study based on realistic distribution and transmission systems in the San Francisco Bay Area, California. Our results indicate the possibility of dramatic savings in transmission investments by incorporating the proposed NRCC-integrated T&D coordination framework.

Li, Yujia↗
DOE OSTI2022

CrossSim Inference Manual v2.0

Neural networks are largely based on matrix computations. During forward inference, the most heavily used compute kernel is the matrix-vector multiplication (MVM): $W \vec{x} $. Inference is a first frontier for the deployment of next-generation hardware for neural network applications, as it is more readily deployed in edge devices, such as mobile devices or embedded processors with size, weight, and power constraints. Inference is also easier to implement in analog systems than training, which has more stringent device requirements. The main processing kernel used during inference is the MVM.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Gray-Box Modeling for Distribution Systems With Inverter-Based Resources: Integrating Physics-Based and Data-Driven Approaches

Here, in this paper, we develop a novel gray-box modeling approach for distribution systems with inverter-based resources (IBRs). The proposed gray-box modeling method aims to improve estimation accuracy by taking advantages of both physics-based (white-box) and data-driven (black-box) modeling approaches. To this end, we utilize partial physical knowledge of the system, including the inverters’ structures and control diagrams, as well as the equivalent network model simplified through Kron reduction. The white-box model containing unknown parameters is then constructed with mathematical equations and an optimization-based method is subsequently employed to identify these unknown parameters within the white-box model. Next, the graybox modeling framework is then constructed by embedding the output variables of the white-box model into the input vector of a black-box model (represented using a neural network). Finally, the black-box section is trained using the collected input-output datasets and the gray-box model is then obtained. Furthermore, case studies demonstrate that our gray-box modeling approach effectively improves estimation accuracy compared to purely physics-based or data-driven methods.

42 ENGINEERING↗
DOE OSTI2023

Multifidelity Neural Network Formulations for Prediction of Reactive Molecular Potential Energy Surfaces

Here, this paper focuses on the development of multifidelity modeling approaches using neural network surrogates, where training data arising from multiple model forms and resolutions are integrated to predict high-fidelity response quantities of interest at lower cost. We focus on the context of quantum chemistry and the integration of information from multiple levels of theory. Important foundations include the use of symmetry function-based atomic energy vector constructions as feature vectors for representing structures across families of molecules and single-fidelity neural network training capabilities that learn the relationships needed to map feature vectors to potential energy predictions. These foundations are embedded within several multifidelity topologies that decompose the high-fidelity mapping into model-based components, including sequential formulations that admit a general nonlinear mapping across fidelities and discrepancy-based formulations that presume an additive decomposition. Methodologies are first explored and demonstrated on a pair of simple analytical test problems and then deployed for potential energy prediction for C 5 H 5 using B2PLYP-D3/6-311++G(d,p) for high-fidelity simulation data and Hartree–Fock 6-31G for low-fidelity data. For the common case of limited access to high-fidelity data, our computational results demonstrate that multifidelity neural network potential energy surface constructions achieve roughly an order of magnitude improvement, either in terms of test error reduction for equivalent total simulation cost or reduction in total cost for equivalent error.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

On the influence of over-parameterization in manifold based surrogates and deep neural operators

Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. Here, we demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Implicit neural representation (INR) has surfaced as a promising direction for solving different scientific visualization tasks due to its continuous representation and flexible input and output settings. We present STSR-INR, an INR solution for generating simultaneous spatiotemporal super-resolution for multivariate time-varying volumetric data. Inheriting the benefits of the INR-based approach, STSR-INR supports unsupervised learning and permits data upscaling with arbitrary spatial and temporal scale factors. Unlike existing GAN- or INR-based super-resolution methods, STSR-INR focuses on tackling variables or ensembles and enabling joint training across datasets of various spatiotemporal resolutions. Here we achieve this capability via a variable embedding scheme that learns latent vectors for different variables. In conjunction with a modulated structure in the network design, we employ a variational auto-decoder to optimize the learnable latent vectors to enable latent-space interpolation. To combat the slow training of INR, we leverage a multi-head strategy to improve training and inference speed with significant speedup. We demonstrate the effectiveness of STSR-INR with multiple scalar field datasets and compare it with conventional tricubic+linear interpolation and state-of-the-art deep-learning-based solutions (STNet and CoordNet).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

An image classification deep-learning algorithm for shrapnel detection from ultrasound images

Ultrasound imaging is essential for non-invasively diagnosing injuries where advanced diagnostics may not be possible. However, image interpretation remains a challenge as proper expertise may not be available. In response, artificial intelligence algorithms are being investigated to automate image analysis and diagnosis. Here, we highlight an image classification convolutional neural network for detecting shrapnel in ultrasound images. As an initial application, different shrapnel types and sizes were embedded first in a tissue mimicking phantom and then in swine thigh tissue. The algorithm architecture was optimized stepwise by minimizing validation loss and maximizing F1 score. The final algorithm design trained on tissue phantom image sets had an F1 score of 0.95 and an area under the ROC curve of 0.95. It maintained higher than a 90% accuracy for each of 8 shrapnel types. When trained only on swine image sets, the optimized algorithm format had even higher metrics: F1 and area under the ROC curve of 0.99. Overall, the algorithm developed resulted in strong classification accuracy for both the tissue phantom and animal tissue. This framework can be applied to other trauma relevant imaging applications such as internal bleeding to further simplify trauma medicine when resources and image interpretation are scarce.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

SBND Analysis using ML Reconstruction Chain

As part of the Short Baseline Neutrino (SBN) Program at Fermilab, the Short Baseline Near Detector (SBND) is positioned in the Booster Neutrino Beam (BNB) and explores neutrino-argon interactions with unprecedented statistics. SBND is a Liquid Argon Time Projection Chamber (LArTPC). Electrons produced through ionization drift toward three wire planes, providing signals that form 2D images of particle trajectories. I introduce the Scalable Particle Imaging using Neural Embeddings (SPINE) framework, which employs a Machine Learning (ML)-based 3D reconstruction using a series of neural networks. Here, we present SPINE’s reconstruction chain, analysis approaches, and results from our latest simulation samples.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Privacy-Preserving Control of Partitioned Energy Resources

Distributed energy resources are an increasingly important part of the electric grid. We examine the problem of partitioning a distributed energy resource among many users while providing privacy to them. In this model, clients can send requests to a server, the server can verify that the requests are valid and aggregate them, but it cannot see the actual values in the requests. Without privacy, each user is forced to reveal their daily schedule or energy use. Energy resources add a novel challenge that prior systems do not address: they require verifying limits on private power (a rate over time) and energy (a sum) values. Furthermore, the cryptographic mechanisms must run on embedded energy control systems. We describe Weft, a novel cryptographic system that verifies both power (rate) and energy (integral) constraints on private client values and aggregates them. The key insight behind the approach is to rely on additively homomorphic secret shares, which allows servers to compute sums from rates. We present 3 cryptographic proof systems with different system trade-off for embedded systems: bit-splitting proofs minimize memory use, sorting proofs minimize computation, and commitment proofs minimize network communication. Using bit-splitting proofs, it takes an IoT client using a CortexM microcontroller 4 minutes of compute time to privately control its share of an energy resource for a day at 20s granularity.

Laufer, Evan↗