Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “layered networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Validation of the Sentinel-5 Precursor TROPOMI cloud data with Cloudnet, Aura OMI O 2 –O 2 , MODIS, and Suomi-NPP VIIRS

Accurate knowledge of cloud properties is essential to the measurement of atmospheric composition from space. In this work we assess the quality of the cloud data from three Copernicus Sentinel-5 Precursor (S5P) TROPOMI cloud products: (i) S5P OCRA/ROCINN_CAL (Optical Cloud Recognition Algorithm/Retrieval of Cloud Information using Neural Networks;Clouds-As-Layers), (ii) S5P OCRA/ROCINN_CRB (Clouds-as-Reflecting Boundaries), and (iii) S5P FRESCO-S (Fast Retrieval Scheme for Clouds from Oxygen absorption bands – Sentinel). Target properties of this work are cloud-top height and cloud optical thickness (OCRA/ROCINN_CAL), cloud height (OCRA/ROCINN_CRB and FRESCO-S), and radiometric cloud fraction (all three algorithms). The analysis combines (i) the examination of cloud maps for artificial geographical patterns, (ii) the comparison to other satellite cloud data (MODIS, NPP-VIIRS, and OMI O2–O2), and (iii) ground-based validation with respect to correlative observations (30 April 2018 to 27 February 2020) from the Cloudnet network of ceilometers, lidars, and radars. Zonal mean latitudinal variation of S5P cloud properties is similar to that of other satellite data. S5P OCRA/ROCINN_CAL agrees well with NPP VIIRS cloud-top height and cloud optical thickness and with Cloudnet cloud-top height, especially for the low (mostly liquid) clouds. For the high clouds, S5P OCRA/ROCINN_CAL cloud-top height is below the cloud-top height of VIIRS and of Cloudnet, while its cloud optical thickness is higher than that of VIIRS. S5P OCRA/ROCINN_CRB and S5P FRESCO cloud height are well below the Cloudnet cloud mean height for the low clouds but match on average better with the Cloudnet cloud mean height for the higher clouds. As opposed to S5P OCRA/ROCINN_CRB and S5P FRESCO, S5P OCRA/ROCINN_CAL is well able to match the lowest CTH mode of the Cloudnet observations. Peculiar geographical patterns are identified in the cloud products and will be mitigated in future releases of the cloud data products.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Classification of G -invariant Shallow Neural Networks

When trying to fit a deep neural network (DNN) to a G-invariant target function with G a group, it only makes sense to constrain the DNN to be G-invariant as well. However, there can be many different ways to do this, thus raising the problem of “G-invariant neural architecture design”: What is the optimal Ginvariant architecture for a given problem? Before we can consider the optimization problem itself, we must understand the search space, the architectures in it, and how they relate to one another. In this paper, we take a first step towards this goal; we prove a theorem that gives a classification of all G-invariant single-hidden-layer or “shallow” neural network (G-SNN) architectures with ReLU activation for any finite orthogonal group G, and we prove a second theorem that characterizes the inclusion maps or “network morphisms” between the architectures that can be leveraged during neural architecture search (NAS). The proof is based on a correspondence of every G-SNN to a signed permutation representation of G acting on the hidden neurons; the classification is equivalently given in terms of the first cohomology classes of G, thus admitting a topological interpretation. The G-SNN architectures corresponding to nontrivial cohomology classes have, to our knowledge, never been explicitly identified in the literature previously. Using a code implementation, we enumerate the G-SNN architectures for some example groups G and visualize their structure. Lastly, we prove that architectures corresponding to inequivalent cohomology classes coincide in function space only when their weight matrices are zero, and we discuss the implications of this for NAS.

Agrawal, Devanshu↗
NASA NTRS1995

Reducing neural network training time with parallel processing

Obtaining optimal solutions for engineering design problems is often expensive because the process typically requires numerous iterations involving analysis and optimization programs. Previous research has shown that a near optimum solution can be obtained in less time by simulating a slow, expensive analysis with a fast, inexpensive neural network. A new approach has been developed to further reduce this time. This approach decomposes a large neural network into many smaller neural networks that can be trained in parallel. Guidelines are developed to avoid some of the pitfalls when training smaller neural networks in parallel. These guidelines allow the engineer: to determine the number of nodes on the hidden layer of the smaller neural networks; to choose the initial training weights; and to select a network configuration that will capture the interactions among the smaller neural networks. This paper presents results describing how these guidelines are developed.

Rogers, James L., Jr.↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Evaluation of data driven low-rank matrix factorization for accelerated solutions of the Vlasov equation

Low-rank methods have shown success in accelerating simulations of a collisionless plasma described by the Vlasov equation, but still rely on computationally costly linear algebra every time step. We propose a data-driven factorization method using artificial neural networks, specifically with convolutional layer architecture, that trains on existing simulation data. At inference time, the model outputs a low-rank decomposition of the distribution field of the charged particles, and we demonstrate that this step is faster than the standard linear algebra technique. Numerical experiments show that the method achieves comparable reconstruction accuracy for interpolation tasks, generalizing to unseen test data in a manner beyond just memorizing training data; patterns in factorization also inherently followed the same numerical trend as those within algebraic methods (e.g., truncated singular-value decomposition). However, when training on the first 70% of a time-series data and testing on the remaining 30%, the method fails to meaningfully extrapolate. Despite this limiting result, the technique may have benefits for simulations in a statistical steady-state or otherwise showing temporal stability. These results suggest that while the model offers a computationally efficient alternative for datasets with temporal stability, its current formulation is best suited for interpolation rather than for predicting future states in time-evolving systems. This study thus lays the groundwork for further refinement of neural network-based approaches to low-rank matrix factorization in high-dimensional plasma simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Illinois Express Quantum Network (IEQNET): metropolitan-scale experimental quantum networking over deployed optical fiber

The Illinois Express Quantum Network (IEQNET) is a program to realize metro-scale quantum networking over deployed optical fiber using currently available technology. IEQNET consists of multiple sites that are geographically dispersed in the Chicago metropolitan area. Each site has one or more quantum nodes (Qnodes) representing the communication parties in a quantum network. Q-nodes generate or measure quantum signals such as entangled photons and communicate the results via standard, classical, means. The entangled photons in IEQNET nodes are generated at multiple wavelengths, and are selectively distributed to the desired users via optical switches. Here we describe the network architecture of IEQNET, including the Internet-inspired layered hierarchy that leverages software-defined-networking (SDN) technology to perform traditional wavelength routing and assignment between the Q-nodes. Specifically, SDN decouples the control and data planes, with the control plane being entirely classical. Issues associated with synchronization, calibration, network monitoring, and scheduling will be discussed. An important goal of IEQNET is demonstrating the extent to which the control plane can coexist with the data plane using the same fiber lines. This goal is furthered by the use of tunable narrow-band optical filtering at the receivers and, at least in some cases, a wide wavelength separation between the quantum and classical channels. We envision IEQNET to aid in developing robust and practical quantum networks by demonstrating metro-scale quantum communication tasks such as entanglement distribution and quantum-state teleportation.

Chung, Joaquin↗
DOE OSTI2022

Implementation Aspects of Smart Grids Cyber-Security Cross-Layered Framework for Critical Infrastructure Operation

Communication networks in power systems are a major part of the smart grid paradigm. It enables and facilitates the automation of power grid operation as well as self-healing in contingencies. Such dependencies on communication networks, though, create a roam for cyber-threats. An adversary can launch an attack on the communication network, which in turn reflects on power grid operation. Attacks could be in the form of false data injection into system measurements, flooding the communication channels with unnecessary data, or intercepting messages. Using machine learning-based processing on data gathered from communication networks and the power grid is a promising solution for detecting cyber threats. In this paper, a co-simulation of cyber-security for cross-layer strategy is presented. The advantage of such a framework is the augmentation of valuable data that enhances the detection as well as identification of anomalies in the operation of the power grid. The framework is implemented on the IEEE 118-bus system. The system is constructed in Mininet to simulate a communication network and obtain data for analysis. A distributed three controller software-defined networking (SDN) framework is proposed that utilizes the Open Network Operating System (ONOS) cluster. According to the findings of our suggested architecture, it outperforms a single SDN controller framework by a factor of more than ten times the throughput. This provides for a higher flow of data throughout the network while decreasing congestion caused by a single controller’s processing restrictions. Furthermore, our CECD-AS approach outperforms state-of-the-art physics and machine learning-based techniques in terms of attack classification. The performance of the framework is investigated under various types of communication attacks.

cross-layered↗
DOE OSTI2021

Low-Power Deep Learning Inference using the SpiNNaker Neuromorphic Platform

n this presentation we will discuss recent results on using the SpiNNaker neuromorphic platform (48-chip model) for deep learning neural network inference. We use the Sandia Labs developed Whet stone spiking deep learning library to train deep multi-layer perceptrons and convolutional neural networks suitable for the spiking substrate on the neural hardware architecture. By using the massively parallel nature of SpiNNaker, we are able to achieve, under certain network topologies, substantial network tiling and consequentially impressive inference throughput. Such high-throughput systems may have eventual application in remote sensing applications where large images need to be chipped, scanned, and processed quickly. Additionally, we explore complex topologies that push the limits of the SpiNNaker routing hardware and investigate how that impacts mapping software-implemented networks to on-hardware instantiations.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2004

Algorithm for Training a Recurrent Multilayer Perceptron

An improved algorithm has been devised for training a recurrent multilayer perceptron (RMLP) for optimal performance in predicting the behavior of a complex, dynamic, and noisy system multiple time steps into the future. [An RMLP is a computational neural network with self-feedback and cross-talk (both delayed by one time step) among neurons in hidden layers]. Like other neural-network-training algorithms, this algorithm adjusts network biases and synaptic-connection weights according to a gradient-descent rule. The distinguishing feature of this algorithm is a combination of global feedback (the use of predictions as well as the current output value in computing the gradient at each time step) and recursiveness. The recursive aspect of the algorithm lies in the inclusion of the gradient of predictions at each time step with respect to the predictions at the preceding time step; this recursion enables the RMLP to learn the dynamics. It has been conjectured that carrying the recursion to even earlier time steps would enable the RMLP to represent a noisier, more complex system.

Parlos, Alexander G.↗
DOE OSTI2020

Probing for Artifacts: Detecting Imagenet Model Evasions

While deep learning models have made incredible progress across a variety of machine learning tasks, they remain vulnerable to adversarial examples crafted to fool otherwise trustworthy models. In this work we approach this problem through the lens of a detection framework. We propose a classification network that uses the hidden layer activations of a trained model as inputs to detect adversarial artifacts in an input. We train this classification network simultaneously against multiple adversarial algorithms to create a more robust detector and show higher detection rates than several alternatives. The novelty of our approach is in the scale and scope of probing Imagenet models for adversarial artifacts. In addition, we propose an improvement to feature squeezing, another common adversarial example detection method.

Rounds, Jeremiah↗
NASA NTRS1978

Photospheric models of solar active regions and the network based on the Mg II h and k line wings

From a comparison between observed and computed wings of the Mg II resonance lines, distributions of temperature versus mass column density for solar photospheric layers in plages and in the chromospheric network are derived. The theoretical profiles are computed assuming partial coherent scattering. In the active regions, temperatures exceed those in the quiet sun by up to 200 K near the temperature minimum and up to 400 K in deeper layers. In the observed network structure, the temperature is enhanced by 200 K at the temperature minimum but is the same as that in the quiet sun at greater depths. The difference in the slope of the temperature distribution between the network and plages is real, but may refer only to long elements of the network rather than to the brightest portions. Adjacent to the network is a region in which the temperatures are similar to those in the quiet sun, except immediately below the temperature minimum, where the temperatures are depressed by 150 K.

Morrison, N. D.↗
DOE OSTI2022

DeepZipper: A Novel Deep-learning Architecture for Lensed Supernovae Identification

Large-scale astronomical surveys have the potential to capture data on large numbers of strongly gravitationally lensed supernovae (LSNe). To facilitate timely analysis and spectroscopic follow-up before the supernova fades, an LSN needs to be identified soon after it begins. To quickly identify LSNe in optical survey data sets, we designed ZipperNet, a multibranch deep neural network that combines convolutional layers (traditionally used for images) with long short-term memory layers (traditionally used for time series). We tested ZipperNet on the task of classifying objects from four categories—no lens, galaxy-galaxy lens, lensed Type-Ia supernova, lensed core-collapse supernova—within high-fidelity simulations of three cosmic survey data sets: the Dark Energy Survey, Rubin Observatory’s Legacy Survey of Space and Time (LSST), and a Dark Energy Spectroscopic Instrument (DESI) imaging survey. Among our results, we find that for the LSST-like data set, ZipperNet classifies LSNe with a receiver operating characteristic area under the curve of 0.97, predicts the spectroscopic type of the lensed supernovae with 79% accuracy, and demonstrates similarly high performance for LSNe 1–2 epochs after first detection. We anticipate that a model like ZipperNet, which simultaneously incorporates spatial and temporal information, can play a significant role in the rapid identification of lensed transient systems in cosmic survey experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Tempering kinetics during multilayer laser additive manufacturing of a ferritic steel

Grade 91 steel forms martensite during additive manufacturing and the extent of tempering of martensite significantly affects the mechanical properties of parts. Currently, there is a lack of quantitative understanding of the tempering kinetics for Grade 91 steel, and as a result, the effects of repeated thermal cycles on properties for different processing conditions cannot be determined. Here we evaluate the tempering kinetics by determining the constant terms in the Johnson Mehl Avrami kinetic equation from the tempering data available in the literature and the thermal cycles computed using a rigorously-tested heat and fluid flow model of multi-layer additive manufacturing. The raw tempering data are cleaned using a neural network to enhance accuracy. The lower layers experience repeating cycles of heating and cooling when the upper layers are added. As a result, the hardness is reduced owing to the tempering of martensite. In contrast, martensite formed in the upper layers is not tempered to the same extent and the hardness remains high. Therefore, the hardness of the part increases with the distance from the substrate. Variations in the heat input at different laser powers and scanning speeds significantly affect the extent of tempering. Finally, since the method used here can provide a quantitative understanding of the tempering of martensite and the spatial variation in hardness, it can be used to tailor the microstructure and hardness of heat treatable printed metallic parts.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Regulating the Third Metal to Design and Engineer Multilayered NiFeM (M: Co, Mn, and Cu) Nanofoam Anode Catalysts for Anion-Exchange Membrane Water Electrolyzers

In this study, alkaline anion-exchange membrane water electrolyzers (AEMWEs) for green hydrogen production have received intensive attention due to their feasibility of using earth-abundant platinum group metal (PGM)-free catalysts. Herein, the third metal is incorporated into NiFe-based catalysts to regulate their electronic structures and morphologies, aiming to achieve sufficient oxygen evolution reaction (OER) activity and performance in AEMWEs. The ternary NiFeM (M: Cu, Co, or Mn) catalysts are featured with multiple layered structures and nanofoam network morphologies, consisting of highly OER-active amorphous Ni-rich oxide shells and electrically conductive metallic alloy cores. The physical and electronic perturbations to the NiFe induced by a third element lead to a fine-tuning of the redox ability of the metal sites at the reaction centers, which breaks the scaling relationship between OH* and O* intermediates at the reaction centers. Thus, the unique structural configuration and electronic regulation simultaneously benefit catalytic activity and performance improvements. These NiFeM nanofoam catalysts demonstrated promising anode performance in actual AEMWEs, comparable to the IrO 2 reference, especially at high current densities. Notably, using various electrolytes (e.g., KOH solution or pure water) for AEMWEs exhibited a different performance trend among studied NiFeM catalysts, likely due to dynamic changes of catalysts under various OER environments.

25 ENERGY STORAGE↗
DOE OSTI2024

Hydrogen embrittlement of Zircaloy-4 fabricated by ultrasonic additive manufacturing

Ultrasonic additive manufacturing (UAM) was successfully applied to the zirconium material system to create a planar geometry. Following fabrication, SS-J3 type tensile specimens of the UAM and wrought Zircaloy-4 with a nominal gage section of 5×1.2×0.75 mm were machined for hydriding studies and mechanical testing. The SS-J3 type tensile specimens were gas-charged with hydrogen using a custom system that precisely controls hydrogen gas flow rate, hydrogen partial pressure, and temperature. To avoid altering the UAM material, the maximum process temperature was limited to 550°C. Using different initial hydrogen gas pressures and flow rates, various hydrogen contents (70–1755 wppm) were achieved for Zircaloy-4 specimens. Tensile testing shows that, regardless of hydrogen content, all UAM specimens measured yield strengths in the range of 557±16 MPa and ultimate tensile strengths of 660±4MPa. However, total elongation clearly decreased as a function of increasing hydrogen content. Further, at the lowest hydrogen content, the total plastic elongation measured 21.5%, and this value decreased to less than 2% when the hydrogen content was increased to 1000 wppm. Cross-sectional optical microscopy images revealed that the hydride distributions are randomly oriented. For the low hydrogen content specimen, these randomly distributed hydrides are isolated from each other. A sandwiched structure, consisting of high-density and low-density layers, was also observed for the specimens with hydrogen content between 200 and 400 wppm. As the hydrogen content increases, the hydrides diffuse into the low-density hydride layers to form a network across the whole specimen. Although the tensile properties of UAM and wrought Zircaloy-4 exhibit the same behavior with increasing hydrogen content, the distinctions in grain orientations led to differences in the hydride orientations at low and intermediate hydrogen concentrations.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Hybrid Machine Learning for Scanning Near-Field Optical Spectroscopy

The underlying physics behind an experimental observation often lacks a simple analytical description. This is especially the case for scanning probe microscopy techniques, where the interaction between the probe and the sample is nontrivial. Realistic modeling to include the details of the probe is always exponentially more difficult than its "spherical cow" counterparts. On the other hand, a well-trained artificial neural network based on real data can grasp the hidden correlation between the signal and sample properties. In this work, we show that, via a combination of model calculation and experimental data acquisition, a physics-infused hybrid neural network can predict the tip-sample interaction in the widely used scattering-type scanning near-field optical microscope. This hybrid network provides a long-sought solution for accurate extraction of material properties from tip-specific raw data. The methodology can be extended to other scanning probe microscopy techniques as well as other data-oriented physical problems in general.

36 MATERIALS SCIENCE↗
DOE OSTI2026

From Reproducible Edge–Cloud Experimentation to Real-World Practice: The E2Clab Experience

Reproducibility is already difficult in distributed systems; on the computing continuum, it becomes substantially harder. Applications that span sensing devices, edge and fog resources, and cloud platforms must be evaluated across heterogeneous hardware, variable network conditions, cross-layer orchestration decisions, and long-running workflow lifecycles. We use E2Clab as a case study to examine these challenges and their implications for experimental methodology. We explain why reproducible experimentation is harder on the continuum, then revisit E2Clab as an initial response based on explicit modeling of infrastructure, workflow lifecycle, and artifacts. Lastly, we discuss how its evolution toward more realistic application settings can be understood through the lens of Translational Computer Science. We argue that reproducible continuum experimentation requires methods that are rigorous enough for research while remaining adaptable to real-world practice.

42 ENGINEERING↗
DOE OSTI2020

TorchBraid

Torchbraid is a python package designed for interaction with the PyTorch library. It enables training of neural networks in a layer-parallel fashion.SAND2020-12228 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Cyr, Eric↗