Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network regularization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Cognition at the Point of Sensing

Over the last 15 years, compressive sensing techniques have been developed which have the potential to greatly reduce the amount of data collected by systems while preserving the amount of information obtained. A cost of this efficiency is that a computationally-intensive optimization routine must be used to put the sensed data into a form that a person can interpret. At the same time, machine learning techniques have experienced tremendous growth as well. Machines have demonstrated the ability learn how to effectively perform tasks such as detection and classification at speeds much faster than humanly possible. Our goal in this project was to study the feasibility of using compressive sensing systems "at the edge." That is, how can compressive sensing sensors be deployed such that information is created at the remote sensor rather than sending raw data to a central processing location? Studies were performed to analyze whether machine learning could be done on the compressively sensed data in its raw form. If a machine is performing the task, is it possible to do so without putting the data into a human interpretable form? We show that this is possible for some systems, in particular a compressive sensing snapshot imaging spectrometer. Machine learning tasks were demonstrated to be more effective and more robust to noise when the machine learning algorithm worked on data in its raw form. This system is shown to outperform a traditional spectrometer. Techniques for reducing the complexity of the reconstruction routine were also analyzed. Techniques for such as data regularization, deep neural networks, and matrix completion were studied and shown to have benefits over traditional reconstruction techniques. In this project we showed that compressive sensing sensors are indeed feasible at the edge. As always, sensors and algorithms must be carefully tuned to work in the constrained environment. In this project we developed tools and techniques to enable those analyses.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Improved pore network models to simulate single-phase flow in porous media by coupling with lattice Boltzmann method

In this paper, different pore network models to simulate single-phase flow in porous media are built and their accuracy is evaluated. In addition to the conventional pore network model (CPNM) which consists of regular pore bodies and throat bonds, three improved pore network models (IPNMs) are developed allowing to better describing the real pore and throat geometry. The first improved pore network model (IPNM1) replaces the regular throat bond with a throat bond showing the real throat cross section. The second improvement (IPNM2) uses a series of sub-throat bonds with varying cross sections to better describe the real throat geometry, which is firstly proposed in this paper. The third model (IPNM3) extracts the real pore-throat-pore geometry without simplification. The conductance of fluid flow through these more realistic throat bonds is calculated by the lattice Boltzmann method (LBM). The accuracy and computational efficiency of the different pore network models are evaluated taking the LBM simulation over the whole porous medium as reference solution. The global permeability and detailed pressure distributions in the pores for the different pore network models are validated. The results show that the accuracy of the pore network model increases from CPNM to IPNM3, but at the expense of increasing computational cost. This study suggests that IPNM3 can replace a whole-domain LBM simulation with similar accuracy but much lower computational cost. As a first-order approximation the newly proposed IPNM2 is suggested as good compromise between accuracy and computational cost.

42 ENGINEERING↗
DOE OSTI2020

Bilevel optimization, deep learning and fractional Laplacian regularization with applications in tomography

Here we consider a generalized bilevel optimization framework for solving inverse problems. We introduce fractional Laplacian as a regularizer to improve the reconstruction quality, and compare it with the total variation regularization. We emphasize that the key advantage of using fractional Laplacian as a regularizer is that it leads to a linear operator, as opposed to the total variation regularization which results in a nonlinear degenerate operator. Inspired by residual neural networks, to learn the optimal strength of regularization and the exponent of fractional Laplacian, we develop a dedicated bilevel optimization neural network with a variable depth for a general regularized inverse problem. We illustrate how to incorporate various regularizer choices into our proposed network. As an example, we consider tomographic reconstruction as a model problem and show an improvement in reconstruction quality, especially for limited data, via fractional Laplacian regularization. We successfully learn the regularization strength and the fractional exponent via our proposed bilevel optimization neural network. We observe that the fractional Laplacian regularization outperforms total variation regularization. This is specially encouraging, and important, in the case of limited and noisy data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Dynamic Low-Rank Training with Spectral Regularization: Achieving Robustness in Compressed Representations

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94\% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗
DOE OSTI2025

Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94 compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗
DOE OSTI2020

Stochasticity and robustness in spiking neural networks

Despite drawing inspiration from biological systems which are inherently noisy and variable, artificial neural networks have been shown to require precise weights to carry out the task which they are trained to accomplish. This creates a challenge when adapting these artificial networks to specialized execution platforms which may encode weights in a manner which restricts their accuracy and/or precision.Reflecting back on the non-idealities which are observed in biological systems, we investigated the effect these properties have on the robustness of spiking neural networks under perturbations to weights. First, we examined techniques extant in conventional neural networks which resemble noisy processes, and postulated they may produce similar beneficial effects in spiking neural networks. Second, we evolved a set of spiking neural networks utilizing biological non-idealities to solve a pole-balancing task, and estimated their robustness. We showed it is higher in networks using noisy neurons, and demonstrated that one of these networks can perform well under the variance expected when a hafnium-oxide based resistive memory is used to encode synaptic weights. Lastly, we trained a series of networks using a surrogate gradient method on the MNIST classification task. We confirmed that these networks demonstrate similar trends in robustness to the evolved networks. We discuss these results and argue that they display empirical evidence supporting the role of noise as a regularizer which can increase network robustness.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

O3BNN-R: An Out-Of-Order Architecture for High-Performance and Regularized BNN Inference

Binarized Neural Networks (BNN) have drawn tremendous attention due to significantly reduced computational complexity and memory demand. They have especially shown great potential in cost- and power-restricted domains, such as IoT and smart edge-devices, where reaching a certain accuracy bar is often sufficient, and real-time is highly desired.In this work, we demonstrate that the highly-condensed BNN model can be shrunk significantly further by dynamically pruning irregular redundant edges. Based on two new observations on BNN-specific properties, an out-of-order (OoO) architecture – O3BNN-R, can curtail edge evaluation in cases where the binary output of a neuron can be determined early. Similar to Instruction-Level-Parallelism(ILP), these fine-grained, irregular, runtime pruning opportunities are traditionally presumed to be difficult to exploit. In order to increase the pruning opportunities, we also optimize the training process by adding 2 regularization items in the loss function (1) for pooling pruning and (2) for threshold pruning. We evaluate our design on an FPGA platform using three well-known networks, including VggNet-16, AlexNet for ImageNet, and a VGG-like network for Cifar-10.

Geng, Tong↗
NASA NTRS1995

Increasing cellular coverage within integrated terrestrial/satellite mobile networks

When applying the hierarchical cellular concept, the satellite acts as giant umbrella cell covering a region with some terrestrial cells. If a mobile terminal traversing the region arrives to the border-line or limits of a regular cellular ground service, network transition occurs and the satellite system continues the mobile coverage. To adequately assess the boundaries of service of a mobile satellite system an a cellular network within an integrated environment, this paper provides an optimized scheme to predict when a network transition may be necessary. Under the assumption of a classified propagation phenomenon and Lognormal shadowing, the study applies an analytical approach to estimate the location of a mobile terminal based on a reception of the signal strength emitted by a base station.

Castro, Jonathan P.↗
DOE OSTI2020

Application of a long short-term memory for deconvoluting conductance contributions at charged ferroelectric domain walls

Ferroelectric domain walls are promising quasi-2D structures that can be leveraged for miniaturization of electronics components and new mechanisms to control electronic signals at the nanoscale. Despite the significant progress in experiment and theory, however, most investigations on ferroelectric domain walls are still on a fundamental level, and reliable characterization of emergent transport phenomena remains a challenging task. Here, we apply a neural-network-based approach to regularize local I ( V )-spectroscopy measurements and improve the information extraction, using data recorded at charged domain walls in hexagonal (Er 0.99 ,Zr 0.01 )MnO 3 as an instructive example. Using a sparse long short-term memory autoencoder, we disentangle competing conductivity signals both spatially and as a function of voltage, facilitating a less biased, unconstrained and more accurate analysis compared to a standard evaluation of conductance maps. The neural-network-based analysis allows us to isolate extrinsic signals that relate to the tip-sample contact and separating them from the intrinsic transport behavior associated with the ferroelectric domain walls in (Er 0.99 ,Zr 0.01 )MnO 3 . Our work expands machine-learning-assisted scanning probe microscopy studies into the realm of local conductance measurements, improving the extraction of physical conduction mechanisms and separation of interfering current signals.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Neural posterior unfolding

Differential cross section measurements are the currency of scientific exchange in particle and nuclear physics. A key challenge for these analyses is the correction for detector distortions, known as deconvolution or unfolding. Binned unfolding of cross section measurements traditionally rely on the regularized inversion of the response matrix that represents the detector response, mapping pre-detector (`particle level') observables to post-detector (`detector level') observables. In this paper we introduce Neural Posterior Unfolding, a modern, Bayesian approach that leverages normalizing flows for unfolding. By using normalizing flows for neural posterior estimation, NPU offers several key advantages including implicit regularization through the neural network architecture, fast amortized inference that eliminates the need for repeated retraining, and direct access to the full uncertainty in the unfolded result. In addition to introducing NPU, we implement a classical Bayesian unfolding method called Fully Bayesian Unfolding (FBU) in modern Python so it can also be studied. These tools are validated on simple Gaussian examples and then tested on simulated jet substructure examples from the Large Hadron Collider (LHC). We find that the Bayesian methods are effective and worth additional development to be analysis ready for cross section measurements at the LHC and beyond.

Analysis and statistical methods↗
DOE OSTI2023

GRIDS-Net: Inverse shape design and identification of scatterers via geometric regularization and physics-embedded deep learning

This study presents a deep learning based methodology for both remote sensing and design of acoustic scatterers. The ability to determine the shape of a scatterer, either in the context of material design or sensing, plays a critical role in many practical engineering problems. This class of inverse problems is extremely challenging due to their high-dimensional, nonlinear, and ill-posed nature. To overcome these technical hurdles, we introduce a geometric regularization approach for deep neural networks (DNN) based on non-uniform rational B-splines (NURBS) and capable of predicting complex 2D scatterer geometries in a parsimonious dimensional representation. Then, this geometric regularization is combined with physics-embedded learning and integrated within a robust convolutional autoencoder (CAE) architecture to accurately predict the shape of 2D scatterers in the context of identification and inverse design problems. Further, an extensive numerical study is presented in order to showcase the remarkable ability of this approach to handle complex scatterer geometries while generating physically-consistent acoustic fields. The study also assesses and contrasts the role played by the (weakly) embedded physics in the convergence of the DNN predictions to a physically consistent inverse design.

42 ENGINEERING↗
NASA NTRS2001

A Regularized Neural Net Approach for Retrieval of Atmospheric and Surface Temperatures with the IASI Instrument

Abstract In this paper, a fast atmospheric and surface temperature retrieval algorithm is developed for the high resolution Infrared Atmospheric Sounding Interferometer (IASI) space-borne instrument. This algorithm is constructed on the basis of a neural network technique that has been regularized by introduction of a priori information. The performance of the resulting fast and accurate inverse radiative transfer model is presented for a large divE:rsified dataset of radiosonde atmospheres including rare events. Two configurations are considered: a tropical-airmass specialized scheme and an all-air-masses scheme.

Aires, F.↗
DOE OSTI2021

Learning to Correct Climate Projection Biases

The fidelity of climate projections is often undermined by biases in climate models due to their simplification or misrepresentation of unresolved climate processes. While various bias correction methods have been developed to post-process model outputs to match observations, existing approaches usually focus on limited, low-order statistics, or break either the spatiotemporal consistency of the target variable, or its dependency upon model resolved dynamics. We develop a Regularized Adversarial Domain Adaptation (RADA) methodology to overcome these deficiencies, and enhance efficient identification and correction of climate model biases. Instead of pre-assuming the spatiotemporal characteristics of model biases, we apply discriminative neural networks to distinguish historical climate simulation samples and observation samples. The evidences based on which the discriminative neural networks make distinctions are applied to train the domain adaptation neural networks to bias correct climate simulations. We regularize the domain adaptation neural networks using cycle-consistent statistical and dynamical constraints. An application to daily precipitation projection over the contiguous United States shows that our methodology can correct all the considered moments of daily precipitation at approximately $1^\circ$ resolution, ensures spatiotemporal consistency and inter-field correlations, and can discriminate between different dynamical conditions. Our methodology offers a powerful tool for disentangling model parameterization biases from their interactions with the chaotic evolution of climate dynamics, opening a novel avenue toward big-data enhanced climate predictions.

58 GEOSCIENCES↗
DOE OSTI2023

Accelerating network layouts using graph neural networks

Graph layout algorithms used in network visualization represent the first and the most widely used tool to unveil the inner structure and the behavior of complex networks. Current network visualization software relies on the force-directed layout (FDL) algorithm, whose high computational complexity makes the visualization of large real networks computationally prohibitive and traps large graphs into high energy configurations, resulting in hard-to-interpret “hairball” layouts. Here we use Graph Neural Networks (GNN) to accelerate FDL, showing that deep learning can address both limitations of FDL: it offers a 10 to 100 fold improvement in speed while also yielding layouts which are more informative. We analytically derive the speedup offered by GNN, relating it to the number of outliers in the eigenspectrum of the adjacency matrix, predicting that GNNs are particularly effective for networks with communities and local regularities. Finally, we use GNN to generate a three-dimensional layout of the Internet, and introduce additional measures to assess the layout quality and its interpretability, exploring the algorithm’s ability to separate communities and the link-length distribution. The novel use of deep neural networks can help accelerate other network-based optimization problems as well, with applications from reaction-diffusion systems to epidemics.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2013

Data Center at NICT

The Data Center at the National Institute of Information and Communications Technology (NICT) archives and releases the databases and analysis results processed at the Correlator and the Analysis Center at NICT. Regular VLBI sessions of the Key Stone Project VLBI Network were the primary objective of the Data Center. These regular sessions continued until the end of November 2001. In addition to the Key Stone Project VLBI sessions, NICT has been conducting geodetic VLBI sessions for various purposes, and these data are also archived and released by the Data Center.

Ichikawa, Ryuichi↗
DOE OSTI2024

Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics

Data-driven constitutive modeling with neural networks has received increased interest in recent years due to its ability to easily incorporate physical and mechanistic constraints and to overcome the challenging and time-consuming task of formulating phenomenological constitutive laws that can accurately capture the observed material response. However, even though neural network-based constitutive laws have been shown to generalize proficiently, the generated representations are not easily interpretable due to their high number of trainable parameters. Sparse regression approaches exist that allow for obtaining interpretable expressions, but the user is tasked with creating a library of model forms which by construction limits their expressiveness to the functional forms provided in the libraries. Here, in this work, we propose to train regularized physics-augmented neural network-based constitutive models utilizing a smoothed version of $L^0$-regularization. This aims to maintain the trustworthiness inherited by the physical constraints, but also enables interpretability which has not been possible thus far on any type of machine learning-based constitutive model where model forms were not assumed a priori but were actually discovered. During the training process, the network simultaneously fits the training data and penalizes the number of active parameters, while also ensuring constitutive constraints such as thermodynamic consistency. We show that the method can reliably obtain interpretable and trustworthy constitutive models for compressible and incompressible hyperelasticity, yield functions, and hardening models for elastoplasticity, using synthetic and experimental data. This work aims to set a new paradigm for interpretable machine learning models in the broad area of solid mechanics where low and limited data is available along with prior knowledge of physical constraints that the learned maps need to obey. This paradigm can potentially be extended to a broader spectrum of scientific exploration.

Data-driven constitutive models↗
DOE OSTI2026

Regularization via f -Divergence: An Application to Multi-Oxide Spectroscopic Analysis

In this paper, we explore the application of convolutional neural networks (CNNs) for predicting the chemical composition of complex geologic samples in a simulated Martian atmospheric environment. Specifically, we aim to characterize oxide weight percentages (wt.%) of rock samples analyzed by remote Laser-Induced Breakdown Spectroscopy (LIBS), framing the problem as a multi-target regression task . Neural networks trained on LIBS spectra are prone to overfitting due to high spectral complexity, limited labeled data, and measurement noise. While regularization is critical for improving generalization, common methods (e.g., ℓ 2 regularization) impose constraints not directly tied to data distribution properties. We propose a novel regularization method based on a specific ƒ-divergence induced by a graph-based estimator, designed to constrain the distributional discrepancy between predictions and targets. This regularizer serves a dual purpose: (a) mitigating overfitting by enforcing a constraint on the distributional difference between predictions and noisy targets, and (b) acting as an auxiliary loss that penalizes large divergences. To enable backpropagation, we develop a differentiable approximation of this particular ƒ-divergence, making the method feasible for neural networks. Experiments on ChemCam and SuperCam LIBS calibration spectra show that mathematical equation-divergence regularization outperforms or matches standard regularization methods (ℓ 1 , ℓ 2 , dropout) and the classical baseline, partial least squares (PLS). Combining ƒ-divergence regularization with standard regularization yields further performance gains, indicating that distributional regularization is useful in this context giving a promising direction for robust model training in planetary science applications. Source code is publicly available at Klein and Li (2025), https://doi.org/10.11578/dc.20250530.7.

58 GEOSCIENCES↗
DOE OSTI2025

An adaptive, data-driven multiscale approach for dense granular flows

The accuracy of coarse-grained continuum models of dense granular flows is limited by the lack of high-fidelity closure models for granular rheology. One approach to addressing this issue, referred to as the hierarchical multiscale method, is to use a high-fidelity fine-grained model to compute the closure terms needed by the coarse-grained model. The difficulty with this approach is that the overall model can become computationally intractable due to the high computational cost of the high-fidelity model. In this work, we describe a multiscale modeling approach for dense granular flows that utilizes neural networks trained using high-fidelity discrete element method (DEM) simulations to approximate the constitutive granular rheology for a continuum incompressible flow model. Our approach leverages an ensemble of neural networks to estimate predictive uncertainty that allows us to determine whether the rheology at a given point is accurately represented by the neural network model. Additional DEM simulations are only performed when needed, minimizing the number of additional DEM simulations required when updating the rheology. This adaptive coupling significantly reduces the overall computational cost of the approach while controlling the error. In addition, the neural networks are customized to learn regularized rheological behavior to ensure well-posedness of the continuum solution. We first validate the approach using two-dimensional steady-state and decelerating inclined flows. We then demonstrate the efficiency of our approach by modeling three-dimensional sub-aerial granular column collapse for varying initial column aspect ratios, where our multiscale method compares well with the computationally expensive computational fluid dynamics (CFD)-DEM simulation.

Dense granular flows↗