Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “network regularization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Deep learning-based noise filtering toward millisecond order imaging by using scanning transmission electron microscopy

Application of scanning transmission electron microscopy (STEM) to in situ observation will be essential in the current and emerging data-driven materials science by taking STEM’s high affinity with various analytical options into account. As is well known, STEM’s image acquisition time needs to be further shortened to capture a targeted phenomenon in real-time as STEM’s current temporal resolution is far below the conventional TEM’s. However, rapid image acquisition in the millisecond per frame or faster generally causes image distortion, poor electron signals, and unidirectional blurring, which are obstacles for realizing video-rate STEM observation. Here we show an image correction framework integrating deep learning (DL)-based denoising and image distortion correction schemes optimized for STEM rapid image acquisition. By comparing a series of distortion corrected rapid scan images with corresponding regular scan speed images, the trained DL network is shown to remove not only the statistical noise but also the unidirectional blurring. This result demonstrates that rapid as well as high-quality image acquisition by STEM without hardware modification can be established by the DL. The DL-based noise filter could be applied to in-situ observation, such as dislocation activities under external stimuli, with high spatio-temporal resolution.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Single Cell RNA-Seq and Machine Learning Reveal Novel Subpopulations in Low-Grade Inflammatory Monocytes With Unique Regulatory Circuits

Subclinical doses of LPS (SD-LPS) are known to cause low-grade inflammatory activation of monocytes, which could lead to inflammatory diseases including atherosclerosis and metabolic syndrome. Sodium 4-phenylbutyrate is a potential therapeutic compound which can reduce the inflammation caused by SD-LPS. To understand the gene regulatory networks of these processes, we have generated scRNA-seq data from mouse monocytes treated with these compounds and identified 11 novel cell clusters. We have developed a machine learning method to integrate scRNA-seq, ATAC-seq, and binding motifs to characterize gene regulatory networks underlying these cell clusters. Using guided regularized random forest and feature selection, our method achieved high performance and outperformed a traditional enrichment-based method in selecting candidate regulatory genes. Our method is particularly efficient in selecting a few candidate genes to explain observed expression pattern. In particular, among 531 candidate TFs, our method achieves an auROC of 0.961 with only 10 motifs. Finally, we found two novel subpopulations of monocyte cells in response to SD-LPS and we confirmed our analysis using independent flow cytometry experiments. Our results suggest that our new machine learning method can select candidate regulatory genes as potential targets for developing new therapeutics against low grade inflammation.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Multidimensional Distributional Neural Network Output Demonstrated in Super‐Resolution of Surface Wind Speed

Accurate quantification of uncertainty in neural network predictions remains a central challenge for scientific applications involving high-dimensional, correlated data. While existing methods capture either aleatoric or epistemic uncertainty, few offer closed-form, multidimensional distributions that preserve spatial correlation while remaining computationally tractable. In this work, we present a framework for training neural networks with a multidimensional Gaussian loss, generating a closed-form predictive distribution over outputs informed by non-identically distributed training data. Our approach captures aleatoric uncertainty by iteratively estimating the means and covariance matrices, and is demonstrated on a super-resolution example out-of-training-sample. We leverage a Fourier representation of the covariance matrix to stabilize network training and preserve spatial correlation. We introduce a novel regularization strategy—referred to as information sharing—that interpolates between image-specific and global covariance estimates, enabling convergence of the super-resolution downscaling network trained on image-specific distributional loss functions. This framework allows for efficient sampling, explicit correlation modeling, and extensions to more complex distribution families all without disrupting prediction performance. We demonstrate the method on a surface wind speed downscaling task and discuss its broader applicability to uncertainty-aware prediction in scientific models.

17 WIND ENERGY↗
DOE OSTI2023

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ~85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Additionally, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Objectively Identifying Transverse Cirrus Bands in Tropical Cyclones using a Convolutional Neural Network

Transverse cirrus bands (TCBs) are bands of upper-level clouds regularly seen in mesoscale and synoptic-scale weather systems. In tropical cyclones, their appearance has been subjectively linked to intensification and the diurnal cycle. However, these hypothesized relationships have not been rigorously tested due to the subjective nature of TCBs in satellite images. A machine learning technique that successfully identifies TCBs objectively in imagery from the GOES-16 Advanced Baseline Imager (ABI) has been developed to solve this problem. The technique uses a U-Net convolutional neural network (CNN) that assigns a probability to each pixel in an image based on the likelihood of the pixel being associated with a TCB. Using the U-Net CNN, a database of TCBs from 2019 to 2022 was developed for the Atlantic tropical cyclone basin by defining an appropriate probability threshold that defines the difference between TCB and non-TCB pixels. This threshold is where the Jaccard score, calculated using manually identified TCBs and model identified TCBs, is maximized. Statistics for TCB occurrence will also be presented, including the relationships between TCBs and storm relative motion, shear relative direction, cardinal direction, tropical cyclone intensity, tropical cyclone intensification rates, and time of day.

John Mark Mayhall↗
NASA NTRS1978

Rocket ozone sounding network data

During the period December 1976 through February 1977, three regular monthly ozone profiles were measured at Wallops Flight Center, two special soundings were taken at Antigua, West Indies, and at the Churchill Research Range, monthly activities were initiated to establish stratospheric ozone climatology. This report presents the data results and flight profiles for the period covered.

Wright, D. U.↗
NASA NTRS1979

Rocket ozone sounding network data

During the period March 1977 through May 1977, three regular monthly ozone profiles were measured at Wallops Flight Center and three regular monthly ozone profiles were measured at the Churchill Research Range. One additional flight was conducted at Wallops Flight Center in support of Nimbus 4 SBUV. Data results and flight profiles for the period covered are presented.

Wright, D. U.↗
NASA NTRS1998

Analysis of record-breaking low ozone values during the 1997 winter over NDSC Station Lauder, New Zealand

During early August 1997, the ozone column density measured over Lauder was unusually low, with a minimum value of 222 Dobson Units (DU) at August 10. These observations are striking since in August, during the Austral winter, the ozone column density should be heading towards its yearly maximum; The August mean ozone column density measured over Lauder between 1987 and 1996 was 348(+/-28) DU, the lowest monthly average in these ten years was 255 DU. Regular altitude profile measurements of ozone, performed at Network for the Detection of Stratospheric Change (NDSC) station Lauder, make it possible to do a detailed, altitude-resolved, study of the low ozone observations. The measurements show ozone poor air in two altitude regions of the stratosphere: A 'high region', extending from the 600 K to the 1050 K isentrope (25 to 34 km), and a 'low region', below about 550 K (22 km). High resolution reverse trajectory maps of potential vorticity (PV) and ozone mixing ratio, based on the assumption of passive advection by the large-scale three-dimensional winds, show that in the 'high region' the ozone poor air was part of the polar vortex, which was centered off the pole and extended over Lauder for several days, while in the 'low region' the ozone poor air was mixed in from low latitudes. A rapid recovery of the ozone column density, by more than 110 DU within 24 hours, was observed when in the low region an ozone rich filament of the polar vortex moved over Lauder, while in the high region the (ozone poor) high part of the vortex moved away.

Brinksma, E. J.↗
DOE OSTI2020

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRSDate not supplied

Objectively Identifying Transverse Cirrus Bands in Tropical Cyclones using a Convolutional Neural Network

Transverse cirrus bands (TCBs) are bands of upper-level clouds that are regularly seen in mesoscale and synoptic-scale weather systems. In tropical cyclones, their appearance has been subjectively linked to intensification and the diurnal cycle, but these hypothesized relationships have not been rigorously tested due to the difficulty of objectively identifying TCBs in satellite images. This presentation describes a machine learning technique that successfully identifies TCBs in imagery from the GOES-16 Advanced Baseline Imager. The technique uses a convolutional neural network (CNN) that assigns a probability of each pixel in the image being associated with a TCB. Using the CNN, a database of TCBs from 2018 to 2022 was developed for the Atlantic basin. Statistics using this database will be presented, including the relationship between TCBs and deep-layer vertical wind shear, intensity change, and the TC diurnal cycle.

John Mark Mayhall↗
DOE OSTI2024

Graph Analytics on Jellyfish topology

Because large unstructured datasets is important for many science domains, distributed graph analytics is critical to many scientists. Unfortunately, obtaining scaling and performance for irregular communication is challenging because contemporary network interconnects are primarily designed to maximize bandwidths of fixed-neighborhoods large-message exchanges (e.g., stencils). Although there is no consensus on the “best” network topologies for irregular communication, unstructured graph-based interconnects can be more suitable. We analyze three popular graph workloads – clustering, pattern enumeration, and traversal — on comparable networks (in terms of resources and costs) constructed from Jellyfish Random Regular, Dragonfly and Fat tree topologies, varying the routing algorithms. Using packet-level simulations, we demonstrate up to 60% improvement in communication time with Jellyfish due to diversity of the short paths between arbitrary endpoints, which can reduce overall network stalls and congestion.

Graph Analytics, network topology, interconnect, H↗
DOE OSTI2023

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

The ability of recurrent neural networks (RNN) to model nonlinear dynamics of high dimensional process data has enabled data-driven RNN-based fault detection algorithms. Previous studies have focused on detecting faults by identifying the discrepancies in data distribution between the faulty and normal data, as reflected in prediction errors generated by RNN models. However, in industrial processes, variations in data distribution can also result from changes in normal control setpoints and compensatory control adjustments in response to disturbances, making it hard to differentiate between normal and faulty conditions. This paper proposes a fault detection method utilizing a long short-term memory (LSTM) encoder–decoder structure with regularized hidden dynamics and reversible instance normalization (RevIN) to compactly represent high-dimensional measurements for effective monitoring. During training, the hidden states of the model are regularized to form a low-dimensional latent space representation of the original multivariate time series data. As a result, the prediction errors of the latent states can be used to monitor the abnormal dynamic variations, while the reconstruction errors of the measured variables are used to monitor the abnormal static variations. Furthermore, the proposed indices can reflect operating conditions, even when the distribution of test data changes, which helps distinguish faults from normal adjustments and disturbances that controllers can settle. Here, data from numerical simulation and the Tennessee Eastman process are used to illustrate the effectiveness of the proposed fault detection method.

42 ENGINEERING↗
DOE OSTI2020

PANNA: Properties from Artificial Neural Network Architectures

We report prediction of material properties from first principles is often a computationally expensive task. Recently, artificial neural networks and other machine learning approaches have been successfully employed to obtain accurate models at a low computational cost by leveraging existing example data. Here, we present a software package “Properties from Artificial Neural Network Architectures” (PANNA) that provides a comprehensive toolkit for creating neural network models for atomistic systems following the Behler–Parrinello topology. Besides the core routines for neural network training, it includes data parser, descriptor builder for Behler–Parrinello class of symmetry functions and force-field generator suitable for integration within molecular dynamics packages. PANNA offers a variety of activation and cost functions, regularization methods, as well as the possibility of using fully-connected networks with custom size for each atomic species. PANNA benefits from the optimization and hardware-flexibility of the underlying TensorFlow engine which allows it to be used on multiple CPU/GPU/TPU systems, making it possible to develop and optimize neural network models based on large datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

5-year Infrastructure plan at FNAL

Fermilab's 5-year infrastructure plan aims to create a sustainable, safe, and long-lasting pavement network by prioritizing lower-rated pavements first for regular maintenance. FNAL should address the most distressed areas promptly, to extend sitewide pavement longevity and reduce overall maintenance costs. This approach compounds improvements by freeing future year’s capital for more routine pavement maintenance. This allows greater surface areas to be addressed each year, until a sustainable cycle can be achieved. The continually improvements enhance safety, benefit employees, visitors, and equipment transportation while aligning with Fermilab's commitment to environmental responsibility.

Washington, Davion↗
NASA NTRS1987

Meteor velocity distribution and an optimum monitoring mathematical model

At present, there are a great number of radio meteor, ionosphere and rocket observation data for the altitude range of 80 to 100 km which indicate the existence of large scale circulation systems in the mesopause to low thermosphere range which change regularly with season and latitude. But the existing observation network and observation programs are not optimal for revealing the main factors forming the circulation mode at these altitudes. A generalized optimum monitoring mathematical model is offered for consideration. The model input data are distribution density, response function, individual measurement root mean square uncertainty and detection effectiveness function. The model makes it possible to obtain the observation distribution density, the minimal possible dispersion and optimized system effectiveness.

Volkov, N. G.↗
NASA NTRS2008

System Engineering on the Use for Ares I,V - the Simpler, the Better

The Ares I and Ares V Vehicles will utilize the J-2X rocket engine developed for NASA by the Pratt & Whitney Rocketdyne Company. The J-2X is an improved higher power version of the original J-2 engine used during the Apollo program. With higher power and updated requirements for safety and performance, the J-2X becomes a new engine using state-of-the-art design methodology, materials and manufacturing processes. The implementation of Systems Engineering (SE) principles enables the rapid J-2X development program to remain aligned with the ARES I and V vehicle programs, Meeting the aggressive development schedule is a challenge. Coordinating the best expertise thai NASA and PWR have to offer requires effectively utilizing resources at multiple sites. This presents formidable communication challenges. SE allows honest and open discussions of issues and problems. This simple idea is often overlooked in large and complex SE programs. Regular and effective meetings linking SE objectives to component designs are used to voice differences of opinions with customer and contractor in attendance so that the best mutual decisions can be made on the shortest possible schedule. Regular technical interchange meetings on secure program wide computer networks and CM processes are effective,in the "Controlled Change" process that exemplifies good SE. Good communication is a key effective SE implementation. The System of Systems approach is the vision of the Orion program which facilitates the establishment of dynamic SE processes at all levels including the engine. SE enables requirements evolution by facilitating organizational and process agility. Flow down and distribution of requirements is controlled by Allocation Reports which breakdown numerical design objectives (weight, reliability, etc.) into quanta goals for each component area. Linked databases of design and verification requirements helps eliminate redundancy and potential mistakes inherent m separated systems. Another tool, the Architecture Design Description, is being used to control J-2X system architecture and effectively communicate configuration changes to those involved in the design process. But the proof is in successful program accomplishment. The SE is the methodology being used to meet the challenge of completing J-2X engine certification 2 years ahead of any engine program ever developed at PWR. The Ares I SE system of systems has delivered according to expectations thus far. All major design reviews (SRR. PDR, CDR) have been successfully conducted to satisfy overall program objectives using SE as the basis for accomplishment. The paper describes SE tools and techniques utilized to achieve this success.

Kelly, William↗
NASA NTRSDate not supplied

Evaluating Long-Term Changes in Atmospheric Ozone

Ozonesondes have made inexpensive, accurate measurements of ozone from the ground to 30km for more than 50 years. The data are used extensively for trend analyses and for evaluation of satellite and model data products, and are also part of climatologies that are used as a priori data for satellite retrievals. They are essential as a transfer standard when merging shorter satellite-derived time series, and are the most important source of trend-quality long-term records below about 18 km. The importance of long-term ozonesonde records as a stable reference has led to increased attention to quantifying uncertainties and changes in ozonesonde data. Based on past intercomparison data, ECC sondes show a modest (~1-5%) high bias with respect to UV-absorption measurements (including MOZAIC-IAGOS) in the troposphere, with an uncertainty of 5%, but no evidence of a change with time. Other sonde types show an increase of 5-20% in sensitivity to tropospheric ozone from 1970-1995. Agreement in the stratosphere is much better. In the past 20 years ozonesonde precision has improved by a factor of 2, primarily through the adoption of strict standard operating procedures. In addition, many of the most important ozonesonde records have been re-evaluated and corrected, and detailed estimates made of their biases and uncertainties. Corrections to historical data for known issues may reduce biases but simultaneously introduce additional uncertainties. The uncertainty budget for the ozone partial pressure reading has contributions from stoichiometry, cell background current, pump efficiency and temperature, sensing solution type and volume. The much-discussed “background current” has recently been shown to be primarily related to reaction stoichiometry. Several quality assurance issues remain, but are tractable problems that can be addressed with further research. This will be required if the present goal of better than 5% overall uncertainty throughout the global ozonesonde network is to be achieved. Finally, the importance of regular sonde intercomparisons, employing UV photometers traceable to the modern UV-absorption standard, and of regular comparison of operational station records with multiple satellite sensors, is emphasized.

Ozonesondes↗
DOE OSTI2020

Optimization-based control using input convex neural networks

Input convex neural networks (ICNNs) are a family of deep learning models where the outputs are constructed to be convex functions of the inputs. By parameterizing system models using ICNNs, optimization-based control problems can be solved as convex optimization problems, leading to improved performance and robustness. This work proposes a novel framework where the control objective function and constraints are modelled using ICNNs. A case study of optimization-based control with output constraints is conducted on a process with input multiplicity and nonminimum phase behavior. The simulation results demonstrate improved economic yield compared with normal neural networks. Additionally, the input convexity formulation is compared with simple regularization techniques, and unique benefits such as improved data efficiency and robustness of the proposed formulation are shown. Finally, by explicitly incorporating prior knowledge about convexity, this framework provides a good balance between the universal approximation power of deep learning and computational feasibility required by control.

42 ENGINEERING↗