Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Loss functions”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Fast and accurate learned multiresolution dynamical downscaling for precipitation

Abstract. This study develops a neural-network-based approach for emulating high-resolution modeled precipitation data with comparable statistical properties but at greatly reduced computational cost. The key idea is to use combination of low- and high-resolution simulations (that differ not only in spatial resolution but also in geospatial patterns) to train a neural network to map from the former to the latter. Specifically, we define two types of CNNs, one that stacks variables directly and one that encodes each variable before stacking, and we train each CNN type both with a conventional loss function, such as mean square error (MSE), and with a conditional generative adversarial network (CGAN), for a total of four CNN variants. We compare the four new CNN-derived high-resolution precipitation results with precipitation generated from original high-resolution simulations, a bilinear interpolater and the state-of-the-art CNN-based super-resolution (SR) technique. Results show that the SR technique produces results similar to those of the bilinear interpolator with smoother spatial and temporal distributions and smaller data variabilities and extremes than the original high-resolution simulations. While the new CNNs trained by MSE generate better results over some regions than the interpolator and SR technique do, their predictions are still biased from the original high-resolution simulations. The CNNs trained by CGAN generate more realistic and physically reasonable results, better capturing not only data variability in time and space but also extremes such as intense and long-lasting storms. The new proposed CNN-based downscaling approach can downscale precipitation from 50 to 12 km in 14 min for 30 years once the network is trained (training takes 4 h using 1 GPU), while the conventional dynamical downscaling would take 1 month using 600 CPU cores to generate simulations at the resolution of 12 km over the contiguous United States.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

DSGAN

This study develops a neural network-based approach for emulating high-resolution modeled precipitation data with comparable statistical properties but at greatly reduced computational cost. The key idea is to use combination of low-and high- resolution simulations to train a neural network to map from the former to the latter. Specifically, we define two types of CNNs, one that stacks variables directly and one that encodes each variable before stacking, and we train each CNN type both with a conventional loss function, such as mean square error (MSE), and with a conditional generative adversarial network (CGAN), for a total of four CNN variants. We compare the four new CNN-derived high-resolution precipitation results with precipitation generated from original high resolution simulations, a bilinear interpolater and the state-of-the-art CNN-based super-resolution (SR) technique. Results show that the SR technique produces results similar to those of the bilinear interpolator with smoother spatial and temporal distributions and smaller data variabilities and extremes than the original high resolution simulations. While the new CNNs trained by MSE generate better results over some regions than the interpolator and SR technique do, their predictions are still not as close as the original high resolution simulations. The CNNs trained by CGAN generate more realistic and physically reasonable results, better capturing not only data variability in time and space but also extremes such as intense and long-lasting storms. The new proposed CNN-based downscaling approach can downscale precipitation from 50~km to 12~km in 14~min for 30~years

LIU, ZHENGCHUN↗
DOE OSTI2020

A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Geosteering inversion, which can be viewed as a nonlinear inverse problem, is an important technique used by directional drilling. Traditional methods that rely on iterative procedures and regularization are sensitive to the selection of initial values and can be slow due to convergence issues. In industrial applications, a lookup table is used to produce fast predictions. However, performance is not guaranteed by this approach due to the limitation of the hardware. In this paper, we propose a novel physics-driven deep-learning framework for providing a fast, accurate surrogate to solve the inverse problem. Particularly, leveraged by the forward physical model and 1D convolutional neural network (1D-CNN), the proposed method provides more reliable solutions to the inverse problem with improved performance. In addition, a new physics-driven loss function is introduced to accommodate both the model misfit and the data misfit. In conclusion, our experiments demonstrate the effectiveness of our method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Anderson Acceleration for Distributed Training of Deep Learning Models

Anderson acceleration (AA) is an extrapolation technique that has recently gained interest in the deep learning (DL) community to speed-up the sequential training of DL models. However, when performed at large scale, the DL training is exposed to a higher risk of getting trapped into steep local minima of the training loss function, and standard AA does not provide sufficient acceleration to escape from these steep local minima. This results in poor generalizability and makes AA ineffective. To restore AA’s advantage to speed-up the training of DL models on large scale computing platforms, we combine AA with an adaptive moving average procedure that boosts the training to escape from steep local minima. By monitoring the relative standard deviation between consecutive iterations, we also introduce a criterion to automatically assess whether the moving average is needed. We applied the method to the following DL instantiations for image classification: (i) ResNet50 trained on the open-source CIFAR100 dataset and (ii) ResNet50 trained on the open-source ImageNet1k dataset. Numerical results obtained using up to 1,536 NVIDIA V100 GPUs on the OLCF supercomputer Summit showed the stabilizing effect of the moving average on AA for all the problems above.

Lupo Pasini, Massimiliano↗
DOE OSTI2023

Refining fast simulation using machine learning

At the CMS experiment, a growing reliance on the fast Monte Carlo application (FastSim) will accompany the high luminosity and detector granularity expected in Phase 2. The FastSim chain is roughly 10 times faster than the application based on the GEANT4 detector simulation and full reconstruction referred to as FullSim. However, this advantage comes at the price of decreased accuracy in some of the final analysis observables. In this contribution, a machine learning-based technique to refine those observables is presented. We employ a regression neural network trained with a sophisticated combination of multiple loss functions to provide post-hoc corrections to samples produced by the FastSim chain. The results show considerably improved agreement with the FullSim output and an improvement in correlations among output observables and external parameters. This technique is a promising replacement for existing correction factors, providing higher accuracy and thus contributing to the wider usage of FastSim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Exploring gauge-fixing conditions with gradient-based optimization

Lattice gauge fixing is required to compute gauge-variant quantities, for example those used in RI-MOM renormalization schemes or as objects of comparison for model calculations. Recently, gauge-variant quantities have also been found to be more amenable to signal-to-noise optimization using contour deformations. These applications motivate systematic parameterization and exploration of gauge-fixing schemes. This work introduces a differentiable parameterization of gauge fixing which is broad enough to cover Landau gauge, Coulomb gauge, and maximal tree gauges. The adjoint state method allows gradient-based optimization to select gauge-fixing schemes that minimize an arbitrary target loss function.

Detmold, William↗
DOE OSTI2024

Uncertainty-aware Continuous Implicit Neural Representations for Remote Sensing Object Counting

Many existing object counting methods rely on density map estimation (DME) of the discrete grid representation by decoding extracted image semantic features from designed convolutional neural networks (CNNs). Relying on discrete density maps not only leads to information loss dependent on the original image resolution, but also has a scalability issue when analyzing high-resolution images with cubically increasing memory complexity. Furthermore, none of the existing methods can offer reliable uncertainty quantification (UQ) for the derived count estimates. To overcome these limitations, we design UNcertainty-aware, hypernetwork-based Implicit neural representations for Counting (UNIC) to assign probabilities and the corresponding counting confidence over continuous spatial coordinates. We derive a sampling-based Bayesian counting loss function and develop the corresponding model training algorithm. UNIC outperforms existing methods on the Remote Sensing Object Counting (RSOC) dataset with reliable UQ and improved interpretability of the derived count estimates. Our code is available at https://github.com/SiyuanXu-tamu/UNIC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration

Federated learning (FL) is a distributed machine learning approach that enables multiple local clients and a central server to collaboratively train a model while keeping the data on their own devices. First-order methods, particularly those incorporating variance reduction techniques, are the most widely used FL algorithms due to their simple implementation and stable performance. However, these methods tend to be slow and require a large number of communication rounds to reach the global minimizer. We propose FedOSAA, a novel approach that preserves the simplicity of first-order methods while achieving the rapid convergence typically associated with second-order methods. Our approach applies one Anderson acceleration (AA) step following classical local updates based on first-order methods with variance reduction, such as FedSVRG and SCAFFOLD, during local training. This AA step is able to leverage curvature information from the history points and gives a new update that approximates the Newton-GMRES direction, thereby significantly improving the convergence. We establish a local linear convergence rate to the global minimizer of FedOSAA for smooth and strongly convex loss functions. Numerical comparisons show that FedOSAA substantially improves the communication and computation efficiency of the original first-order methods, achieving performance comparable to second-order methods like GIANT.

Feng, Xue [University of California, Davis]↗
DOE OSTI2021

hippynn Python Package

hippynn is a python package for defining, training, and applying neural networks to atomistic systems. In particular, it focuses on Hierarchical Interacting Particle Neural Networks (HIP-NNs), a deep learning architecture for atomistic systems. HIP-NNs take input data describing the properties of atomistic systems (often generated using ab-initio quantum mechanics) to learn fast and accurate models. A trained HIP-NN can predict potential energy surfaces, atomic charges, and more. hippynn uses PyTorch for portable high-performance code, including both CPU and GPU support. hippynn allows for extensive customization, including user-defined models and loss functions, to facilitate future research into extensions of HIP-NN as well as other atomistic deep learning models.

Lubbers, Nicolas↗
DOE OSTI2023

Real-space Green's function approach for intrinsic losses in x-ray spectra

Intrinsic inelastic losses in x-ray spectra originate from excitations in an interacting electron system due to a suddenly created core-hole. These losses characterize the features observed in x-ray photoemission spectra (XPS), as well as many-body effects such as satellites and edge-singularities in x-ray absorption spectra (XAS). However, they are usually neglected in practical calculations. As shown by Langreth these losses can be treated within linear response in terms of a cumulant Green's function in momentum space. Here we present a complementary ab initio real-space Green's function generalization of the Langreth cumulant in terms of the dynamically screened core-hole interaction W c (ω) and the independent particle response function. Here we find that the cumulant kernel β⁡(ω) is analogous to XAS, but with the transition operator replaced by the core-hole potential with monopole selection rules. The behavior reflects the analytic structure of the loss function, with peaks near the zeros of the dielectric function, consistent with delocalized quasiboson excitations. The approach simplifies when W c (ω) is localized and spherically symmetric. In conclusion, illustrative results and comparisons are presented for the electron gas, sodium, and some early transition metal compounds.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Application of a deep learning semantic segmentation model to helium bubbles and voids in nuclear materials

Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. Additionally, we performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model’s application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles.

42 ENGINEERING↗
DOE OSTI2021

Electronic structure of reduced CeO 2 (111) surfaces interacting with hydrogen as revealed through electron energy loss spectroscopy in comparison with theoretical investigations

Based on both, ab-initio cluster calculations as well as periodic density functional based loss function calculations, we have assigned the origins of the valence electron excitation regime in electron energy loss spectra of well-ordered ceria films in (111) orientation at various states of reduction as well as after exposing the reduced films to hydrogen from the gas phase (Li et al, Angew. Chem. Int. Ed., 58 (2019) 14686–14693). The explicit calculation of intensity distributions using the dipole approximation allow us to draw conclusions about the nature of oxygen vacancies, which occur upon reduction and how those interact with hydrogen to form hydride species. Herein, we also reports a brief discussion of vibrational excitations of hydrogen loaded ceria and corroborates previous interpretations, based on inelastic neutron scattering.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models

Recent progress in scientific machine learning (SciML) has opened up the possibility of training novel neural network architectures that solve complex partial differential equations (PDEs). Several (nearly data free) approaches have been recently reported that successfully solve PDEs, with examples including deep feed forward networks, generative networks, and deep encoder-decoder networks. However, practical adoption of these approaches is limited by the difficulty in training these models, especially to make predictions at large output resolutions (≥1024×1024). Here we report on a software framework for data parallel distributed deep learning that resolves the twin challenges of training these large SciML models - training in reasonable time as well as distributing the storage requirements. Our framework provides several out of the box functionality including (a) loss integrity independent of number of processes, (b) synchronized batch normalization, and (c) distributed higher-order optimization methods. We show excellent scalability of this framework on both cloud as well as HPC clusters, and report on the interplay between bandwidth, network topology and bare metal vs cloud. We deploy this approach to train generative models of sizes hitherto not possible, showing that neural PDE solvers can be viably trained for practical applications. We also demonstrate that distributed higher-order optimization methods are 2-3× faster than stochastic gradient-based methods and provide minimal convergence drift with higher batch-size.

PDEs↗
DOE OSTI2022

Loss of Multiple ABCB Auxin Transporters Recapitulates the Major twisted dwarf 1 Phenotypes in Arabidopsis thaliana

FK506-BINDING PROTEIN 42/TWISTED DWARF 1 (FKBP42/TWD1) directly regulates cellular trafficking and activation of multiple ATP-BINDING CASSETTE (ABC) transporters from the ABCB and ABCC subfamilies. abcb1 abcb19 double mutants exhibit remarkable phenotypic overlap with twd1 including severe dwarfism, stamen elongation defects, and compact circinate leaves; however, twd1 mutants exhibit greater loss of polar auxin transport and additional helical twisting of roots, inflorescences, and siliques. As abcc1 abcc2 mutants do not exhibit any visible phenotypes and TWD1 does not interact with PIN or AUX1/LAX auxin transporters, loss of function of other ABCB auxin transporters is hypothesized to underly the remaining morphological phenotypes. Here, gene expression, mutant analyses, pharmacological inhibitor studies, auxin transport assays, and direct auxin quantitations were used to determine the relative contributions of loss of other reported ABCB auxin transporters (4, 6, 11, 14, 20, and 21) to twd1 phenotypes. From these analyses, the additional reduction in plant height and the twisted inflorescence, root, and silique phenotypes observed in twd1 compared to abcb1 abcb19 result from loss of ABCB6 and ABCB20 function. Additionally, abcb6 abcb20 root twisting exhibited the same sensitivity to the auxin transport inhibitor 1-napthalthalamic acid as twd1 suggesting they are the primary contributors to these auxin-dependent organ twisting phenotypes. The lack of obvious phenotypes in higher order abcb4 and abcb21 mutants suggests that the functional loss of these transporters does not contribute to twd1 root or shoot twisting. Analyses of ABCB11 and ABCB14 function revealed capacity for auxin transport; however, their activities are readily outcompeted by other substrates, suggesting alternate functions in planta, consistent with a spectrum of relative substrate affinities among ABCB transporters. Overall, the results presented here suggest that the ABCB1/19 and ABCB6/20 pairs represent the primary long-distance ABCB auxin transporters in Arabidopsis and account for all reported twd1 morphological phenotypes. Other ABCB transporters appear to participate in highly localized auxin streams or mobilize alternate transport substrates.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Dynamic Disruption Resilience in Intermodal Transport Networks: Integrating Flow Weighting and Centrality Measures

Resilient intermodal freight networks are vital for sustaining supply chains amid increasing threats from natural hazards and cyberattacks. Transportation resilience has been widely studied; understanding how random and targeted disruptions affect structural connectivity and functional performance remains a key challenge. To address this, this study evaluates the robustness of the US intermodal freight network, which consists of rail and water modes, using a simulation-based framework that integrates graph-theoretic metrics with flow-weighted centrality measures. Disruption scenarios are examined, including random failures as well as targeted node and edge removals based on static and dynamically updated degree and betweenness centrality. To reflect more realistic conditions, flow-weighted degree centralities (WDC) and partial node degradation are considered. Two resilience indicators are used: (1) the size of the giant connected component to measure structural connectivity; and (2) flow-weighted network efficiency (NE) to assess freight mobility under disruption. The results show that progressively degrading nodes ranked by WDC to 60% of their original functionality causes a sharper decline in normalized NE, for up to approximately 45 affected nodes, than complete failure (100% loss of functionality) applied to nodes targeted by weighted betweenness centrality or selected at random. This highlights how partial degradation of high-tonnage hubs can produce disproportionately large functional losses. The findings emphasize the need for resilience strategies that go beyond network topology to incorporate freight flow dynamics.

42 ENGINEERING↗
DOE OSTI2021

DYNC1H1 regulates NSCLC cell growth and metastasis by IFN-γ-JAK-STAT signaling and is associated with an aberrant immune response

Highlights: • DYNC1H1 is upregulated and its high expression predicted poor outcomes in NSCLC. • Loss of function of DYNC1H1 inhibits cell growth and metastasis of NSCLC cells in vitro. • DYNC1H1 knockdown inhibited metastasis of NSCLC cells in vivo. • DYNC1H1 loss of function suppresses IFN-γ-JAK-STAT signaling pathway in NSCLC. • DYNC1H1 expression is correlated with aberrant immune response in NSCLC indicated by analysis of TCGA immune data. It is urgent to identify new biomarkers and therapeutic targets to ameliorate the clinical prognosis of patients with lung cancer. The functional significance and molecular mechanism of dynein cytoplasmic 1 heavy chain 1 (DYNC1H1) in nonsmall cell lung cancer (NSCLC) progression is still elusive. In our current study, publicly available data and Western blotting experiments confirmed that DYNC1H1 expression was upregulated in lung cancer samples compared with noncancerous samples. Quantitative real-time PCR (qPCR) results indicated that high DYNC1H1 expression in lung cancer tissues was significantly associated with clinical tumor stage and distal metastasis; moreover, its high expression was negatively correlated with prognosis. Functional experiments demonstrated that DYNC1H1 loss of function caused a significant decrease in cell viability and cell proliferative ability, inhibition of the cell cycle, and promotion of both migration potential and invasion potential in vitro. Animal experiments by tail vein injection of lung cancer cells showed that DYNC1H1 knockdown significantly decreased lung cancer metastasis. Mechanistically, the results from a human protein array showed changes in the IFN-γ-JAK-STAT signaling pathway, and analysis of The Cancer Genome Atlas (TCGA) immune data demonstrated that disturbance of the immune microenvironment might be involved in the impaired growth and metastatic ability mediated by DYNC1H1 loss in NSCLC. DYNC1H1 might serve as a promising biological marker of prognosis and a potential clinical therapeutic target for patients with NSCLC.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Elevated resistance to insects and plant pathogens without compromising seed production

Described herein are modified plants, plant cells, and plant seeds that have at least one mutant loss-of-function JAZ gene and at least one loss-of-function cdk8 gene. Such plants are highly resistant to pests and environmental stress and have restored growth and increased seed yield compared to plant lines with a jazD genetic background.

Howe, Gregg A.↗
DOE OSTI2022

A positive feedback loop involving the Spa2 SHD domain contributes to focal polarization

Focal polarization is necessary for finely arranged cell-cell interactions. The yeast mating projection, with its punctate polarisome, is a good model system for this process. We explored the critical role of the polarisome scaffold protein Spa2 during yeast mating with a hypothesis motivated by mathematical modeling and tested by in vivo experiments. Our simulations predicted that two positive feedback loops generate focal polarization, including a novel feedback pathway involving the N-terminal domain of Spa2. We characterized the latter using loss-of-function and gain-of-function mutants. The N-terminal region contains a Spa2 Homology Domain (SHD) which is conserved from yeast to humans, and when mutated largely reproduced the spa2Δ phenotype. Our work together with published data show that the SHD domain recruits Msb3/4 that stimulates Sec4-mediated transport of Bud6 to the polarisome. There, Bud6 activates Bni1-catalyzed actin cable formation, recruiting more Spa2 and completing the positive feedback loop. We demonstrate that disrupting this loop at any point results in morphological defects. Gain-of-function perturbations partially restored focal polarization in a spa2 loss-of-function mutant without restoring localization of upstream components, thus supporting the pathway order. Thus, we have collected data consistent with a novel positive feedback loop that contributes to focal polarization during pheromone-induced polarization in yeast.

59 BASIC BIOLOGICAL SCIENCES↗