Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “loss function regularization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

CFM-ID 4.0 – a web server for accurate MS-based metabolite identification

The CFM-ID 4.0 web server (https://cfmid.wishartlab.com) is an online tool for predicting, annotating and interpreting tandem mass (MS/MS) spectra of small molecules. It is specifically designed to assist researchers pursuing studies in metabolomics, exposomics and analytical chemistry. More specifically, CFM-ID 4.0 supports the: 1) prediction of electrospray ionization quadrupole time-of-flight tandem mass spectra (ESI-QTOF-MS/MS) for small molecules over multiple collision energies (10 eV, 20 eV, and 40 eV); 2) annotation of ESI-QTOF-MS/MS spectra given the structure of the compound; and 3) identification of a small molecule that generated a given ESI-QTOF-MS/MS spectrum at one or more collision energies. The CFM-ID 4.0 web server makes use of a substantially improved MS fragmentation algorithm, a much larger database of experimental and in silico predicted MS/MS spectra and improved scoring methods to offer more accurate MS/MS spectral prediction and MS/MS-based compound identification. Compared to earlier versions of CFM-ID, this new version has an MS/MS spectral prediction performance that is ~22% better and a compound identification accuracy that is ~35% better on a standard (CASMI 2016) testing dataset. CFM-ID 4.0 also features a neutral loss function that allows users to identify similar or substituent compounds where no match can be found using CFM-ID’s regular MS/MS-to-compound identification utility. Finally, the CFM-ID 4.0 web server now offers a much more refined user interface that is easier to use, supports molecular formula identification (from MS/MS data), provides more interactively viewable data (including proposed fragment ion structures) and displays MS mirror plots for comparing predicted with observed MS/MS spectra. These improvements should make CFM-ID 4.0 much more useful to the community and should make small molecule identification much easier, faster, and more accurate.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Multidimensional Distributional Neural Network Output Demonstrated in Super‐Resolution of Surface Wind Speed

Accurate quantification of uncertainty in neural network predictions remains a central challenge for scientific applications involving high-dimensional, correlated data. While existing methods capture either aleatoric or epistemic uncertainty, few offer closed-form, multidimensional distributions that preserve spatial correlation while remaining computationally tractable. In this work, we present a framework for training neural networks with a multidimensional Gaussian loss, generating a closed-form predictive distribution over outputs informed by non-identically distributed training data. Our approach captures aleatoric uncertainty by iteratively estimating the means and covariance matrices, and is demonstrated on a super-resolution example out-of-training-sample. We leverage a Fourier representation of the covariance matrix to stabilize network training and preserve spatial correlation. We introduce a novel regularization strategy—referred to as information sharing—that interpolates between image-specific and global covariance estimates, enabling convergence of the super-resolution downscaling network trained on image-specific distributional loss functions. This framework allows for efficient sampling, explicit correlation modeling, and extensions to more complex distribution families all without disrupting prediction performance. We demonstrate the method on a surface wind speed downscaling task and discuss its broader applicability to uncertainty-aware prediction in scientific models.

17 WIND ENERGY↗
DOE OSTI2026

FunDiff: diffusion models over function spaces for physics-informed generative modeling

Recent advances in generative modeling-particularly diffusion models and flow matching-have been widely used for synthesizing discrete data such as images and videos. However, adapting these models to physical applications remains challenging, as the quantities of interest are continuous functions governed by complex physical laws. To address this, we introduce FunDiff, an efficient and robust framework for generative modeling in function spaces. FunDiff combines a latent diffusion process with a function autoencoder architecture to handle input functions with varying discretizations, generates continuous functions that can be evaluated at arbitrary locations, and seamlessly incorporate physical priors. These priors are enforced through architectural constraints or physics-informed loss functions, ensuring that generated samples satisfy fundamental physical laws. We theoretically establish minimax optimality guarantees for density estimation in function spaces, demonstrating that diffusion-based estimators achieve optimal convergence rates under suitable regularity conditions. We further demonstrate the practical effectiveness of FunDiff across diverse applications in fluid dynamics and solid mechanics. Empirical results indicate that our method can generate physically consistent samples with high fidelity to the target distribution, and exhibit robustness to noisy and low-resolution data.

Wang, Sifan [Yale University, New Haven, CT (Unite↗
DOE OSTI2023

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Abstract Physics-informed neural networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schrödinger and Burgers’ equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

How Well Does Kohn–Sham Regularizer Work for Weakly Correlated Systems?

Kohn–Sham regularizer (KSR) is a differentiable machine learning approach to finding the exchange-correlation functional in Kohn–Sham density functional theory that works for strongly correlated systems. Here we test KSR for a weak correlation. We propose spin-adapted KSR (sKSR) with trainable local, semilocal, and nonlocal approximations found by minimizing density and total energy loss. We assess the atoms-to-molecules generalizability by training on one-dimensional (1D) H, He, Li, Be, and Be 2+ and testing on 1D hydrogen chains, LiH, BeH 2 , and helium hydride complexes. The generalization error from our semilocal approximation is comparable to other differentiable approaches, but our nonlocal functional outperforms any existing machine learning functionals, predicting ground-state energies of test systems with a mean absolute error of 2.7 mH.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

AADL: Anderson Accelerated Deep Learning

We propose a stable, distributed approach to perform AA that accelerates the convergence rate of stochastic first-order optimizers to train neural networks. Differently from previous works, we do not alter neither the scheme to perform AA nor the loss function minimized during the training. To improve robustness against stagnation, we customize general guidelines that suggest to relax the frequency of AA corrections by performing AA only at the end of an entire training epoch. To improve robustness of AA against the stochastic oscillations of first-order optimizers, we average the gradients computed on consecutive stochastic optimization updates. The improved regularity of the converging sequence and the reduced amplitude of stochastic oscillations across consecutive optimization steps allows AA to efficiently extrapolate an improved converging sequence, thereby overcoming limitations of existing approaches to perform AA on stochastic optimization.

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗
DOE OSTI2025

An end-to-end deep learning method for solving nonlocal Allen–Cahn and Cahn–Hilliard phase-field models

Here, we propose an efficient end-to-end deep learning method for solving nonlocal Allen–Cahn (AC) and Cahn–Hilliard (CH) phase-field models. One motivation for this effort emanates from the fact that discretized partial differential equation-based AC or CH phase-field models result in diffuse interfaces between phases, with the only recourse for remediation is to severely refine the spatial grids in the vicinity of the true moving sharp interface whose width is determined by a grid-independent parameter that is substantially larger than the local grid size. In this work, we introduce non-mass conserving nonlocal AC or CH phase-field models with regular, logarithmic, or obstacle double-well potentials. Because of non-locality, some of these models feature totally sharp interfaces separating phases. The discretization of such models can lead to a transition between phases whose width is only a single grid cell wide. Another motivation is to use deep learning approaches to ameliorate the otherwise high cost of solving discretized nonlocal phase-field models. To this end, loss functions of the customized neural networks are defined using the residual of the fully discrete approximations of the AC or CH models, which results from applying a Fourier collocation method and a temporal semi-implicit approximation. To address the long-range interactions in the models, we tailor the architecture of the neural network by incorporating a nonlocal kernel as an input channel to the neural network model. We then provide the results of extensive computational experiments to illustrate the accuracy, predictive capabilities, and cost reductions of the proposed method.

42 ENGINEERING↗
DOE OSTI2022

ARPIST: Provably accurate and stable numerical integration over spherical triangles

Numerical integration on spherical triangles, including the computation of their areas, is a core computation in geomathematics. The commonly used techniques sometimes suffer from instabilities and significant loss of accuracy. We describe a new algorithm, called ARPIST, for accurate and stable integration of functions on spherical triangles. ARPIST is based on an easy-to-implement transformation to the spherical triangle from its corresponding linear triangle via radial projection to achieve high accuracy and efficiency. More importantly, ARPIST overcomes potential instabilities in computing the Jacobian of the transformation, even for poorly shaped triangles that may occur at poles in regular longitude-latitude meshes, by avoiding potential catastrophic rounding errors. We compare our proposed technique with L’Huilier’s Theorem for computing the area of spherical triangles, and also compare it with the recently developed LSQST method (Beckmann et al., 2014) and a radial-basis-function-based technique (Reeger and Fornberg, 2016) for integration of smooth functions on spherical triangulations. In conclusion, our results show that ARPIST enables better or comparable accuracy over previous methods while being easier to implement, significantly faster, and more tolerant of poor element shapes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

CRAGE-CRISPR facilitates rapid activation of secondary metabolite biosynthetic gene clusters in bacteria

With the advent of genome sequencing and mining technologies, secondary metabolite biosynthetic gene clusters (BGCs) within bacterial genomes are becoming easier to predict. For subsequent BGC characterization, clustered regularly interspaced short palindromic repeats (CRISPR) has contributed to knocking out target genes and/or modulating their expression; however, CRISPR is limited to strains for which robust genetic tools are available. Here we present a strategy that combines CRISPR with chassis-independent recombinase-assisted genome engineering (CRAGE), which enables CRISPR systems in diverse bacteria. To demonstrate CRAGE-CRISPR, we select 10 polyketide/non-ribosomal peptide BGCs in Photorhabdus luminescens as models and create their deletion and activation mutants. Subsequent loss- and gain-of-function studies confirm 22 secondary metabolites associated with the BGCs, including a metabolite from a previously uncharacterized BGC. These results demonstrate that the CRAGE-CRISPR system is a simple yet powerful approach to rapidly perturb expression of defined BGCs and to profile genotype-phenotype relationships in bacteria.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

CaloTrilogy: Toward a Breakthrough in One-Step, End-to-End, Physics-Guided Shower Generation for Modern Calorimeters

High-precision calorimeter simulation at current and future colliders imposes rapidly growing computational demands, motivating the development of machine-learning surrogates for traditional Monte Carlo tools such as Geant4. Flow matching and diffusion-based generative models have become leading approaches for high-dimensional fast simulation because of their sample quality, but typically require ${\cal O}(100)$ function evaluations at inference and often rely on auxiliary networks to constrain global observables, compromising streamlined end-to-end generation. We introduce a unified framework that improves the balance between speed, shower quality, and physics fidelity. The method combines: (i) an average velocity field integrator that enables sampling in one or a few evaluations; (ii) a learned generative prior in shower space, constructed from data rather than random noise; and (iii) physics-guided loss terms that impose inductive biases on key observables during training. These elements are training time regularizers, preserving end-to-end inference with no additional cost. With only one or a few evaluation steps, the model achieves shower quality competitive with state-of-the-art flow and diffusion approaches, tested on several public high granularity calorimeter datasets. The results demonstrate inter-layer shower structure consistent with the underlying physics, providing a strong candidate for future fast simulation workflows.

Jiang, Cheng [Edinburgh U.]↗
DOE OSTI2025

Physics-guided dual implicit neural representations for source separation

Significant challenges exist in efficient data analysis of most advanced experimental and observational techniques because the collected signals often include unwanted contributions, such as background and signal distortions, that can obscure the physically relevant information of interest. To address this, we have developed a self-supervised machine-learning approach for source separation using a dual implicit neural representation framework that jointly trains two neural networks: one for approximating distortions of the physical signal of interest and the other for learning the effective background contribution. Our method learns directly from the raw data by minimizing a reconstruction-based loss function without requiring labeled data or pre-defined dictionaries. We demonstrate the effectiveness of our framework by considering a challenging case study involving large-scale simulated, as well as experimental, momentum-energy-dependent inelastic neutron scattering data in a four-dimensional parameter space, characterized by heterogeneous background contributions and unknown distortions to the target signal. The method is found to successfully separate physically meaningful signals from a complex or structured background even when the signal characteristics vary across all four dimensions of the parameter space. An analytical approach that informs the choice of the regularization parameter is presented. Our method offers a versatile framework for addressing source separation problems across diverse domains, ranging from superimposed signals in astronomical measurements to structural features in biomedical image reconstructions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Enhancing Interpretability in Generative Modeling: Statistically Disentangled Latent Spaces Guided by Generative Factors in Scientific Datasets

This study addresses the challenge of statistically extracting generative factors from complex, high-dimensional datasets in unsupervised or semi-supervised settings. We investigate encoder-decoder-based generative models for nonlinear dimensionality reduction, focusing on disentangling low-dimensional latent variables corresponding to independent physical factors. Introducing Aux-VAE, a novel architecture within the classical Variational Autoencoder framework, we achieve disentanglement with minimal modifications to the standard VAE loss function by leveraging prior statistical knowledge through auxiliary variables. These variables guide the shaping of the latent space by aligning latent factors with learned auxiliary variables. We validate the efficacy of Aux-VAE through comparative assessments on multiple datasets, including astronomical simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Inverter Reliability Estimation for Advanced Inverter Functionality

In the near future, grid operators are expected to regularly use advanced distributed energy resource (DER) functions, defined in IEEE 1547-2018, to perform a range of grid-support operations. Many of these functions adjust the active and reactive power of the device through commanded or autonomous modes, which will produce new stresses on the grid-interfacing power electronics components, such as DC/AC inverters. In previous work, multiple DER devices were instrumented to evaluate additional component stress under multiple reactive power setpoints. We utilize quasi-static time-series simulations to determine voltage-reactive power mode (volt-var) mission profile of inverters in an active power system. Mission profiles and loss estimates are then combined to estimate the reduction of the useful life of inverters from different reactive power profiles. It was found that the average lifetime reduction was approximately 0.15% for an inverter between standard unity power factor operation and the IEEE 1547 default volt-var curve based on thermal damage due to switching in the power transistors. For an inverter with an expected 20-year lifetime, the 1547 volt-var curve would reduce the expected life of the device by 12 days. This framework for determining an inverter's useful life from experimental and modeling data can be applied to any failure mechanism and advanced inverter operation.

component degradation↗
DOE OSTI2022

Automating Component-Level Stress Measurements for Inverter Reliability Estimation

In the near future, grid operators are expected to regularly use advanced distributed energy resource (DER) functions, defined in IEEE 1547-2018, to perform a range of grid-support operations. Many of these functions adjust the active and reactive power of the device through commanded or autonomous operating modes which induce new stresses on the power electronics components. In this work, an experimental and theoretical framework is introduced which couples laboratory-measured component stress with advanced inverter functionality and derives a reduction in useful lifetime based on an applicable reliability model. Multiple DER devices were instrumented to calculate the additional component stress under multiple reactive power setpoints to estimate associated DER lifetime reductions. A clear increase in switch loss was demonstrated as a function of irradiance level and power factor. This is replicated in the system-level efficiency measurements, although magnitudes were different—suggesting other loss mechanisms exist. Using an approximate Arrhenius thermal model for the switches, the experimental data indicate a lifetime reduction of 1.5% when operating the inverter at 0.85 PF—compared to unity PF—assuming the DER failure mechanism thermally driven within the H-bridge. If other failure mechanisms are discovered for a set of power electronics devices, this testing and calculation framework can easily be tailored to those failure mechanisms.

Flicker, Jack (ORCID:0000000212232154)↗
DOE OSTI2022

Microbial community shifts correspond with suppression of decomposition 25 years after liming of acidic forest soils

Microbial community structure and function regularly covary with soil pH, yet effects of these interactions on soil carbon are rarely tested experimentally within natural ecosystems. We investigated the enduring (25 year) impacts of liming on microbial community structure and decomposition at an acidic northern hardwood forest, where experimental liming increased pH one unit and surprisingly doubled the organic carbon stocks of the forest floor. Here we show that this increase in carbon storage corresponded with restructuring of the bacterial and fungal communities that drive decomposition. In the Oe horizon, liming reduced the activities of five extracellular enzymes that mediate decomposition, while the Oa horizon showed an especially large (64%) reduction in the activity of a sixth, peroxidase, which is an oxidative enzyme central to lignocellulose degradation. Decreased enzyme activities corresponded with loss of microbial taxa important for lignocellulose decay, including large reductions in the dominant ectomycorrhizal genera Russula and Cenococcum, saprotrophic and wood decaying fungi, and Actinobacteria (Thermomonosporaceae). These results demonstrate the importance of pH as a dominant regulator of microbial community structure and illustrate how changes to this structure can produce large, otherwise unexpected increases in carbon storage in forest soils.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Robust wind farm layout optimization

Wake interactions in wind farms cause losses in annual energy production (AEP) on the order of 10%. Wind farm designers optimize the layout of the farm to mitigate wake losses, especially in the dominant site-specific wind directions. As wind turbines and wind farms grow in scale, optimization becomes more complex. Offshore wind farms regularly comprise more than 100 wind turbines and are characterized by complex boundaries due to shipping lanes, neighboring wind farms, and other constraints. Layout optimization methods are broadly split between gradient-based and gradient-free approaches. Gradient-based approaches can converge quickly and perform well for smaller, academic problems but are often sensitive to initial conditions and tuning parameters and require expert knowledge to use. On the other hand, gradient-free approaches can be more robust to problem complexities. We present a robust layout optimization approach based on a random search algorithm. The algorithm is intended for those who are not optimization experts and has few tuning parameters that need specification to achieve satisfactory results. Unlike off-the-shelf methods, which use generally available, non-domain-specific optimization routines that accept as inputs an optimization function and constraint definitions, this approach takes advantage of the relative computational costs of the different evaluations by evaluating cheaper computations first (boundary and minimum distance constraints) and running expensive AEP evaluations only if all other checks pass. Moreover, an outer genetic algorithm allows multiple solutions to evolve in parallel, enabling rapid solution development on high-performance computers. We discuss the relative ease of selecting necessary tuning parameters and demonstrate the efficacy of the genetic random search on a complex layout problem consisting of placing 70 turbines in a nonconvex and unconnected boundary region.

17 WIND ENERGY↗
DOE OSTI2022

A fourth-order phase-field fracture model: Formulation and numerical solution using a continuous/discontinuous Galerkin method

Modeling crack initiation and propagation in brittle materials is of great importance to be able to predict sudden loss of load-carrying capacity and prevent catastrophic failure under severe dynamic loading conditions. Second-order phase-field fracture models have gained wide adoption given their ability to capture the formation of complex fracture patterns, e.g. via crack merging and branching, and their suitability for implementation within the context of the conventional finite element method. Higher-order phase-field models have also been proposed to increase the regularity of the exact solution and thus increase the spatial convergence rate of its numerical approximation. However, they require special numerical techniques to enforce the necessary continuity of the phase field solution. In this paper, we derive a fourth-order phase-field model of fracture in two independent ways; namely, from Hamilton’s principle and from a higher-order micromechanics-based approach. The latter approach is novel, and provides a physical interpretation of the higher-order terms in the model. In addition, we propose a continuous/discontinuous Galerkin (C/DG) method for use in computing the approximate phase-field solution. This method employs Lagrange polynomial shape functions to guarantee -continuity of the solution at inter-element boundaries, and enforces the required regularity with the aid of additional variational and interior penalty terms in the weak form. Finally, the phase-field equation is coupled with the momentum balance equation to model dynamic fracture problems in hyper-elastic materials. Two benchmark problems are presented to compare the numerical behavior of the C/DG method with mixed finite element methods.

42 ENGINEERING↗