Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “gradient methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Chemically Amplified, Dry-Develop Poly(aldehyde) Photoresist

The catalytic decomposition of poly(phthalaldehyde) with a photoacid generator can be used as dry-develop photoresist, where the exposed film depolymerizes into small molecules to allow the development of features via controlled vaporization. Higher temperatures enabled shorter dry-development times, but also promoted faster photoacid diffusion that compromised pattern fidelity. Trihexylamine was used as a base quencher to counteract acid diffusion in a phthalaldehyde-propanal co-polymer photoresist. The propanal co-monomer in the polymer improves the vaporization rate because it has a higher vapor pressure than phthalaldehyde. Addition of the base quencher was found to improve the contrast, pattern fidelity, and ease-of-handling of the dry-develop resist in a direct-write UV lithography tool. The dry-development of 4 μ m features was achieved with no appreciable residue. For large area features, a spatially variable exposure method was used to direct the residue away from the exposed area. The gradient exposure method was used to produce 100 μ m features. Plasma etching after dry-development was also used to achieve residue-free dry-developed patterns. These results show the benefits of incorporating base additives into a dry-develop depolymerizable resist system and highlight the need for addressing residue formation.

Materials Science↗
DOE OSTI2021

Preliminary Theoretical Analysis of Mixed Precision Krylov Subspace Methods (Q3 Report)

The third quarter of the project was spent performing theoretical finite precision analysis of Krylov subspace method variants that use mixed precision. Our focus here is on the Conjugate Gradient (CG) method and the Lanczos method. We have performed an analysis of maximum attainable accuracy for the classical CG method in which 3 precisions are used: a working precision ε, a precision ε IP for the inner product computations, and a precision ε MV for the matrix-vector products. Our results show that performing inner product computations in lower precision does not affect the attainable accuracy. Further, we have performed a complete error analysis of the s-step Lanczos algorithm. In this case, we show that the numerical behavior of the algorithm can be significantly improved by using extra precision in a small part of the computation. We summarize the main theorems in the remainder of the document. Other activities include attending biweekly xSDK meetings. The subsequent quarter will be spent finalizing these results into technical reports and/or manuscripts for submission to journals, as well as identifying opportunities for future work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Significance of Low‐Velocity Zones on Solute Retention in Rough Fractures

Natural fractures are characterized by high internal heterogeneity. This internal variability is the cause of flow channeling, which in turn leads to contaminant transport taking place primarily along the high-velocity channels. Mass exchange between the high-velocity channels and the low-velocity zones has the potential to enhance contaminant retention, due to solute diffusion into the low-velocity zones and subsequent exposure to additional surface area for diffusion into the bordering rock matrix. Here, we derive a random walk particle tracking method for heterogeneous fractures, which includes an additional term to account for the aperture gradient. The method takes into account advection, diffusion in the fracture and matrix diffusion. The developed numerical framework is applied to assess the effect of low-velocity zones in rough self-affine fractures. The results show that diffusion into low-velocity zones has a visible but modest impact on contaminant retention. The magnitude of this impact does not change considerably, regardless of whether diffusion into the rock matrix is considered in the model, and increases for a decreasing average Péclet number of the fracture.

58 GEOSCIENCES↗
DOE OSTI2024

A Hierarchical OPF Algorithm with Improved Gradient Evaluation in Three-Phase Networks

Linear approximation commonly used in solving alternating-current optimal power flow (AC-OPF) simplifies the system models but incurs accumulated voltage errors in large power networks. Such errors will make the primal-dual type gradient algorithms converge to solutions with voltage violation. In this paper, we improve a recent hierarchical OPF algorithm that rested on primal-dual gradients evaluated with a linearized distribution power flow model. Specifically, we propose a more accurate gradient evaluation method based on an unbalanced three-phase nonlinear distribution power flow model to mitigate the errors arising from linearization. The resultant gradients feature a blocked structure that enables our development of an improved hierarchical primal-dual algorithm to solve the OPF problem. Numerical results on the IEEE 123-bus test feeder and a 4,518-node test feeder show that the proposed method can enhance voltage safety at comparable computational efficiency with the linearized algorithm.

approximation algorithms↗
DOE OSTI2025

Accelerating eigenvalue computation for nuclear structure calculations via perturbative corrections

Subspace projection methods utilizing perturbative corrections have been proposed for computing the lowest few eigenvalues and corresponding eigenvectors of large Hamiltonian matrices. In this paper, we build upon these methods and introduce the term Subspace Projection with Perturbative Corrections (SPPC) method to refer to this approach. We tailor the SPPC for nuclear many-body Hamiltonians represented in a truncated configuration interaction subspace, i.e., the no-core shell model (NCSM). We use the hierarchical structure of the NCSM Hamiltonian to partition the Hamiltonian as the sum of two matrices. The first matrix corresponds to the Hamiltonian represented in a small configuration space, whereas the second is viewed as the perturbation to the first matrix. Eigenvalues and eigenvectors of the first matrix can be computed efficiently. Because of the split, perturbative corrections to the eigenvectors of the first matrix can be obtained efficiently from the solutions of a sequence of linear systems of equations defined in the small configuration space. These correction vectors can be combined with the approximate eigenvectors of the first matrix to construct a subspace from which more accurate approximations of the desired eigenpairs can be obtained. We show by numerical examples that the SPPC method can be more efficient than conventional iterative methods for solving large-scale eigenvalue problems such as the Lanczos, block Lanczos and the locally optimal block preconditioned conjugate gradient (LOBPCG) method. The method can also be combined with other methods to avoid convergence stagnation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

DP-TwoLevel: two-stage gradient subspace learning for differentially private federated learning

Federated learning (FL) enables collaborative model training across distributed data sources without sharing raw data, but faces fundamental challenges in communication efficiency and privacy. Differentially private (DP) training mitigates information leakage but introduces noise that degrades model performance, especially in high-dimensional settings. We propose DP-TwoLevel, a hierarchical gradient projection method that improves utility under fixed DP constraints by exploiting low-dimensional structure in model updates. Our approach learns a two-level PCA-based representation of gradients and applies DP noise in a reduced-dimensional subspace, thereby lowering the effective noise magnitude while preserving dominant signal components. We evaluate the method across three datasets (MNIST, Fashion-MNIST, CIFAR-10) and three privacy regimes (ϵ∈0.5, 1.0, 2.0). Across nine experimental settings, DP-TwoLevel consistently outperforms DP-FedAvg, achieving an average accuracy improvement of 9.44%, with larger gains observed in lower ϵ(higher-noise) regimes (up to +22.31%). We further analyze scalability across models ranging from 100K to 1.49M parameters and identify a variance-based success criterion: performance remains strong when the projection preserves more than 75% of gradient variance, degrades in a marginal regime (65–75%), and fails below this threshold. Our results demonstrate that structure-aware dimensionality reduction can significantly improve the privacy–utility tradeoff in FL without modifying formal privacy guarantees. We also provide empirical evidence of scaling limitations for global projections and motivate per-layer extensions for larger models.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗
DOE OSTI2024

AI-Driven Detector Design for the EIC (Final Technical Report)

We developed an optimization workflow based on DNN-based fast-simulation and reconstruction algorithms. We used these methods to advance the design of calorimeter systems for the Electron-Ion Collider (EIC). This DNN-driven optimization provides a blueprint for integrating gradient-based methods into detector-design workflows. All software pipelines and methods have been released publicly and incorporated into the EIC collaboration’s physics studies, broadening their impact. Three journal articles detailing the methods developed here serve as a reference for the design and optimal use of next generation high-granularity calorimeter systems in nuclear and particle physics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Nonlinear Matrix Approximation with Radial Basis Function Components

We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L2 error for a wide range of matrix types. For example, it leads to 2 to 6 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

Rebrova, Elizaveta↗
DOE OSTI2023

Securely Aggregated Coded Matrix Inversion

Coded computing is a method for mitigating straggling workers in a centralized computing network, by using erasure-coding techniques. Federated learning is a decentralized model for training data distributed across client devices. In this work we propose approximating the inverse of an aggregated data matrix, where the data is generated by clients; similar to the federated learning paradigm, while also being resilient to stragglers. To do so, we propose a coded computing method based on gradient coding. We modify this method so that the coordinator does not access the local data at any point; while the clients access the aggregated matrix in order to complete their tasks. Here, the network we consider is not centrally administrated, and the communications which take place are secure against potential eavesdroppers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗
DOE OSTI2022

Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems

Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Additionally, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.

42 ENGINEERING↗
DOE OSTI2021

Large gradients via correlation in random parameterized quantum circuits

Scaling of variational quantum algorithms to large problem sizes requires efficient optimization of random parameterized quantum circuits. For such circuits with uncorrelated parameters, the presence of exponentially vanishing gradients in cost function landscapes is an obstacle to optimization by gradient descent methods. In this work, we prove that reducing the dimensionality of the parameter space by utilizing circuit modules containing spatially or temporally correlated gate layers can allow one to circumvent the vanishing gradient phenomenon. Here, examples are drawn from random separable circuits and asymptotically optimal variational versions of Grover's algorithm based on the quantum alternating operator ansatz. In the latter scenario, our bounds on cost function variation imply a transition between vanishing gradients and efficient trainability as the number of layers is increased toward $\mathcal{O}\left({2}^{n/2}\right)$, the optimal oracle complexity of quantum unstructured search.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Distributed-Memory Parallel JointNMF

Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.

Eswar, Srinivas↗
DOE OSTI2024

A data-driven sensor placement approach for detecting voltage violations in distribution systems

Stochastic fluctuations in power injections from distributed energy resources (DERs) combined with load variability can cause constraint violations (e.g., exceeded voltage limits) in electric distribution systems. To monitor grid operations, sensors are placed to measure important quantities such as the voltage magnitudes. Here, in this paper, we consider a sensor placement problem which seeks to identify locations for installing sensors that can capture all possible violations of voltage magnitude limits. We formulate a bilevel optimization problem that minimizes the number of sensors and avoids false sensor alarms in the upper level while ensuring detection of any voltage violations in the lower level. This problem is challenging due to the nonlinearity of the power flow equations and the presence of binary variables. Accordingly, we employ recently developed conservative linear approximations of the power flow equations that overestimate or underestimate the voltage magnitudes. By replacing the nonlinear power flow equations with conservative linear approximations, we can ensure that the resulting sensor locations and thresholds are sufficient to identify any constraint violations. Additionally, we apply various problem reformulations to significantly improve computational tractability while simultaneously ensuring an appropriate placement of sensors. Lastly, we improve the quality of the results via an approximate gradient descent method that adjusts the sensor thresholds. We demonstrate the effectiveness of our proposed method for several test cases, including a system with multiple switching configurations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Scale setting the Möbius domain wall fermion on gradient-flowed HISQ action using the omega baryon mass and the gradient-flow scales 𝑡 0 and 𝑤 0

We report on a subpercent scale determination using the omega baryon mass and gradient-flow methods. The calculations are performed on 22 ensembles of 𝑁 𝑓 =2 +1 +1 highly improved, rooted staggered sea-quark configurations generated by the MILC and CalLat Collaborations. The valence quark action used is Möbius domain wall fermions solved on these configurations after a gradient-flow smearing is applied with a flowtime of 𝑡 gf = 1 in lattice units. The ensembles span four lattice spacings in the range 0.06 ≲ 𝑎 ≲0.15 fm, six pion masses in the range 130 ≲ 𝑚 𝜋 ≲ 400 MeV and multiple lattice volumes. On each ensemble, the gradient-flow scales 𝑡 0 /𝑎 2 and 𝑤 0 /𝑎 and the omega baryon mass 𝑎⁢𝑚 Ω are computed. The dimensionless product of these quantities is then extrapolated to the continuum and infinite volume limits and interpolated to the physical light, strange and charm quark mass point in the isospin limit, resulting in the determination of $\sqrt{t}_0$ = 0.1422⁢(14) fm and 𝑤 0 = 0.1709⁢(11) fm with all sources of statistical and systematic uncertainty accounted for. The dominant uncertainty in both results is the stochastic uncertainty, though for $\sqrt{t}_0$ there are comparable continuum extrapolation uncertainties. For 𝑤 0 , there is a clear path for a few-per-mille uncertainty just through improved stochastic precision, as recently obtained by the Budapest-Marseille-Wuppertal Collaboration.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Artifact identification in X-ray diffraction data using machine learning methods

In situ synchrotron high-energy X-ray powder diffraction (XRD) is highly utilized by researchers to analyze the crystallographic structures of materials in functional devices ( e.g. battery materials) or in complex sample environments ( e.g. diamond anvil cells or syntheses reactors). An atomic structure of a material can be identified by its diffraction pattern along with a detailed analysis of the Rietveld refinement which yields rich information on the structure and the material, such as crystallite size, microstrain and defects. For in situ experiments, a series of XRD images is usually collected on the same sample under different conditions ( e.g. adiabatic conditions) yielding different states of matter, or is simply collected continuously as a function of time to track the change of a sample during a chemical or physical process. In situ experiments are usually performed with area detectors and collect images composed of diffraction patterns. For an ideal powder, the diffraction pattern should be a series of concentric Debye–Scherrer rings with evenly distributed intensities in each ring. For a realistic sample, one may observe different characteristics other than the typical ring pattern, such as textures or preferred orientations and single-crystal diffraction spots. Textures or preferred orientations usually have several parts of a ring that are more intense than the rest, whereas single-crystal diffraction spots are localized intense spots owing to diffraction of large crystals, typically >10 µm. In this work, an investigation of machine learning methods is presented for fast and reliable identification and separation of the single-crystal diffraction spots in XRD images. The exclusion of artifacts during an XRD image integration process allows a precise analysis of the powder diffraction rings of interest. When it is trained with small subsets of highly diverse datasets, the gradient boosting method can consistently produce high-accuracy results. The method dramatically decreases the amount of time spent identifying and separating single-crystal diffraction spots in comparison with the conventional method.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Qualitative and quantitative enhancement of parameter estimation for model-based diagnostics using automatic differentiation with an application to inertial fusion

Parameter estimation using observables is a fundamental concept in the experimental sciences. Mathematical models that represent the physical processes can enable reconstructions of the experimental observables and greatly assist in parameter estimation by turning it into an optimization problem which can be solved by gradient-free or gradient-based methods. In this work, the recent rise in flexible frameworks for developing differentiable scientific computing programs is leveraged in order to dramatically accelerate data analysis of a common experimental diagnostic relevant to laser–plasma and inertial fusion experiments, Thomson scattering. A differentiable Thomson-scattering data analysis tool is developed that uses reverse-mode automatic differentiation (AD) to calculate gradients. By switching from finite differencing to reverse-mode AD, three distinct outcomes are achieved. First, gradient descent is accelerated dramatically to the extent that it enables near real-time usage in laser–plasma experiments. Second, qualitatively novel quantities which require $\mathcal{O}(10^3)$ parameters can now be included in the analysis of data which enables unprecedented measurements of small-scale laser–plasma phenomena. Third, uncertainty estimation approaches that leverage the value of the Hessian become accurate and efficient because reverse-mode AD can be used for calculating the Hessian.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Optimal control of coupled quantum systems based on the first-order Magnus expansion: Application to multiple dipole-dipole-coupled molecular rotors

This paper presents a method for performing approximate optimal control simulations for quantum systems with multiple coupled degrees of freedom. In this work, the time evolution is simulated using the first-order Magnus expansion in the interaction picture, where the couplings between different degrees of freedom are treated as the perturbation. A numerical implementation procedure is presented that leverages upon pairwise couplings and the separability of the zeroth-order time evolution operator to achieve a reduced computational cost, which is analyzed with respect to the number of degrees of freedom. The formulation is compatible with gradient-free methods to optimize the control field, and a stochastic hill climbing algorithm is adopted for this purpose. As illustrations, optimal control simulations are performed for systems of two and three dipole-dipole-coupled molecular rotors under the influence of a control field. For the two-rotor system, the field is optimized to achieve either orientation or entanglement objectives. For the three-rotor system, the field is optimized either to orient all three rotors in the same direction or to orient one rotor in a particular direction while the other two rotors point in the opposite direction.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗