Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “iterative methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Revisiting Dropout: Escaping Pressure for Training Neural Networks with Multiple Costs

A common approach to jointly learn multiple tasks with a shared structure is to optimize the model with a combined landscape of multiple sub-costs. However, gradients derived from each sub-cost often conflicts in cost plateaus, resulting in a subpar optimum. In this work, we shed light on such gradient conflict challenges and suggest a solution named Cost-Out, which randomly drops the sub-costs for each iteration. We provide the theoretical and empirical evidence of the existence of escaping pressure induced by the Cost-Out mechanism. While simple, the empirical results indicate that the proposed method can enhance the performance of multi-task learning problems, including two-digit image classification sampled from MNIST dataset and machine translation tasks for English from and to French, Spanish, and German WMT14 datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Preconditioned least‐squares Petrov–Galerkin reduced order models

Abstract In this article, we introduce a methodology for improving the accuracy and efficiency of reduced order models (ROMs) constructed using the least‐squares Petrov–Galerkin (LSPG) projection method through the introduction of preconditioning. Unlike prior related work, which focuses on preconditioning the linear systems arising within the ROM numerical solution procedure to improve linear solver performance, our approach leverages a preconditioning matrix directly within the minimization problem underlying the LSPG formulation. Applying preconditioning in this way has the potential to improve ROM accuracy for several reasons. First, preconditioning the LSPG formulation changes the norm defining the residual minimization, which can improve the residual‐based stability constant bounding the ROM solution's error. The incorporation of a preconditioner into the LSPG formulation can have the additional effect of scaling the components of the residual being minimized to make them roughly of the same magnitude, which can be beneficial when applying the LSPG method to problems with disparate scales (e.g., dimensional equations, multi‐physics problems). Importantly, we demonstrate that an “ideal preconditioned” LSPG ROM (a ROM in which the preconditioner is the inverse of the Jacobian of its corresponding full order model) emulates projection of the full order model solution increment onto the reduced basis. This quantity defines a lower bound on the error of a ROM solution for a given reduced basis. By designing preconditioners that approximate the Jacobian inverse—as is common in designing preconditioners for solving linear systems—it is possible to obtain a ROM whose error approaches this lower bound. The proposed approach is evaluated on several mechanical and thermo‐mechanical problems implemented within the Albany HPC code and run in the predictive regime, with prediction across material parameter space. We demonstrate numerically that the introduction of simple Jacobi, Gauss‐Seidel, and ILU preconditioners into the proper orthogonal decomposition/LSPG formulation reduces significantly the ROM solution error, the reduced Jacobian condition number, the number of nonlinear iterations required to reach convergence, and the wall time (thereby improving efficiency). Moreover, our numerical results reveal that the introduction of preconditioning can deliver a robust and accurate solution for test cases in which the unpreconditioned LSPG method fails to converge.

Lindsay, Payton↗
DOE OSTI2022

Procedure Parsing: A Method for Parsing Handwritten Documents into Computer-Based Procedures

The nuclear industry is heavily procedure driven, where almost everything has a step-by-step instruction that is expected to be followed in detail. Historically, these procedures were printed on paper copies. Recently, the industry transitioned towards electronic copies (i.e., PDFs on tablets). One major drive for this transition is the introduction of human error and loss of situation awareness when using paper copies. However, electronic copies of documents inherently have the same error traps as their paper cousins. Therefore, there is an increased interest in a way to utilize the information in the step-by-step guidance, but to present it in a dynamic manner that guides the user and adapts to any encountered conditions. Researchers at Idaho National Laboratory propose a flexible, automated method based on document parsing and augmented by natural language processing (NLP) techniques, to address these shortcomings and capitalize on these recent advancements in machine learning. The proposed method provides a cost-effective solution for computer-assisted procedure parsing of hand-written control room procedures, originally authored in Word or PDF formats, into instructions that can be displayed as computer-based procedures (CBP) in a modern graphical user interface. The researchers devised, implemented and demonstrated the Operating Procedure Extender for Novel Systems (OPENS) method in 2020. The key to OPENS is to map the original procedure text into a context-free grammar, tying content to equipment, locations, and other steps, actions, etc. This formal grammar is then used to isolate and define keywords and actions verbs, such as “measure” or “evaluate” and tie them to specific equipment referenced within that step or located in other steps, substeps, actions, subactions and tables throughout the procedure. OPENS generates an abstract syntax tree from the document which it uses to store a copy of this information in the open-standard, machine-readable and human-readable file formats XML and JSON. The XML is useful to preserve the relational aspects of the procedure for referencing tables and branching information so the user can be directed to the next appropriate active step based on the values entered for that step and previous steps. The JSON is useful for storing and exchanging data objects used to track responses to previous steps and state changes in simulated environments. In future iterations, these formats can also be used for storing more detailed information about input during plant operation or simulation. The techniques the researcher developed could further be improved by integration of recent advancements in machine learning. NLP methods could standardize documents, correct for grammatical error, and provide automated semantic validation. The researcher expects that self-supervised techniques applied to collections of natural language instructions could strengthen the model with broader context. All these methods together give us a practical way to automatically extract protocols from documents and user interactions, empowering researchers, procedure writers and nuclear operators while moving the industry forward.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2023

Generalized spin σ -SCF method

We introduce a generalization of the σ-SCF method to approximate noncollinear spin ground and excited single-reference electronic states by minimizing the Hamiltonian variance. The new method is based on the σ-SCF method, originally proposed by Ye et al. [J. Chem. Phys. 147, 214104 (2017)], and provides a prescription to determine ground and excited noncollinear spin states on an equal footing. Our implementation was carried out utilizing an initial simulated annealing stage followed by a mean-field iterative self-consistent approach to simplify the cumbersome search introduced by generalizing the spin degrees of freedom. The simulated annealing stage ensures a broad exploration of the Hilbert space spanned by the generalized spin single-reference states with random complex element-wise rotations of the generalized density matrix elements in the simulated annealing stage. The mean-field iterative self-consistent stage employs an effective Fockian derived from the variance, which is utilized to converge tightly to the solutions. This process helps us to easily find complex spin structures, avoiding manipulating the initial guess. As proof-of-concept tests, we present results for Hn (n = 3–7) planar rings and polyhedral clusters with geometrical spin frustration. We show that most of these systems have noncollinear spin excited states that can be interpreted in terms of geometric spin frustration. These states are not directly targeted by energy minimization methods, which are meant to converge to the ground state. This stresses the capability of the σ-SCF methodology to find approximate noncollinear spin structures as mean-field excited states.

Chemistry↗
DOE OSTI2020

Reducing communication in algebraic multigrid with multi-step node aware communication

Algebraic multigrid (AMG) is often viewed as a scalable [Formula: see text] solver for sparse linear systems. Yet, AMG lacks parallel scalability due to increasingly large costs associated with communication, both in the initial construction of a multigrid hierarchy and in the iterative solve phase. This work introduces a parallel implementation of AMG that reduces the cost of communication, yielding improved parallel scalability. It is common in Message Passing Interface (MPI), particularly in the MPI-everywhere approach, to arrange inter-process communication, so that communication is transported regardless of the location of the send and receive processes. Performance tests show notable differences in the cost of intra- and internode communication, motivating a restructuring of communication. In this case, the communication schedule takes advantage of the less costly intra-node communication, reducing both the number and the size of internode messages. Node-centric communication extends to the range of components in both the setup and solve phase of AMG, yielding an increase in the weak and strong scaling of the entire method.

Computer Science↗
DOE OSTI2022

Bayesian Optimization Framework for Imperfect Data or Models

Conventional Bayesian optimization methods implicitly assume that the data and model being optimized are “perfect.” This assumption leads to inaccurate posterior probability distribution functions (PDFs) when applied to “imperfect” data or models. The new Bayesian optimization framework presented in this report provides a way to parameterize the effect of imperfections usually encountered in a prior PDF of generalized data or a model on the posterior PDF. The effects of imperfections are parameterized by a set of constraints imposed on the posterior expectation values of deviations between the data and the model and on their covariance matrix elements. A particular set of values for these constraints conveys an evaluator’s best estimate of the effect of imperfections on the corresponding posterior expectation values. When a prior PDF of generalized data is assumed to be normal, an expression for a posterior PDF satisfying an arbitrary set of constraints is derived analytically for linear models. An analogous iterative algorithm is given for nonlinear models. The corresponding posterior PDF should be used to estimate any posterior expectation values in the presence of imperfections parameterized by that set of constraints. A posterior PDF of a conventional Bayesian optimization method is recovered analytically when all evaluator-specified constraints are set to zero (i.e., in the absence of any imperfections). The analytical expressions derived in this report for normal PDFs and linear models were verified numerically by a Metropolis–Hastings Monte Carlo method. The methods presented herein could be applied to any kind of data or models, including differential cross-section data or integral benchmark experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Standards, dissemination, and best practices in systems biology

In this study, the reproducibility of scientific research is crucial to the success of the scientific method. Here, we review the current best practices when publishing mechanistic models in systems biology. We recommend, where possible, to use software engineering strategies such as testing, verification, validation, documentation, versioning, iterative development, and continuous integration. In addition, adhering to the Findable, Accessible, Interoperable, and Reusable modeling principles allows other scientists to collaborate and build off of each other’s work. Existing standards such as Systems Biology Markup Language, CellML, or Simulation Experiment Description Markup Language can greatly improve the likelihood that a published model is reproducible, especially if such models are deposited in well-established model repositories. Where models are published in executable programming languages, the source code and their data should be published as open-source in public code repositories together with any documentation and testing code. For complex models, we recommend container-based solutions where any software dependencies and the run-time context can be easily replicated.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

WinnowML: Stable feature selection for maximizing prediction accuracy of time-based system modeling

Online deep learning (ODL) has become an important methodology for modeling time-based performance of computer systems. An open problem is the intelligent selection of features from raw workload traces of computer systems. The best methods are overly sensitive to noisy data, causing frequent feature changes and re-training. Using all available features inflates training time and introduces model artifacts if some features should have been dropped. We present WinnowML, a method for automatically determining the most relevant feature subset for a predictive time-series model. WinnowML combines existing feature ranking algorithms and a history of each feature's ranking to iteratively rank a feature set to lower prediction error and maximize long term relevance. From this ranked feature set, the most relevant and stable subset is selected to train a model. Experimentally, we show how WinnowML can lower a model's mean absolute relative error up to 42% on average compared to the closest performing approach. Additionally, we lower the fluctuation in feature ranking and selection up to 65%. We also demonstrate how to combine WinnowML and a model search tool to provide improvements in performance of up to 14.5% when compared to using all the feature available.

Bel, Oceane MS↗
DOE OSTI2021

Space to Air High-Altitude Region Adjoint Neutron Transport

Neutrons from an atmospheric nuclear explosion can be detected by sensors in orbit. Current tools for characterizing the neutron energy spectrum assume a known source and use forward transport to recreate the detector response. In realistic scenarios the true source is unknown, making this an inefficient, iterative approach. In contrast, the adjoint approach directly solves for the source spectrum, enabling source reconstruction. The time–energy fluence at the satellite and adjoint transport equation allow a Monte Carlo method to characterize the neutron source’s energy spectrum directly in a new model: the Space to High-Altitude Region Adjoint (SAHARA) model. A new adjoint source event estimator was developed in SAHARA to find feasible solutions to the neutron transport problem given the constraints of the adjoint environment. This work explores SAHARA’s development and performance for mono-energetic and continuous neutron energy sources. In general, the identified spectra were shifted towards energies approximately 5% lower than the true source spectra, but SAHARA was able to capture the correct spectral shapes. Continuous energy sources, including real-world sources Fat Man and Little Boy, resulted in identifiable spectra that could have been produced by the same distribution as the true sources as demonstrated by two-dimensional (2D) Kolmogorov–Smirnov tests.

LaMere, Zachary W.↗
DOE OSTI2020

Environmental creep-fatigue and weld creep cracking: a summary of design and fitness-for-service practices

This report surveys current creep-fatigue design and fitness-for-service assessment procedures, with a particular focus on environmental effects and creep-fatigue cracking near weldments. Included in this assessment are the American Society of Mechanical Engineers Boiler and Pressure Vessel Code Section III, Division 5, the British R5 standard, the French RCC-MRx design code, the ITER Structural Design Criteria, and the API-579/ASME FFS-1 fitness-for-service rules for petrochemical components. The focus of this review is to identify gaps in the available methods that could prevent regulators from evaluating future high temperature reactor designs. The report makes recommendations, where appropriate, on actions that could be taken to fill these gaps by either providing adequate assessment procedures for creep-fatigue damage in harsh environments or by evaluating strategies vendors could use to mitigate regulatory concerns with creep-fatigue damage in high temperature reactor components.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

Technical Report on Subsurface Monitoring of the Brady Hot Spring Geothermal Site, Nevada, based upon Full Waveform Inversion

Abilities to accurately characterize the subsurface in a geothermal setting is key to assess and support production. An important element of geothermal reservoir monitoring is also the ability to investigate fluid transport within fracture network. This report focuses on improving subsurface imaging and monitoring in geothermal settings using full waveform inversion based on the adjoint method and time-lapse imaging. To assess our method, we rely on a dense seismic dataset collected in 2016 at the Brady Hot Springs geothermal site in Nevada for the DOE-funded project Poroelastic Tomography by Adjoint Inverse Modeling of Data from Seismology, Geodesy, and Hydrology. This dataset captures subsurface changes across four stages of geothermal power plant operations, which involve varying rates of fluid injection and extraction. Two velocity models were previously derived from this dataset using different methods: one based on travel times and another on sweep interferometry. Our first step is to refine these models using adjoint tomography, which has been applied successfully at global and regional-scales but is less common at the reservoir-scale. Two approaches are then explored for time-lapse analysis: directly comparing refined tomographic models from different stages or backpropagating waveform differences relative to a baseline tomographic model. The main take away is that both approaches highlight similar reservoir behaviors, but the latter approach is more computationally effective in capturing small-scale changes in subsurface properties. For this work, we leverage the use of Salvus (www.mondaic.com), an end-to-end seismic imaging solution, relying on the spectral element method to compute forward and adjoint simulations, and developed by Mondaic Ltd. It includes integrated workflow management that handles waveform and metadata, launches simulations, computes waveform misfits and adjoint sources, and iterates for model updates by nonlinear optimization.

15 GEOTHERMAL ENERGY↗
DOE OSTI2020

Data Driven Online Learning of Power System Dynamics

With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model.

Sinha, Subhrajit↗
DOE OSTI2022

Ultra-rapid, physics-based development pathway for reactor-relevant RF antenna materials

This paper presents a rapid, atomistically-informed, experimental development pathway for fusion reactor-relevant radio frequency (RF) antenna materials in the Cu-Cr-(Nb,Al,Zr) composition system, with the goal of improving upon GRCop-84. RF antennas in a tokamak fusion reactor will face a unique set of challenges as both structural and functional materials. The desired material must simultaneously achieve and maintain high electrical conductivity, high strength, high thermal conductivity, resist high temperatures, possess low nuclear activation, and incur low damage due to neutron bombardment. The GRCop-84 alloy serves as a starting point for iterative improvement, with the desire to reduce or eliminate Nb from the material to minimize nuclear activation. The rapid development pathway makes use of a multi-target combinatorial thick film sputtering process to produce full ternary phase diagrams on a Si wafer substrate. Transient grating spectroscopy (TGS), a laser-ultrasonic method, will determine spatially-varying thermo-elastic properties, while four terminal electrical conductivity measurements will map out the best per- forming regions of the sample for in-depth study at larger length scales. High energy proton and self-ion irradiation emulates the effects of neutron damage on the thermal/electric properties. With rapid turnaround time (∼days) in terms of mapping radiation damage-induced material property changes in the full ternary system, these techniques allow rapid iteration towards an optimal material, testing hundreds of nearby compositions in the time it took to test one. Focused testing of larger, single composition samples (produced in an arc furnace or by laser sintering) provides data on structural and high power RF properties, and validates our thick-film based workflow.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

From Deposition to Encapsulation: Roll-to-roll manufacturing of organic light emitting devices for lighting (Final Report)

Organic light emitting devices (OLEDs) are promising solid state light sources due to their high efficiency, high color quality and flexible form factors. The key to enable low cost OLED lighting, is to rapidly fabricate thin film organic layers on a continuous flexible roll, called R2R processing. This project aims to investigate the feasibility of mass production of OLED lighting using the R2R process, from deposition to encapsulation. A high efficiency white OLED is fabricated on 10 cm-wide substrate rolls in a pilot R2R tool comprising of two different organic deposition: vacuum thermal evaporation (VTE) and organic vapor phase deposition (OVPD). A high quality encapsulation process to package OLEDs is demonstrated using an atomic layer deposition tool integrated to the R2R system without air exposure. The method to achieve ultrahigh deposition rates required by R2R processing is demonstrated by OVPD. Uniform organic semiconductor thin films grown by OVPD at rates as high as 50 Å/s are achieved. A comprehensive numerical model that is capable of simulating complex, multilayer WOLED structures is developed to provide an alternative to experimental iterations of OLED design and tests. A cost estimate on the R2R production of WOLEDs for lighting is developed. Assuming a WOLED luminance of 10 klm/m 2 , the cost of a WOLED light engine is anticipated to be $\$ 12.5$ /klm. With incremental reduction in material and driver costs and improved luminance, the cost of WOLED lighting can be reduced to $\$ 6.3$ /klm in the near term. These findings suggest OLED lighting can be volume manufactured by R2R vapor deposition methods with much reduced costs compared to current batch processing methods, potentially positioning WOLEDs for use in numerous premium lighting applications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Mixed-precision numerics in scientific applications: survey and perspectives

The explosive demand for artificial intelligence (AI) workloads has led to a significant increase in silicon area dedicated to lower-precision computations on recent high-performance computing hardware designs. However, mixed-precision capabilities, which can achieve performance improvements of up to 8x compared to double-precision in extreme compute-intensive workloads, remain largely untapped in most scientific applications. A growing number of efforts have shown that mixed-precision algorithmic innovations can deliver superior performance without sacrificing accuracy. These developments should prompt computational scientists to seriously consider whether their scientific modeling and simulation applications could benefit from the acceleration offered by new hardware and mixed-precision algorithms. In this survey, we (1) review progress across diverse scientific domains—fluid dynamics, weather and climate, quantum chemistry, and computational genomics—that have begun adopting mixed-precision strategies; (2) examine state-of-the-art algorithmic techniques such as iterative refinement, splitting and emulation schemes, and adaptive precision solvers; (3) assess their implications for accuracy, performance, and resource utilization; and (4) survey the emerging software ecosystem that enables mixed-precision methods at scale. We conclude with perspectives and recommendations on cross-cutting opportunities, domain-specific challenges, and the role of co-design between application scientists, numerical analysts, and computer scientists. Collectively, this survey underscores that mixed-precision numerics can reshape computational science by aligning algorithms with the evolving landscape of hardware capabilities.

Graphics processing units↗
DOE OSTI2025

Discovering nuclear models from symbolic machine learning

Numerous phenomenological nuclear models have been proposed to describe specific observables within different regions of the nuclear chart. However, developing a unified model that describes the complex behavior of all nuclei remains an open challenge. Here, we explore whether symbolic Machine Learning (ML) can rediscover traditional nuclear physics models or identify alternatives with improved simplicity, fidelity, and predictive power. To address this challenge, we developed a Multi-objective Iterated Symbolic Regression approach that handles symbolic regressions over multiple target observables, accounts for experimental uncertainties and is robust against high-dimensional problems. As a proof of principle, we applied this method to describe the nuclear binding energies and charge radii of light and medium mass nuclei. Our approach identified simple analytical relationships based on the number of protons and neutrons, providing interpretable models with precision comparable to state-of-the-art nuclear models. Additionally, we integrated this ML-discovered model with an existing complementary model to estimate the limits of nuclear stability. These results highlight the potential of symbolic ML to develop accurate nuclear models and guide our description of complex many-body problems.

Nuclear structure↗
DOE OSTI2022

A deep learning-enhanced framework for multiphysics joint inversion

Joint inversion has drawn considerable attention due to the availability of multiple geophysical data sets, ever-increasing computational resources, the development of advanced algorithms, and its ability to reduce inversion uncertainty. A key issue of joint inversion is to develop effective strategies to link different geophysical data in a unified mathematical framework, in which the information obtained from different models can complement each other. We have developed a deep learning-enhanced joint inversion framework to simultaneously reconstruct different physical models by fusing different types of geophysical data. Traditionally, structure similarity constraints are pursued by joint inversion algorithms using manually crafted formulations (e.g., cross gradient). The constraint is constructed by a deep neural network (DNN) during the learning process. The framework is designed to combine the DNN and a traditional independent inversion workflow and improve the joint inversion result iteratively. The network can be easily extended to incorporate multiphysics without structural changes. Numerical experiments on the joint inversion of 2D DC resistivity data and seismic traveltime are used to validate our method. In addition, this learning-based framework demonstrates excellent generalization abilities when tested on data sets using different geologic structures. It also can handle different sensing configurations and nonconforming discretization.

Geochemistry & Geophysics↗
DOE OSTI2024

Advancing attenuation estimation through integration of the Hessian in multiparameter viscoacoustic full-waveform inversion

Accurate seismic attenuation models of subsurface structures not only enhance subsequent migration processes by improving fidelity, resolution, and facilitating amplitude-compliant angle gather generation but also provide valuable constraints on subsurface physical properties. Leveraging full-wavefield information, multiparameter viscoacoustic full-waveform inversion ( Q-FWI) simultaneously estimates seismic velocity and attenuation ( Q) models. However, a major challenge in Q-FWI is the contamination of crosstalk artifacts, where inaccuracies in the velocity model are mistakenly mapped to the inverted attenuation model. While incorporating the Hessian is expected to mitigate these artifacts, the explicit implementation is prohibitively expensive due to its formidable computational cost. In this study, we formulate and develop a Q-FWI algorithm via the Newton-conjugate gradient (CG) framework, where the search direction at each iteration is determined through an internal CG loop. In particular, the Hessian is integrated into each CG step in a matrix-free fashion using the second-order adjoint-state method. We find through synthetic experiments that our Newton-CG Q-FWI significantly mitigates crosstalk artifacts compared with the limited-memory Broyden-Fletcher-Goldfarb-Shanno method and the CG method, albeit with a notable computational cost. In the discussion of several key implementation details, we also determine the significance of the approximate Gauss-Newton Hessian, the second-order adjoint-state method, and the two-stage inversion strategy.

Geochemistry & Geophysics↗