SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
High Performance Computing (HPC) enables problem solving at the grandest scales, on the most complex challenges that face humanity. The problems often tackled by scientific computing are: Very complex; Cool; Using the fastest, most powerful supercomputers on the planet; Often important, impactful, effecting real lives.
Since their introduction in 2017 with Attention is All You Need¹, transformers have established themselves as the state of the art for natural language processing (NLP). In 2021, An Image is Worth 16x16 Words successfully adapted transformers for computer vision tasks. Since then, numerous transformer-based architectures have been proposed for computer vision. This article walks through the Vision Transformer (ViT) as laid out in An Image is Worth 16x16 Words.
The Xyce™ Parallel Electronic Simulator has been written to support the simulation needs of Sandia National Laboratories’ electrical designers. Xyce™ is a SPICE-compatible simulator with the ability to solve extremely large circuit problems on large-scale parallel computing platforms, but also includes support for most popular parallel and serial computers.
Superconducting quantum computers are a particular technology that involve the realization of qubits by means of LC circuits with a Josephson junction. As shown in Fig. 1, this latter addition has the effect of introducing a nonlinearity (anharmonicity) that makes the frequency spacing between each energy level non-uniform. This is a desired effect for separating the two lowest levels, used for computation, from the higher ones, that are theoretically infinite and which are not desired to be used.
This final technical report summarizes the research conducted under DOE Grant DE-SC0002623, "Properties of Electronic Materials," led by Principal Investigator Shengbai Zhang at Rensselaer Polytechnic Institute. Over the 16-year period, the project employed first-principles computational methods to investigate the structural, electronic, and dynamic properties of a wide range of electronic materials, with applications in energy technologies, optoelectronics, and data storage. Key areas included topological insulators, phase-change materials, graphene and two-dimensional systems, perovskites for photovoltaics, defect engineering in semiconductors, kagome lattices, and ultrafast carrier dynamics. The research resulted in 115 peer-reviewed publications, advancing fundamental understanding of material behaviors at the atomic scale and contributing to innovations in renewable energy, memory devices, and quantum materials. Findings have implications for improving energy efficiency, developing lead-free solar cells, and enabling high-speed data processing. The work has trained numerous graduate students and postdocs, fostering the next generation of computational materials scientists. The original goals were to develop theoretical models and computational tools to predict and optimize electronic properties of materials for energy applications. All objectives were accomplished, with no major departures from planned methodologies. Challenges in computational scaling were addressed through access to high-performance computing resources.
Lawrence Livermore National Security, LLC (LLNS) is interested in receiving information about technologies that could be available in the 2029-2030 timeframe that may serve to enable the vision for a Future Generation High Performance Computing (HPC) Center (FG-HPCC) described in this document. The future HPC Center vision has been conceived to meet the future mission needs of the Advanced Simulation and Computing (ASC) Program within the National Nuclear Security Administration (NNSA). LLNS envisions a center composed not of many independent clusters, but of heterogeneous elements accessible to users as a single system. The capabilities will be integrated to create a scalable, flexible, yet tightly coupled computing center capable of integrated HPC, AI, and cloud-like workloads.
Bayesian inference is used to calibrate a bottom-up home PLC network model with unknown loads and wires at frequencies up to 30 MHz. A network topology with over 50 parameters is calibrated using global sensitivity analysis and transitional Markov Chain Monte Carlo (TMCMC). The sensitivity-informed Bayesian inference computes Sobol indices for each network parameter and applies TMCMC to calibrate the most sensitive parameters for a given network topology. A greedy random search with TMCMC is used to refine the discrete random variables of the network. This results in a model that can accurately compute the transfer function despite noisy training data and a high dimensional parameter space. The model is able to infer some parameters of the network used to produce the training data, and accurately computes the transfer function under extrapolative scenarios.
Liquid crystal elastomers (LCEs) are responsive materials that can undergo large reversible deformations upon exposure to external stimuli, such as electrical and thermal fields. Controlling the alignment of their liquid crystals mesogens to achieve desired shape changes unlocks a new design paradigm that is unavailable when using traditional materials. While experimental measurements can provide valuable insights into their behavior, computational analysis is essential to exploit their full potential. Accurate simulation is not, however, the end goal; rather, it is the means to achieve their optimal design. Such design optimization problems are best solved with algorithms that require gradients, i.e., sensitivities, of the cost and constraint functions with respect to the design parameters, to efficiently traverse the design space. In this work, a nonlinear LCE model and adjoint sensitivity analysis are implemented in a scalable and flexible finite element-based open source framework and integrated into a gradient-based design optimization tool. To display the versatility of the computational framework, LCE design problems that optimize both the material, i.e., liquid crystal orientation, and structural shape to reach a target actuated shapes or maximize energy absorption are solved. Multiple parameterizations, customized to address fabrication limitations, are investigated in both 2D and 3D. The case studies are followed by a discussion on the simulation and design optimization hurdles, as well as potential avenues for improving the robustness of similar computational frameworks for applications of interest.
Large-scale multivariate regression is a fundamental statistical tool with a wide range of applications. Here, this study considers the problem of simultaneously testing a large number of general linear hypotheses, encompassing covariate-effect analysis, analysis of variance, and model comparisons. The challenge that accompanies a large number of tests is the ubiquitous presence of heavy-tailed and/or highly skewed measurement noise, which is the main reason for the failure of conventional least squares-based methods. For large-scale multivariate regression, we develop a set of robust inference methods to explore data features such as heavy tailedness and skewness, which are not visible to least squares methods. The new testing procedure is based on the data-adaptive Huber regression and a new covariance estimator of regression estimates. Under mild conditions, we show that our methods produce consistent estimates of the false discovery proportion. Extensive numerical experiments and an empirical study on quantitative linguistics demonstrate the advantage of the proposed method over many state-of-the-art methods when the data are generated from heavy-tailed and/or skewed distributions.
An analytical approach was developed to evaluate the electrodynamic interactions affecting a thin, bare metallic wire moving in the ionosphere. The wire's diameter was smaller than the Debye length; therefore, the plasma sheath around the wire was taken into account in computing inducing drag force and torque. Computer programs were prepared for the numerical evaluation of mathematical functions that were required to compute the distribution of the potential along the wire and of the current in the wire. Numerical calculations based on this software are shown.
Quantum analog computing is based upon similarity between mathematical formalism of quantum mechanics and phenomena to be computed. It exploits a dynamical convergence of several competing phenomena to an attractor which can represent an externum of a function, an image, a solution to a system of ODE, or a stochastic process.
Here, we investigate a projection-based reduced order model of the steady incompressible Navier–Stokes equations for moderate Reynolds numbers. In particular, we construct an “embedded” reduced basis space, by applying proper orthogonal decomposition to the Shifted Boundary Method, a high-fidelity embedded method recently developed. We focus on the geometrical parametrization through level-set geometries, using a fixed Cartesian background geometry and the associated mesh. This approach avoids both remeshing and the development of a reference domain formulation, as typically done in fitted mesh finite element formulations. Two-dimensional computational examples for one and three parameter dimensions are presented to validate the convergence and the efficacy of the proposed approach.
We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .
Here, the classical Newton-Raphson (NR) method for solving nonlinear equations is enhanced in two ways through the use of hypercomplex variables and algebra. In particular, i) the Jacobian is computed in a highly accurate and automated way, and ii) the derivative of the solution to the nonlinear equations is computed with respect to any parameter contained within the system of equations. These advances provide two significant enhancements in that it is straightforward to provide an accurate Jacobian and to construct a reduced order model (ROM) of arbitrary order with respect to any parameter of the system. The ROM can then be used to approximate the solution for other parameter values without requiring additional solutions of the nonlinear equations. Several case studies are presented including 1D and 2D academic examples with fully functioning Python code provided. Additionally, a case of study of the catenary of an elastic cable subject to its own weight and a vertical point load. Derivatives up to 10th order were computed with respect to material, loading, and geometrical parameters. The derivatives were used to generate reduced order models of the cable deformation and reaction forces at its ends with respect to multiple input parameters. Results show that from a single hypercomplex evaluation of the cable under a single vertical point load, it is possible to generate an accurate reduced order model capable of predicting the cable deformation with 1.5 times the load in the opposite direction and with 3.5 times the load in the same direction without resolving the system of equations.
Phase field models for ductile fracture have gained significant attention in the last two decades due to their ability in implicitly tracking the nucleation and propagation of cracks. However, most crack phase field formulations for elastoplastic solids focus only on the effects of plastic deformation, and do not consider the different multi-axial stress states that may arise in practical designs. In this work, a thermodynamically consistent phase field approach coupled with finite strain plasticity, considering multi-axial stress states is presented. In order to account for the coupling between plasticity and stress states, the Stress-Weighted Ductile Fracture Model (SWDFM) is utilized. The SWDFM represents a criterion for predicting ductile crack initiation under both monotonic and cyclic loadings based on histories of an internal plastic variable, stress triaxiality, and the Lode angle parameter. The excellent performance of the SWDFM for predicting ductile crack initiation motivates for its incorporation into a phase field approach for predicting both crack initiation and propagation through degradation of the fracture toughness. Moreover, based on the second law of thermodynamics, exact requirements are imposed on the rate at which the fracture toughness can evolve. Here, a novel function for degrading the plastic yield surface during the evolution of damage is introduced. This function, in line with experimental observations, leads to an accumulation of plastic deformation in damaged regions of a solid, and avoids numerical instabilities arising from concentrations of large plastic deformations in severely damaged regions. For validating the proposed model, results of computational simulations are compared to data from selected tests considering different multi-axial stress states. Comparisons of the numerical results with data from laboratory experiments demonstrate the capabilities of the proposed framework.
Spectral analyses of angle-modulated communication systems is studied by: (1) performing a literature survey of candidate power spectrum computational techniques, determining the computational requirements, and formulating a mathematical model satisfying these requirements; (2) implementing the model on UNIVAC 1230 digital computer as the Spectral Analysis Program (SAP); and (3) developing the hardware specifications for a data acquisition system which will acquire an input modulating signal for SAP. The SAP computational technique uses extended fast Fourier transform and represents a generalized approach for simple and complex modulating signals.
Riverine floods pose a considerable risk to many communities. Improving flood hazard projections has the potential to inform the design and implementation of flood risk management strategies. Current flood hazard projections are uncertain, especially due to uncertain model parameters. Calibration methods use observations to quantify model parameter uncertainty. With limited computational resources, researchers typically calibrate models using either relatively few expensive model runs at high spatial resolutions or many cheaper runs at lower spatial resolutions. This leads to an open question: is it possible to effectively combine information from the high and low resolution model runs? We propose a Bayesian emulation–calibration approach that assimilates model outputs and observations at multiple resolutions. As a case study for a riverine community in Pennsylvania, we demonstrate our approach using the LISFLOOD-FP flood hazard model. Here, the multiresolution approach results in improved parameter inference over the single resolution approach in multiple scenarios. Results vary based on the parameter values and the number of available models runs. Our method is general and can be used to calibrate other high dimensional computer models to improve projections.
Large language models (LLMs) have arisen rapidly to the center stage of artificial intelligence as the foundation models applicable to many downstream learning tasks. However, how to effectively build, train, and serve such models for many high-stake and first-principle-based scientific use cases are both of great interests and of great challenges. Moreover, pre-training LLMs with billions or even trillions of parameters can be prohibitively expensive not just for academic institutions, but also for well-funded industrial and government labs. Furthermore, the energy cost and the environmental impact of developing LLMs must be kept in mind. Here, in this work, we conduct a first-of-its-kind performance analysis to understand the time and energy cost of pre-training LLMs on the Department of Energy (DOE)’s leadership-class supercomputers. Employing state-of-the-art distributed training techniques, we evaluate the computational performance of various parallelization approaches at scale for a range of model sizes, and establish a projection model for the cost of full training. Our findings provide baseline results, best practices, and heuristics for pre-training such large models that should be valuable to HPC community at large. We also offer insights and optimization strategies for using the first exascale computing system, Frontier, to train models of the size of GPT-3 and beyond.