Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large system simulation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Time sequence of energetic particle spectra in quasiparallel shocks in large simulation systems

We have performed collisionless shock simulations using a one-dimensional hybrid particle-in-cell method to investigate the energy spectra of the differential intensity around the quasiparallel shocks. The system size is sufficiently large (200 000 ion inertia length) in order to eliminate the unphysical effect caused by the upstream boundary. The obtained spectrum of the differential intensity have the shape of the power-law with exponentially falling off in higher energy as predicted in previous simulations, however, the power-law indices and e-folding energy do not depend on the shock parameters, the shock Mach number M{sub A} (7.1-11.7) and the shock angle {theta}{sub BN} (10 deg. - 40 deg.). The power-law index is {approx}1.0. This number is close to the prediction by the standard diffusive shock acceleration theory. However, the reason for the agreements between the values is not clear because an additional acceleration process is also observed in the present runs; this process was reported in [Sugiyama and Terasawa, Adv. Space Res. 24, 73 (1999)] and [Sugiyama et al., J. Geophys. Res. 106, 21657 (2001)]. The e-folding energy linearly increases in time for all runs. One of the reasons for these independent profiles on shock parameters is that the pitch angle distribution in the upstream region shows similar profiles in each run.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A computational study of the use of an optimization-based method for simulating large multibody systems.

The present work aims at comparing the performance of several quadratic programming (QP) solvers for simulating large-scale frictional rigid-body systems. Traditional time-stepping schemes for simulation of multibody systems are formulated as linear complementarity problems (LCPs) with copositive matrices. Such LCPs are generally solved by means of Lemke-type algorithms and solvers such as the PATH solver proved to be robust. However, for large systems, the PATH solver or any other pivotal algorithm becomes unpractical from a computational point of view. The convex relaxation proposed by one of the authors allows the formulation of the integration step as a QPD, for which a wide variety of state-of-the-art solvers are available. In what follows we report the results obtained solving that subproblem when using the QP solvers MOSEK, OOQP, TRON, and BLMVM. OOQP is presented with both the symmetric indefinite solver MA27 and our Cholesky reformulation using the CHOLMOD package. We investigate computational performance and address the correctness of the results from a modeling point of view. We conclude that the OOQP solver, particularly with the CHOLMOD linear algebra solver, has predictable performance and memory use patterns and is far more competitive for these problems than are the other solvers.

97 MATHEMATICS AND COMPUTING↗

Networked HIL Simulation System for Modeling Large-scale Power Systems

This paper presents a networked hardware-in-the-loop (HIL) simulation system for modeling large-scale power systems. Researchers have developed many HIL test systems for power systems in recent years. Those test systems can model both microsecond-level dynamic responses of power electronic systems and millisecond-level transients of transmission and distribution grids. By integrating individual HIL test systems into a network of HIL test systems, we can create large-scale power grid digital twins with flexible structures at required modeling resolution that fits for a wide range of system operating conditions. This will not only significantly reduce the need for field tests when developing new technologies but also greatly shorten the model development cycle. In this paper, we present a networked OPAL-RT based HIL test system for developing transmission-distribution coordinative Volt-VAr regulation technologies as an example to illustrate system setups, communication requirements among different HIL simulation systems, and system connection mechanisms. Impacts of communication delays, information exchange cycles, and computing delays are illustrated. Simulation results show that the performance of a networked HIL test system is satisfactory.

Xie, Fuhong↗

A fast and accurate modeling approach for water and steam thermodynamics with practical applications in district heating system simulation

In U.S. district heating (DH) systems, steam is the most common heat transport medium. Industry demand for new advanced modeling capabilities of complete steam DH systems is increasing; however, the existing models for water/steam thermodynamics are too slow for large system simulations because of computationally expensive algebraic loops that require the solution to nonlinear systems of equations. For practical applications, this work presents a novel split-medium approach that implements numerically efficient liquid water models alongside various water/steam models, breaking costly algebraic loops by decoupling mass and energy balance equations. New component models for steam DH systems are also presented. We implemented the models in the equation based Modelica language and evaluated accuracy and computing speed across multiple scales: from fundamental thermodynamic properties to complete districts featuring 10 to 200 buildings. Compared to district models with the IF97 water/steam model and equipment models from the Modelica Standard Library, the new implementation improves the scaling rate for large districts from cubic to quadratic with negligible compromise to accuracy. Additionally, for an annual simulation with 180 buildings, this translates to a computing time reduction from 33 to 1-1.5 h. These results are critically important for industry practitioners to simulate steam DH systems at large scales.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Linear and Nonlinear Solvers for Simulating Multiphase Flow within Large-Scale Engineered Subsurface Systems

Simulation of multiphase flow in the subsurface is well-known to be computationally challenging. While there have been many studies that have explored approaches to overcoming these challenges, they often utilize relatively simple case studies. In this paper, we focus on the unique numerical challenges posed by modeling large-scale engineered subsurface systems, characterized by discrete features embedded in a heterogeneous natural subsurface setting. The man-made features such as shafts, tunnels, and barriers often cause multiple challenges in modeling the domain for multiphase porous media flow. This flow scenario can have a wide range of applications such as nuclear waste repositories, enhanced recovery of a petroleum reservoir, geothermal engineering, and carbon sequestration. An example of these severe numerical challenges is the case of performance assessment (PA) for Waste Isolation Pilot Plant (WIPP), the only operating deep geological repository in the US, which simulates extreme material properties of bedded salt rock formation and extreme contrast due to open excavation next to the formation. The models have extremes not only of permeability and porosity but also of the constitutive models needed for multiphase flow; additionally, they have process models like salt creep closure reducing porosity over time, fracturing in clay and anhydrite interbeds of the bedded salt, gas generation from the waste materials, and unintentional human borehole intrusions in some scenarios. Numerical simulations require the solution of coupled systems of nonlinear PDEs; in our work, we use the open-source simulator PFLOTRAN which is based on Finite Volume discretization. The solution of the nonlinear equations requires use of the Newton-Raphson iteration at each time step, which entails the solution of the linearized Jacobian system at each iteration. The effects of all the processes (i.e., large number of unknowns, highly nonlinear constitutive relations, large contrasts in material properties in short distances) lead to an ill-conditioned Jacobian matrix that severely challenges traditional linear solver, i.e., stabilized biconjugate gradient with block Jacobi incomplete LU preconditioner (BCGS-ILU) leading to non-convergence for traditional Newton-Raphson nonlinear solver causing unacceptably long computation time for each model. This paper presents linear solvers such as constrained pressure residual (CPR) two-stage preconditioner with alternate-block-factorization (ABF) and quasi- implicit pressure and explicit saturation (QIMPES) decouplers and flexible generalized residual solver (FGMRES). The new general-purpose nonlinear solver, Newton trust-region dogleg Cauchy (NTRDC), is also introduced to resolve extreme nonlinearities in the models. We demonstrate the effectiveness of each method relative to the default BCGS-Newton solver. The two best cases had nearly 50 times speed-up and achieved completion of a simulation in 14 hours that never completed due to non-convergence with the default solver. We also investigate the strong scalability of each method and discuss some of the deficiencies found for Block Jacobi preconditioner using parallel domain decomposition, and node packing effects of modern processor architecture.

Preconditioner, Nonlinear, Porous media, Multiphas↗

Non-Empirical and Self-Interaction Corrections for DFTB: Towards Accurate Quantum Simulations for Large Mesoscale Systems (Final Report)

This project was comprised of two complementary (but parallel) thrusts: (1) implementing massively-parallelized computing hardware (with new computational hardware that may replace GPUs) in the density functional tight binding (DFTB) approach and (2) developing new capabilities in DFTB to calculate the electronic structure and dynamics of large chemical systems. While classical molecular dynamics can handle hundreds of thousands of atoms, it cannot provide a first-principles based description of chemical systems at the quantum level. At the other extreme, conventional Kohn-Sham DFT methods can probe the true quantum mechanical nature of chemical systems; however, these methods cannot tackle the large sizes and length scales relevant to dynamics simulations of realistic systems. The DFTB formalism utilized in this project provides a viable approach for probing these large systems at a quantum mechanical level of detail. However, to utilize the DFTB approach for accurate calculations of electronic properties, it is crucial to incorporate quantum-based corrections in DFTB since exchange-correlation effects can still remain very strong in these large systems. At the same time, enhancing the computational efficiency of DFTB is also essential since optimal computational performance is required for addressing the large size scales associated with realistic chemical systems. As such, the new non-empirical corrections and computing hardware enhancements implemented in this project will enable accurate and computationally efficient approaches to directly probe electronic properties in these large, complex systems.

36 MATERIALS SCIENCE↗

Bond breaking in epoxy systems: A combined QM/MM approach

A novel method to combine quantum mechanics (QM) and molecular mechanics has been developed to accurately and efficiently account for covalent bond breaking in polymer systems under high strain without the use of predetermined break locations. Use of this method will provide a better fundamental understanding of the mechano-chemical origins of fracture in thermosets. Since classical force fields cannot accurately account for bond breaking, and QM is too demanding to simulate large systems, a hybrid approach is required. In the method presented here, strain is applied to the system using a classical force field, and all bond lengths are monitored. When a bond is stretched past a threshold value, a zone surrounding the bond is used in a QM energy minimization to determine which, if any, bonds break. The QM results are then used to reconstitute the system to continue the classical simulation at progressively larger strain until another QM calculation is triggered. In this way, a QM calculation is only computed when and where needed, allowing for efficient simulations. A robust QM method for energy minimization has been determined, as well as appropriate values for the QM zone size and the threshold bond length. Compute times do not differ dramatically from classical molecular mechanical simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Hydrodynamic Interactions in Topologically Linked Ring Polymers.

Despite decades of interdisciplinary research on topologically linked ring polymers, their dynamics remain largely unstudied. These systems represent a major scientific challenge as they are often subject to both topological and hydrodynamic interactions (HI), which render dynamical solutions either mathematically intractable or computationally prohibitive. Here we circumvent these limitations by preaveraging the HI of linked rings. We show that the symmetry of ring polymers leads to a hydrodynamic decoupling of ring dynamics. This decoupling is valid even for nonideal polymers and nonequilibrium conditions. Physically, our findings suggest that the effects of topology and HI are nearly independent and do not act cooperatively to influence polymer dynamics. We use this result to develop highly efficient Brownian dynamics algorithms that offer enormous performance improvements over conventional methods and apply these algorithms to simulate catenated ring polymers at equilibrium, confirming the independence of topological effects and HI. The methods developed here can be used to study and simulate large systems of linked rings with HI, including kinetoplast DNA, Olympic gels, and poly[n]catenanes.

Rauscher, Phillip M.↗

Heterogeneous CPU+GPU-Enabled Simulations for DFTB Molecular Dynamics of Large Chemical and Biological Systems

We introduce a new heterogeneous CPU+GPU-enhanced DFTB approach for the routine and efficient simulation of large chemical and biological systems. Here, compared to homogenous computing with conventional CPUs, heterogeneous computing approaches exhibit substantial performance with only a modest increase in power consumption, both of which are essential to upcoming exascale computing initiatives. We show that DFTB-based molecular dynamics is a natural candidate for heterogeneous computing since the computational bottleneck in these simulations is the diagonalization of the Hamiltonian matrix, which is performed several times during a single molecular dynamics trajectory. To thoroughly test and understand the performance of our heterogeneous CPU+GPU approach, we examine a variety of algorithmic implementations, benchmarks of different hardware configurations, and applications of this methodology on several large chemical and biological systems. Finally, to demonstrate the capability of our implementation, we conclude with a large-scale DFTB MD simulation of explicitly solvated HIV protease (3,974 atoms total) as a proof-of-concept example of an extremely large/complex system which, to the best of our knowledge, is the first time that an entire explicitly-solvated protein has been treated at a quantum-based MD level of detail.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Influence of disorder on antidot vortex Majorana states in three-dimensional topological insulators

Topological insulator/superconductor two-dimensional heterostructures are promising candidates for realizing topological superconductivity and Majorana modes. In these systems, a vortex pinned by a prefabricated antidot in the superconductor can host Majorana zero-energy modes (MZMs), which are exotic quasiparticles that may enable quantum information processing. However, a major challenge is to design devices that can manipulate the information encoded in these MZMs. One of the key factors is to create small and clean antidots, so the MZMs, localized in the vortex core, have a large gap to other excitations. If the antidot is too large or too disordered, the level spacing for the subgap vortex states may become smaller than temperature. In this paper, we numerically investigate the effects of disorder, chemical potential, and antidot size on the subgap vortex spectrum, using a two-dimensional effective model of the topological insulator surface. Our model allows us to simulate large system sizes with vortices up to 1.8 µ⁢m in diameter (with a 6 nm lattice constant). We also compare our disorder model with the transport data from existing experiments. As a result, we find that the spectral gap can exhibit a nonmonotonic behavior as a function of disorder strength, and that it can be tuned by applying a gate voltage.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Characteristics of locational uncertainty marginal price for correlated uncertainties of variable renewable generation and demands

With the rapid increase of variable renewable energy sources in power systems, how to manage and price the uncertainty of renewable resources’ power outputs is becoming an urgent issue. Current market designs considering the uncertainties are mainly based on the probabilistic scenario set of demand and renewable energy resources power outputs. This consideration makes market designs vulnerable to three significant challenges when put into practice. First, the accurate probability distribution of renewable generation is hard to obtain in real-time. Second, it is challenging to clear the market timely with many scenarios to guarantee accuracy. Third, generation cost recovery cannot be guaranteed for some scenarios. To overcome these challenges, this paper proposes a locational uncertainty marginal price model to price the uncertainty explicitly based on a scenario-free stochastic market-clearing model. Instead of using the probabilistic scenario set, the uncertainty of renewable energy sources and loads is modeled with distributionally-robust chance constraints. The correlation of uncertainties can be endogenously modeled in both the market-clearing and the locational uncertainty marginal price formation. Furthermore, this paper proves that generation cost recovery, revenue adequacy, and partial market equilibrium can be achieved using the locational uncertainty marginal price model. Numerical results from both the small and large systems simulations validate that the generation cost recovery is maintained no matter the generation participates in uncertainty mitigation or not. The transmission congestion surplus is also allocated appropriately among loads, renewable energy sources, and financial transmission right owners.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Molecular Modeling and Molecular Dynamics Simulation of a Packed and Intact Bacterial Microcompartment

Bacterial microcompartments (BMCs) are protein-bound organelles found in some bacteria which encapsulate enzymes for enhanced catalytic activity. These compartments spatially sequester enzymes within semipermeable shell proteins and are packed full of enzyme cargoes and metabolites as they fulfill their function. Coupling together recent SAXS and proteomics work, it is possible to develop molecular models for these microcompartments and interrogate enzyme and metabolite dynamics within. Our primary goal of this study is to quantify the permeability of metabolite glyceraldehyde-3-phosphate (G3P) and dihydroxyacetone phosphate (DHAP) across the BMC shell through classical molecular dynamics simulation. The Haliangium ochraceum model of BMC shell (PDB: 6MZX) was used to model an intact BMC of approximately 10 million atoms. Working at this scale presented its own challenges in managing large data sets, with multiple challenges and hardware advances discussed that facilitated this work. Over approximately 750 ns of aggregate simulation, we see multiple permeation events for these metabolites that were added at high concentration through the pores present within BMC shell tiles. When compared to independent permeability estimates for the same metabolites determined through replica exchange umbrella sampling simulations, the permeabilities varied by approximately 3 orders of magnitude. Regardless, the permeability coefficients for both G3P and DHAP are highly similar and very high, such that only very small concentration gradients can be maintained across the BMC shell between the cytosol and BMC interior. The large simulation systems also facilitated comparisons for molecular diffusivity in the crowded environment within the BMC shell. By our estimates, the viscosity within a packed BMC shell is at least 10-fold higher than it would be in neat solution and is the real driver for varying permeability estimates we obtained through simulation. These findings will be used as design inputs for future bioengineering efforts to make products from BMCs, highlighting how permeable BMC shells can be.

Diffusion↗

Almost strong zero modes at finite temperature

Interacting fermionic chains exhibit extended regions of topological degeneracy of their ground states as a result of the presence of Majorana or parafermionic zero modes localized at the edges. In the opposite limit of infinite temperature, the corresponding nonintegrable spin chains, obtained via generalized Jordan-Wigner mapping, are known to host so-called almost strong zero modes, which are long-lived with respect to any bulk excitations. Here we study the fairly unexplored territory that bridges these two extreme cases of zero and infinite temperature. We blend two established techniques for states, the Lanczos series expansion and a tensor network ansatz, uplifting them to the level of operator algebra. This allows us to efficiently simulate large system sizes for arbitrarily long timescales and to extract the temperature-dependent decay rates. We observe that for the Kitaev-Hubbard model, the decay rate of the edge mode depends exponentially on the inverse temperature 𝛽, and on an effective energy scale Δ eff that is greater than the thermodynamic gap of the system Δ.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Constant-pH Molecular Dynamics Simulations for Large Biomolecular Systems

We report that an increasingly important endeavor is to develop computational strategies that enable molecular dynamics (MD) simulations of biomolecular systems with spontaneous changes in protonation states under conditions of constant pH. The present work describes our efforts to implement the powerful constant-pH MD simulation method, based on a hybrid nonequilibrium MD/Monte Carlo (neMD/MC) technique within the highly scalable program NAMD. The constant-pH hybrid neMD/MC method has several appealing features; it samples the correct semigrand canonical ensemble rigorously, the computational cost increases linearly with the number of titratable sites, and it is applicable to explicit solvent simulations. The present implementation of the constant-pH hybrid neMD/MC in NAMD is designed to handle a wide range of biomolecular systems with no constraints on the choice of force field. Furthermore, the sampling efficiency can be adaptively improved on-the-fly by adjusting algorithmic parameters during the simulation. Finally, illustrative examples emphasizing medium- and large-scale applications on next-generation supercomputing architectures are provided.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Enabling Scalable VQE Simulation on Leading HPC Systems

Large-scale simulations of quantum circuits pose significant challenges, especially in the context of quantum chemistry, due to the number of qubits, circuit depth, and the number of circuits needed per problem. High-performance computing (HPC) systems offer massive computational capabilities that could help overcome these obstacles. We developed a high-performance quantum circuit simulator, called NWQ-Sim, and demonstrate its capability to simulate large quantum chemistry problems on NERSC's Perlmutter supercomputer. Integrating NWQ-Sim with XACC, we have executed QPE and VQE algorithms for downfolded quantum chemistry systems at unprecedented scales. Our work demonstrates the potential of leveraging HPC resources to advance quantum chemistry and other applications of near-term quantum devices.

Wang, Meng↗

Distributed Parareal in Time with Adaptive Coarse Solver for Large Scale Power System Simulations

This paper describes a distributed Parareal in time algorithm with the adaptive coarse solver. In the distributed algorithm, the propagation of a coarse solution is distributed among processors in addition to the fine solution propagation. In the adaptive coarse solver the complexity of the model is changed based on the size of generator rotor angle deviation. In addition, non-iterative approach to model a ZIP load is presented. The proposed method is tested on the large-scale 5617-machine 70285-bus Eastern Interconnection system.

Osipov, Denis↗