Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scalable Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗
DOE OSTI2024

EFIT-Prime: Probabilistic and physics-constrained reduced-order neural network model for equilibrium reconstruction in DIII-D

We introduce EFIT-Prime, a novel machine learning surrogate model for EFIT (Equilibrium FIT) that integrates probabilistic and physics-informed methodologies to overcome typical limitations associated with deterministic and ad hoc neural network architectures. EFIT-Prime utilizes a neural architecture search-based deep ensemble for robust uncertainty quantification, providing scalable and efficient neural architectures that comprehensively quantify both data and model uncertainties. Physically informed by the Grad–Shafranov equation, EFIT-Prime applies a constraint on the current density J tor and a smoothness constraint on the first derivative of the poloidal flux, ensuring physically plausible solutions. Furthermore, the spatial location of the diagnostics is explicitly incorporated in the inputs to account for their spatial correlation. Extensive evaluations demonstrate EFIT-Prime's accuracy and robustness across diverse scenarios, most notably showing good generalization on negative-triangularity discharges that were excluded from training. Timing studies indicate an ensemble inference time of 15 ms for predicting a new equilibrium, offering the possibility of plasma control in real-time, if the model is optimized for speed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Blueprint for DOE Quantum Supercomputing: Ensuring U.S. Leadership in the Quantum Decade

Quantum computing stands at the threshold of a transformative decade, where the field will evolve from small-scale demonstrations toward practical scientific computing at scale. This Blueprint identifies fault-tolerant quantum computers (FTQCs) as a viable, scalable, and broadly applicable path to achieving “quantum scientific utility,” defined as solving scientifically valuable problems beyond the reach of conventional, classical computers. This capability is expected to show scientific demonstrations in the late 2020s and to mature in the early-to-mid 2030s. This Blueprint outlines a strategy to prepare the U.S. Department of Energy (DOE) for FTQCs and their integration into the U.S. national scientific computing infrastructure. Its purpose is to identify the steps, milestones, and research directions necessary for DOE to enable initial deployment of FTQCs in 2028 as a scientific tool for the nation and mature this capability into the 2030s. DOE has a long history of supporting quantum information science and technology, contributing significantly to research advancements, training a quantum-ready workforce, and providing access to early small-scale quantum hardware. Given recent demonstrations of logical operations on error-corrected logical qubits and the advancement of commercial hardware roadmaps, DOE should begin preparations for large-scale, fault-tolerant quantum computing deployment for DOE science missions. This Blueprint proposes that DOE focus on (1) deploying first-generation scientifically relevant quantum computers with at least 100 logical qubits and performing at least 10,000 to 100,000 hard logical operations in scientifically relevant calculations; (2) developing essential FTQC programming competencies, system software, and facility readiness; and (3) investing in cutting edge focused R&D that fosters breakthroughs in scientific applications, algorithms, and logical architectures needed to accelerate the advent of scientific utility. This effort will position DOE to transition to larger systems: production-scale quantum computers that comprise 1,000 to 10,000 logical qubits, perform 1 to 10 billion hard logical operations, and execute scientifically useful computations at scale. Achieving these goals will require DOE facilities to evolve with urgency to support scientific campaigns that integrate quantum and classical computing resources into efficient workflows, novel software and firmware environments for compiling and routing quantum programs on FTQC machines, and suitable infrastructure for quantum hardware. It will also require further development and optimization of scientific applications from the fields of materials science, quantum chemistry, and high-energy and nuclear physics. The Blueprint calls for transformative R&D and collective action to accelerate the advent of scientific quantum utility and bring it within reach by 2028.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Adiabatic quantum support vector machines

Adiabatic quantum computers can solve difficult optimization problems (e.g., the quadratic unconstrained binary optimization problem), and they seem well suited to train machine learning models. In this paper, we describe an adiabatic quantum approach for training support vector machines. We show that the time complexity of our quantum approach is an order of magnitude better than the classical approach. Next, we compare the test accuracy of our quantum approach against a classical approach that uses the Scikit-learn library in Python across five benchmark datasets (Iris, Wisconsin Breast Cancer (WBC), Wine, Digits, and Lambeq). We show that our quantum approach obtains accuracies on par with the classical approach. Finally, we perform a scalability study in which we compute the total training times of the quantum approach and the classical approach with an increasing number of features and an increasing number of data points in the training dataset. In conclusion, our scalability results show that the quantum approach obtains a 3.5–4.5x speedup over the classical approach on datasets with many (millions of) features.

Computational Complexity↗
DOE OSTI2022

A phenology- and trend-based approach for accurate mapping of sea-level driven coastal forest retreat

The rapid replacement of upland forest by encroaching marshland is a striking manifestation of global sea-level rise (SLR). Timely and high-resolution information on the location and extent of transition forest (the ecotone between upland forest and marsh where tree mortality due to seawater intrusion begins) is fundamental to understanding the processes and patterns of SLR-driven landscape reorganization. Despite its significance, accurate characterization of salt-impacted transition forest remains challenging due to the complexity of coastal environments, scarcity of ground-truth data, and the lack of effective mapping algorithms. Here we use the full archive of Landsat images between 1984 and 2021 to investigate the spectral, temporal, and phenological characteristics of transition forest, and develop a robust framework for monitoring coastal vegetation shifts in the mid-Atlantic U.S., a global SLR hotspot. Here, we found that transition forest exhibits strong negative NDVI trends and a deviation of land surface phenology from marsh and upland forest that distinguishes itself from surrounding vegetation. By integrating temporal trends and land surface phenology, our results demonstrate superior discrimination between marsh and coastal forests to existing map products (e.g. NOAA Coastal Change Analysis Program, National Land Cover Database) that allows a reliable identification of the coastal treeline. We applied the approach to map regional land cover in 1985, 2000 and 2020 (overall classification accuracy >92%) and found that the area of coastal forest decreased by 22.0% from 1985 to 2020, the majority of which transitioned to marshland (92.3%, 5.3 × 10 3 ha). Based upon fine-scale patterns of coastal transgression, we created a practical workflow for spatially explicit quantification of forest retreat rates. Concurrent with rising sea level, coastal forests migrated upslope from 0.63 (± 0.27) m above sea level in 1985 to 0.78 (± 0.32) m above sea level in 2020, and horizontal forest retreat rates accelerated from 3.1 (range of 0–36) m yr -1 during 1985–2000 to 4.7 (0–55) m yr -1 during 2001–2020. As SLR continues to accelerate, our study may serve as a scalable solution for consistent tracking of coastal landscape evolution that is urgently needed for sustainable forest and wetland management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Benders Decomposition Using Graph Modeling and Multi-Parametric Programming

Benders decomposition is a widely used method for solving large and structured optimization problems, but its performance is affected by the repeated solution of subproblems. We propose a flexible and modular algorithmic framework for accelerating Benders decomposition. Specifically, we express the problem structure by using a graph-theoretic modeling abstraction in which nodes represent optimization subproblems and edges represent connectivity between subproblems. A key innovation of our approach is that we embed multiparametric programming (mp) surrogates for node subproblems, which maps the exact analytical map of the subproblem solution space. The use of mp surrogates allows us to replace subproblem solves with fast look-ups and function evaluations for primal and dual variables during the iterative Benders process. We formally show the equivalence between classical Benders cuts and those derived from the mp solution. We implement our framework in the open-source PlasmoBenders.jl software package. To demonstrate the capabilities of the proposed framework, we apply it to a two-stage stochastic programming problem, which aims to make optimal capacity expansion decisions under market uncertainty. We evaluate both single-cut and multicut variants of Benders decomposition and show that the use of mp surrogates achieves substantial speedups in subproblem solve time, while preserving the convergence guarantees of Benders decomposition. We highlight advantages in solution analysis and interpretability that is enabled by mp critical region tracking; specifically, we show that these reveal how decisions evolve geometrically across the Benders search. Our results aim to demonstrate that combining surrogate modeling with graph modeling offers a promising and extensible foundation for structure-exploiting decomposition. In addition, by decomposing the problem into more tractable subproblems, the proposed approach also aims to overcome scalability issues of mp. Finally, the use of mp surrogates provides a unifying and modular optimization framework that enables the representation of heterogeneous node subproblems as modeling objects with a homogeneous structure.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

Updates in Developing a Prototype Science Pipeline and Full-Volume, Global Hyperspectral Synthetic Data Sets for NASA’s Earth System Observatory’s Upcoming Surface, Biology and Geology Mission

The Surface Biology and Geology (SBG) mission recently passed mission confirmation review and has entered phase A – design and development. SBG will acquire high resolution solar-reflected spectroscopy and thermal infrared observations at a data rate of ~2.5 TB/day and generate products at ~40 TB/day. Given that the per-day volume is greater than NASA’s total extant airborne hyperspectral data collection, collecting, processing, disseminating, and exploiting the SBG data present new challenges. To meet these challenges, we have developed a prototype science pipeline and a full-volume global hyperspectral synthetic data set to help prepare for SBG’s flight (see poster GC42D-0730). Our science pipeline is based on the science processing technology developed for NASA’s Kepler and TESS planet-hunting missions. The pipeline infrastructure, Ziggy, provides a scalable architecture for robust, repeatable, and replicable science and application products that can be run on a range of systems from a laptop to the cloud or a supercomputer. Ziggy is compliant with NASA Procedural Requirement (NPR) 7150.2C, is at a technical readiness level (TRL) of 7 and has been released to github.com/nasa/ziggy. We integrated Ziggy with EO-1/Hyperion workflows to build a prototype pipeline and ingested the 17-year mission archive that provides globally sampled visible through shortwave infrared spectra that are representative of SBG data types and volumes. We fully implemented the first stage and processed the entire 55 TB Hyperion data set from the raw data (Level 0) to top-of-the-atmosphere radiance (Level 1R). We are currently evaluating the ISOFIT atmospheric correction module to convert the L1R data to surface reflectance (Level 2) before reprocessing the full data set to L2. Crosschecks are being performed with RadCalNet as well as with coincident observations by AVIRIS. We are also investigating modern methods for georectifying the Hyperion scenes. Finally, we describe an analysis of the cost to conduct forward processing and reprocessing campaigns for SBG on HECC with dedicated compute and storage resources using the resurrected Hyperion pipeline as a proxy for full-volume SBG data. The analysis demonstrates that SBG L0 data can be processed to L2 on HECC with full reprocessing campaigns every two years for ~$2.6M over a 7-year lifespan. Moreover, 69% of the system capacity would be available for other activities, possibly enabling future open-source science activities, including algorithm development, L3+ processing, .etc.

ESD↗
NASA NTRS2000

Use Computer-Aided Tools to Parallelize Large CFD Applications

Porting applications to high performance parallel computers is always a challenging task. It is time consuming and costly. With rapid progressing in hardware architectures and increasing complexity of real applications in recent years, the problem becomes even more sever. Today, scalability and high performance are mostly involving handwritten parallel programs using message-passing libraries (e.g. MPI). However, this process is very difficult and often error-prone. The recent reemergence of shared memory parallel (SMP) architectures, such as the cache coherent Non-Uniform Memory Access (ccNUMA) architecture used in the SGI Origin 2000, show good prospects for scaling beyond hundreds of processors. Programming on an SMP is simplified by working in a globally accessible address space. The user can supply compiler directives, such as OpenMP, to parallelize the code. As an industry standard for portable implementation of parallel programs for SMPs, OpenMP is a set of compiler directives and callable runtime library routines that extend Fortran, C and C++ to express shared memory parallelism. It promises an incremental path for parallel conversion of existing software, as well as scalability and performance for a complete rewrite or an entirely new development. Perhaps the main disadvantage of programming with directives is that inserted directives may not necessarily enhance performance. In the worst cases, it can create erroneous results. While vendors have provided tools to perform error-checking and profiling, automation in directive insertion is very limited and often failed on large programs, primarily due to the lack of a thorough enough data dependence analysis. To overcome the deficiency, we have developed a toolkit, CAPO, to automatically insert OpenMP directives in Fortran programs and apply certain degrees of optimization. CAPO is aimed at taking advantage of detailed inter-procedural dependence analysis provided by CAPTools, developed by the University of Greenwich, to reduce potential errors made by users. Earlier tests on NAS Benchmarks and ARC3D have demonstrated good success of this tool. In this study, we have applied CAPO to parallelize three large applications in the area of computational fluid dynamics (CFD): OVERFLOW, TLNS3D and INS3D. These codes are widely used for solving Navier-Stokes equations with complicated boundary conditions and turbulence model in multiple zones. Each one comprises of from 50K to 1,00k lines of FORTRAN77. As an example, CAPO took 77 hours to complete the data dependence analysis of OVERFLOW on a workstation (SGI, 175MHz, R10K processor). A fair amount of effort was spent on correcting false dependencies due to lack of necessary knowledge during the analysis. Even so, CAPO provides an easy way for user to interact with the parallelization process. The OpenMP version was generated within a day after the analysis was completed. Due to sequential algorithms involved, code sections in TLNS3D and INS3D need to be restructured by hand to produce more efficient parallel codes. An included figure shows preliminary test results of the generated OVERFLOW with several test cases in single zone. The MPI data points for the small test case were taken from a handcoded MPI version. As we can see, CAPO's version has achieved 18 fold speed up on 32 nodes of the SGI O2K. For the small test case, it outperformed the MPI version. These results are very encouraging, but further work is needed. For example, although CAPO attempts to place directives on the outer- most parallel loops in an interprocedural framework, it does not insert directives based on the best manual strategy. In particular, it lacks the support of parallelization at the multi-zone level. Future work will emphasize on the development of methodology to work in a multi-zone level and with a hybrid approach. Development of tools to perform more complicated code transformation is also needed.

Jin, H.↗
NASA NTRS2016

Transitioning Autonomous Systems Technology Research to a Flight Software Environment

NASA has developed methods and algorithms for autonomous spacecraft operations,including automated planning and scheduling, fault diagnostics and impact determination,procedure management and display. Making the transition from technology research tooperational flight software requires overcoming significant technical, programmatic andcultural challenges. Technology research is aimed at developing methods that performspecific functions correctly, but the resulting software may not be designed for flightprocessors with limited CPU, memory and network resources, and may not be easilyintegrated into spacecraft flight software. Our objective in the Autonomous Systems andOperations Project is to make significant strides toward the transformation from technologyto operational use. Our focus was twofold: maturing research grade autonomy software intoa flight software environment using broadly accepted languages and tools; and integratingautonomy applications with each other and with representative systems and their data andcommand interfaces. For a target flight software environment, we chose Core FlightSoftware, developed by Goddard Space Flight Center as a common operating systemindependent framework. Our hardware integration environment was provided by theIntegrated Power and Avionics Systems (iPAS) Lab at Johnson Space Center, in whichvarious subsystem development has been conducted to address engineering challenges forthe vehicles and systems required for long-duration missions into the solar system. The iPASand its network of connected facilities provides realistic subsystem hardware or simulationsof spacecraft power, life support, guidance, navigation and control, and command and datahandling subsystems. Interfaces between autonomy applications and the subsystems beingassessed and controlled were developed, assessed and refined. The hardware and softwareenvironment using CFS and the iPAS facility has proven to be a highly flexible and realisticenvironment in which to rapidly integrate applications in an iterative, low cost setting. Usingthe integration environment we have developed, we will turn our focus to performance andsizing analysis to determine the computational requirements for full-scale deployment ofautonomy technology. Scalability of reasoners and the spacecraft models upon which theyoperate, and robustness across the full range of spacecraft conditions and environments willbe explored and improved. We are making significant contributions to the future programsthat will build the spacecraft that will take humans beyond the Earth-Moon system, in whichprogram Systems Engineers will be able to accurately and confidently design in accurate,robust and mature autonomous operations systems.

Flight Software↗
DOE OSTI2019

Modeling Mesoscale Processes of Scalable Synthesis (Final Report)

The main objective of the center is to develop tools that can help for the understanding of equilibrium and non-equilibrium properties of nanoparticles in aqueous electrolytes. To achieve this goal, we need to develop new computational methodologies that can integrate molecular level description such as the electronic structure analysis to mesoscale description such as generalized hydrodynamics. The challenge is to develop systematic coarse graining techniques that can adaptively choose the level of description needed for the particular task. For example, if bonding breaking is involved, then one should use appropriate models from quantum mechanics to accurately model the time scale and molecular mechanism for the breaking process. However, to model transport process or the process of self-assembly, much less detailed models can be used. There are two main components to this task. One is to develop accurate models that bridge different levels of description. The classical example is the QM/MM model that bridges quantum mechanics and molecular mechanics for studying chemical reactions of large molecules. This work was awarded the Nobel prize this year. For our purpose, we also need to develop methods that bridge molecular and continuum levels of description. Another main task is to develop systematic coarse-graining methods that can give rise to accurate mesoscale and macroscale models, starting from more detailed models at the atomic or molecular scale. None of these ideas are new. By now they have been fashionable topics for quite some time. But neither problem is satisfactorily resolved. The main problem is that uncontrolled approximations are used. As a result, we have very little idea about the validity and accuracy of these models. The main purpose of the center is to exploit the Mori-Zwanzig formalism to achieve the goals stated above. The Mori-Zwanzig formalism is a general tool for eliminating degrees of freedom from existing models. What is attractive about this formalism is its generality. The flip side is that it is also somewhat of a tautology. In order to make it a practical tool, one has to make approximations and this is where the trouble comes. Instead of focusing on Mori-Zwanzig formalism, the PI proposes to study the closely related renormalization group formalism (RG). Like Mori-Zwanzig, RG is a general strategy for eliminating small scale degrees of freedom in a model. Unlike Mori-Zwanzig, RG has the advantage that at least for some problems, one can write a PDE (partial differential equation) type of model for the exact RG flow, without any approximation. This gives us a solid starting point for discussing and analyzing approximations. The PI studied approximate RG flows for turbulent diffusion and related problems. However, extending it to more practical problems proved to be hard. The second line that the PI has pursued is to develop special numerical algorithms for studying particularly important events for dynamics processes in meso-scale systems. The PI succeeded in doing so for the problem of understanding phase transition in metallic systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Fast and scalable quantum Monte Carlo simulations of electron-phonon models

We introduce methodologies for highly scalable quantum Monte Carlo simulations of electron-phonon models, and report benchmark results for the Holstein model on the square lattice. The determinant quantum Monte Carlo (DQMC) method is a widely used tool for simulating simple electron-phonon models at finite temperatures, but incurs a computational cost that scales cubically with system size. Alternatively, near-linear scaling with system size can be achieved with the hybrid Monte Carlo (HMC) method and an integral representation of the Fermion determinant. Here, we introduce a collection of methodologies that make such simulations even faster. To combat "stiffness" arising from the bosonic action, we review how Fourier acceleration can be combined with time-step splitting. To overcome phonon sampling barriers associated with strongly-bound bipolaron formation, we design global Monte Carlo updates that approximately respect particle-hole symmetry. To accelerate the iterative linear solver, we introduce a preconditioner that becomes exact in the adiabatic limit of infinite atomic mass. Finally, we demonstrate how stochastic measurements can be accelerated using fast Fourier transforms. Here, these methods are all complementary and, combined, may produce multiple orders of magnitude speedup, depending on model details.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

SpecSims: A Scalable Speculative Tree-based Simulation Cloning Framework for Finite Memory Machines

Simulation cloning is a technique in which cloned simulations whose state spaces differ partially from their parent simulation due to intervening events are spawned at runtime and concurrently advanced. It is a powerful method to carry out what-if analysis by speculatively exploring and evaluating the impact of various permutations of intervening cascade of events. Due to the exponential growth in the number of possible clones even for a small number of distinct intervening events, the practical efficacy of the approach is often severely limited by the maximum available memory of the computing host. In this paper, we introduce a novel speculative simulation cloning framework that executes a simulation cloning campaign capable of efficiently exploring an exponentially large space of clone simulations created by permutation of intervening events under a finite memory constraint. We provide a theoretical analysis of the runtime characteristics of our proposed approach and highlight its novel advantages such as memory-aware and as-long-as-needed execution. Furthermore, in support of our analytical findings and to demonstrate its practical feasibility, we implement a prototype of the cloning framework on a shared memory system and report its performance characteristics in the context of a heat diffusion simulation, and a power grid simulation subject to cascading disruptions from geomagnetic disturbances.

Simulation framework↗
DOE OSTI2017

Atomistic Simulation Derived Insight on the Irreversible Structural Changes of Si Electrode during Fast and Slow Delithiation

Quantifying the irreversible chemical and structural changes of Si during cycling remains challenging. In this study, a continuous reactive molecular dynamics delithiation algorithm, with well-controlled potential gradient and delithiation rate, was developed and used to investigate the “natural” delithiation responses of an aluminum-oxide coated silicon thin-film. Fast delithiation led to the formation of dense Si network near the surface and nanoporosity inside the a-Li x Si, resulting in 141% volume dilation and significant amount of Li trapped inside (a-Li 1.2 Si) at the end of delithiation process. In contrast, slow delithiation allowed the a-Li x Si to shrink by near-equilibrium condition, demonstrating no permanent inner pore with nearly Li-free structure (a-Li 0.2 Si) and minimal volume dilation (44%). However, even without trapped Li, the delithiated a-LixSi still exhibited higher volume (lower density) than the equilibrium structure with the same Li concentration, despite delithiation rate. The origin of this excess volume is the loss of directly bonded Si–Si pairs, which made the subsequent relithiation faster. On the basis of the atomistic modeling and the quantified degradation mechanism, battery operating guidelines, including the delithiation rate and the depth of charge to avoid trapped Li and coating delamination, were suggested to improve the durability Si electrodes.

25 ENERGY STORAGE↗
DOE OSTI2021

Real Time - Optimal Power Flow Based Distributed Energy Resources Management System (DERMS) (CRADA Number CRD-20-16909 Final Report)

The integration of behind-the-meter distributed energy resources (DERs) into distribution systems brings transformative changes to power systems. This requires operators and planners to find solutions to modernize electric grids and to effectively manage DERs for grid services. NREL developed novel DER management algorithms (referred to as Real-Time Optimal Power Flow, RT-OPF) through U.S. Department of Energy (DOE)-funded efforts, including Advanced Research Projects Agency–Energy (ARPA-E) Network Optimized Distributed Energy Systems (NODES) funding. This cutting-edge control technology aims to modernize distribution systems with large amounts of DER integration, which will help utilities solve issues brought by renewable integration and build resilient and renewable-based electric grids nationwide. Utilidata worked with NREL to investigate the commercialization opportunity of this RT-OPF-based distributed energy resource management system (DERMS). In this project, NREL performs the technology transfer of the RT-OPF to Utilidata to help them fully understand the RT-OPF solution, to identify potential engineering hurdles, and to assess the expected commercial value of various RT-OPF use cases and deployment. The technology transfer work includes two major tasks. First, NREL performs an in-depth knowledge transfer of the entire RT-OPF solution to Utilidata to help them gain an extensive and detailed understanding of the complete RT-OPF solution. In this task, NREL provides exhaustive information (e.g., documentation, code packages, laboratory and field trial data, performance results) while conducting in-depth training sessions to provide a thorough explanation of the entire solution. NREL hosts meetings to present different topics related to the RT-OPF solution, and question-and-answer sessions are included in each meeting to better explain the RT-OPF-related work. Second, the RT-OPF simulations are performed in a laboratory environment. The main objectives of this task are to walk through with Utilidata engineers how to set up a simulation of RT-OPF, identifying each RT-OPF code block/component in operation, learning how these components interact with each other, and eventually running RT-OPF simulations under various system conditions. This task helps Utilidata engineers understand performance limitations and constraints while also quantifying the commercial value of the RT-OPF for different use cases. Based on these two tasks, Utilidata engineers should be able to define and prioritize the next steps of RT-OPF implementation with an eye toward commercial success and scalability of the solution. The next steps are expected to be part of a new project following the conclusion of this project.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2015

Python-Based Scientific Analysis and Visualization of Precipitation Systems at NASA Marshall Space Flight Center

At NASA Marshall Space Flight Center (MSFC), Python is used several different ways to analyze and visualize precipitating weather systems. A number of different Python‐based software packages have been developed, which are available to the larger scientific community. The approach in all these packages is to utilize pre‐existing Python modules as well as to be object‐oriented and scalable. The first package that will be described and demonstrated is the Python Advanced Microwave Precipitation Radiometer (AMPR) Data Toolkit, or PyAMPR for short. PyAMPR reads geolocated brightness temperature data from any flight of the AMPR airborne instrument over its 25‐year history into a common data structure suitable for user‐defined analyses. It features rapid, simplified (i.e., one line of code) production of quick‐look imagery, including Google Earth overlays, swath plots of individual channels, and strip charts showing multiple channels at once. These plotting routines are also capable of significant customization for detailed, publication‐ready figures. Deconvolution of the polarization‐varying channels to static horizontally and vertically polarized scenes is also available. Examples will be given of PyAMPR's contribution toward real‐time AMPR data display during the Integrated Precipitation and Hydrology Experiment (IPHEx), which took place in the Carolinas during May‐June 2014. The second software package is the Marshall Multi‐Radar/Multi‐Sensor (MRMS) Mosaic Python Toolkit, or MMM‐Py for short. MMM‐Py was designed to read, analyze, and display three‐dimensional national mosaicked reflectivity data produced by the NOAA National Severe Storms Laboratory (NSSL). MMM‐Py can read MRMS mosaics from either their unique binary format or their converted NetCDF format. It can also read and properly interpret the current mosaic design (4 regional tiles) as well as mosaics produced prior to late July 2013 (8 tiles). MMM‐Py can easily stitch multiple tiles together to provide a larger regional or national picture of precipitating weather systems. Composites, horizontal and vertical crosssections, and combinations thereof are easily displayed using as little as one line of code. MMM‐Py can also write to the native MRMS binary format, and sub‐sectioning of tiles (or multiple stitched tiles) is anticipated to be in place by the time of this meeting. Thus, MMM‐Py also can be used to power the creation of custom mosaics for targeted regional studies. Overlays of other data (e.g., lightning observations) are easily accomplished. Demonstrations of MMM‐Py, including the creation of animations, will be shown. Finally, Marshall has done significant work to interface Python‐based analysis routines with the U.S. Department of Energy's Py‐ART software package for radar data ingest, processing, and analysis. One example of this is the Python Turbulence Detection Algorithm (PyTDA), an MSFC‐based implementation of the National Center for Atmospheric Research (NCAR) Turbulence Detection Algorithm (NTDA) for the purposes of convective‐scale analysis, situational awareness, and forensic meteorology. PyTDA exploits Py‐ART's radar data ingest routines and data model to rapidly produce aviation‐relevant turbulence estimates from Doppler radar data. Work toward processing speed optimization and better integration within the Py‐ART framework will be highlighted. Python‐based analysis within the Py‐ART framework is also being done for new research related to intercomparison of ground‐based radar data with satellite estimates of ocean winds, as well as research on the electrification of pyrocumulus clouds.

Lang, Timothy J.↗
DOE OSTI2024

The Effective Fragment Molecular Orbital Method: Achieving High Scalability and Accuracy for Large Systems

The effective fragment molecular orbital (EFMO) method has been developed to predict the total energy of a very large molecular system accurately (with respect to the underlying quantum mechanical method) and efficiently by taking advantage of the locality of strong chemical interactions and employing a two-level hierarchical parallelism. The accuracy of the EFMO method is partly attributed to the accurate and robust intermolecular interaction prediction between distant fragments, in particular, the many-body polarization and dispersion effects, which require the generation of static and dynamic polarizability tensors by solving the coupled perturbed Hartree–Fock (CPHF) and time-dependent HF (TDHF) equations, respectively. Solving the CPHF and TDHF equations is the main EFMO computational bottleneck due to the inefficient (serial) and I/O-intensive implementation of the CPHF and TDHF solvers. In this work, the efficiency and scalability of the EFMO method are significantly improved with a new CPU memory-based implementation for solving the CPHF and TDHF equations that are parallelized by either message passing interface (MPI) or hybrid MPI/OpenMP. Here, the accuracy of the EFMO method is demonstrated for both covalently bonded systems and noncovalently bound molecular clusters by systematically examining the effects of basis sets and a key distance-related cutoff parameter, R cut . R cut determines whether a fragment pair (dimer) is treated by the chosen ab initio method or calculated using the effective fragment potential (EFP) method (separated dimers). Decreasing the value of Rcut increases the number of separated (EFP) dimers, thereby decreasing the computational effort. It is demonstrated that excellent accuracy (<1 kcal/mol error per fragment) can be achieved when using a sufficiently large basis set with diffuse functions coupled with a small R cut value. With the new parallel implementation, the total EFMO wall time is substantially reduced, especially with a high number of MPI ranks. Given a sufficient workload, nearly ideal strong scaling is achieved for the CPHF and TDHF parts of the calculation. For the first time, EFMO calculations with the inclusion of long-range polarization and dispersion interactions on a hydrated mesoporous silica nanoparticle with explicit water solvent molecules (more than 15k atoms) are achieved on a massively parallel supercomputer using nearly 1000 physical nodes. In addition, EFMO calculations on the carbinolamine formation step of an amine-catalyzed aldol reaction at the nanoscale with explicit solvent effects are presented.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

Generation of Continental Scale Percent Tree Cover Product Using Deep-learning and Multi-scale Remote Sensing Data

Spatially explicit percent tree cover (TC) estimation is critical for mapping forest aboveground biomass and its dynamics. While various TC products have been developed, there has not been a generalized framework that can be applied to diverse terrestrial ecosystems due to underlain extreme complexities. Deep learning algorithms can learn a spatial pattern and radiometric characteristics of tree canopy as a robust approximation of physical or empirical models, and thus have emerged as promising and efficient tools for large-scale TC mapping. In this study, we synergistically use very high-resolution aerial imageries (National Agriculture Imagery Program, NAIP) and medium resolution Landsat data to map continental-scale TC (CONUS and Mexico) through a hierarchical deep learning approach (Convolutional Neural Network), i.e., NAIP TC generated from a NAIP model is utilized to train a Landsat model. The produced TC product (hereafter, NEX-TC) is able to capture the spatial pattern of TC distribution and its changes driven by natural disturbance and human land management. We further explore and analyze the reliability and potential uncertainty of the NEX-TC by comparing it to lidar- (lidar-TC), National Land Cover Database (NLCD-TC), and MODIS Vegetation Continuous Field (MODIS-TC). This evaluation practice reveals that TC products based on passive optical sensors tend to underestimate TC across all land cover types while Landsat-based TCs (i.e., NEX-TC & NLCD-TC) perform better than the coarser MODIS TC estimate. Our results show that the NEX-TC is generally comparable to NLCD-TC but it particularly outperforms NLCD-TC and MODIS-TC over the dense forests where lidar-TC indicates >80% TC. These results indicate that our hierarchical deep learning approach and TC product will be effective and useful for characterizing large-scale tree cover and possibly associated carbon dynamics.

Landsat↗
DOE OSTI2022

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗