Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Technical Note: NASAaccess – A Tool for Access, Reformatting, and Visualization of Remotely Sensed Earth Observation and Climate Data

The National Aeronautics and Space Administration (NASA) has launched a new initiative, the Open-Source Science Initiative (OSSI), to enable and support science towards openness. The OSSI supports open-source software development and dissemination. In this work, we present NASAaccess, which is an open-source software package and web-based environmental modeling application for earth observation data accessing, reformatting, and presenting quantitative data products. The main objective of developing the NASAaccess platform is to facilitate exploration, modeling, and understanding of earth data for scientists, stakeholders, and concerned citizens whose objectives align with the new OSSI goals. The NASAaccess platform is available as software packages (i.e., the R and conda packages) as well as an interactive-format web-based environmental modeling application for earth observation data developed with Tethys Platform. NASAaccess has been envisioned as lowering the technical barriers and simplifying the process of accessing scalable distributed computing resources and leveraging additional software for data and computationally intensive modeling frameworks. Specifically, NASAaccess has been developed to meet the need for seamless earth observation remote-sensing and climate data ingestion into various hydrological modeling frameworks. Moreover, NASAaccess is also contributing to keeping interested parties and stakeholders engaged with environmental modeling, accessing the information available in various remote-sensing products. NASAaccess' current capabilities cover various NASA datasets and products that include the Global Precipitation Measurement (GPM) data products, the Global Land Data Assimilation System (GLDAS) land surface states and fluxes, and the NASA Earth Exchange Global Daily Downscaled Projections (NEX-GDDP) Coupled Model Intercomparison Project Phase 5 (CMIP5) and Coupled Model Intercomparison Project Phase 6 (CMIP6) climate change dataset products.

Ibrahim Nourein Mohammed↗
DOE OSTI2019

A year in the life of a parallel file system

I/O performance is a critical aspect of data-intensive scientific computing. We seek to advance the state of the practice in understanding and diagnosing I/O performance issues through investigation of a comprehensive I/O performance data set that captures a full year of production storage activity at two leadership-scale computing facilities. We demonstrate techniques to identify regions of interest, perform focused investigations of both long-term trends and transient anomalies, and uncover the contributing factors that lead to performance fluctuation. We find that a year in the life of a parallel file system is comprised of distinct regions of long-term performance variation in addition to short-term performance transients. We demonstrate how systematic identification of these performance regions, combined with comprehensive analysis, allows us to isolate the factors contributing to different performance maladies at different time scales. From this, we present specific lessons learned and important considerations for HPC storage practitioners.

Lockwood, GK↗
DOE OSTI2016

Enabling NVM for Data-Intensive Scientific Services

Specialized, transient data services are playing an increasingly prominent role in data-intensive scientific computing. These services offer flexible, on-demand pairing of applications with storage hardware using semantics that are optimized for the problem domain. Concurrent with this trend, upcoming scientific computing and big data systems will be deployed with emerging NVM technology to achieve the highest possible price/productivity ratio. Clearly, therefore, we must develop techniques to facilitate the confluence of specialized data services and NVM technology. In this work we explore how to enable the composition of NVM resources within transient distributed services while still retaining their essential performance characteristics. Our approach involves eschewing the conventional distributed file system model and instead projecting NVM devices as remote microservices that leverage user-level threads, RPC services, RMA-enabled network transports, and persistent memory libraries in order to maximize performance. We describe a prototype system that incorporates these concepts, evaluate its performance for key workloads on an exemplar system, and discuss how the system can be leveraged as a component of future data-intensive architectures.

Carns, Philip↗
DOE OSTI2020

A simple and fast algorithm for estimating the capacity credit of solar and storage

Energy storage is a leading option to enhance the resource adequacy contribution of solar energy. Detailed analysis of the capacity credit of solar energy and energy storage is limited in part due to the data intensive and computationally complex nature of probabilistic resource adequacy assessments. This paper presents a simple algorithm for calculating the capacity credit of energy-limited resources that, due to the low computational and data needs, is well suited to exploratory analysis. Validation against benchmarks based on probabilistic techniques shows that it can yield similar insights. The method is used to evaluate the impact of different solar and storage configurations, particularly with respect to the strategy for coupling storage and solar photovoltaic systems. Furthermore, application of the method to a case study of utilities in Florida, where solar is rapidly growing and demand peaks in the winter and summer, demonstrates that it can improve on rules of thumb used in practice by some utilities. If storage is required to charge only from solar, periods of high demand driven by cold weather events accompanied by lower solar production can result in a capacity credit of solar and storage that is less than the capacity credit of storage alone.

14 SOLAR ENERGY↗
NASA NTRSDate not supplied

Increasing accessibility to deep learning-based analytics for space biology: pretrained models, transfer learning, and analytics platform development

Biological systems react in complex ways to the stressors of spaceflight, and the data capturing these relationships is concomitantly high-dimensional and complex. Deep learning and machine learning approaches are increasingly popular as an analytical approach for space biosciences, due to their ability to model complex relationships in complex data. However, such approaches often require large datasets and extensive computational resources. New approaches that minimize data sizes and computational power needed to leverage machine learning, and resources that make these approaches accessible, are needed to increase accessibility and adoption of machine learning in the space biosciences. Transfer learning, in which a pretrained model of broad utility is trained on a large dataset, and subsequently reused on downstream applications for which data is more limited, is one approach to minimizing data and computational intensity of deep learning applications. This transfer learning approach results in more performant models in high-dimensional, low-sample-size settings such as space biology, as compared to training models on limited data from scratch. This presentation will outline efforts to generate pretrained models for the space biology community, and highlight transfer learning applications modeling microbial antibiotic resistance during spaceflight. Finally, in order to increase accessibility of these models and tools, as well as others, for the broader space biology community, we present a modeling and analysis platform facilitating machine learning applications in space biology. This platform streamlines machine learning training and analysis in a notebook format, facilitates download and use of space biology data from the NASA GeneLab database, and can be utilized on NASA-hosted servers or downloaded and hosted locally. This effort, as part of the AI4LS (Artificial Intelligence for Life in Space) working group, will increase accessibility, feasibility, and performance of machine learning approaches for the space biology community.

Adrienne Hoarfrost↗
DOE OSTI2026

Neural Posterior Estimation for Scalable and Accurate Inverse Parameter Inference in Li-Ion Batteries

Diagnosing the internal state of Li-ion batteries is critical for battery research, operation of real-world systems, and prognostic evaluation of remaining lifetime. By using physics-based models to perform probabilistic parameter estimation via Bayesian calibration, diagnostics can account for the uncertainty due to model fitness, data noise, and the observability of any given parameter. However, Bayesian calibration in Li-ion batteries using electrochemical data is computationally intensive even when using a fast surrogate in place of physics-based models, requiring many thousands of model evaluations. A fully amortized alternative is neural posterior estimation (NPE). NPE shifts the computational burden from the parameter estimation step to data generation and model training, reducing the parameter estimation time from minutes to milliseconds, enabling real-time applications. The present work shows that NPE can infer parameters equally or more accurately than Bayesian calibration, even if it leads to higher voltage reconstruction errors. We also demonstrate that the higher computational costs for data generation are tractable even in high-dimensional cases (ranging from 6 to 27 estimated parameters). The NPE method also offers several interpretability advantages over Bayesian calibration, such as local parameter sensitivity to specific regions of the voltage curve. The NPE method is demonstrated using an experimental fast charge dataset, with parameter estimates validated against measurements of loss of lithium inventory and loss of active material. The implementation is made available in a companion repository (https://github.com/NatLabRockies/BatFIT).

25 ENERGY STORAGE↗
DOE OSTI2018

Computing environments for reproducibility: Capturing the “Whole Tale”

The act of sharing scientific knowledge is rapidly evolving away from traditional articles and presentations to the delivery of executable objects that integrate the data and computational details (e.g., scripts and workflows) upon which the findings rely. This envisioned coupling of data and process is essential to advancing science but faces technical and institutional barriers. The Whole Tale project aims to address these barriers by connecting computational, data-intensive research efforts with the larger research process transforming the knowledge discovery and dissemination process into one where data products are united with research articles to create "living publications" or tales. The Whole Tale focuses on the full spectrum of science, empowering users in the long tail of science, and power users with demands for access to big data and compute resources. We report here on the design, architecture, and implementation of the Whole Tale environment.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2012

Partial Overhaul and Initial Parallel Optimization of KINETICS, a Coupled Dynamics and Chemistry Atmosphere Model

KINETICS is a coupled dynamics and chemistry atmosphere model that is data intensive and computationally demanding. The potential performance gain from using a supercomputer motivates the adaptation from a serial version to a parallelized one. Although the initial parallelization had been done, bottlenecks caused by an abundance of communication calls between processors led to an unfavorable drop in performance. Before starting on the parallel optimization process, a partial overhaul was required because a large emphasis was placed on streamlining the code for user convenience and revising the program to accommodate the new supercomputers at Caltech and JPL. After the first round of optimizations, the partial runtime was reduced by a factor of 23; however, performance gains are dependent on the size of the data, the number of processors requested, and the computer used.

KINETICS↗
DOE OSTI2014

Collective input/output under memory constraints

Compared with current high-performance computing (HPC) systems, exascale systems are expected to have much less memory per node, which can significantly reduce necessary collective input/output (I/O) performance. In this study, we introduce a memory-conscious collective I/O strategy that takes into account memory capacity and bandwidth constraints. The new strategy restricts aggregation data traffic within disjointed subgroups, coordinates I/O accesses in intranode and internode layers, and determines I/O aggregators at run time considering memory consumption among processes. We have prototyped the design and evaluated it with commonly used benchmarks to verify its potential. The evaluation results demonstrate that this strategy holds promise in mitigating the memory pressure, alleviating the contention for memory bandwidth, and improving the I/O performance for projected extreme-scale systems. Given the importance of supporting increasingly data-intensive workloads and projected memory constraints on increasingly larger scale HPC systems, this new memory-conscious collective I/O can have a significant positive impact on scientific discovery productivity.

collective input/output↗
DOE OSTI2015

Simplified predictive models for CO 2 sequestration performance assessment

CO2 sequestration in deep saline formations is increasingly being considered as a viable strategy for the mitigation of greenhouse gas emissions from anthropogenic sources. In this context, detailed numerical simulation based models are routinely used to understand key processes and parameters affecting pressure propagation and buoyant plume migration following CO2 injection into the subsurface. As these models are data and computation intensive, the development of computationally-efficient alternatives to conventional numerical simulators has become an active area of research. Such simplified models can be valuable assets during preliminary CO2 injection project screening, serve as a key element of probabilistic system assessment modeling tools, and assist regulators in quickly evaluating geological storage projects. We present three strategies for the development and validation of simplified modeling approaches for CO2 sequestration in deep saline formations: (1) simplified physics-based modeling, (2) statisticallearning based modeling, and (3) reduced-order method based modeling. In the first category, a set of full-physics compositional simulations is used to develop correlations for dimensionless injectivity as a function of the slope of the CO2 fractional-flow curve, variance of layer permeability values, and the nature of vertical permeability arrangement. The same variables, along with a modified gravity number, can be used to develop a correlation for the total storage efficiency within the CO2 plume footprint. Furthermore, the dimensionless average pressure buildup after the onset of boundary effects can be correlated to dimensionless time, CO2 plume footprint, and storativity contrast between the reservoir and caprock. In the second category, statistical “proxy models” are developed using the simulation domain described previously with two approaches: (a) classical Box-Behnken experimental design with a quadratic response surface, and (b) maximin Latin Hypercube sampling (LHS) based design with a multidimensional kriging metamodel fit. For roughly the same number of simulations, the LHS-based metamodel yields a more robust predictive model, as verified by a k-fold cross-validation approach (with data split into training and test sets) as well by validation with an independent dataset. In the third category, a reduced-order modeling procedure is utilized that combines proper orthogonal decomposition (POD) for reducing problem dimensionality with trajectory-piecewise linearization (TPWL) in order to represent system response at new control settings from a limited number of training runs. Significant savings in computational time are observed with reasonable accuracy from the PODTPWL reduced-order model for both vertical and horizontal well problems – which could be important in the context of history matching, uncertainty quantification and optimization problems. The simplified physics and statistical learning based models are also validated using an uncertainty analysis framework. Reference cumulative distribution functions of key model outcomes (i.e., plume radius and reservoir pressure buildup) generated using a 97-run full-physics simulation are successfully validated against the CDF from 10,000 sample probabilistic simulations using the simplified models. The main contribution of this research project is the development and validation of a portfolio of simplified modeling approaches that will enable rapid feasibility and risk assessment for CO2 sequestration in deep saline formations.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1985

MAX: A space station computer option

Information on Max, a space station computer option, is given in viewgraph form. The computer option is characterized by embedded, real-time applications; synchronous, cyclic operation and asynchronous, event driven operation; computationally intensive and data intensive processing; a wide range of throughput and memory requirements; a range of fault tolerant requirements from none to full; and maintainability, including capability for on-line substitution in critical systems.

Smith, D. B.↗
DOE OSTI2013

EINSTEIN@HOME DISCOVERY OF FOUR YOUNG GAMMA-RAY PULSARS IN FERMI LAT DATA

We report the discovery of four gamma-ray pulsars, detected in computing-intensive blind searches of data from the Fermi Large Area Telescope (LAT). The pulsars were found using a novel search approach, combining volunteer distributed computing via Einstein@Home and methods originally developed in gravitational-wave astronomy. The pulsars PSRs J0554+3107, J1422–6138, J1522–5735, and J1932+1916 are young and energetic, with characteristic ages between 35 and 56 kyr and spin-down powers in the range 6 × 10{sup 34}—10{sup 36} erg s{sup –1}. They are located in the Galactic plane and have rotation rates of less than 10 Hz, among which the 2.1 Hz spin frequency of PSR J0554+3107 is the slowest of any known gamma-ray pulsar. For two of the new pulsars, we find supernova remnants coincident on the sky and discuss the plausibility of such associations. Deep radio follow-up observations found no pulsations, suggesting that all four pulsars are radio-quiet as viewed from Earth. These discoveries, the first gamma-ray pulsars found by volunteer computing, motivate continued blind pulsar searches of the many other unidentified LAT gamma-ray sources.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Report of the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing: Harnessing Community, Software, and AI for Cross-Disciplinary Team Science

Scientific computing is undergoing rapid transformation as advances in artificial intelligence, heterogeneous computing, automation, and data-intensive research reshape not only computational tools but also the institutions, workforce models, and collaborative practices that support scientific discovery. This report synthesizes insights from the 2026 Workshop on Next-Generation Ecosystems for Scientific Computing, the second in a three-year series focused on strengthening scientific computing ecosystems through socio-technical co-design. Workshop discussions identified four interdependent strategic themes: software ecosystems for AI-enabled scientific discovery; trust, validation, and traceability; human-AI teaming and paradigm shifts; and workforce, pedagogy, and governance. The report translates these themes into eight priorities for community action spanning shared research infrastructure, trust and traceability, user experience, human-AI teaming, workforce development, cross-sector coordination, stewardship and sustainability, and evaluation of scientific value. Together, these priorities outline directions for building scientific computing ecosystems that remain trustworthy, sustainable, innovative, and resilient as AI assumes a growing role in scientific work.

AI↗
DOE OSTI2019

Active Subspace Methods for Data-Intensive Inverse Problems

Fostered by advances in computing, simulation and data science have emerged as the third and fourth paradigms enabling scientific discovery. Many mathematical challenges arise when integrating these paradigms to produce new knowledge. One of the most important challenges is the inverse or calibration problem: find the inputs of a simulation such that its outputs agree with a given set of measurements. As both the cost of the complex simulations and the volume of measurement data sets increase, intuition-driven trial-and-error methods for calibration quickly become untenable; modern scientists need more computationally efficient and mathematically rigorous methods. When measurements and simulated predictions contain uncertainty, the Bayesian perspective on statistical inference provides a popular mathematical framework for developing computational methods to solve inverse problems. Markov chain Monte Carlo (MCMC) methods are a mainstay of this framework; they characterize a probability distribution on the space of model inputs that is conditioned on the given measurements. However, despite recent advances, MCMC methods remain impractical for expensive simulations with more than a handful of inputs. At its heart, MCMC searches the space of inputs for corresponding outputs that match measurements, and so it is afflicted with the curse of dimensionality. Additionally, MCMC methods become less efficient as the volume of the measurement data increases, particularly if the measurements are redundant. To make MCMC—and Bayesian methods, in general—amenable to today’s computer models with millions of inputs and terabytes of measurement data, one must devise methods to reduce the dimension of both the simulation inputs and the measurements. For many models used in practice, the outputs vary primarily on a low-dimensional manifold of the nominally high-dimensional input space. Recently developed active subspace methods discover this low-dimensional manifold and exploit it to enable analyses typically suited for low-dimensional settings (e.g., statistical inversion of complex systems). We hypothesize that active subspace methods can provide the necessary dimension reduction, enabling existing Bayesian methods to find solutions for a large class of otherwise intractable statistical inverse problems. We will test the hypothesis with a series of mathematical analyses and computational experiments that will proceed along three research veins: (i) integrating active subspace methods into methods for statistical inversion, (ii) addressing current challenges in active subspace methods, and (iii) incorporating methods for nonlinear dimensionality reduction into active subspace methods. We will apply the methods we develop to real inverse problems in chemical kinetics and turbulent flame modeling. Our team combines expertise in methods for inverse problems, dimension reduction, and reduced order modeling. With a history of active collaboration and thoughtful engagement, we bring enthusiasm and fresh perspectives to a well studied subject. We are excited by the potential opportunity to work together to extend and apply active subspace methods to data-intensive inverse problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Accurate optimization of amino acid form factors for computing small-angle X-ray scattering intensity of atomistic protein structures

Structure modelling via small-angle X-ray scattering (SAXS) data generally requires intensive computations of scattering intensity from any given biomolecular structure, where the accurate evaluation of SAXS profiles using coarse-grained (CG) methods is vital to improve computational efficiency. To date, most CG SAXS computing methods have been based on a single-bead-per-residue approximation but have neglected structural correlations between amino acids. To improve the accuracy of scattering calculations, accurate CG form factors of amino acids are now derived using a rigorous optimization strategy, termed electron-density matching (EDM), to best fit electron-density distributions of protein structures. This EDM method is compared with and tested against other CG SAXS computing methods, and the resulting CG SAXS profiles from EDM agree better with all-atom theoretical SAXS data. By including the protein hydration shell represented by explicit CG water molecules and the correction of protein excluded volume, the developed CG form factors also reproduce the selected experimental SAXS profiles with very small deviations. Taken together, these EDM-derived CG form factors present an accurate and efficient computational approach for SAXS computing, especially when higher molecular details (represented by the q range of the SAXS data) become necessary for effective structure modelling.

Tong, Dudu↗
NASA NTRS2004

The Montage architecture for grid-enabled science processing of large, distributed datasets

Montage is an Earth Science Technology Office (ESTO) Computational Technologies (CT) Round III Grand Challenge investigation to deploy a portable, compute-intensive, custom astronomical image mosaicking service for the National Virtual Observatory (NVO). Although Montage is developing a compute- and data-intensive service for the astronomy community, we are also helping to address a problem that spans both Earth and Space science, namely how to efficiently access and process multi-terabyte, distributed datasets. In both communities, the datasets are massive, and are stored in distributed archives that are, in most cases, remote from the available Computational resources. Therefore, state of the art computational grid technologies are a key element of the Montage portal architecture. This paper describes the aspects of the Montage design that are applicable to both the Earth and Space science communities.

virtual observatory↗
DOE OSTI2014

A Large-Scale, High-Resolution Hydrological Model Parameter Data Set for Climate Change Impact Assessment for the Conterminous US

To extend geographical coverage, refine spatial resolution, and improve modeling efficiency, a computation- and data-intensive effort was conducted to organize a comprehensive hydrologic dataset with post-calibrated model parameters for hydro-climate impact assessment. Several key inputs for hydrologic simulation—including meteorologic forcings, soil, land class, vegetation, and elevation—were collected from multiple best-available data sources and organized for 2107 hydrologic subbasins (8-digit hydrologic units, HUC8s) in the conterminous United States at refined 1/24° (~4 km) spatial resolution. Using high-performance computing for intensive model calibration, a high-resolution parameter dataset was prepared for the macro-scale Variable Infiltration Capacity (VIC) hydrologic model. The VIC simulation was driven by DAYMET daily meteorological forcing and was calibrated against USGS WaterWatch monthly runoff observations for each HUC8. The results showed that this new parameter dataset may help reasonably simulate runoff at most US HUC8 subbasins. Based on this exhaustive calibration effort, it is now possible to accurately estimate the resources required for further model improvement across the entire conterminous United States. We anticipate that through this hydrologic parameter dataset, the repeated effort of fundamental data processing can be lessened, so that research efforts can emphasize the more challenging task of assessing climate change impacts. The pre-organized model parameter dataset will be provided to interested parties to support further hydro-climate impact assessment.

Oubeidillah, Abdoul A.↗