Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Scientific Computing.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,639 records · Page 91
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Trace: a high-throughput tomographic reconstruction engine for large-scale datasets

Abstract Background Modern synchrotron light sources and detectors produce data at such scale and complexity that large-scale computation is required to unleash their full power. One of the widely used imaging techniques that generates data at tens of gigabytes per second is computed tomography (CT). Although CT experiments result in rapid data generation, the analysis and reconstruction of the collected data may require hours or even days of computation time with a medium-sized workstation, which hinders the scientific progress that relies on the results of analysis. Methods We present Trace, a data-intensive computing engine that we have developed to enable high-performance implementation of iterative tomographic reconstruction algorithms for parallel computers. Trace provides fine-grained reconstruction of tomography datasets using both (thread-level) shared memory and (process-level) distributed memory parallelization. Trace utilizes a special data structure called replicated reconstruction object to maximize application performance. We also present the optimizations that we apply to the replicated reconstruction objects and evaluate them using tomography datasets collected at the Advanced Photon Source. Results Our experimental evaluations show that our optimizations and parallelization techniques can provide 158× speedup using 32 compute nodes (384 cores) over a single-core configuration and decrease the end-to-end processing time of a large sinogram (with 4501 × 1 × 22,400 dimensions) from 12.5 h to <5 min per iteration. Conclusion The proposed tomographic reconstruction engine can efficiently process large-scale tomographic data using many compute nodes and minimize reconstruction times.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1994

Six Years of Parallel Computing at NAS (1987 - 1993): What Have we Learned?

In the fall of 1987 the age of parallelism at NAS began with the installation of a 32K processor CM-2 from Thinking Machines. In 1987 this was described as an "experiment" in parallel processing. In the six years since, NAS acquired a series of parallel machines, and conducted an active research and development effort focused on the use of highly parallel machines for applications in the computational aerosciences. In this time period parallel processing for scientific applications evolved from a fringe research topic into the one of main activities at NAS. In this presentation I will review the history of parallel computing at NAS in the context of the major progress, which has been made in the field in general. I will attempt to summarize the lessons we have learned so far, and the contributions NAS has made to the state of the art. Based on these insights I will comment on the current state of parallel computing (including the HPCC effort) and try to predict some trends for the next six years.

Simon, Horst D.↗
DOE OSTI2019

Measuring the impact of burst buffers on data-intensive scientific workflows

Science applications frequently produce and consume large volumes of data, but delivering this data to and from compute resources can be concerning, as parallel file system performance is not keeping up with compute and memory performance. To alter this I/O bottleneck, some systems have deployed burst buffers, but their impact on performance for real-world scientific workflow applications is still not clear. In this paper, we examine the impact of burst buffers through the remote-shared, allocatable burst buffers on the Cori system at NERSC. By running two data-intensive workflows, a high-throughput genome analysis workflow, and a subset of the SCEC high-performance CyberShake workflow, a production seismic hazard analysis workflow, we find that using burst buffers offers read and write improvements of an order of magnitude, and these improvements lead to increased job performance, and thereby increased overall workflow performance, even for long-running CPU-bound jobs.

42 ENGINEERING↗
NASA NTRS1984

The 3SPACE digitizer: A new input device for solid geometry modeling

The 3SPACE Digitizer is a derivation of SPASYN, which is a patented magnetic transducing system offering an accurate method of calculating the location of one object relative to another. SPASYN uses low-frequency magnetic fields to measure both the position and orientation of its sensor relative to its transmitter. The 3SPACE Digitizer unit consists of a system's electronics unit, a model table, a stylus, and a keypad. In operation, the 3SPACE Digitizer determines the X, Y, Z coordinates of any point located on a low-conductive, three-dimensional model. This data, along with the orientation of the Digitizer's stylus, is immediately available for transmission to a host computer or graphics terminal. Digitization is effected by touching the point to be measured with the tip of a handheld stylus. The 3SPACE Digitizer presents new opportunities for interacting with computers in a large number of engineering and scientific applications. Developed from proven military technology, the 3SPACE Digitizer significantly simplifies three-dimensional digitization and overcomes many of the problems that once plagued such procedures. Its ability to digitize solid objects makes it a valuable new tool for computer-aided design and manufacturing.

Scully, J.↗
DOE OSTI2016

Large-scale optimization-based non-negative computational framework for diffusion equations: Parallel implementation and performance studies

It is well-known that the standard Galerkin formulation, which is often the formulation of choice under the finite element method for solving self-adjoint diffusion equations, does not meet maximum principles and the non-negative constraint for anisotropic diffusion equations. Recently, optimization-based methodologies that satisfy maximum principles and the non-negative constraint for steady-state and transient diffusion-type equations have been proposed. To date, these methodologies have been tested only on small-scale academic problems. The purpose of this paper is to systematically study the performance of the non-negative methodology in the context of high performance computing (HPC). PETSc and TAO libraries are, respectively, used for the parallel environment and optimization solvers. For large-scale problems, it is important for computational scientists to understand the computational performance of current algorithms available in these scientific libraries. The numerical experiments are conducted on the state-of-the-art HPC systems, and a single-core performance model is used to better characterize the efficiency of the solvers. Furthermore, our studies indicate that the proposed non-negative computational framework for diffusion-type equations exhibits excellent strong scaling for real-world large-scale problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

Lazy Checkpointing: Exploiting Temporal Locality in Failures to Mitigate Checkpointing Overheads on Extreme-Scale Systems, In: 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks

Continuing increase in the computational power of supercomputers has enabled large-scale scientific applications in the areas of astrophysics, fusion, climate and combustion to run larger and longer-running simulations, facilitating deeper scientific insights. However, these long-running simulations are often interrupted by multiple system failures. Therefore, these applications rely on "check pointing'" as a resilience mechanism to store application state to permanent storage and recover from failures. Unfortunately, check pointing incurs excessive I/O overhead on supercomputers due to large size of checkpoints, resulting in a sub-optimal performance and resource utilization. In this paper, we devise novel mechanisms to show how check pointing overhead can be mitigated significantly by exploiting the temporal characteristics of system failures. We provide new insights and detailed quantitative understanding of the check pointing overheads and trade-offs on large-scale machines. Our prototype implementation shows the viability of our approach on extreme-scale machines.

Computer Science↗
DOE OSTI2026

Visualization for Insight and Data Analysis in Energy Research

This talk explores how advanced visualization technologies are transforming analytical reasoning and knowledge discovery in energy research, drawing on recent work at the National Laboratory of the Rockies' Computational Science Center. Through a series of scientific case studies, we demonstrate how immersive and high-resolution visualization environments enable scientists and engineers to identify previously unseen patterns and features - insights that often remain hidden in traditional desktop-based analysis. By embedding richer information into interactive analytics tools, these approaches support the exploration of complex, multivariate parameter spaces, where interaction itself catalyzes understanding. Beyond capability, we emphasize the critical role of visualization design grounded in perception and cognition, showing how visual encodings directly influence analytical outcomes. Spanning applications from materials science to integrated energy systems, these visualization approaches accelerate innovation and improve decision-making by enabling deeper, more reliable insight into increasingly complex energy data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

A feasibility study on porting the community land model onto accelerators using OpenACC

As environmental models (such as Accelerated Climate Model for Energy (ACME), Parallel Reactive Flow and Transport Model (PFLOTRAN), Arctic Terrestrial Simulator (ATS), etc.) became more and more complicated, we are facing enormous challenges regarding to porting those applications onto hybrid computing architecture. OpenACC appears as a very promising technology, therefore, we have conducted a feasibility analysis on porting the Community Land Model (CLM), a terrestrial ecosystem model within the Community Earth System Models (CESM)). Specifically, we used automatic function testing platform to extract a small computing kernel out of CLM, then we apply this kernel into the actually CLM dataflow procedure, and investigate the strategy of data parallelization and the benefit of data movement provided by current implementation of OpenACC. Even it is a non-intensive kernel, on a single 16-core computing node, the performance (based on the actual computation time using one GPU) of OpenACC implementation is 2.3 time faster than that of OpenMP implementation using single OpenMP thread, but it is 2.8 times slower than the performance of OpenMP implementation using 16 threads. On multiple nodes, MPI_OpenACC implementation demonstrated very good scalability on up to 128 GPUs on 128 computing nodes. This study also provides useful information for us to look into the potential benefits of “deep copy” capability and “routine” feature of OpenACC standards. In conclusion, we believe that our experience on the environmental model, CLM, can be beneficial to many other scientific research programs who are interested to porting their large scale scientific code using OpenACC onto high-end computers, empowered by hybrid computing architecture.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Regression with Small Data Sets: A Case Study using Code Surrogates in Additive Manufacturing

There has been an increasing interest in recent years in the mining of massive data sets whose sizes are measured in terabytes. While it is easy to collect such large data sets in some application domains, there are others where collecting even a single data point can be very expensive, so the resulting data sets have only tens or hundreds of samples. For example, when complex computer simulations are used to understand a scientific phenomenon, we want to run the simulation for many different values of the input parameters and analyze the resulting output. The data set relating the simulation inputs and outputs is typically quite small, especially when each run of the simulation is expensive. However, regression techniques can still be used on such data sets to build an inexpensive \surrogate" that could provide an approximate output for a given set of inputs. A good surrogate can be very useful in sensitivity analysis, uncertainty analysis, and in designing experiments. In this paper, we compare different regression techniques to determine how well they predict melt-pool characteristics in the problem domain of additive manufacturing. Our analysis indicates that some of the commonly used regression methods do perform quite well even on small data sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Data And Informatics Working Group On Virtual Data Integration Workshop Report

This report is the outcome of a workshop that was commissioned by the Department of Energy’s Climate and Environmental Sciences Division (CESD) to examine current and future data infrastructure requirements that would be foundational to achieving CESD’s scientific mission goals. Over the past several years, data volumes in CESD disciplines have risen sharply to unprecedented levels (tens of petabytes). So too has the complexity and diversity of the research data (simulation, observation, and reanalysis) needing to be captured, stored, verified, analyzed, and integrated. With the trends of increased data volume (in the hundreds of petabytes), more complex analysis processes, and growing crossdisciplinary collaborations, it is timely to investigate whether the CESD community has the right computational and data support to realize the full scientific potential from its data collections. In recognition of the challenges, a partnership is forming across CESD and with national and international agencies to investigate the viability of creating an integrated, collaborative data infrastructure: a virtual laboratory. The overarching goal of this report is to identify the community’s key data technology requirements and high-priority development needs for sustaining and growing their scientific discovery potential. The report also aims to map these requirements to existing solutions and to identify gaps in current services, tools, and infrastructure that will need to be addressed in the short, medium, and long term so as not to impede scientific progress

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2023

Ocean Carbon From Space: Current Status and Priorities for the Next Decade

The ocean plays a central role in modulating the Earth’s carbon cycle. Monitoring how the ocean carbon cycle is changing is fundamental to managing climate change. Satellite remote sensing is currently our best tool for viewing the ocean surface globally and systematically, at high spatial and temporal resolutions, and the past few decades have seen an exponential growth in studies utilising satellite data for ocean carbon research. Satellite-based observations must be combined with in-situ observations and models, to obtain a comprehensive view of ocean carbon pools and fluxes. To help prioritise future research in this area, a workshop was organised that assembled leading experts working on the topic, from around the world, including remote-sensing scientists, field scientists and modellers, with the goal to articulate a collective view of the current status of ocean carbon research, identify gaps in knowledge, and formulate a scientific roadmap for the next decade, with an emphasis on evaluating where satellite remote sensing may contribute. A total of 449 scientists and stakeholders participated (with balanced gender representation), from North and South America, Europe, Asia, Africa, and Oceania. Sessions targeted both inorganic and organic pools of carbon in the ocean, in both dissolved and particulate form, as well as major fluxes of carbon between reservoirs (e.g., primary production) and at interfaces (e.g., air-sea and land–ocean). Extreme events, blue carbon and carbon budgeting were also key topics discussed. Emerging priorities identified include: expanding the networks and quality of in-situ observations; improved satellite retrievals; improved uncertainty quantification; improved understanding of vertical distributions; integration with models; improved techniques to bridge spatial and temporal scales of the different data sources; and improved fundamental understanding of the ocean carbon cycle, and of the interactions among pools of carbon and light. We also report on priorities for the specific pools and fluxes studied, and highlight issues and concerns that arose during discussions, such as the need to consider the environmental impact of satellites or space activities; the role satellites can play in monitoring ocean carbon dioxide removal approaches; economic valuation of the satellite based information; to consider how satellites can contribute to monitoring cycles of other important climatically-relevant compounds and elements; to promote diversity and inclusivity in ocean carbon research; to bring together communities working on different aspects of planetary carbon; maximising use of international bodies; to follow an open science approach; to explore new and innovative ways to remotely monitor ocean carbon; and to harness quantum computing. Overall, this paper provides a comprehensive scientific roadmap for the next decade on how satellite remote sensing could help monitor the ocean carbon cycle, and its links to the other domains, such as terrestrial and atmosphere.

oceanography↗
DOE OSTI2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Highlights of X-Stack ExM Deliverable Swift/T

Swift/T is a key success from the ExM: System support for extreme-scale, many-task applications1 X-Stack project, which proposed to use concurrent dataflow as an innovative programming model to exploit extreme parallelism in exascale computers. The Swift/T component of the project reimplemented the Swift language from scratch to allow applications that compose scientific modules together to be build and run on available petascale computers (Blue Gene, Cray). Swift/T does this via a new compiler and runtime that generates and executes the application as an MPI program. We assume that mission-critical emerging exascale applications will be composed as scalable applications using existing software components, connected by data dependencies. Developers wrap native code fragments using a higherlevel language, then build composite applications to form a computational experiment. This exemplifies hierarchical concurrency: lower-level messaging libraries are used for fine-grained parallelism; highlevel control is used for inter-task coordination. These patterns are best expressed with dataflow, but static DAGs (i.e., other workflow languages) limit the applications that can be built; they do not provide the expressiveness of Swift, such as conditional execution, iteration, and recursive functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Jörg Langowski: his scientific legacy and the future it promises

Background: With the passing of Jörg Langowski 6 May 2017 in a sailplane accident, the scientific community was deprived of a strident and effective voice for DNA and chromatin molecular and computational biophysics, for open access publishing and for the creation of effective scientific research networks. Methods: In this work, after reviewing some of Jörg’s key research contributions and ideas, we offer through the personal remembrance of his closest collaborators, a deep analysis of the major results of his research and the future directions they have engendered. Conclusions: The legacy of Jörg Langowski has been to propel a way of viewing biological function that considers living systems as dynamic and in three dimensions. This physical view of biology that he pioneered is now, finally, becoming established also because of his great effort.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2019

High-performance End-to-End Integrity Verification on Big Data Transfer

The scale of scientific data generated by experimental facilities and simulations in high-performance computing facilities has been proliferating with the emergence of IoT-based big data. In many cases, this data must be transmitted rapidly and reliably to remote facilities for storage, analysis, or sharing, for the Internet of Things (IoT) applications. Simultaneously, IoT data can be verified using a checksum after the data has been written to the disk at the destination to ensure its integrity. However, this end-to-end integrity verification inevitably creates overheads (extra disk I/O and more computation). Thus, the overall data transfer time increases. In this article, we evaluate strategies to maximize the overlap between data transfer and checksum computation for astronomical observation data. Specifically, we examine file-level and block-level (with various block sizes) pipelining to overlap data transfer and checksum computation. We analyze these pipelining approaches in the context of GridFTP, a widely used protocol for scientific data transfers. Theoretical analysis and experiments are conducted to evaluate our methods. The results show that block-level pipelining is effective in maximizing the overlap mentioned above, and can improve the overall data transfer time with end-to-end integrity verification by up to 70% compared to the sequential execution of transfer and checksum, and by up to 60% compared to file-level pipelining.

data integrity↗
NASA NTRSDate not supplied

Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform (MAAP)

The scientific community is faced with a need for greatly improved data sharing, analysis, visualization and advanced collaboration based firmly on open science principles. Recent and upcoming launches of new satellite missions with more complex and voluminous data, as well as the ever more urgent need to better understand the global carbon budget and related ecological processes, provided the immediate rational for the ESA-NASA Multi-mission Algorithm and Analysis Platform (MAAP). This highly collaborative joint project of ESA and NASA established a framework between ESA and NASA to share data, science algorithms and compute resources in order to foster and accelerate scientific research conducted by ESA and NASA EO data users. Presented to the public in October 2021, the current version of MAAP provides a common cloud-based platform with computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of global above-ground biomass. Data from the Global Ecosystem Dynamics Investigation (GEDI) mission on the International Space Station and the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) have been instrumental in the first products of MAAP including the first comprehensive map of Boreal above-ground Biomass and a current Global Biomass Harmonization Activity, but the platform is also being specifically designed to support the forthcoming ESA Biomass mission and incorporate data from the upcoming NASA-ISRO SAR (NISAR) mission. While these missions and the corresponding research which includes airborne, field, and calibration/validation data collection and analyses, provide a wealth of data and information relating to global biomass estimation, they also present data storing, processing and sharing challenges. The NISAR mission alone will produce about 80TB/day. These large data volumes present a challenge that would otherwise place accessibility limits on the scientific community and impact scientific progress. Other challenges being addressed by MAAP include: 1) Enabling researchers to easily discover, process, visualize and analyze large volumes of data from both agencies; 2) Providing a wide variety of data in the same coordinate reference frame to enable comparison, analysis, data evaluation, and data generation; 3) Providing a version-controlled science algorithm development environment that supports tools, co-located data and processing resources; and 4) Addressing intellectual property and sharing challenges related to collaborative algorithm development and sharing of data and algorithms. MAAP products can be explored on the MAAP Dashboard at https://earthdata.nasa.gov/maap-biomass or the joint platform entrance at scimaap.net. MAAP also can be accessed through individual NASA (https://maap-project.org) and ESA (https://esa-maap.org/) landing pages.

cloud computing↗
DOE OSTI2015

Load‐balanced and locality‐aware scheduling for data‐intensive workloads at extreme scales

Summary Data‐driven programming models such as many‐task computing (MTC) have been prevalent for running data‐intensive scientific applications. MTC applies over‐decomposition to enable distributed scheduling. To achieve extreme scalability, MTC proposes a fully distributed task scheduling architecture that employs as many schedulers as the compute nodes to make scheduling decisions. Achieving distributed load balancing and best exploiting data locality are two important goals for the best performance of distributed scheduling of data‐intensive applications. Our previous research proposed a data‐aware work‐stealing technique to optimize both load balancing and data locality by using both dedicated and shared task ready queues in each scheduler. Tasks were organized in queues based on the input data size and location. Distributed key‐value store was applied to manage task metadata. We implemented the technique in MATRIX, a distributed MTC task execution framework. In this work, we devise an analytical suboptimal upper bound of the proposed technique, compare MATRIX with other scheduling systems, and explore the scalability of the technique at extreme scales. Results show that the technique is not only scalable but can achieve performance within 15% of the suboptimal solution. Copyright © 2015 John Wiley & Sons, Ltd.

Wang, Ke↗
DOE OSTI2023

Experiences Readying Applications for Exascale

The advent of Exascale computing invites an assessment of existing best practices for developing application readiness on the world's largest supercomputers. This work details observations from the last four years in preparing scientific applications to run on the Oak Ridge Leadership Computing Facility's (OLCF) Frontier system. This paper addresses a range of topics in software including programmability, tuning, and portability considerations that are key to moving applications from existing systems to future installations. A set of representative workloads provides case studies for general system and software testing. We evaluate the use of early access systems for development across several generations of hardware. Finally, we discuss how best practices were identified and disseminated to the community through a wide range of activities including user-guides and trainings. We conclude with recommendations for ensuring application readiness on future leadership computing systems.

exascale↗