Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Performance analysis of data intensive cloud systems based on data management and replication: a survey

As we delve deeper into the ‘Digital Age’, we witness an explosive growth in the volume, velocity, and variety of the data available on the Internet. For example, in 2012 about 2.5 quintillion bytes of data was created on a daily basis that originated from myriad of sources and applications including mobiledevices, sensors, individual archives, social networks, Internet of Things, enterprises, cameras, software logs, etc. Such ‘Data Explosions’ has led to one of the most challenging research issues of the current Information and Communication Technology era: how to optimally manage (e.g., store, replicated, filter, and the like) such large amount of data and identify new ways to analyze large amounts of data for unlocking information. It is clear that such large data streams cannot be managed by setting up on-premises enterprise database systems as it leads to a large up-front cost in buying and administering the hardware and software systems. Therefore, next generation data management systems must be deployed on cloud. The cloud computing paradigm provides scalable and elastic resources, such as data and services accessible over the Internet Every Cloud Service Provider must assure that data is efficiently processed and distributed in a way that does not compromise end-users’ Quality of Service (QoS) in terms of data availability, data search delay, data analysis delay, and the like. In the aforementioned perspective, data replication is used in the cloud for improving the performance (e.g., read and write delay) of applications that access data. Through replication a data intensive application or system can achieve high availability, better fault tolerance, and data recovery. In this paper, we survey data management and replication approaches (from 2007 to 2011) that are developed by both industrial and research communities. The focus of the survey is to discuss and characterize the existing approaches of data replication and management that tackle the resource usage and QoS provisioning with different levels of efficiencies. Moreover, the breakdown of both influential expressions (data replication and management) to provide different QoS attributes is deliberated. Furthermore, the performance advantages and disadvantages of data replication and management approaches in the cloud computing environments are analyzed. Open issues and future challenges related to data consistency, scalability, load balancing, processing and placement are also reported.

Malik, Saif Ur Rehman↗
DOE OSTI2024

Generic and ML Workloads in an HPC Datacenter: Node Energy, Job Failures, and Node-Job Analysis

HPC datacenters offer a backbone to the modern digital society. Increasingly, they run Machine Learning (ML) jobs next to generic, compute-intensive workloads, supporting science, business, and other decision-making processes. However, understanding how ML jobs impact the operation of HPC datacenters, relative to generic jobs, remains desirable but understudied. In this work, we leverage long-term operational data, collected from a national-scale production HPC datacenter, and statistically compare how ML and generic jobs can impact the performance, failures, resource utilization, and energy consumption of HPC datacenters. Our study provides key insights, e.g., ML-related power usage causes GPU nodes to run into temperature limitations, median/mean runtime and failure rates are higher for ML jobs than for generic jobs, both ML and generic jobs exhibit highly variable arrival processes and resource demands, significant amounts of energy are spent on unsuccessfully terminating jobs, and concurrent jobs tend to terminate in the same state. We open-source our cleaned-up data traces on Zenodo (https://doi. org/10.5281/zenodo.13685426), and provide our analysis toolkit as software hosted on GitHub (https://github.com/atlarge-research/2024-icpads-hpc-workload-characterization). This study offers multiple benefits for data center administrators, who can improve operational efficiency, and for researchers, who can further improve system designs, scheduling techniques, etc.

crossanalysis↗
DOE OSTI2024

Transformational Regional-Scale Earthquake Simulations with the DOE EarthQuake SIMulation Exascale Framework

Earthquakes present worldwide risk to economic and human safety. The 2023 earthquakes in Turkiye provided a reminder of the potential for catastrophic consequences with 50,700 deaths and 15.7 million people affected. The ability to predict ground motions and infrastructure damage for earthquakes continues to be a challenging problem for scientists and engineers. Until now, estimates of ground motions have been performed empirically by looking at sparse data from past earthquakes. This approach can provide statistical information on intensity amplitudes but cannot inform site-specific ground motions essential to developing the most effective resilience. Interest has grown in large-scale computational models to simulate earthquakes at regional scale. The U.S. Department of Energy EarthQuake SIMulation (EQSIM) framework was developed for regional-scale earthquake simulations at unprecedented fidelity, taking advantage of emerging GPU-accelerated systems. This article describes the EQSIM workflow and demonstrates regional-scale simulations with the new computational capability available to scientists in their quest to mitigate future disasters.

58 GEOSCIENCES↗
DOE OSTI2011

Velo: Riding the Knowledge Management Wave for Simulation and Modeling

Modern scientific enterprises are inherently knowledge-intensive. In general, scientific studies in domains such as geosciences, climate, and biology require the acquisition and manipulation of large amounts of experimental and field data in order to create inputs for large-scale computational simulations. The results of these simulations must then be analyzed, leading to refinements of inputs and models and further simulations. Further, these results must be managed and archived to provide justifications for publications and regulatory decisions that are based on these models. In this paper we describe our Velo framework that is designed as a reusable, domain independent knowledge management infrastructure for modeling and simulation. Velo leverages, integrates, and extends open source collaborative and content management technologies to create a scalable and flexible core platform that can be tailored to specific scientific domains. In this paper we describe the architecture of Velo for managing and associating the various types of data that are used and created in modeling and simulation projects, as well as the framework for integrating domain-specific tools. To demonstrate a realization of Velo, we describe the Geologic Sequestration Software Suite (GS3) that has been developed to support geologic sequestration modeling. This provides a concrete example of the inherent extensibility and utility of our approach.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2014

Developing a Massively Parallel Forward Projection Radiography Model for Large-Scale Industrial Applications

This project utilizes Graphics Processing Units (GPUs) to compute radiograph simulations for arbitrary objects. The generation of radiographs, also known as the forward projection imaging model, is computationally intensive and not widely utilized. The goal of this research is to develop a massively parallel algorithm that can compute forward projections for objects with a trillion voxels (3D pixels). To achieve this end, the data are divided into blocks that can each t into GPU memory. The forward projected image is also divided into segments to allow for future parallelization and to avoid needless computations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1983

Information theory analysis of sensor-array imaging systems for computer vision

Information theory is used to assess the performance of sensor-array imaging systems, with emphasis on the performance obtained with image-plane signal processing. By electronically controlling the spatial response of the imaging system, as suggested by the mechanism of human vision, it is possible to trade-off edge enhancement for sensitivity, increase dynamic range, and reduce data transmission. Computational results show that: signal information density varies little with large variations in the statistical properties of random radiance fields; most information (generally about 85 to 95 percent) is contained in the signal intensity transitions rather than levels; and performance is optimized when the OTF of the imaging system is nearly limited to the sampling passband to minimize aliasing at the cost of blurring, and the SNR is very high to permit the retrieval of small spatial detail from the extensively blurred signal. Shading the lens aperture transmittance to increase depth of field and using a regular hexagonal sensor-array instead of square lattice to decrease sensitivity to edge orientation also improves the signal information density up to about 30 percent at high SNRs.

Huck, F. O.↗
NASA NTRS1987

Digital image classification approach for estimating forest clearing and regrowth rates and trends

A technique is presented to monitor vegetation changes for a selected study area in Costa Rica. A normalized difference vegetation index was computed for three dates of Landsat satellite data and a modified parallelipiped classifier was employed to generate a multitemporal greenness image representing all three dates. A second-generation image was created by partitioning the intensity levels at each date into high, medium, and low and thereby reducing the number of classes to 21. A sampling technique was applied to describe forest and other land cover change occurring between time periods based on interpretation of aerial photography that closely matched the dates of satellite acquisition. Comparison of the Landsat-derived classes with the photo-interpreted sample areas can provide a basis for evaluating the satellite monitoring technique and the accuracy of estimating forest clearing and regrowth rates and trends.

Sader, Steven A.↗
NASA NTRS1990

Multicolor pyrometer for materials processing in space

This report documents the work performed by Physical Sciences Inc. (PSI), under contract to NASA JPL, during a 2.5-year SBIR Phase 2 Program. The program goals were to design, construct, and program a prototype passive imaging pyrometer capable of measuring, as accurately as possible, and controlling the temperature distribution across the surface of a moving object suspended in space. These goals were achieved and the instrument was delivered to JPL in November 1989. The pyrometer utilizes an optical system which operates at short wavelengths compared to the peak of the black-body spectrum for the temperature range of interest, thus minimizing errors associated with a lack of knowledge about the heated sample's emissivity. To cover temperatures from 900 to 2500 K, six wavelengths are available. The preferred wavelength for measurement of a particular temperature decreases as the temperature increases. Images at all six wavelengths are projected onto a single CCD camera concurrently. The camera and optical system have been calibrated to relate the measured intensity at each pixel to the temperature of the heated object. The output of the camera is digitized by a frame grabber installed in a personal computer and analyzed automatically to yield temperature information. The data can be used in a feedback loop to alter the status of computer-activated switches and thereby control a heating system.

Frish, M. B.↗
NASA NTRS2011

Current Status on Radiation Modeling for the Hayabusa Re-entry

On June 13, 2010 the Japanese Hayabusa capsule performed its reentry into the Earths atmosphere over Australia after a seven year journey to the asteroid Itokawa. The reentry was studied by numerous imaging and spectroscopic instruments onboard NASA's DC-8 Airborne Laboratory and from three sites on the ground, in order to measure surface and plasma radiation generated by the Hayabusa Sample Return Capsule (SRC). Post flight, the flow solutions were recomputed to include the whole flow field around the capsule at 11 points along the reentry trajectory using updated trajectory information. Again, material response was taken into account to obtain most reliable surface temperature information. These data will be used to compute thermal radiation of the glowing heat shield and plasma radiation by the shock/post-shock layer system to support analysis of the experimental observation data. For this purpose, lines of sight data are being extracted from the flow field volume grids and plasma radiation will be computed using NEQAIR [4] which is a line-by-line spectroscopic code with one-dimensional transport of radiation intensity. The procedures being used were already successfully applied to the analysis of the observation of the Stardust reentry [5].

Winter, Michael W.↗
DOE OSTI2019

Autonomous Radiological Survey Systems for Locating Fired Depleted Uranium Munitions - 19212

An ensemble of autonomous radiological survey platforms that utilize a suite of scintillation detectors that can be used to conduct radiological site characterization, assist with remediation efforts, or conduct site clearance in accordance with the Multi-Agency Radiologic Survey and Site Investigation Manual are described. The overall systems are composed of inorganic scintillators, light detection and ranging, global positioning, inertial guidance, wireless telemetry, and signal processing systems. Two primary designs are discussed: a three-meter wide platform and multiple half-meter-wide systems. The survey systems are used to conduct radiological surveys over large areas autonomously, without human involvement. The three-meter system has been designed to cover large, open areas free of obstacles. Smaller robotic platforms are designed to handle a large range of terrain types and to conduct surveys in areas that are plagued with obstacles, such as bushes and trees. The primary design criteria for these systems were accurate locating of fired depleted uranium (DU) munitions. Several U.S. Army facilities have remnant DU material on active firing ranges where DU rounds have been used in performance evaluations. Many ranges also have unexploded ordinance (UXO) challenges which are problematic for conducting walkover surveys. The combination of having active DU impacted firing ranges with UXO make conducting site characterization surveys very difficult and hazardous. The autonomous surveying systems allow radiological scanning data collection to be scheduled around range activities and reduces human exposure to ionizing radiation and UXO. All of the robotic platforms act independently, and are controlled by a centralized computer system via a wireless telemetry system. The central computer system is responsible for setting up survey units, survey paths, and scheduling which platform is to complete the scheduled survey. Unit selection and scheduling are completed by technicians, and the work is conducted autonomously by these survey platforms. The survey areas and survey paths are transmitted to each platform through the telemetry system. Radiological and positional data are collected by the systems and transmitted back to the central computer system in real-time. Collected data is saved to a database for post-survey analysis or can be analyzed in real-time. Analysis of the radiological data is used to construct radiation intensity and estimated DU concentration maps. Automatically generated reports for each survey unit will provide range managers with: survey coverage data maps, estimated uranium concentration maps, and a detailed list of areas that are above a user defined concentration. This paper describes the components of each of the surveying systems, their capabilities, and limitations. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS1972

UK-5 Van Allen belt radiation exposure: A special study to determine the trapped particle intensities on the UK-5 satellite with spatial mapping of the ambient flux environment

Vehicle encountered electron and proton fluxes were calculated for a set of nominal UK-5 trajectories with new computational methods and new electron environment models. Temporal variations in the electron data were considered and partially accounted for. Field strength calculations were performed with an extrapolated model on the basis of linear secular variation predictions. Tabular maps for selected electron and proton energies were constructed as functions of latitude and longitude for specified altitudes. Orbital flux integration results are presented in graphical and tabular form; they are analyzed, explained, and discussed.

Stassinopoulos, E. G.↗
DOE OSTI2017

Experience in using commercial clouds in CMS

Historically high energy physics computing has been performed on large purpose-built computing systems. In the beginning there were single site computing facilities, which evolved into the Worldwide LHC Computing Grid (WLCG) used today. The vast majority of the WLCG resources are used for LHC computing and the resources are scheduled to be continuously used throughout the year. In the last several years there has been an explosion in capacity and capability of commercial and academic computing clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest amongst the cloud providers to demonstrate the capability to perform large scale scientific computing. In this presentation we will discuss results from the CMS experiment using the Fermilab HEPCloud Facility, which utilized both local Fermilab resources and Amazon Web Services (AWS). The goal was to work with AWS through a matching grant to demonstrate a sustained scale approximately equal to half of the worldwide processing resources available to CMS. We will discuss the planning and technical challenges involved in organizing the most IO intensive CMS workflows on a large-scale set of virtualized resource provisioned by the Fermilab HEPCloud. We will describe the data handling and data management challenges. Also, we will discuss the economic issues and cost and operational efficiency comparison to our dedicated resources. At the end we will consider the changes in the working model of HEP computing in a domain with the availability of large scale resources scheduled at peak times.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2015

Predicting X-ray diffuse scattering from translation–libration–screw structural ensembles

A method of simulating X-ray diffuse scattering from multi-model PDB files is presented. Despite similar agreement with Bragg data, different translation–libration–screw refinement strategies produce unique diffuse intensity patterns. Identifying the intramolecular motions of proteins and nucleic acids is a major challenge in macromolecular X-ray crystallography. Because Bragg diffraction describes the average positional distribution of crystalline atoms with imperfect precision, the resulting electron density can be compatible with multiple models of motion. Diffuse X-ray scattering can reduce this degeneracy by reporting on correlated atomic displacements. Although recent technological advances are increasing the potential to accurately measure diffuse scattering, computational modeling and validation tools are still needed to quantify the agreement between experimental data and different parameterizations of crystalline disorder. A new tool, phenix.diffuse, addresses this need by employing Guinier’s equation to calculate diffuse scattering from Protein Data Bank (PDB)-formatted structural ensembles. As an example case, phenix.diffuse is applied to translation–libration–screw (TLS) refinement, which models rigid-body displacement for segments of the macromolecule. To enable the calculation of diffuse scattering from TLS-refined structures, phenix.tls-as-xyz builds multi-model PDB files that sample the underlying T, L and S tensors. In the glycerophosphodiesterase GpdQ, alternative TLS-group partitioning and different motional correlations between groups yield markedly dissimilar diffuse scattering maps with distinct implications for molecular mechanism and allostery. These methods demonstrate how, in principle, X-ray diffuse scattering could extend macromolecular structural refinement, validation and analysis.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Integrating Ultra-Coarse-Grained Protein Models into Accessible Workflows for Multiscale Molecular Dynamics

To capture protein conformational transitions using molecular dynamics (MD), several simulation resolutions covering different spatial and temporal scales are typically needed. All-atom (AA) simulations provide fine resolution, but are computationally infeasible for large systems over longer durations. Coarse-grained (CG) and ultra-coarse-grained (UCG) models have a lower resolution and computational cost while still being able to conserve essential protein features. Prior work on a Multiscale Machinelearned Modeling Infrastructure (MuMMI) combined both AA and CG simulations to study RAS-RAF protein interactions, leveraging CG models for longer time scales and using AA to investigate unusual conformations in greater detail. However, MuMMI is still resource-intensive, and this study aims to maximize exploration of the protein conformational space while reducing computational cost. In this paper, we build on prior work that integrates UCG models based on heterogeneous elastic network modeling (hENM) into the MuMMI workflow. We demonstrate that UCG models enable accurate sampling of protein conformations, focusing on simulating RAS-RAF protein interactions. Using higher-resolution CG Martini simulation data, we can automatically refine intramolecular interactions in UCG models. We present a scalable Python package that uses fluctuations observed in higher-resolution CG Martini simulations to estimate bond coefficients of the UCG model. We built novel machine learning-based backmapping methods to recover more detailed CG Martini structures from UCG structures, using diffusion models to learn the mapping between scales. Finally, we present UCG-mini-MuMMI, an accessible and less compute-intensive version of MuMMI as a resource for the scientific community. Incorporating UCG models into MD studies is applicable to a broad range of systems and proteins, and our study offers insights into the advantages and limitations of these methods.

Chemical structure↗
NASA NTRS2014

Highly Accurate Potential Energy Surface, Dipole Moment Surface, Rovibrational Energy Levels, and Infrared Line List for (32)S(16)O2 up to 8000 cm(exp -1)

A purely ab initio potential energy surface (PES) was refined with selected (32)S(16)O2 HITRAN data. Compared to HITRAN, the root-mean-squares error (RMS) error for all J=0-80 rovibrational energy levels computed on the refined PES (denoted Ames-1) is 0.013 cm(exp -1). Combined with a CCSD(T)/aug-cc-pV(Q+d)Z dipole moment surface (DMS), an infrared (IR) line list (denoted Ames-296K) has been computed at 296K and covers up to 8,000 cm(exp -1). Compared to the HITRAN and CDMS databases, the intensity agreement for most vibrational bands is better than 85-90%. Our predictions for (34)S(16)O2 band origins, higher energy (32)S(16)O2 band origins and missing (32)S(16)O2 IR bands have been verified by most recent experiments and available HITRAN data. We conclude that the Ames-1 PES is able to predict (32/34)S(16)O2 band origins below 5500 cm(exp -1) with 0.01-0.03 cm(exp -1) uncertainties, and the Ames-296K line list provides continuous, reliable and accurate IR simulations. The Ka-dependence of both line position and line intensity errors is discussed. The line list will greatly facilitate SO2 IR spectral experimental analysis, as well as elimination of SO2 lines in high-resolution astronomical observations.

Sulfur Dioxide (SO2)↗
DOE OSTI2019

Modeling pulse-cleaning plasma mirrors from dielectric response to saturation: A particle-in-cell approach

Pulse-cleaning plasma mirrors are widely employed to improve ultraintense laser contrast, but much of the literature concerning their effect on the reflected pulse is empirical. A simulation study of pulse-cleaning plasma mirrors using the particle-in-cell code large scale plasma is presented here. The importance of capturing initial ionization from neutral atoms, collisional effects, and simulation dimensionality is considered. Excellent agreement with experimental data is obtained when a multiphoton ionization model is employed. Furthermore, a series of 2D simulations is shown to accurately replicate both the reflected light intensity and mode obtained from full 3D simulations at significantly reduced computation cost.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1972

An interdisciplinary analysis of ERTS data for Colorado mountain environments using ADP techniques. An early analysis of ERTS-1 data

There are no author-identified significant results in this report. The principal problem encountered has been the lack of good quality, small scale baseline photography for the test areas. Analysis of the ERTS-1 data for the San Juan Site will emphasize development of a preliminary spectral classification defining grass cover categories, and then selection of subframes for intensive investigation of the forestry, geologic, and hydrologic properties of the area. Primary work has been devoted to the selection and digitization of areas for topographic modeling, and compilation of ground based data maps necessary for computer analysis. Study effort has emphasized: geomorphic features; macro-vegetation; micro-vegetation; snow-hydrology; insect/disease damage; and blow-down. Analysis of a frame of the Lake Texoma area indicates a great deal of potential in the analysis and interpretation of ERTS imagery. Preliminary results of investigations of geologic, forest, range, cropland, and water resources of the area are summarized.

Hoffer, R. M.↗