Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Green Computing Opportunities & Strategy

Computation is critical to emerging fields of data intensive research, enabling new methodologies, approaches and tools. As the rate of hardware efficiency gains slows, computational time and energy costs increase. To match pace with computation demand, new approaches are needed to keep the opportunity for impact open. Charles Tripp, lead of the Green Computing Catalyzer, discusses research efforts to improve software efficiency to enable faster, less energy-intensive computing.

algorithmic efficiency↗
DOE OSTI2015

Data Intensive Scientific Workflows on a Federated Cloud: CRADA Final Report

The Fermilab Scientific Computing Division and the KISTI Global Science Experimental Data Hub Center have built a prototypical large-scale infrastructure to handle scientific workflows of stakeholders to run on multiple cloud resources. The demonstrations have been in the areas of (a) Data-Intensive Scientific Workflows on Federated Clouds, (b) Interoperability and Federation of Cloud Resources, and (c) Virtual Infrastructure Automation to enable On-Demand Services.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS1982

Ultraviolet continuum absorption /less than about 1000 A/ above the quiet sun transition region

Lyman continuum absorption shortward of 912 A in the quiet sun solar transition region is investigated by combining spectra obtained from the Apollo Telescope Mount experiments on Skylab. The most recent atomic data are used to compute line intensities for lines that fall on both sides of the Lyman limit. Lines of O III, O IV, O V, and S IV are considered. The computed intensity ratios of most lines from O IV, O V, and S IV agree with the experimental ratios to within a factor of 2. However, the discrepancies show no apparent wavelength dependence. From this fact, it is concluded that at least part of the discrepancy between theory and observation for lines of these ions can be accounted for by uncertainties in instrumental calibration and atomic data. However, difficulties remain in reconciling observation and theory, particularly for lines of O III, and one line of S IV. The other recent results of Schmahl and Orrall (1979) are also discussed in terms of newer atomic data.

Doschek, G. A.↗
NASA NTRS2010

An Improved Wake Vortex Tracking Algorithm for Multiple Aircraft

The accurate tracking of vortex evolution from Large Eddy Simulation (LES) data is a complex and computationally intensive problem. The vortex tracking requires the analysis of very large three-dimensional and time-varying datasets. The complexity of the problem is further compounded by the fact that these vortices are embedded in a background turbulence field, and they may interact with the ground surface. Another level of complication can arise, if vortices from multiple aircrafts are simulated. This paper presents a new technique for post-processing LES data to obtain wake vortex tracks and wake intensities. The new approach isolates vortices by defining "regions of interest" (ROI) around each vortex and has the ability to identify vortex pairs from multiple aircraft. The paper describes the new methodology for tracking wake vortices and presents application of the technique for single and multiple aircraft.

Switzer, George F.↗
DOE OSTI2025

Refining Jets for CMS Run 3 using Fast Simulation

As the LHC moves into its high-luminosity phase, the CMS experiment must handle more complex data collected at much higher rates. While the Geant4-based simulation application (FullSim) provides highly accurate simulation to complement real data, FullSim’s intensive consumption of computing resources becomes an increasing liability as the rates increase, while faster tools offer an advantage. The fast MC production application (FastSim) delivers a complete simulation with a factor of 10 speedup over FullSim, but introduces inaccuracies in some observables. A specialized refinement method, Fast Perfekt, employs machine learning to improve the accuracy of FastSim. An initial report of this work focused on the refinement of jet flavor tagging observables. This article presents an update on the refinement, focusing on PUPPI jets with Run 3 data-taking conditions. Refinement is extended to include jet transverse momentum as well as its propagation to missing transverse momentum. A gridbased framework and real-time monitoring system have been developed to facilitate optimization and scaling of the refinement to a large number of target variables.

Güngördü, Açelya Deniz [Istanbul Tech. U.]↗
DOE OSTI2018

SharP Data Constructs: Data Constructs to Enable Data-Centric Computing

Extreme-scale applications (i.e., Big-Compute) are becoming increasingly data-intensive, i.e., producing and consuming increasingly large amounts of data. The HPC systems traditionally used for these applications are now used for Big-Data applications such as data analytics, social network analysis, machine learning, and genomics. As a consequence of these trends, the system architecture should be flexible and data-centric. This can already be witnessed in the pre-exascale systems with TBs of on-node hierarchical and heterogeneous memories, PBs of system memory, low-latency, high-throughput networks, and many threaded cores. As such, the pre-exascale systems suit the needs of both Big-Compute and Big-Data applications. Though the system architecture is flexible enough to support both Big-Compute and Big-Data, we argue there is a software gap. Particularly, we need data-centric abstractions to leverage the full potential of the system, i.e., there is a need for native support for data resilience, the ability to express data locality and affinity, mechanisms to reduce data movement, the ability to share data, and abstractions to express User's data usage and data access patterns. In this paper, we (i) show the need for taking a holistic approach towards data-centric abstractions, (ii) show how these approaches were realized in the SHARed data-structure centric Programming abstraction (SharP) library, a data-structure centric programming abstraction, and (iii) apply these approaches to a variety of applications that demonstrate its usefulness. Particularly, we apply these approaches to QMCPack and the Graph500 benchmark and demonstrate the advantages of this approach on extreme-scale systems.

Aderholdt, William Ferrol↗
DOE OSTI2023

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

The Estimation Surface Latent Heat Flux Over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method utilizes aerosol lidar backscatter data, multi-channel infrared radiometer data and microwave scatterometer data acquired onboard the NASA P-3B research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from the scatterometers and the lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via bulk aerodynamic formulae. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement. However, the SSM/I values are biased high by about 30 W/sq m. In addition, the MABL height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone top, bottom and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.62, 0.67 and 0.61, respectively.

Palm, Stephen P.↗
NASA NTRS2001

The Estimation of Surface Latent Heat Flux over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method utilizes aerosol lidar backscatter data, multi-channel infrared radiometer data, and microwave scatterometer data acquired onboard the NASA P-313 research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from scatterometers and lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via a bulk aerodynamic formula. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement. However, the SSM/I values are biased low by about 15 W/sq m. In addition, the Marine Atmospheric Boundary Layer (MABL) height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone depth, MABL height and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.39, 0.43 and 0.71, respectively.

Palm, Stephen P.↗
NASA NTRS2000

The Estimation of Surface Latent Heat Flux Over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method uses aerosol lidar backscatter data, multi-channel infrared radiometer data and microwave scatterometer data acquired onboard the NASA P-3B research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from the scatterometers and the lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via bulk aerodynamic formulae. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement with an rms error and bias of about 50 and 25 W per square meters, respectively. In addition, the MABL height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone top, bottom and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.62, 0.67 and 0.61, respectively.

Palm, Stephen P.↗
DOE OSTI2017

Active Subspace Methods for Data-Intensive Inverse Problems

The project has developed theory and computational tools to exploit active subspaces to reduce the dimension in statistical calibration problems. This dimension reduction enables MCMC methods to calibrate otherwise intractable models. The same theoretical and computational tools can also reduce the measurement dimension for calibration problems that use large stores of data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2006

Application of the Convolution Formalism to the Ocean Tide Potential: Results from the Gravity and Recovery and Climate Experiment (GRACE)

A computationally efficient approach to reducing omission errors in ocean tide potential models is derived and evaluated using data from the Gravity Recovery and Climate Experiment (GRACE) mission. Ocean tide height models are usually explicitly available at a few frequencies, and a smooth unit response is assumed to infer the response across the tidal spectrum. The convolution formalism of Munk and Cartwright (1966) models this response function with a Fourier series. This allows the total ocean tide height, and therefore the total ocean tide potential, to be modeled as a weighted sum of past, present, and future values of the tide-generating potential. Previous applications of the convolution formalism have usually been limited to tide height models, but we extend it to ocean tide potential models. We use luni-solar ephemerides to derive the required tide-generating potential so that the complete spectrum of the ocean tide potential is efficiently represented. In contrast, the traditionally adopted harmonic model of the ocean tide potential requires the explicit sum of the contributions from individual tidal frequencies. It is therefore subject to omission errors from neglected frequencies and is computationally more intensive. Intersatellite range rate data from the GRACE mission are used to compare convolution and harmonic models of the ocean tide potential. The monthly range rate residual variance is smaller by 4-5%, and the daily residual variance is smaller by as much as 15% when using the convolution model than when using a harmonic model that is defined by twice the number of parameters.

models↗
DOE OSTI2014

HERCULES: Strong Patterns towards More Intelligent Predictive Modeling, In: 2014 43rd International Conference on Parallel Processing

Recent work has shown that program analysis techniques to select meaningful code features of programs are important in the task of deciding the best compiler optimizations. Although, there are many successful state-of-the-art program analysis techniques, they often do not provide a simple method to extract the most expressive information about loops, especially when a target program is computationally intensive with complex loops and data dependencies. In this paper, we introduce a static technique to characterize a program using a pattern-driven system named HERCULES. This characterization technique not only helps a user to understand programs by searching pattern-of-interests, but also can be used for a predictive model that effectively selects the proper compiler optimizations. We formulated 35 loop patterns, then evaluated our characterization technique by comparing the predictive models constructed using HERCULES to three other state-of-the-art characterization methods. We show that our models outperform three state-of-the-art program characterization techniques on two multicore systems in selecting the best optimization combination from a given loop transformation space. We achieved up to 67% of the best possible speedup achievable with the optimization search space we evaluated.

Computer Science↗
DOE OSTI2017

SharP

The pre-exascale systems are expected to have a significant amount of hierarchical and heterogeneous on-node memory, and this trend of system architecture in extreme-scale systems is expected to continue into the exascale era. along with hierarchical-heterogeneous memory, the system typically has a high-performing network ad a compute accelerator. This system architecture is not only effective for running traditional High Performance Computing (HPC) applications (Big-Compute), but also for running data-intensive HPC applications and Big-Data applications. As a consequence, there is a growing desire to have a single system serve the needs of both Big-Compute and Big-Data applications. Though the system architecture supports the convergence of the Big-Compute and Big-Data, the programming models and software layer have yet to evolve to support either hierarchical-heterogeneous memory systems or the convergence. A programming abstraction to address this problem. The programming abstraction is implemented as a software library and runs on pre-exascale and exascale systems supporting current and emerging system architecture. Using distributed data-structures as a central concept, it provides (1) a simple, usable, and portable abstraction for hierarchical-heterogeneous memory and (2) a unified programming abstraction for Big-Compute and Big-Data applications.

Venkata, Manjunath Gorentla↗
NASA NTRS1981

Ground-based detection of stratospheric ClO and average column density measurement over 17 days of observation

A very sensitive mm-wave heterodyne receiver of a type originally developed for radio astronomy is used in combination with a broadband filter bank spectrometer to detect and measure the 204-GHz rotational transition of stratospheric ClO. Data is acquired in the form of a dimensionless ratio and is converted to units expressing signal intensity. Computed line shapes are matched against the data for a comparison with theoretical stratospheric models and previous measurements. The ClO distribution in the stratosphere measured at 42 deg N latitude in early spring is significantly different from that expected based on existing theoretical models. There is much less ClO at latitudes below about 36 km than predicted, and average ClO column density found for 42 deg N is 20-30% less than that from existing balloon flight measurements at 32 deg N.

Dezafra, R.↗
NASA NTRS1979

Turbulence in planetary occultations. IV - Power spectra of phase and intensity fluctuations

Power spectra of phase and intensity scintillations during occultation by turbulent planetary atmospheres are significantly affected by the inhomogeneous background upon which the turbulence is superimposed. Such coupling is particularly pronounced in the intensity, where there is also a marked difference in spectral shape between a central and grazing occultation. While the former has its structural features smoothed by coupling to the inhomogeneous background, such features are enhanced in the latter. Indeed, the latter power spectrum peaks around the characteristic frequency that is determined by the size of the free-space Fresnel zone and the ray velocity in the atmosphere; at higher frequencies strong fringes develop in the power spectrum. A confrontation between the theoretical scintillation spectra computed here and those calculated from the Mariner 5 Venus mission by Woo et al. (1974) is inconclusive, mainly because of insufficient statistical resolution. Phase and/or intensity power spectra computed from occultation data may be used to deduce characteristics of the turbulence and to distinguish turbulence from other perturbations in the refractive index. Such determinations are facilitated if observations are made at two or more frequencies (radio occultation) or in two or more colors (stellar occultation).

Haugstad, B. S.↗
NASA NTRS1989

F.T. spectroscopy of OH produced in a microwave surface plasma

Emission spectra of the hydroxyl radical have been recorded between 450 and 4200/cm at 0.0027/cm unapodized resolution from an air+oxygen+water vapor plasma, using a surface microwave discharge. The characteristics of the source are described. The emission spectra were recorded with the F.T. spectrometer using a GeCu detector and KCl beams plitter. High J values were observed both for the pure rotational transition in v=0 and v=1 vibrational levels, and for the rovibrational transition in the 0-1 and 1-2 bands. The set of line intensity data will be used to compute the electric dipole moment function of OH.

Benidar, A.↗
NASA NTRS2000

Finite Element Analysis of Reverberation Chambers

The primary motivating factor behind the initiation of this work was to provide a deterministic means of establishing the validity of the statistical methods that are recommended for the determination of fields that interact in -an avionics system. The application of finite element analysis to reverberation chambers is the initial step required to establish a reasonable course of inquiry in this particularly data-intensive study. The use of computational electromagnetics provides a high degree of control of the "experimental" parameters that can be utilized in a simulation of reverberating structures. As the work evolved there were four primary focus areas they are: 1. The eigenvalue problem for the source free problem. 2. The development of a complex efficient eigensolver. 3. The application of a source for the TE and TM fields for statistical characterization. 4. The examination of shielding effectiveness in a reverberating environment. One early purpose of this work was to establish the utility of finite element techniques in the development of an extended low frequency statistical model for reverberation phenomena. By employing finite element techniques, structures of arbitrary complexity can be analyzed due to the use of triangular shape functions in the spatial discretization. The effects of both frequency stirring and mechanical stirring are presented. It is suggested that for the low frequency operation the typical tuner size is inadequate to provide a sufficiently random field and that frequency stirring should be used. The results of the finite element analysis of the reverberation chamber illustrate io-W the potential utility of a 2D representation for enhancing the basic statistical characteristics of the chamber when operating in a low frequency regime. The basic field statistics are verified for frequency stirring over a wide range of frequencies. Mechanical stirring is shown to provide an effective frequency deviation.

Bunting, Charles F.↗