Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Intensive Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Refining Jets for CMS Run 3 using Fast Simulation

As the LHC moves into its high-luminosity phase, the CMS experiment must handle more complex data collected at much higher rates. While the Geant4-based simulation application (FullSim) provides highly accurate simulation to complement real data, FullSim’s intensive consumption of computing resources becomes an increasing liability as the rates increase, while faster tools offer an advantage. The fast MC production application (FastSim) delivers a complete simulation with a factor of 10 speedup over FullSim, but introduces inaccuracies in some observables. A specialized refinement method, Fast Perfekt, employs machine learning to improve the accuracy of FastSim. An initial report of this work focused on the refinement of jet flavor tagging observables. This article presents an update on the refinement, focusing on PUPPI jets with Run 3 data-taking conditions. Refinement is extended to include jet transverse momentum as well as its propagation to missing transverse momentum. A gridbased framework and real-time monitoring system have been developed to facilitate optimization and scaling of the refinement to a large number of target variables.

Güngördü, Açelya Deniz [Istanbul Tech. U.]↗
DOE OSTI2018

SharP Data Constructs: Data Constructs to Enable Data-Centric Computing

Extreme-scale applications (i.e., Big-Compute) are becoming increasingly data-intensive, i.e., producing and consuming increasingly large amounts of data. The HPC systems traditionally used for these applications are now used for Big-Data applications such as data analytics, social network analysis, machine learning, and genomics. As a consequence of these trends, the system architecture should be flexible and data-centric. This can already be witnessed in the pre-exascale systems with TBs of on-node hierarchical and heterogeneous memories, PBs of system memory, low-latency, high-throughput networks, and many threaded cores. As such, the pre-exascale systems suit the needs of both Big-Compute and Big-Data applications. Though the system architecture is flexible enough to support both Big-Compute and Big-Data, we argue there is a software gap. Particularly, we need data-centric abstractions to leverage the full potential of the system, i.e., there is a need for native support for data resilience, the ability to express data locality and affinity, mechanisms to reduce data movement, the ability to share data, and abstractions to express User's data usage and data access patterns. In this paper, we (i) show the need for taking a holistic approach towards data-centric abstractions, (ii) show how these approaches were realized in the SHARed data-structure centric Programming abstraction (SharP) library, a data-structure centric programming abstraction, and (iii) apply these approaches to a variety of applications that demonstrate its usefulness. Particularly, we apply these approaches to QMCPack and the Graph500 benchmark and demonstrate the advantages of this approach on extreme-scale systems.

Aderholdt, William Ferrol↗
DOE OSTI2023

PROTEUS: Machine Learning Driven Resilience for Extreme-scale Systems

The objective of this project is to design, develop, and evaluate scalable software to enhance resilience, data checkpointing, program restart, and analysis. The proposed tasks are to 1) develop scalable machine learning techniques to learn temporal change patterns in a scalable and in-situ manner, and to minimize data movement and maximize learning locally closest to data; 2) design a concise data representation and indexing mechanism to capture the distribution of changes in data that can guarantee point-wise user-defined tolerable errors while reducing the data storage requirements by an order of magnitude or more; 3) develop data reduction techniques as library modules; 4) exploit local SSD for minimizing data movement in storage hierarchy; 5) develop anomaly detection algorithms that can predict corruptions based on learning of emerging patterns; 6) develop software libraries to be incorporated within widely used data formats and APIs; and 7) evaluate the proposed software using DOE scientific applications. The outcomes of the proposed work are to satisfy many synergistic data reduction and resilience requirements for large-scale data intensive applications executed on extreme-scale computing systems. The developed mechanism for error-bound data approximation is directly applicable to existing scientific applications. Through machine learning from historical events and change distribution, this work will enable anomaly detection for DOE computer facility.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

The Estimation Surface Latent Heat Flux Over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method utilizes aerosol lidar backscatter data, multi-channel infrared radiometer data and microwave scatterometer data acquired onboard the NASA P-3B research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from the scatterometers and the lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via bulk aerodynamic formulae. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement. However, the SSM/I values are biased high by about 30 W/sq m. In addition, the MABL height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone top, bottom and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.62, 0.67 and 0.61, respectively.

Palm, Stephen P.↗
NASA NTRS2001

The Estimation of Surface Latent Heat Flux over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method utilizes aerosol lidar backscatter data, multi-channel infrared radiometer data, and microwave scatterometer data acquired onboard the NASA P-313 research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from scatterometers and lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via a bulk aerodynamic formula. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement. However, the SSM/I values are biased low by about 15 W/sq m. In addition, the Marine Atmospheric Boundary Layer (MABL) height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone depth, MABL height and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.39, 0.43 and 0.71, respectively.

Palm, Stephen P.↗
NASA NTRS2000

The Estimation of Surface Latent Heat Flux Over the Ocean and its Relationship to Marine Atmospheric Boundary Layer (MABL) Structure

A new technique combining active and passive remote sensing instruments for the estimation of surface latent heat flux over the ocean is presented. This synergistic method uses aerosol lidar backscatter data, multi-channel infrared radiometer data and microwave scatterometer data acquired onboard the NASA P-3B research aircraft during an extended field campaign over the Atlantic ocean in support of the Lidar In-space Technology Experiment (LITE) in September of 1994. The 10 meter wind speed derived from the scatterometers and the lidar-radiometer inferred near-surface moisture are used to obtain an estimate of the surface flux of moisture via bulk aerodynamic formulae. The results are compared with the Special Sensor Microwave Imager (SSM/I) daily average latent heat flux and show reasonable agreement with an rms error and bias of about 50 and 25 W per square meters, respectively. In addition, the MABL height, entrainment zone thickness and integrated lidar backscatter intensity are computed from the lidar data and compared with the magnitude of the surface fluxes. The results show that the surface latent heat flux is most strongly correlated with entrainment zone top, bottom and the integrated MABL lidar backscatter, with corresponding correlation coefficients of 0.62, 0.67 and 0.61, respectively.

Palm, Stephen P.↗
DOE OSTI2017

Active Subspace Methods for Data-Intensive Inverse Problems

The project has developed theory and computational tools to exploit active subspaces to reduce the dimension in statistical calibration problems. This dimension reduction enables MCMC methods to calibrate otherwise intractable models. The same theoretical and computational tools can also reduce the measurement dimension for calibration problems that use large stores of data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2006

Application of the Convolution Formalism to the Ocean Tide Potential: Results from the Gravity and Recovery and Climate Experiment (GRACE)

A computationally efficient approach to reducing omission errors in ocean tide potential models is derived and evaluated using data from the Gravity Recovery and Climate Experiment (GRACE) mission. Ocean tide height models are usually explicitly available at a few frequencies, and a smooth unit response is assumed to infer the response across the tidal spectrum. The convolution formalism of Munk and Cartwright (1966) models this response function with a Fourier series. This allows the total ocean tide height, and therefore the total ocean tide potential, to be modeled as a weighted sum of past, present, and future values of the tide-generating potential. Previous applications of the convolution formalism have usually been limited to tide height models, but we extend it to ocean tide potential models. We use luni-solar ephemerides to derive the required tide-generating potential so that the complete spectrum of the ocean tide potential is efficiently represented. In contrast, the traditionally adopted harmonic model of the ocean tide potential requires the explicit sum of the contributions from individual tidal frequencies. It is therefore subject to omission errors from neglected frequencies and is computationally more intensive. Intersatellite range rate data from the GRACE mission are used to compare convolution and harmonic models of the ocean tide potential. The monthly range rate residual variance is smaller by 4-5%, and the daily residual variance is smaller by as much as 15% when using the convolution model than when using a harmonic model that is defined by twice the number of parameters.

models↗
DOE OSTI2014

HERCULES: Strong Patterns towards More Intelligent Predictive Modeling, In: 2014 43rd International Conference on Parallel Processing

Recent work has shown that program analysis techniques to select meaningful code features of programs are important in the task of deciding the best compiler optimizations. Although, there are many successful state-of-the-art program analysis techniques, they often do not provide a simple method to extract the most expressive information about loops, especially when a target program is computationally intensive with complex loops and data dependencies. In this paper, we introduce a static technique to characterize a program using a pattern-driven system named HERCULES. This characterization technique not only helps a user to understand programs by searching pattern-of-interests, but also can be used for a predictive model that effectively selects the proper compiler optimizations. We formulated 35 loop patterns, then evaluated our characterization technique by comparing the predictive models constructed using HERCULES to three other state-of-the-art characterization methods. We show that our models outperform three state-of-the-art program characterization techniques on two multicore systems in selecting the best optimization combination from a given loop transformation space. We achieved up to 67% of the best possible speedup achievable with the optimization search space we evaluated.

Computer Science↗
DOE OSTI2011

Designing the Cloud-based DOE Systems Biology Knowledgebase

Systems Biology research, even more than many other scientific domains, is becoming increasingly data-intensive. Not only have advances in experimental and computational technologies lead to an exponential increase in scientific data volumes and their complexity, but increasingly such databases themselves are providing the basis for new scientific discoveries. To engage effectively with these community resources, integrated analyses, synthesis and simulation software is needed, regularly supported by scientific workflows. In order to provide a more collaborative, community driven research environment for this heterogeneous setting, the Department of Energy (DOE) has decided to develop a federated, cloud based cyber infrastructure - the Systems Biology Knowledgebase (Kbase). Pacific Northwest National Laboratory (PNNL) with its long tradition in data intensive science lead two of the five initial pilot projects, these two focusing on defining and testing the basic federated cloud-based system architecture and develop a prototype implementation. Hereby the community wide accessibility of biological data and the capability to integrate and analyze this data within its changing research context were seen as key technical functionalities the Kbase needed to enable. In this paper we describe the results of our investigations into the design of a cloud based federated infrastructure for: (1) Semantics driven data discovery, access and integration; (2) Data annotation, publication and sharing; (3) Workflow enabled data analysis; and (4) Project based collaborative working. We describe our approach, exemplary use cases and our prototype implementation that demonstrates the feasibility of this approach.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2017

SharP

The pre-exascale systems are expected to have a significant amount of hierarchical and heterogeneous on-node memory, and this trend of system architecture in extreme-scale systems is expected to continue into the exascale era. along with hierarchical-heterogeneous memory, the system typically has a high-performing network ad a compute accelerator. This system architecture is not only effective for running traditional High Performance Computing (HPC) applications (Big-Compute), but also for running data-intensive HPC applications and Big-Data applications. As a consequence, there is a growing desire to have a single system serve the needs of both Big-Compute and Big-Data applications. Though the system architecture supports the convergence of the Big-Compute and Big-Data, the programming models and software layer have yet to evolve to support either hierarchical-heterogeneous memory systems or the convergence. A programming abstraction to address this problem. The programming abstraction is implemented as a software library and runs on pre-exascale and exascale systems supporting current and emerging system architecture. Using distributed data-structures as a central concept, it provides (1) a simple, usable, and portable abstraction for hierarchical-heterogeneous memory and (2) a unified programming abstraction for Big-Compute and Big-Data applications.

Venkata, Manjunath Gorentla↗
NASA NTRS1981

Ground-based detection of stratospheric ClO and average column density measurement over 17 days of observation

A very sensitive mm-wave heterodyne receiver of a type originally developed for radio astronomy is used in combination with a broadband filter bank spectrometer to detect and measure the 204-GHz rotational transition of stratospheric ClO. Data is acquired in the form of a dimensionless ratio and is converted to units expressing signal intensity. Computed line shapes are matched against the data for a comparison with theoretical stratospheric models and previous measurements. The ClO distribution in the stratosphere measured at 42 deg N latitude in early spring is significantly different from that expected based on existing theoretical models. There is much less ClO at latitudes below about 36 km than predicted, and average ClO column density found for 42 deg N is 20-30% less than that from existing balloon flight measurements at 32 deg N.

Dezafra, R.↗
NASA NTRS1979

Turbulence in planetary occultations. IV - Power spectra of phase and intensity fluctuations

Power spectra of phase and intensity scintillations during occultation by turbulent planetary atmospheres are significantly affected by the inhomogeneous background upon which the turbulence is superimposed. Such coupling is particularly pronounced in the intensity, where there is also a marked difference in spectral shape between a central and grazing occultation. While the former has its structural features smoothed by coupling to the inhomogeneous background, such features are enhanced in the latter. Indeed, the latter power spectrum peaks around the characteristic frequency that is determined by the size of the free-space Fresnel zone and the ray velocity in the atmosphere; at higher frequencies strong fringes develop in the power spectrum. A confrontation between the theoretical scintillation spectra computed here and those calculated from the Mariner 5 Venus mission by Woo et al. (1974) is inconclusive, mainly because of insufficient statistical resolution. Phase and/or intensity power spectra computed from occultation data may be used to deduce characteristics of the turbulence and to distinguish turbulence from other perturbations in the refractive index. Such determinations are facilitated if observations are made at two or more frequencies (radio occultation) or in two or more colors (stellar occultation).

Haugstad, B. S.↗
NASA NTRS1989

F.T. spectroscopy of OH produced in a microwave surface plasma

Emission spectra of the hydroxyl radical have been recorded between 450 and 4200/cm at 0.0027/cm unapodized resolution from an air+oxygen+water vapor plasma, using a surface microwave discharge. The characteristics of the source are described. The emission spectra were recorded with the F.T. spectrometer using a GeCu detector and KCl beams plitter. High J values were observed both for the pure rotational transition in v=0 and v=1 vibrational levels, and for the rovibrational transition in the 0-1 and 1-2 bands. The set of line intensity data will be used to compute the electric dipole moment function of OH.

Benidar, A.↗
NASA NTRS2000

Finite Element Analysis of Reverberation Chambers

The primary motivating factor behind the initiation of this work was to provide a deterministic means of establishing the validity of the statistical methods that are recommended for the determination of fields that interact in -an avionics system. The application of finite element analysis to reverberation chambers is the initial step required to establish a reasonable course of inquiry in this particularly data-intensive study. The use of computational electromagnetics provides a high degree of control of the "experimental" parameters that can be utilized in a simulation of reverberating structures. As the work evolved there were four primary focus areas they are: 1. The eigenvalue problem for the source free problem. 2. The development of a complex efficient eigensolver. 3. The application of a source for the TE and TM fields for statistical characterization. 4. The examination of shielding effectiveness in a reverberating environment. One early purpose of this work was to establish the utility of finite element techniques in the development of an extended low frequency statistical model for reverberation phenomena. By employing finite element techniques, structures of arbitrary complexity can be analyzed due to the use of triangular shape functions in the spatial discretization. The effects of both frequency stirring and mechanical stirring are presented. It is suggested that for the low frequency operation the typical tuner size is inadequate to provide a sufficiently random field and that frequency stirring should be used. The results of the finite element analysis of the reverberation chamber illustrate io-W the potential utility of a 2D representation for enhancing the basic statistical characteristics of the chamber when operating in a low frequency regime. The basic field statistics are verified for frequency stirring over a wide range of frequencies. Mechanical stirring is shown to provide an effective frequency deviation.

Bunting, Charles F.↗
NASA NTRS2006

Radiation Tolerant Intelligent Memory Stack (RTIMS)

The Radiation Tolerant Intelligent Memory Stack (RTIMS), suitable for both geostationary and low earth orbit missions, has been developed. The memory module is fully functional and undergoing environmental and radiation characterization. A self-contained flight-like module is expected to be completed in 2006. RTIMS provides reconfigurable circuitry and 2 gigabits of error corrected or 1 gigabit of triple redundant digital memory in a small package. RTIMS utilizes circuit stacking of heterogeneous components and radiation shielding technologies. A reprogrammable field programmable gate array (FPGA), six synchronous dynamic random access memories, linear regulator, and the radiation mitigation circuitries are stacked into a module of 42.7mm x 42.7mm x 13.00mm. Triple module redundancy, current limiting, configuration scrubbing, and single event function interrupt detection are employed to mitigate radiation effects. The mitigation techniques significantly simplify system design. RTIMS is well suited for deployment in real-time data processing, reconfigurable computing, and memory intensive applications.

Ng, Tak-kwong↗
NASA NTRS2010

Combining Model-Based and Feature-Driven Diagnosis Approaches - A Case Study on Electromechanical Actuators

Model-based diagnosis typically uses analytical redundancy to compare predictions from a model against observations from the system being diagnosed. However this approach does not work very well when it is not feasible to create analytic relations describing all the observed data, e.g., for vibration data which is usually sampled at very high rates and requires very detailed finite element models to describe its behavior. In such cases, features (in time and frequency domains) that contain diagnostic information are extracted from the data. Since this is a computationally intensive process, it is not efficient to extract all the features all the time. In this paper we present an approach that combines the analytic model-based and feature-driven diagnosis approaches. The analytic approach is used to reduce the set of possible faults and then features are chosen to best distinguish among the remaining faults. We describe an implementation of this approach on the Flyable Electro-mechanical Actuator (FLEA) test bed.

Narasimhan, Sriram↗
NASA NTRS1981

A digital Seasat SAR correlation-simulation program

A correlation-simulation program for processing Seasat SAR data is presented, featuring the use of FORTRAN code to allow simple transference to another computer. The highest attainable azimuthal resolution for Seasat SAR experiments and the ability to generate images with varying quality by varying resolution or the number of looks are additional features. A fiber optics link transfers the Seasat data from high intensity digital tapes to an SEL 32/55 computer, and the least significant bits of the 5 bit data samples are truncated, thus the input data have 4 bits per sample. A total 8192 echoes are processed each time and stored on 8 separate data files on the disk. In addition, a program operating on 1-look high resolution images for performance simulations of proposed radar systems and processing schemes has been completed. Finally, the coding is simple, and 8 hours are required to process one image using this program.

Li, F.↗