SEARCH · Engineering Papers

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Electron-Hole Theory of the Effect of Quantum Nuclei on the X-Ray Absorption Spectra of Liquid Water

Electron-hole excitation theory is used to unveil the role of nuclear quantum effects on the x-ray absorption spectral signatures of water, whose structure is computed via path-integral molecular dynamics with the MB-pol intermolecular potential model. Compared to spectra generated from the classically modeled water, quantum nuclei introduce important effects on the spectra in terms of both the energies and line shapes. Fluctuations due to delocalized protons influence the short-range ordering of the hydrogen bond network via changes in the intramolecular covalence, which broaden the preedge spectra. For intermediate-range and long-range ordering, quantum nuclei approach the neighboring oxygen atoms more closely than classical protons, promoting an “icelike” spectral feature with the intensities shifted from the main edge to the postedge. Computed spectra are in nearly quantitative agreement with the available experimental data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2025

GreenThrift: Optimizing Carbon and Cost for Flexible Residential Loads

Reducing buildings’ carbon emissions is an important sustainability challenge. While scheduling flexible building loads has been previously used for a variety of grid and energy optimizations, carbon footprint reduction using such flexible loads poses new challenges since such methods need to balance both energy and carbon costs while also reducing user inconvenience from delaying such loads. This paper highlights the potential conflict between electricity prices and carbon emissions and the resulting trade-offs in carbon-aware and cost-aware load scheduling. To address this trade-off, we propose GreenThrift, a home automation system that leverages the scheduling capabilities of smart appliances and knowledge of future carbon intensity and cost to reduce both the carbon emissions and costs of flexible energy loads. At the heart of GreenThrift is an optimization technique that automatically computes schedules based on user configurations and preferences. We evaluate the effectiveness of GreenThrift using real-world carbon intensity data, electricity prices, and load traces from multiple locations and across different scenarios and objectives. Our results show that GreenThrift can replicate the offline optimal and retains 97% of the savings when optimizing the carbon emissions. Moreover, we show how GreenThrift can balance the conflict between carbon and cost and retain 95.3% and 85.5% of the potential carbon and cost savings, respectively.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2019

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

The Mochi project began in 2015 in recognition of a set of changes that were set to alter the landscape of storage and I/O for DOE science. On the one hand, an influx of new classes of applications including an initial set of data intensive and machine learning applications were operating with new types of data and in new ways, as compared to traditional computational science simulation codes. Simultaneously, new memory, storage, and networking technologies were driving a need to adapt software designs to a rapidly changing design space. The goal of the three-year Mochi project was to develop and demonstrate an approach for composing storage services which provides new levels of functionality, performance, and reliability for science applications at extreme scale. The team, including a mix of Laboratory, university, and industry collaborators, focused on systems and applications of interest to DOE Office of Science, tracking developments at NERSC and the LCFs as well as application trends within ECP and SciDAC. Technical work focused on three themes: composition (understand the building blocks needed to provide storage services that meet DOE mission needs and how these components are flexibly composed), application interfacing (what service compositions are most effective for common DOE application use cases), and consistency and synchronization (taking advantage of knowledge about, and from, the system and scientific applications to provide better service than would otherwise be possible). The project was highly successful, producing 13 publications and tens of thousands of lines of open source software that is now being used by multiple ECP teams, HPC software vendors, and international collaborators. The uptake of Mochi technologies speaks to both the value of the service composition model and the quality of architecture and software developed by the team. Through continued funding under ASCR and ECP, the Mochi team and its technologies are on track to further contribute to DOE science successes into the exascale timeframe.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1973

Results of precision processing (scene correction) of ERTS-1 images using digital image processing techniques

ERTS-1 MSS and RBV data recorded on computer compatible tapes have been analyzed and processed, and preliminary results have been obtained. No degradation of intensity (radiance) information occurred in implementing the geometric correction. The quality and resolution of the digitally processed images are very good, due primarily to the fact that the number of film generations and conversions is reduced to a minimum. Processing times of digitally processed images are about equivalent to the NDPF electro-optical processor.

Bernstein, R.↗

NASA NTRS1994

Moored rainfall measurements during COARE

This presentation discusses mini-ORG rainfall estimates collected from an array of six moornings in the western equatorial Pacific during the TOGA-COARE experiment. The moorings were clustered in the vicinity of the COARE intensive flux array (IFA) centered near 2 deg S, 156 deg E. The basic data set consisted of hourly means computed from 5-second samples.

Mcphaden, Michael J.↗

NASA NTRS1999

Modeling of Flow Transition Using an Intermittency Transport Equation

A new transport equation for intermittency factor is proposed to model transitional flows. The intermittent behavior of the transitional flows is incorporated into the computations by modifying the eddy viscosity, mu(sub t), obtainable from a turbulence model, with the intermittency factor, gamma: mu(sub t, sup *) = gamma.mu(sub t). In this paper, Menter's SST model (Menter, 1994) is employed to compute mu(sub t) and other turbulent quantities. The proposed intermittency transport equation can be considered as a blending of two models - Steelant and Dick (1996) and Cho and Chung (1992). The former was proposed for near-wall flows and was designed to reproduce the streamwise variation of the intermittency factor in the transition zone following Dhawan and Narasimha correlation (Dhawan and Narasimha, 1958) and the latter was proposed for free shear flows and was used to provide a realistic cross-stream variation of the intermittency profile. The new model was used to predict the T3 series experiments assembled by Savill (1993a, 1993b) including flows with different freestream turbulence intensities and two pressure-gradient cases. For all test cases good agreements between the computed results and the experimental data are observed.

Suzen, Y. B.↗

DOE OSTI2014

The Science DMZ: A Network Design Pattern for Data-Intensive Science

The ever-increasing scale of scientific data has become a significant challenge for researchers that rely on networks to interact with remote computing systems and transfer results to collaborators worldwide. Despite the availability of high-capacity connections, scientists struggle with inadequate cyberinfrastructure that cripples data transfer performance, and impedes scientific progress. The Science DMZ paradigm comprises a proven set of network design patterns that collectively address these problems for scientists. We explain the Science DMZ model, including network architecture, system configuration, cybersecurity, and performance tools, that creates an optimized network environment for science. We describe use cases from universities, supercomputing centers and research laboratories, highlighting the effectiveness of the Science DMZ model in diverse operational settings. In all, the Science DMZ model is a solid platform that supports any science workflow, and flexibly accommodates emerging network technologies. As a result, the Science DMZ vastly improves collaboration, accelerating scientific discovery.

Dart, Eli↗

DOE OSTI2013

The Science DMZ: A Network Design Pattern for Data-Intensive Science

42 ENGINEERING↗

NASA NTRS1994

TOPEX/POSEIDON tides estimated using a global inverse model

Altimetric data from the TOPEX/POSEIDON mission will be used for studies of global ocean circulation and marine geophysics. However, it is first necessary to remove the ocean tides, which are aliased in the raw data. The tides are constrained by the two distinct types of information: the hydrodynamic equations which the tidal fields of elevations and velocities must satisfy, and direct observational data from tide gauges and satellite altimetry. Here we develop and apply a generalized inverse method, which allows us to combine rationally all of this information into global tidal fields best fitting both the data and the dynamics, in a least squares sense. The resulting inverse solution is a sum of the direct solution to the astronomically forced Laplace tidal equations and a linear combination of the representers for the data functionals. The representer functions (one for each datum) are determined by the dynamical equations, and by our prior estimates of the statistics or errors in these equations. Our major task is a direct numerical calculation of these representers. This task is computationally intensive, but well suited to massively parallel processing. By calculating the representers we reduce the full (infinite dimensional) problem to a relatively low-dimensional problem at the outset, allowing full control over the conditioning and hence the stability of the inverse solution. With the representers calculated we can easily update our model as additional TOPEX/POSEIDON data become available. As an initial illustration we invert harmonic constants from a set of 80 open-ocean tide gauges. We then present a practical scheme for direct inversion of TOPEX/POSEIDON crossover data. We apply this method to 38 cycles of geophysical data records (GDR) data, computing preliminary global estimates of the four principal tidal constituents, M(sub 2), S(sub 2), K(sub 1) and O(sub 1). The inverse solution yields tidal fields which are simultaneously smoother, and in better agreement with altimetric and ground truth data, than previously proposed tidal models. Relative to the 'default' tidal corrections provided with the TOPEX/POSEIDON GDR, the inverse solution reduces crossover difference variances significantly (approximately 20-30%), even though only a small number of free parameters (approximately equal to 1000) are actually fit to the crossover data.

Egbert, Gary D.↗

NASA NTRS1991

Compiling and editing agricultural strata boundaries with remotely sensed imagery and map attribute data using graphics workstations

The USDA presently uses labor-intensive photographic interpretation procedures to delineate large geographical areas into manageable size sampling units for the estimation of domestic crop and livestock production. Computer software to automate the boundary delineation procedure, called the computer-assisted stratification and sampling (CASS) system, was developed using a Hewlett Packard color-graphics workstation. The CASS procedures display Thematic Mapper (TM) satellite digital imagery on a graphics display workstation as the backdrop for the onscreen delineation of sampling units. USGS Digital Line Graph (DLG) data for roads and waterways are displayed over the TM imagery to aid in identifying potential sample unit boundaries. Initial analysis conducted with three Missouri counties indicated that CASS was six times faster than the manual techniques in delineating sampling units.

Cheng, Thomas D.↗

NASA NTRSDate not supplied

Effect of Electronic Shot Noise on Dynamic Measurements using Optical Techniques: Examples from Rayleigh Scattering and Unsteady PSP

Electronic shot noise is an unavoidable reality in all optical techniques that depend on measuring light intensity. For steady-state, time-averaged measurements the impact of shot noise can be easily reduced by increasing the exposure time, or by averaging over multiple-exposures. That is not the case for unsteady measurements, where time histories of light intensity variations need to be created either by high-speed photography (unsteady PSP) or via photoelectron counting (spectrally resolved Rayleigh scattering) over short time durations using a photo-multiplier tube (PMT) and photon counting electronics. Electronic shot noise introduces a fixed amount of random error, which can overwhelm the light intensity variation caused by turbulent fluctuations. Spectrum computed from such time series shows a fixed noise floor that is independent of the number of data points in the time series. For a fixed optical system, where the collected power of the luminescent light (uPSP), or the scattered light (Rayleigh) is fixed, one needs to resort to special techniques to obtain enough signal-to noise ratio (SNR). For the uPSP application it is shown that averaging of the adjacent pixels improves SNR; although this may lead to a sacrifice of spatial resolution. A second means is to increase the exposure time, which leads to a lowering of the frequency range. For the Rayleigh application, improvements of SNR can be achieved via two different cross-correlation based approaches. The first involves measuring the light intensity using two PMTs using short, contiguous gates; thereby, creating two time-series of data. The second one involves collecting one long time-series of data using one set of measurement device, followed by an odd-even splitting into two time series. When the two time-series are cross-correlated, and a power spectrum is calculated, a significant reduction in the shot noise floor can be achieved. Examples from measurements of density and velocity fluctuations spectra from two different Rayleigh setup are presented to demonstrate the process.

uPSP↗

NASA NTRSDate not supplied

Comparison of Ring Diagrams Based on the Doppler Shifts of Synthetic Data Obtained with Bisector Method and SDO/HMI Pipeline

Ring diagrams are cross-sections of three-dimensional spatiotemporal power spectra of solar oscillations. The rings reveal information about sub-surface flows and represent an important tool for helioseismology. Ring diagrams can be constructed using Doppler velocity or intensity maps of spectral lines. How the velocities are computed is an important factor for accuracy of information we can retrieve on subsurface flows. In our work, the ring diagrams are generated from Doppler shift data of synthesized Fe I 6173 Å line. We compare ring diagrams computed by two methods–HMI line-of-sight pipeline and the bisector of Fe line. Fe I line is synthesized for StellarBox 3D Radiative hydrodynamic simulations under LTE assumption. We aim to answer the following questions: 1.How do power spectra obtained from velocities computed with the HMI pipeline and bisector of the Fe I 6173Å compare? 2.How do the power spectral density retrieved with each method vary with heliocentric angle? 3.What is the effect of changing resolution on the power spectral density in ring diagrams obtained with the two methods?

SMD↗

DOE OSTI2013

What is Data-Intensive Science?

What is Data Intensive Science? Today we are living in a digital world, where scientists often no longer interact directly with the physical object of their research, but do so via digitally captured, reduced, calibrated, analyzed, synthesized and, at times, visualized data. Advances in experimental and computational technologies have lead to an exponential growth in the volumes, variety and complexity of this data and while the deluge is not happening everywhere in an absolute sense, it is in a relative one. Science today is data intensive. Data intensive science has the potential to transform not only how we do science, but how quickly we can translate scientific progress into complete solutions, policies, decisions and ultimately economic success. Critically, data intensive science touches some of the most important challenges we are facing. Consider a few of the grand challenges outlined by the U.S. National Academy of Engineering: make solar energy economical, provide energy from fusion, develop carbon sequestration methods, advance health informatics, engineer better medicines, secure cyberspace, and engineer the tools of scientific discovery. Arguably, meeting any of these challenges requires the collaborative effort of trans-disciplinary teams, but also significant contributions from enabling data intensive technologies. Indeed for many of them, advances in data intensive research will be the single most important factor in developing successful and timely solutions. Simple extrapolations of how we currently interact with and utilize data and knowledge are not sufficient to meet this need. Given the importance of these challenges, a new, bold vision for the role of data in science, and indeed how research will be conducted in a data intensive environment is evolving.

Data Intensive Science↗

DOE OSTI2020

Data Driven Online Learning of Power System Dynamics

With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model.

Sinha, Subhrajit↗

DOE OSTI2026

Uncertain quantum computing futures and potential energy and physical resource impacts at scale

Considerable attention has recently focused on the vast energy and water demands of supercomputing, namely large-scale data centers that underpin artificial intelligence (AI), one of the great disruptors of contemporary society. Looking ahead some years from now, quantum computing is poised to disrupt established computing paradigms once again. Scientists and engineers are now working intensely to bring this century-old dream of physicists to fruition. Yet, as quantum computers begin to be integrated with classical supercomputing architectures, the implications for energy and physical resource use also need to be understood, especially how they compare to today’s AI data centers. These impacts have not yet been quantified by the research community – a notable gap in the literature, even if commercial-scale deployment of Quantum-Accelerated Computing Infrastructure (QuACI) is not expected for a few more years. This study is the first to conduct such an assessment. Using publicly available information from academic sources and private industry, we characterize multiple configurations of superconducting qubit-based, fault-tolerant quantum computers (FTQC) that could plausibly be deployed at scale in the 2030s and into the 2040s. By parameterizing these FTQC systems at a process level, we conduct a prospective scenario analysis to quantify their energy and physical resource needs. While these estimates are uncertain, given the current state of quantum technologies and their unknown future trajectories, important insights can already be drawn. One key finding is that while the electricity needs for a fleet of FTQCs are within the bounds of previous modeling studies that have explored high electricity demand futures, the needs for certain physical resources, namely water and helium-3, could pose bottlenecks to QuACI scale-up.

Computing↗

DOE OSTI2014

Big-Data RHEED analysis for understanding epitaxial film growth processes

Reflection high energy electron diffraction (RHEED) has by now become a standard tool for in-situ monitoring of film growth by pulsed laser deposition and molecular beam epitaxy. Yet despite the widespread adoption and wealth of information in RHEED image, most applications are limited to observing intensity oscillations of the specular spot, and much additional information on growth is discarded. With ease of data acquisition and increased computation speeds, statistical methods to rapidly mine the dataset are now feasible. Here, we develop such an approach to the analysis of the fundamental growth processes through multivariate statistical analysis of RHEED image sequence. This approach is illustrated for growth of LaxCa1-xMnO3 films grown on etched (001) SrTiO3 substrates, but is universal. The multivariate methods including principal component analysis and k-means clustering provide insight into the relevant behaviors, the timing and nature of a disordered to ordered growth change, and highlight statistically significant patterns. Fourier analysis yields the harmonic components of the signal and allows separation of the relevant components and baselines, isolating the assymetric nature of the step density function and the transmission spots from the imperfect layer-by-layer (LBL) growth. These studies show the promise of big data approaches to obtaining more insight into film properties during and after epitaxial film growth. Furthermore, these studies open the pathway to use forward prediction methods to potentially allow significantly more control over growth process and hence final film quality.

Big-Data↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

The DOE High Energy Physics (HEP) program in Neutrino and Collider science drives data-intensive science and simulation on extreme-scale platforms. Modeling and optimizing the complex distributed components from experimental to leadership computing facilities are essential for HEP workflows to achieve required response times and resilience under various conditions. Tachyon proposes a framework for scalable modeling, simulation, and validation of key performance characteristics for the distributed infrastructure between FNAL and ALCF, along with associated HEP workflows.

Carothers, Chris [Rensselaer Poly.]↗

DOE OSTI2026

Tachyon: Intelligent Multi-Scale Modeling of Distributed Resilient Infrastructure and Workflows for Data Intensive HEP Analyses

Carothers, Chris [Rensselaer Poly.]↗