Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,513 records · Page 84

Real-time forecasting of electricity demand in a streams-based architecture with applications

A streams platform is used. Multiple streams of electricity usage data are received, each from an electrical meter providing periodic updates to electrical usage for devices connected to the electrical meter. Weather information is received corresponding to locations where the electrical meters are. Real-time predictive modeling of electricity demand is performed based on the received multiple streams of electricity usage data and the received weather information, at least by performing: updating a state space model for electrical load curves using the usage data from the streams and the weather, wherein the updating uses current load observations for the multiple streams for a current time period; and creating forecast(s) for the electricity demand. The forecast(s) of the electricity demand are output. Appliance-level predictions may be made and used, and substitution effects and load management functions may be performed.

Ghosh, Soumyadip↗

The Pristine survey – XVI. The metallicity of 26 stellar streams around the Milky Way detected with the STREAMFINDER in Gaia EDR3

ABSTRACT We use the photometric metallicities provided by the panoramic Pristine survey to study the veracity and derive the metallicities of the numerous stellar streams found by the application of the STREAMFINDER algorithm to the Gaia Early Data Release 3 data. All 26 streams present in Pristine show a clear metallicity distribution function, which provides an independent check of the reality of these structures, supporting the reliability of STREAMFINDER in finding streams and the power of Pristine to measure precise metallicities. We further present six candidate structures with coherent phase-space and metallicity signals that are very likely streams. The majority of studied streams are very metal-poor (14 structures with [Fe/H] < −2.0) and include three systems with [Fe/H] < −2.9 (C-11, C-19, and C-20). These streams could be the closest debris of low-luminosity dwarf galaxies or may have originated from globular clusters of significantly lower metallicity than any known current Milky Way globular cluster. Our study shows that the promise of the Gaia data for Galactic Archeology studies can be substantially strengthened by quality photometric metallicities, allowing us to peer back into the earliest epochs of the formation of our Galaxy and its stellar halo constituents.

79 ASTRONOMY AND ASTROPHYSICS↗

Using integrated data analysis to extend measurement capability (invited)

The analysis approach called integrated data analysis (IDA) provides a means to exploit all information present in multiple streams of raw data to produce the best inference of a plasma parameter. This contrasts with the typical approach in which information (data) from a single diagnostic is used to measure a given parameter, e.g., visible bremsstrahlung → Z eff . Data from a given diagnostic usually contain information on many parameters. For example, a Thomson scattering diagnostic is sensitive to bremsstrahlung and line emission in addition to electron temperature. This background light is typically subtracted off and discarded but could be used to improve knowledge of Z eff . IDA encourages explicit awareness of such information and provides the quantitative framework to exploit it. This gives IDA the ability to increase spatial and temporal resolution, increase precision and accuracy of inferences, and measure plasma parameters that are difficult or impossible to measure using single diagnostic techniques. One example is the measurement of Z eff on Madison symmetric torus using IDA since no single diagnostic can provide a robust measurement. Thus, as we enter the burning plasma era, application of IDA will be critical to the measurement of certain parameters, as diagnostic access in the harsh fusion environment will be extremely limited.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

STUDIES OF MILLIMETER-WAVE ATMOSPHERIC NOISE ABOVE MAUNA KEA

We report measurements of the fluctuations in atmospheric emission (atmospheric noise) above Mauna Kea recorded with Bolocam at 143 and 268 GHz from the Caltech Submillimeter Observatory. The 143 GHz data were collected during a 40 night observing run in late 2003, and the 268 GHz observations were made in early 2004 and early 2005 over a total of 60 nights. Below approx =0.5 Hz, the data time-streams are dominated by atmospheric noise in all observing conditions. The atmospheric noise data are consistent with a Kolmogorov-Taylor turbulence model for a thin wind-driven screen, and the median amplitude of the fluctuations is 280 mK{sup 2} rad{sup -5/3} at 143 GHz and 4000 mK{sup 2} rad{sup -5/3} at 268 GHz. Comparing our results with previous ACBAR data, we find that the normalization of the power spectrum of the atmospheric noise fluctuations is a factor of approx =80 larger above Mauna Kea than above the South Pole at millimeter wavelengths. Most of this difference is due to the fact that the atmosphere above the South Pole is much drier than the atmosphere above Mauna Kea. However, the atmosphere above the South Pole is slightly more stable as well: the fractional fluctuations in the column depth of precipitable water vapor are a factor of approx =sq root2 smaller at the South Pole compared to Mauna Kea. Based on our atmospheric modeling, we developed several algorithms to remove the atmospheric noise, and the best results were achieved when we described the fluctuations using a low-order polynomial in detector position over the 8' field of view. However, even with these algorithms, we were not able to reach photon-background-limited instrument photometer performance at frequencies below approx =0.5 Hz in any observing conditions. We also observed an excess low-frequency noise that is highly correlated between detectors separated by approx<(f/number sign)lambda; this noise appears to be caused by atmospheric fluctuations, but we do not have an adequate model to explain its source. We hypothesize that the correlations arise from the classical coherence of the electromagnetic field across a distance of approx =(f/number sign)lambda on the focal plane.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

File Specification for GEOS-5 FP (Forward Processing)

The GEOS-5 FP Atmospheric Data Assimilation System (GEOS-5 ADAS) uses an analysis developed jointly with NOAA's National Centers for Environmental Prediction (NCEP), which allows the Global Modeling and Assimilation Office (GMAO) to take advantage of the developments at NCEP and the Joint Center for Satellite Data Assimilation (JCSDA). The GEOS-5 AGCM uses the finite-volume dynamics (Lin, 2004) integrated with various physics packages (e.g, Bacmeister et al., 2006), under the Earth System Modeling Framework (ESMF) including the Catchment Land Surface Model (CLSM) (e.g., Koster et al., 2000). The GSI analysis is a three-dimensional variational (3DVar) analysis applied in grid-point space to facilitate the implementation of anisotropic, inhomogeneous covariances (e.g., Wu et al., 2002; Derber et al., 2003). The GSI implementation for GEOS-5 FP incorporates a set of recursive filters that produce approximately Gaussian smoothing kernels and isotropic correlation functions. The GEOS-5 ADAS is documented in Rienecker et al. (2008). More recent updates to the model are presented in Molod et al. (2011). The GEOS-5 system actively assimilates roughly 2 × 10(exp 6) observations for each analysis, including about 7.5 × 10(exp 5) AIRS radiance data. The input stream is roughly twice this volume, but because of the large volume, the data are thinned commensurate with the analysis grid to reduce the computational burden. Data are also rejected from the analysis through quality control procedures designed to detect, for example, the presence of cloud. To minimize the spurious periodic perturbations of the analysis, GEOS-5 FP uses the Incremental Analysis Update (IAU) technique developed by Bloom et al. (1996). More details of this procedure are given in Appendix A. The assimilation is performed at a horizontal resolution of 0.3125-degree longitude by 0.25- degree latitude and at 72 levels, extending to 0.01 hPa. All products are generated at the native resolution of the horizontal grid. The majority of data products are time-averaged, but four instantaneous products are also available. Hourly data intervals are used for two-dimensional products, while 3-hourly intervals are used for three-dimensional products. These may be on the model's native 72-layer vertical grid or at 42 pressure surfaces extending to 0.1 hPa. This document describes the gridded output files produced by the GMAO near real-time operational FP, using the most recent version of the GEOS-5 assimilation system. Additional details about variables listed in this file specification can be found in a separate document, the GEOS-5 File Specification Variable Definition Glossary. Documentation about the current access methods for products described in this document can be found on the GMAO products page: http://gmao.gsfc.nasa.gov/products/.

GSI↗

File Specification for GEOS-5 FP-IT (Forward Processing for Instrument Teams)

The GEOS-5 FP-IT Atmospheric Data Assimilation System (GEOS-5 ADAS) uses an analysis developed jointly with NOAA's National Centers for Environmental Prediction (NCEP), which allows the Global Modeling and Assimilation Office (GMAO) to take advantage of the developments at NCEP and the Joint Center for Satellite Data Assimilation (JCSDA). The GEOS-5 AGCM uses the finite-volume dynamics (Lin, 2004) integrated with various physics packages (e.g, Bacmeister et al., 2006), under the Earth System Modeling Framework (ESMF) including the Catchment Land Surface Model (CLSM) (e.g., Koster et al., 2000). The GSI analysis is a three-dimensional variational (3DVar) analysis applied in grid-point space to facilitate the implementation of anisotropic, inhomogeneous covariances (e.g., Wu et al., 2002; Derber et al., 2003). The GSI implementation for GEOS-5 FP-IT incorporates a set of recursive filters that produce approximately Gaussian smoothing kernels and isotropic correlation functions. The GEOS-5 ADAS is documented in Rienecker et al. (2008). More recent updates to the model are presented in Molod et al. (2011). The GEOS-5 system actively assimilates roughly 2 × 10(exp 6) observations for each analysis, including about 7.5 × 10(exp 5) AIRS radiance data. The input stream is roughly twice this volume, but because of the large volume, the data are thinned commensurate with the analysis grid to reduce the computational burden. Data are also rejected from the analysis through quality control procedures designed to detect, for example, the presence of cloud. To minimize the spurious periodic perturbations of the analysis, GEOS-5 FP-IT uses the Incremental Analysis Update (IAU) technique developed by Bloom et al. (1996). More details of this procedure are given in Appendix A. The analysis is performed at a horizontal resolution of 0.625-degree longitude by 0.5-degree latitude and at 72 levels, extending to 0.01 hPa. All products are generated at the native resolution of the horizontal grid. The majority of data products are time-averaged, but four instantaneous products are also available. Hourly data intervals are used for two-dimensional products, while 3-hourly intervals are used for three-dimensional products. These may be on the model's native 72-layer vertical grid or at 42 pressure surfaces extending to 0.1 hPa. This document describes the gridded output files produced by the GMAO near real-time operational GEOS-5 FP-IT processing in support of the EOS instrument teams. Additional details about variables listed in this file specification can be found in a separate document, the GEOS-5 File Specification Variable Definition Glossary.

3D Var↗

Data Archive and Portal (DAP) Platform for Solid Phase Processing Technologies

The scale and speed of data generated by modern scientific experiments have constantly challenged the research community to store, curate, manage and optimally use it to drive scientific discoveries. In this work, we have developed a data archive and portal (DAP) platform including analytics capabilities to collect, curate, and manage data and metadata stream for solid phase processing (SPP) techniques. We successfully hosted around ~347K files of data related to processing parameters, microscopic images, and spectroscopic data related to solid phase processing. The DAP platform for SPP will establish an enduring capability to support machine learning and grow collaboration at the intersection of materials science and data science.

36 MATERIALS SCIENCE↗

Seasat altimeter observations of dynamic topography in the Gulf Stream region

A straightforward approach to altimeter data analysis in the Gulf Stream system is presented, using a detailed geoid model to remove the gravitational component. The resulting sea surface height profiles compare remarkably well with independent oceanographic observations. Specific features such as cold rings, warm rings, and no anomaly regions are analyzed and it is shown that known positions of cyclonic and anticyclonic rings correspond with depressions and elevations, respectively, with amplitudes as large as 95 cm. The apparent fluctuation of the Gulf Stream is indicated by the results, as in the finding that on time scales of a few days, surface transport indicated by the sea surface height difference across the stream varied by nearly 30%

Cheney, R. E.↗

Semi–Analytical Modeling of Transient Stream Drawdown and Depletion in Response to Aquifer Pumping

Analytical and semi–analytical models for stream depletion with transient stream stage drawdown induced by groundwater pumping are developed to address a deficiency in existing models, namely, the use of a fixed stream stage condition at the stream–aquifer interface. Here field data are presented to demonstrate that stream stage drawdown does indeed occur in response to groundwater pumping near aquifer–connected streams. A model that predicts stream depletion with transient stream drawdown is developed based on stream channel mass conservation and finite stream channel storage. The resulting models are shown to reduce to existing fixed–stage models in the limit as stream channel storage becomes infinitely large, and to the confined aquifer flow with a no–flow boundary at the streambed in the limit as stream storage becomes vanishingly small. The model is applied to field measurements of aquifer and stream drawdown, giving estimates of aquifer hydraulic parameters, streambed conductance, and a measure of stream channel storage. The results of the modeling and data analysis presented herein have implications for sustainable groundwater management.

54 ENVIRONMENTAL SCIENCES↗

Coolant tube curvature effects on film cooling as detected by infrared imagery

Reported herein are comparative thermal film cooling footprints observed by infrared imagery from straight, curved and looped coolant tube geometries. It was hypothesized that the difference in secondary flow and turbulence structure of flow through these three tubes should influence the mixing properties between the coolant and mainstream. The coolant was injected across an adiabatic plate through a hole angled at 30 deg to the surface in line with the free stream flow. The data cover a range of blowing rates from 0.37 to 1.25 (mass flow per unit area of coolant divided by free stream). Average temperature difference between coolant and tunnel air was 25 C. Data comparisons confirmed that coolant tube curvature significantly influences film cooling effectiveness.

Papell, S. S.↗

Noise Prediction Module for Offset Stream Nozzles

A Modern Design of Experiments (MDOE) analysis of data acquired for an offset stream technology was presented. The data acquisition and concept development were funded under a Supersonics NRA NNX07AC62A awarded to Dimitri Papamoschou at University of California, Irvine. The technology involved the introduction of airfoils in the fan stream of a bypass ratio (BPR) two nozzle system operated at transonic exhaust speeds. The vanes deflected the fan stream relative to the core stream and resulted in reduced sideline noise for polar angles in the peak jet noise direction. Noise prediction models were developed for a range of vane configurations. The models interface with an existing ANOPP module and can be used or future system level studies.

Henderson, Brenda S.↗

Fiber Optic Tactical Local Network (FOTLAN)

A 100 Mbit/s FDDI (fiber distributed data interface) network interface unit is described that supports real-time data, voice and video. Its high-speed interrupt-driven hardware architecture efficiently manages stream and packet data transfer to the FDDI network. Other enhancements include modular single-mode laser-diode fiber optic links to maximize node spacing, optic bypass switches for increased fault tolerance, and a hardware performance monitor to gather real-time network diagnostics.

Bergman, L. A.↗

Data and script associated with “Shifts in Rain-Snow Partitioning Drive Faster Water Transit Times in the US Pacific Northwest”

This data package contains the data and code to use and run the Water Tracer enabled version of the Weather Research and Forecasting Hydrologic model (WT-WRF-Hydro) with the Sequential Precipitation Input Tagging (SPIT) framework. It is associated with the publication “Shifts in Rain-Snow Partitioning Drive Faster Water Transit Times in the US Pacific Northwest” published in Scientific Reports (Butler et al., 2026; https://doi.org/10.1038/s41598-026-46539-1). We use the Continental U.S. (CONUSII; Rasmussen et al., 2021) dataset to force the model with an historical climate (2006–2013) and a future climate (2086–2093) with a representative carbon pathway (RCP) 8.5 scenario. We use the model to calculate water transit times in five headwater catchments within the U.S. Pacific Northwest. We also show key hydrologic and environmental variables that affect water transit times and changes in the future. Finally, we use observed data to validate the model such as stream water isotopes, snowpack characteristics, and stream discharge. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. The data package consists of 11 folders: (1) "Figures" contains the exported figures used in the manuscript; (2) "Model_Isotope_Date" contains the WT-WRF-Hydro isotope date used in model validation; (3) “Model_Outputs_Future” contains the WT-WRF-Hydro future climate outputs; (4) “Model_Outputs_Historical” contains the WT-WRF-Hydro historical climate outputs; (5) “Model_Outputs_Weights_Areas” contains the WT-WRF-Hydro weights per catchment used to calculate water transit times and isotopes in stream water; (6) “MODIS_data_scripts” contains data used to validate snow conditions in the study area; (7) “Observed_Flow_Data” contains the observed streamflow data used in model validation; (8) “Observed_Isotope_Data” contains the observed stream water isotope data used in model validation; (9) “Scripts” contains the Python scripts used to general results and the figures; (10) “Statistic_Outputs” contains the water transit time statistical outputs reported in this manuscript; (11) “Validation_SNOTEL” contains the SNOTEL data used in model validation. The files in this data package have the following file extensions: .tif, .txt, .csv, .pdf, .py, .jpg, and .png.

American River↗

A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization

There presently exist two dominant strategies to reduce data sizes in analysis and visualization: reducing the precision of the data, e.g., through quantization, or reducing its resolution, e.g., by subsampling. Both have advantages and disadvantages and both face fundamental limits at which the reduced information ceases to be useful. The paper explores the additional gains that could be achieved by combining both strategies. In particular, we present a common framework that allows us to study the trade-off in reducing precision and/or resolution in a principled manner. We represent data reduction schemes as progressive streams of bits and study how various bit orderings such as by resolution, by precision, etc., impact the resulting approximation error across a variety of data sets as well as analysis tasks. Additionally, we compute streams that are optimized for different tasks to serve as lower bounds on the achievable error. Scientific data management systems can use the results proposed in this paper as guidance on how to store and stream data to make efficient use of the limited storage and bandwidth in practice.

97 MATHEMATICS AND COMPUTING↗

Insights from Smart Meters. Ramp-up, dependability, and short-term persistence of savings from Home Energy Reports

Smart meters, smart thermostats, and other new technologies provide previously unavailable high-frequency and location-specific energy usage data. Many utilities are now able to capture real-time, customer specific hourly interval usage data for a large proportion of their residential and small commercial customers. These vast, constantly growing streams of rich data (or, “big data”) have the potential to provide novel insights into key policy questions about how people make energy decisions. The richness and granularity of these data enable many types of creative and cutting-edge analytics. Technically sophisticated and rigorous statistical techniques can be used to pull useful insights out of this high-frequency, human-focused data. In this series, we call this “behavior analytics.” This kind of analytics has the potential to provide tremendous value to a wide range of energy programs. For example, disaggregated and heterogeneous information about actual energy use allows energy efficiency (EE) and/or demand response (DR) program implementers to target specific programs to specific households; enables evaluation, measurement and verification (EM&V) of energy efficiency programs to be performed on a much shorter time horizon than was previously possible; and may provide better insights into the energy and peak hour savings associated with EE and DR programs (e.g., behavior-based (BB) programs). The goal of this series is to enable evidence-based and data-driven decision making by policy makers and industry stakeholders, including program planners, program administrators, utilities, state regulatory agencies, and evaluators. We focus on research findings that are immediately relevant.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Insights from Smart Meters: Ramp-Up, Dependability, and Short-Term Persistence of Savings from Home Energy Reports

Smart meters, smart thermostats, and other new technologies provide previously unavailable high-frequency and location-specific energy usage data. Many utilities are now able to capture real-time, customer-specific hourly interval usage data for a large proportion of their residential and small commercial customers. These vast, constantly growing streams of rich data (or big data) have the potential to provide novel insights into key policy questions about how people make energy decisions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A New Source of Space Weather Data

The twin STEREO spacecraft were launched on October 26,2006 and, after spending several weeks in alignment orbits, became operational in heliocentric orbits on January 2 1,2007. The spacecraft are now in orbits similar to Earth's and are separating from each other at a rate of about 45 degrees per year as viewed from the sun. The STEREO spacecraft are each equipped with a complement of EUV, and white light imagers as well as particles and fields in situ instruments and a radio burst tracker. In addition to full resolution data downloaded once per day, each spacecraft broadcasts a real time telemetry 'beacon' stream containing compressed images and typically one minute averages of solar wind measurements and radio intensities. These real time data streams are received by a mix of NOAA and NASA-managed tracking stations and the data is transmitted by these stations to the STEREO Science Center at NASA Goddard Space Flight Center where they are processed into useable parameters with a latency of less than five minutes from receipt. As the two spacecraft recede from the sun-Earth line, triangulation of the imaging and radio measurements should allow better estimate of the speed and direction of coronal mass ejections, allowing a more precise estimate of their arrival at Earth. In situ particles and fields measurements, particularly for the spacecraft (Behind) trailing Earth in its orbit, will sample the solar wind that will eventually co-rotate around to the sun-Earth line. Late in the prime mission when the spacecraft are very far (>90 degrees) from the sun-Earth line, observations from the Behind spacecraft, will be particularly valuable for providing a preview of active regions well beyond the sun's East limb (as seen from Earth). Continually updated STEREO space weather data can be viewed and downloaded at http:/lstereo-ssc.nascom.nasa.govldata/beacon/.

Kaiser, Michael L.↗