Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Lead tungstate calorimeters at Jefferson Lab and perspectives for the Electron–Ion Collider

Electromagnetic calorimeters based on PbWO4 scintillating crystals have a widespread applica- tion in experiments at different accelerator facilities such as CERN, FNAL, GSI, and Jefferson Lab. The unique properties of PbWO4 crystals, including a small radiation length and Molire radius, make them ideal for building high-granularity, radiation-hard detectors. This enables excellent spa- tial separation and energy resolution of reconstructed electromagnetic showers, making PbWO4 crystals the material of choice for numerous experiments. Lead tungstate calorimeters have been successfully used in several experiments at Jefferson Lab. Two large-scale detectors have been re- cently fabricated for future experiments : the Neutral Particle Spectrometer and the lead tungstate calorimeter of the GlueX detector. The future application of PbWO4 crystals in the ElectronIon Collider further highlights their ongoing importance in advancing experimental capabilities. In planning new experiments, the development of calorimeter instrumentation technologies becomes paramount. The integration of modern photodetectors, such as Silicon photomultipliers that are capable of operating in strong magnetic fields, and the implementation of streaming readout data acquisition systems, sophisticated shower reconstruction algorithms, and real-time data analysis are some examples of the continuously growing requirements of experimental setups. I will give an overview of the lead tungstate scintillating calorimeters and discuss some recent advancement in the calorimeter instrumentation.

Somov, Alexander↗
NASA NTRS2009

Software Considerations for Subscale Flight Testing of Experimental Control Laws

The NASA AirSTAR system has been designed to address the challenges associated with safe and efficient subscale flight testing of research control laws in adverse flight conditions. In this paper, software elements of this system are described, with an emphasis on components which allow for rapid prototyping and deployment of aircraft control laws. Through model-based design and automatic coding a common code-base is used for desktop analysis, piloted simulation and real-time flight control. The flight control system provides the ability to rapidly integrate and test multiple research control laws and to emulate component or sensor failures. Integrated integrity monitoring systems provide aircraft structural load protection, isolate the system from control algorithm failures, and monitor the health of telemetry streams. Finally, issues associated with software configuration management and code modularity are briefly discussed.

Murch, Austin M.↗
NASA NTRS2009

On DESTINY Science Instrument Electrical and Electronics Subsystem Framework

Future space missions are going to require large focal planes with many sensing arrays and hundreds of millions of pixels all read out at high data rates'' . This will place unique demands on the electrical and electronics (EE) subsystem design and it will be critically important to have high technology readiness level (TRL) EE concepts ready to support such missions. One such omission is the Joint Dark Energy Mission (JDEM) charged with making precise measurements of the expansion rate of the universe to reveal vital clues about the nature of dark energy - a hypothetical form of energy that permeates all of space and tends to increase the rate of the expansion. One of three JDEM concept studies - the Dark Energy Space Telescope (DESTINY) was conducted in 2008 at the NASA's Goddard Space Flight Center (GSFC) in Greenbelt, Maryland. This paper presents the EE subsystem framework, which evolved from the DESTINY science instrument study. It describes the main challenges and implementation concepts related to the design of an EE subsystem featuring multiple focal planes populated with dozens of large arrays and millions of pixels. The focal planes are passively cooled to cryogenic temperatures (below 140 K). The sensor mosaic is controlled by a large number of Readout Integrated Circuits and Application Specific Integrated Circuits - the ROICs/ASICs in near proximity to their sensor focal planes. The ASICs, in turn, are serviced by a set of "warm" EE subsystem boxes performing Field Programmable Gate Array (FPGA) based digital signal processing (DSP) computations of complex algorithms, such as sampling-up-the-ramp algorithm (SUTR), over large volumes of fast data streams. The SUTR boxes are supported by the Instrument Control/Command and Data Handling box (ICDH Primary and Backup boxes) for lossless data compression, command and low volume telemetry handling, power conversion and for communications with the spacecraft. The paper outlines how the JDEM DESTINY concept instrument EE subsystem can be built now, a design; which is generally U.S. Government work not protected by U.S. copyright IEEEAC paper # 1429. Version 4. Updated October 19, 2009 applicable to a wide variety of missions using large focal planes with lar ge mosaics of sensors.

Kizhner, Semion↗
NASA NTRS2011

Enchanced interference cancellation and telemetry reception in multipath environments with a single paraboic dish antenna using a focal plane array

An Advanced Focal Plane Array ("AFPA") for parabolic dish antennas that exploits spatial diversity to achieve better channel equalization performance in the presence of multipath (better than temporal equalization alone), and which is capable of receiving from two or more sources within a field-of-view in the presence of multipath. The AFPA uses a focal plane array of receiving elements plus a spatio-temporal filter that keeps information on the adaptive FIR filter weights, relative amplitudes and phases of the incoming signals, and which employs an Interference Cancelling Constant Modulus Algorithm (IC-CMA) that resolves multiple telemetry streams simultaneously from the respective aero-nautical platforms. This data is sent to an angle estimator to calculate the target's angular position, and then on to Kalman filters FOR smoothing and time series prediction. The resulting velocity and acceleration estimates from the time series data are sent to an antenna control unit (ACU) to be used for pointing control.

Mukai, Ryan↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as time series data shifts. In this research, a changepoint detection (CPD) algorithm that automatically detects data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window across the labeled data sets. Pending approval, we plan to release the labeled data sets for this research on NREL's DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package. By supplying the training sets and algorithm, we hope to encourage further development in this research space.

data shift↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series: Preprint

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗
DOE OSTI2024

Stellar Population Properties in the Stellar Streams around SPRC047

Abstract We have investigated the properties (e.g., age, metallicity) of the stellar populations of a ringlike tidal stellar stream (or streams) around the edge-on galaxy SPRC047 (z= 0.031) using spectral energy distribution (SED) fits to integrated broadband aperture flux densities. We used visual images in six different bands and Spitzer/IRAC 3.6μm data. We have attempted to derive best-fit stellar population parameters (metallicity, age) in three noncontiguous segments of the stream. Due to the very low surface brightness of the stream, we have performed a deconvolution with a Richardson–Lucy–type algorithm of the low spatial resolution 3.6μm IRAC image, thereby reducing the effect of the point-spread function aliasedemissionfrom the bright edge-on central galaxy at the locations of our three stream segments. Our SED fits that used several different star formation (SF) history priors, from an exponentially decaying SF burst to continuous SF, indicate that the age–metallicity–dust degeneracy is not resolved, most likely because of inadequate wavelength coverage and low signal-to-noise ratios of the low surface brightness features. We also discuss how future deep visual–near-infrared observations, combined with absolute flux calibration uncertainties at or below the 1% level, complemented by equally well absolute flux-calibrated observations in ultraviolet and mid-infrared bands, would improve the accuracy of broadband SED fitting results for low surface brightness targets, such as stellar streams around nearby galaxies that are not resolved into stars.

Astronomy & Astrophysics↗
NASA NTRS2008

Comparison of the MODIS Multilayer Cloud Detection and Thermodynamic Phase Products with CALIPSO and CloudSat

CALIPSO and CloudSat, launched in June 2006, provide global active remote sensing measurements of clouds and aerosols that can be used for validation of a variety of passive imager retrievals derived from instruments flying on the Aqua spacecraft and other A-Train platforms. The most recent processing effort for the MODIS Atmosphere Team, referred to as the "Collection 5" stream, includes a research-level multilayer cloud detection algorithm that uses both thermodynamic phase information derived from a combination of solar and thermal emission bands to discriminate layers of different phases, as well as true layer separation discrimination using a moderately absorbing water vapor band. The multilayer detection algorithm is designed to provide a means of assessing the applicability of 1D cloud models used in the MODIS cloud optical and microphysical product retrieval, which are generated at a 1 h resolution. Using pixel-level collocations of MODIS Aqua, CALIOP, and CloudSat radar measurements, we investigate the global performance of the thermodynamic phase and multilayer cloud detection algorithms.

Platnick, Steven↗
NASA NTRS2018

The Convergence of Analysis Produced by Overlapping Assimilation Streams

When conducting a reanalysis for a long period, it is common to do so using multiple, parallel computational schemes. A period of overlap is included to foster continuity between successive streams. At the GMAO, the overlap period is one year. By the end of that period, the time mean anlysis for corresponding overlapped months are almost identical, as desired. The variances of differences at corresponding analysis times within each month, however, do not converge to zero. Instead their monthly variances converge to values that are a significant fraction of the the estimated variances of analysis errors. This occurs although the overlapping streams use the same observations, assimilation model, and assimilation algorithm, differing only in the background applied information applied at the beginning of the overlap period.

Errico, Ronald M.↗
DOE OSTI2026

Accelerating multigrid with streaming chiral SVD for Wilson fermions in lattice QCD

A modification to the setup algorithm for the multigrid preconditioner of Wilson fermions in lattice QCD is presented. A larger basis of test vectors than that used in regular multigrid is calculated by the smoother and truncated by singular value decomposition on the chiral components of the test vectors. The truncated basis is used to form the prolongation and restriction matrices of the multigrid hierarchy. This modification of the setup method is demonstrated to increase the convergence of linear solvers on an anisotropic lattice with m π ≈ 239 MeV from the Hadron Spectrum Collaboration and an isotropic lattice with m π ≈ 220 MeV from the MILC Collaboration. The lattice volume dependence of the method is also examined. Increasing the number of test vectors improves speedup up to a point, but storing these vectors becomes impossible in limited memory resources such as GPUs. To address storage cost, we implement a streaming singular value decomposition of the basis of test vectors on the chiral components and demonstrate a decrease in the number of fine level iterations by a factor of 1.7 for m q ≈ m crit

Iterative methods↗
NASA NTRS1989

Numerical simulation of three-dimensional transonic turbulent projectile aerodynamics by TVD schemes

The two-dimensional symmetric TVD scheme proposed by Yee has been extended to and investigated for three-dimensional thin-layer Navier-Stokes simulation of complex aerodynamic problems. An existing three-dimensional Navier-stokes code based on the beam and warming algorithm is modified to provide an option of using the TVD algorithm and the flow problem considered is a transonic turbulent flow past a projectile with sting at ten-degree angle of attack. Numerical experiments conducted for three flow cases, free-stream Mach numbers of 0.91, 0.96 and 1.20 show that the symmetric TVD algorithm can provide surface pressure distribution in excellent agreement with measured data; moreover, the rate of convergence to attain a steady state solution is about two times faster than the original beam and warming algorithm.

Shiau, Nae-Haur↗
DOE OSTI2025

Reconstruction framework advancements to support streaming for the ePIC detector at the EIC

The ePIC collaboration adopted the JANA2 framework to manage its reconstruction algorithms. This framework has since evolved substantially in response to ePIC’s needs. There have been three main design drivers: integrating cleanly with the Podio-based data models and other layers of the key4hep stack, enabling external configuration of existing components, and supporting timeframe splitting for streaming readout. The result is a unified component model featuring a new declarative interface for specifying inputs, outputs, parameters, services, and resources. This interface enables the user to instantiate, configure, and wire components via an external file. One critical new addition to the component model is a hierarchical decomposition of data boundaries into levels such as Run, Timeframe, PhysicsEvent, and Subevent. Two new component abstractions, Folder and Unfolder, are introduced in order to traverse this hierarchy, e.g. by splitting or merging. The pre-existing components can now operate at different event levels, and JANA2 will automatically construct the corresponding parallel processing topology. This means that a user may write an algorithm once, and configure it at runtime to operate on timeframes or on physics events. Overall, these changes mean that the user requires less knowledge about the framework internals, obtains greater flexibility with configuration, and gains the ability to reuse the existing abstractions in new streaming contexts.

Brei, Nathan [Thomas Jefferson National Accelerato↗
NASA NTRS2010

Multiple Kernel Learning for Heterogeneous Anomaly Detection: Algorithm and Aviation Safety Case Study

The world-wide aviation system is one of the most complex dynamical systems ever developed and is generating data at an extremely rapid rate. Most modern commercial aircraft record several hundred flight parameters including information from the guidance, navigation, and control systems, the avionics and propulsion systems, and the pilot inputs into the aircraft. These parameters may be continuous measurements or binary or categorical measurements recorded in one second intervals for the duration of the flight. Currently, most approaches to aviation safety are reactive, meaning that they are designed to react to an aviation safety incident or accident. In this paper, we discuss a novel approach based on the theory of multiple kernel learning to detect potential safety anomalies in very large data bases of discrete and continuous data from world-wide operations of commercial fleets. We pose a general anomaly detection problem which includes both discrete and continuous data streams, where we assume that the discrete streams have a causal influence on the continuous streams. We also assume that atypical sequence of events in the discrete streams can lead to off-nominal system performance. We discuss the application domain, novel algorithms, and also discuss results on real-world data sets. Our algorithm uncovers operationally significant events in high dimensional data streams in the aviation industry which are not detectable using state of the art methods

Das, Santanu↗
DOE OSTI2026

CONUS-wide Projected Flood Frequency and Uncertainty Estimates, Version 1.0

This dataset presents a large-ensemble of CONUS-wide projected flood frequency and uncertainty estimates across ~2.7 million NHDPlusV2 river reaches over the CONUS. The framework producing this dataset leverages a multi-model, uncertainty-aware modeling framework that allows evaluating shifts in flood frequences at the stream reach level across the CONUS. CONUS-wide ensemble streamflow projections generated from hydrologic simulations driven by downscaled and bias-corrected Coupled Model Intercomparison Project Phase 6 (CMIP6) outputs are used to derive these flood frequency and uncertainty estimates over the period 1980 - 2099. A spatially consistent regional L-moment algorithm is applied across clusters defined by the US Hydrologic Unit Code Subregions (HUC4s and HUC8s) and NHDPlusV2 stream orders to estimate flood frequencies. The dataset also includes at-site based flood estimates that allow for the comparison between local and regional approach-based estimates, assess projected changes, and characterize their uncertainties. For more reliable estimation of rare flood frequencies such as 500 and 1000-year return periods, super-ensemble based estimates are also included in the dataset. This dataset is derived to support the "Impact-Informed Dam Safety Risk Assessment for Securing Hydropower Assests" project for the US Department of Energy (DOE) Hydropower and Hydrokinetic Office (H2O). For further details, refer to Kao et al. (2022), Ghimire et al. (2023), Ghimire et al. (2025), and Hosking and Wallis (1997).

Ghimire, Ganesh [ORNL] (ORCID:0000000242843941)↗
NASA NTRS1981

Entropy encoding applied to a video DM bit stream

This paper investigates bandwidth compression of delta modulated video signals using a variety of novel techniques. The video signal is encoded at 16 MHz with an adaptive delta modulator (ADM) using the Song video mode algorithm. One method of achieving bandwidth compression is through the detection and reduction of steady state patterns in the DM bit stream. Further reduction is obtained by deleting the horizontal and vertical drive information. Finally, a modified ADM algorithm is introduced to increase the frequency of steady state patterns. When the modified ADM is combined with entropy coding a 30-50% data compression can be realized.

Barba, J.↗
NASA NTRS2014

Lightning Jump Algorithm Development for the GOES·R Geostationary Lightning Mapper

Current work on the lightning jump algorithm to be used in GOES‐R Geostationary Lightning Mapper (GLM)'s data stream is multifaceted due to the intricate interplay between the storm tracking, GLM proxy data, and the performance of the lightning jump itself. This work outlines the progress of the last year, where analysis and performance of the lightning jump algorithm with automated storm tracking and GLM proxy data were assessed using over 700 storms from North Alabama. The cases analyzed coincide with previous semi‐objective work performed using total lightning mapping array (LMA) measurements in Schultz et al. (2011). Analysis shows that key components of the algorithm (flash rate and sigma thresholds) have the greatest influence on the performance of the algorithm when validating using severe storm reports. Automated objective analysis using the GLM proxy data has shown probability of detection (POD) values around 60% with false alarm rates (FAR) around 73% using similar methodology to Schultz et al. (2011). However, when applying verification methods similar to those employed by the National Weather Service, POD values increase slightly (69%) and FAR values decrease (63%). The relationship between storm tracking and lightning jump has also been tested in a real‐time framework at NSSL. This system includes fully automated tracking by radar alone, real‐time LMA and radar observations and the lightning jump. Results indicate that the POD is strong at 65%. However, the FAR is significantly higher than in Schultz et al. (2011) (50‐80% depending on various tracking/lightning jump parameters) when using storm reports for verification. Given known issues with Storm Data, the performance of the real‐time jump algorithm is also being tested with high density radar and surface observations from the NSSL Severe Hazards Analysis & Verification Experiment (SHAVE).

Schultz. E.↗