Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS1980

Algorithm for calculating turbine cooling flow and the resulting decrease in turbine efficiency

An algorithm is presented for calculating both the quantity of compressor bleed flow required to cool the turbine and the decrease in turbine efficiency caused by the injection of cooling air into the gas stream. The algorithm, which is intended for an axial flow, air routine in a properly written thermodynamic cycle code. Ten different cooling configurations are available for each row of cooled airfoils in the turbine. Results from the algorithm are substantiated by comparison with flows predicted by major engine manufacturers for given bulk metal temperatures and given cooling configurations. A list of definitions for the terms in the subroutine is presented.

Gauntner, J. W.↗
NASA NTRS1997

Departure Trajectory Synthesis and the Intercept Problem

Two areas of the departure problem in air traffic control are discussed. The first topic is the generation of climb-out trajectories to a fix. The trajectories would be utilized by a scheduling algorithm to allocate runways, sequence the proposed departures, and assign a departure time. The second area is concerned with finding horizontal trajectories to merge aircraft from the TRACON to an open slot in the en-route environment. Solutions are presented for the intercept problem for two cases: (1) the aircraft is traveling at the speed of the aircraft in the jetway; (2) the merging aircraft has to accelerate to reach the speed of the aircraft in the en-route stream. An algorithm is given regarding the computation of a solution for the latter case. For the former, a set of equations is given that allows us to numerically solve for the coordinate where the merge will occur.

Bolender, Michael A.↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
DOE OSTI2022

Machine learning on FPGA for event selection

Real-time data processing is a frontier field in experimental particle physics. The application of FPGAs at the trigger level is used by many current and planned experiments (CMS, LHCb, Belle2, PANDA). Usually they use conventional processing algorithms. LHCb has implemented Machine Learning (ML) elements for real-time data processing with a triggered readout system that runs most of the ML algorithms on a computer farm. The work described in this article aims to test the ML-FPGA algorithms for streaming data acquisition. Herein, there are many experiments working in this area and they have a lot in common, but there are many specific solutions for detector and accelerator parameters that are worth exploring further. This report describes the purpose of the work and progress in evaluating the ML-FPGA application.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono↗
NASA NTRS2021

Algorithm Theoretical Basis Document (ATBD) - Stream Stage Measurements: V2.5.1 Water Level Products from Satellite Radar Altimetry

In response to the 2018 NASA ROSES Applied Sciences/Water Resources (NASA HQ Program Official: Dr. Brad Doorn) call for proposals, the “Integration of Remotely Sensed Streamflow Data into Alaska Water Resource Management Agency Operations” project with Principal Investigator (PI) Jack Eggleston USGS, was successful, and had the ultimate goal of creating a series of remotely sensed or derived Alaska river parameters for integration into NWIS. These parameters included surface water height and average reach surface water slope (from altimetry), average reach width (from Landsat imagery), and an associated river discharge derived via theoretical means. The surface water height products were required to have both archival and near real time components, noting the availability of ~25years of potential measurements, and accepting the temporal resolution (10-35days) of the suite of radar altimeters. Each surface water level product was expected to be a continuous time series of observation with a sufficient accuracy to highlight monthly, seasonal and interannual variation. The designated set of river reaches were chosen for their geographical distribution, their reach width, and the presence of a radar altimeter mission satellite overpass. This document describes the procedure associated with the creation of these altimetric surface water level products and is relevant to product Version 2.5.1 available from the Global Water Monitor (GWM) web portal.

Altimetry↗
DOE OSTI2020

Scheduling and Control of Flexible Building Loads for Grid Services Based on a Virtual Battery Model

This paper presents a framework for modeling, scheduling, and controlling residential thermostatically controlled loads (TCLs) to provide multiple grid services, such as energy shifting, peak load reduction, and ancillary services. A modeling method is proposed to characterize the aggregate flexibility from heterogeneous TCLs using a battery-equivalent model. Based on the flexibility model, a multi-period optimal scheduling formulation is developed to best utilize the flexibility from building loads and maximize total benefits from stacked value streams. An algorithm is proposed to control individual TCLs to follow the desired power consumption in real-time. The proposed methods are illustrated and validated through simulations.

Wu, Di↗
DOE OSTI2022

A dictionary learning algorithm for compression and reconstruction of streaming data in preset order

There has been an emerging interest in developing and applying dictionary learning (DL) to process massive datasets in the last decade. Many of these efforts, however, focus on employing DL to compress and extract a set of important features from data, while considering restoring the original data from this set a secondary goal. On the other hand, although several methods are able to process streaming data by updating the dictionary incrementally as new snapshots pass by, most of those algorithms are designed for the setting where the snapshots are randomly drawn from a probability distribution. In this paper, we present a new DL approach to compress and denoise massive dataset in real time, in which the data are streamed through in a preset order (instances are videos and temporal experimental data), so at any time, we can only observe a biased sample set of the whole data. Here, our approach incrementally builds up the dictionary in a relatively simple manner: if the new snapshot is adequately explained by the current dictionary, we perform a sparse coding to find its sparse representation; otherwise, we add the new snapshot to the dictionary, with a Gram-Schmidt process to maintain the orthogonality. To compress and denoise noisy datasets, we apply the denoising to the snapshot directly before sparse coding, which deviates from traditional dictionary learning approach that achieves denoising via sparse coding. Compared to full-batch matrix decomposition methods, where the whole data is kept in memory, and other mini-batch approaches, where unbiased sampling is often assumed, our approach has minimal requirement in data sampling and storage: i) each snapshot is only seen once then discarded, and ii) the snapshots are drawn in a preset order, so can be highly biased. Through experiments on climate simulations and scanning transmission electron microscopy (STEM) data, we demonstrate that the proposed approach performs competitively to those methods in data reconstruction and denoising.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗
DOE OSTI2024

Kernelized approaches to streaming compression of scientific data

In this paper three algorithms are developed for the streaming compression of scientific data. The algorithms presented are reliant on the theory of vector-valued reproducing kernel Hilbert spaces and operator valued kernel. Further, the scientific data is modeled as a snapshot of time dependent vector field F(x, t) over a manifold M and the recovery of the data is framed as a learning problem. These processes are then appropriately modified and ana lyzed for the streaming scenario in which data is generated without the ability to revisit past entries.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2002

Navigation Algorithms for the SeaWiFS Mission

The navigation algorithms for the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) were designed to meet the requirement of 1-pixel accuracy-a standard deviation (sigma) of 2. The objective has been to extract the best possible accuracy from the spacecraft telemetry and avoid the need for costly manual renavigation or geometric rectification. The requirement is addressed by postprocessing of both the Global Positioning System (GPS) receiver and Attitude Control System (ACS) data in the spacecraft telemetry stream. The navigation algorithms described are separated into four areas: orbit processing, attitude sensor processing, attitude determination, and final navigation processing. There has been substantial modification during the mission of the attitude determination and attitude sensor processing algorithms. For the former, the basic approach was completely changed during the first year of the mission, from a single-frame deterministic method to a Kalman smoother. This was done for several reasons: a) to improve the overall accuracy of the attitude determination, particularly near the sub-solar point; b) to reduce discontinuities; c) to support the single-ACS-string spacecraft operation that was started after the first mission year, which causes gaps in attitude sensor coverage; and d) to handle data quality problems (which became evident after launch) in the direct-broadcast data. The changes to the attitude sensor processing algorithms primarily involved the development of a model for the Earth horizon height, also needed for single-string operation; the incorporation of improved sensor calibration data; and improved data quality checking and smoothing to handle the data quality issues. The attitude sensor alignments have also been revised multiple times, generally in conjunction with the other changes. The orbit and final navigation processing algorithms have remained largely unchanged during the mission, aside from refinements to data quality checking. Although further improvements are certainly possible, future evolution of the algorithms is expected to be limited to refinements of the methods presented here, and no substantial changes are anticipated.

Hooker, Stanford B.↗
NASA NTRS1992

An architecture for the development of real-time fault diagnosis systems using model-based reasoning

Presented here is an architecture for implementing real-time telemetry based diagnostic systems using model-based reasoning. First, we describe Paragon, a knowledge acquisition tool for offline entry and validation of physical system models. Paragon provides domain experts with a structured editing capability to capture the physical component's structure, behavior, and causal relationships. We next describe the architecture of the run time diagnostic system. The diagnostic system, written entirely in Ada, uses the behavioral model developed offline by Paragon to simulate expected component states as reflected in the telemetry stream. The diagnostic algorithm traces causal relationships contained within the model to isolate system faults. Since the diagnostic process relies exclusively on the behavioral model and is implemented without the use of heuristic rules, it can be used to isolate unpredicted faults in a wide variety of systems. Finally, we discuss the implementation of a prototype system constructed using this technique for diagnosing faults in a science instrument. The prototype demonstrates the use of model-based reasoning to develop maintainable systems with greater diagnostic capabilities at a lower cost.

Hall, Gardiner A.↗
DOE OSTI2021

via machinae : Searching for stellar streams using unsupervised machine learning

ABSTRACT We develop a new machine learning algorithm, via machinae, to identify cold stellar streams in data from the Gaia telescope. via machinae is based on ANODE, a general method that uses conditional density estimation and sideband interpolation to detect local overdensities in the data in a model agnostic way. By applying ANODE to the positions, proper motions, and photometry of stars observed by Gaia, via machinae obtains a collection of those stars deemed most likely to belong to a stellar stream. We further apply an automated line-finding method based on the Hough transform to search for line-like features in patches of the sky. In this paper, we describe the via machinae algorithm in detail and demonstrate our approach on the prominent stream GD-1. Though some parts of the algorithm are tuned to increase sensitivity to cold streams, the via machinae technique itself does not rely on astrophysical assumptions, such as the potential of the Milky Way or stellar isochrones. This flexibility suggests that it may have further applications in identifying other anomalous structures within the Gaia data set, for example debris flow and globular clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1990

Measuring and modeling near-surface reflected and emitted radiation fluxes at the FIFE site

Information is presented pertaining to the measurement and estimation of reflected and emitted components of the radiation balance. Information is included about reflectance and transmittance of solar radiation from and through the leaves of some grass and forb prairie species, bidirectional reflectance from a prairie canopy is discussed and measured and estimated fluxes are described of incoming and outgoing longwave and shortwave radiation. Results of the study showed only very small differences in reflectances and transmittances for the adaxial and abaxial surfaces of grass species in the visible and infrared wavebands, but some differences in the infrared wavebands were noted for the forbs. Reflectance from the prairie canopy changed as a function of solar and view zenith angles in the solar principal plane with definite asymmetry about nadir. The surface temperature of prairie canopies was found to vary by as much as 5 C depending on view zenith and azimuth position and on the solar azimuth. Aerodynamic temperature calculated from measured sensible heat fluxes ranged from 0 to 3 C higher than nadir-viewed temperatures. Models were developed to estimate incoming and reflected shortwave radiation from data collected with a Barnes Modular Multiband Radiometer. Several algorithms for estimating incoming longwave radiation were evaluated and compared to actual measures of that parameter. Net radiation was calculated using the estimated components of the shortwave radiation streams, determined from the algorithms developed, and from the longwave radiation streams provided by the Brunt, modified Deacon, and the Stefan-Boltzmann models. Estimates of net radiation were compared to measured values and found to be within the measurement error of the net radiometers used in the study.

Blad, Blaine L.↗
NASA NTRSDate not supplied

Generation of Land Surface Reflectance with Combined Geo-KOMPSAT-2A AMI and Himawari 8 AHI Observations

The latest generation of geostationary satellites has opened a new era of Earth observations with unprecedented spatiotemporal resolution and spectral range. Together with GOES 16/17 ABI, FY4-A AGRI, and Himawari-8 AHI, a new Korean geostationary satellite (Geo-KOMPSAT-2A AMI) has operationally collected a full-disk image in 16 channels every ten minutes since July 2019, allowing diurnal land surface monitoring over a large proportion of Asia and all of Oceania. Retrieving accurate surface reflectance (SR) over land from GK-2A/AMI is a challenging but high priority objective. One of the challenges is the absence of a spectral band in the 2.2 m SWIR range from AMI, which is required by many atmospheric correction algorithms to retrieve atmospheric aerosol properties. To remedy this issue, we adopt a strategy that combines concurrent GK-2A/AMI and Himawari 8/AHI observations in order to derive AMI SR. We have adapted the NASA Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to process the data stream from Himawari 8/AHI. The advantages of the MAIAC algorithm is its capability to exploit the high temporal frequency and varying illumination geometry of the geostationary data for advanced cloud/snow detection, aerosol retrieval, and characterization of surface bidirectional reflectance properties. Leveraging the similarities of spectral bands and the sun-target-sensor geometry between AMI and AHI, we are able to create denser time series of observations and enhanced BRDF samples over most of the spatial coverage of AMI (and AHI). The combined stereo-type observations not only help derive SR for AMI but also enhance retrievals of the corresponding AHI surface products. We evaluate the resulting AMI SR using ground (AERONET) observations and corresponding MODIS products. Further, we discuss potential challenges in utilizing the geostationary satellite data for land surface monitoring.

geostationary satellite↗
NASA NTRS2011

Accelerated Adaptive MGS Phase Retrieval

The Modified Gerchberg-Saxton (MGS) algorithm is an image-based wavefront-sensing method that can turn any science instrument focal plane into a wavefront sensor. MGS characterizes optical systems by estimating the wavefront errors in the exit pupil using only intensity images of a star or other point source of light. This innovative implementation of MGS significantly accelerates the MGS phase retrieval algorithm by using stream-processing hardware on conventional graphics cards. Stream processing is a relatively new, yet powerful, paradigm to allow parallel processing of certain applications that apply single instructions to multiple data (SIMD). These stream processors are designed specifically to support large-scale parallel computing on a single graphics chip. Computationally intensive algorithms, such as the Fast Fourier Transform (FFT), are particularly well suited for this computing environment. This high-speed version of MGS exploits commercially available hardware to accomplish the same objective in a fraction of the original time. The exploit involves performing matrix calculations in nVidia graphic cards. The graphical processor unit (GPU) is hardware that is specialized for computationally intensive, highly parallel computation. From the software perspective, a parallel programming model is used, called CUDA, to transparently scale multicore parallelism in hardware. This technology gives computationally intensive applications access to the processing power of the nVidia GPUs through a C/C++ programming interface. The AAMGS (Accelerated Adaptive MGS) software takes advantage of these advanced technologies, to accelerate the optical phase error characterization. With a single PC that contains four nVidia GTX-280 graphic cards, the new implementation can process four images simultaneously to produce a JWST (James Webb Space Telescope) wavefront measurement 60 times faster than the previous code.

Lam, Raymond K.↗