Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Enhancements and Deployment of the TDAQ System for the Mu2e Experiment

The Real Time Processing Systems Division at Fermilab has deployed new features to the Off-The-Shelf Data Acquisition framework (otsdaq) for the Mu2e experiment. The Mu2e experiment will search for the coherent neutrino-less conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement of 10,000 times over existing limits. Such a charged lepton flavor-violating reaction probes new physics at a scale unavailable at present or planned high-energy colliders. The Mu2e Trigger and Data Acquisition (TDAQ) system uses otsdaq as its online Data Acquisition System (DAQ) framework. otsdaq integrates the artdaq and art frameworks for event transfer, filtering, and processing. otsdaq is a web-based DAQ software suite focusing on flexibility and scalability and provides a multi-user interface accessible through a web browser. artdaq handles the entire data stream, which is read over the peripheral component interconnect express (PCIe) bus to a software filter algorithm that selects events combined with the data flux coming from a cosmic-ray veto (CRV) system. Detector front-ends are configured through the PCIe bus by customized otsdaq plugins. The otsdaq slow controls infrastructure has been further developed using the experimental physics and industrial control system (EPICS) open-source platform for monitoring, controlling, alarming, and archiving. The detector control system (DCS) for Mu2e has been integrated into otsdaq. The production TDAQ and DCS system has been deployed at the experimental hall and is being debugged and optimized for experiment operations. We report on the feature enhancements and deployment of otsdaq for Mu2e.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Building Battery Energy Storage System Performance Data into an Economic Assessment

Economic assessments of battery energy storage systems (BESSs) rely on optimal dispatch methods to capture temporal interdependency of BESS operations and coupling among different grid applications. Many existing BESS economic assessment studies are based on a simplified first-order scalar model using BESS nameplate numbers. Such a method cannot accurately capture varying capabilities and nonlinear dynamics of a BESS, leading to inaccurate assessment results. This paper presents a practical implementation of performance data into the economic assessment of the BESSs recently developed within the Snohomish Public Utility District system in Washington State. Extensive testing conducted over multiple seasons is used to assess the technical performance and develop high-fidelity models of the BESSs. A dynamic programming algorithm is proposed to optimally dispatch the BESSs and assess the potential benefits from stacked value streams. The proposed method can be adapted for the economic assessment of other BESSs with different applications.

Wu, Di↗
DOE OSTI2022

Comparing Calibration Algorithms for the Rapid Characterization of Pretreated Corn Stover Using Near-Infrared Spectroscopy

Rapid characterization of biomass composition is a key enabling technology for biorefineries—the ability to measure the chemical composition of biomass materials entering the biorefinery as well as the composition of key process intermediate streams would allow real-time process control and the development of robust models to predict process performance. The utility of near-infrared (NIR) spectroscopy for rapid characterization requires multivariate algorithms for building calibration models. The most prevalent algorithm used for building calibration models using NIR spectra is the linear modeling algorithm Partial Least Squares Regression (PLS). Nonlinear regression algorithms (which are typically more computationally intensive than linear modeling approaches) have gained popularity in recent years due to their ability to solve a wide variety of classification and regression problems and the dramatic increase in available computational resources. In this work, we demonstrate that a calibration model can predict the composition of corn stover process intermediate samples pretreated with three different treatments—hot water (HW), dilute acid (DA), and deacetylation followed by dilute acid (DDA). We quantitatively compare three different algorithms for building prediction models based on near-infrared spectroscopy—partial least squares (PLS), support vector machines (SVM), and random forests (RF). We demonstrate the utility of improving model performance by accounting for instrument performance variability using repeated measurements of standard materials (e.g., the “repeatability file” strategy) and investigate its performance with nonlinear regression techniques, and we discuss methods for quantifying the uncertainties of specific predictions among the three methods.

09 BIOMASS FUELS↗
DOE OSTI2023

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real- world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5- based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

azimuth↗
NASA NTRS2004

Autonomous Environment-Monitoring Networks

Autonomous environment-monitoring networks (AEMNs) are artificial neural networks that are specialized for recognizing familiarity and, conversely, novelty. Like a biological neural network, an AEMN receives a constant stream of inputs. For purposes of computational implementation, the inputs are vector representations of the information of interest. As long as the most recent input vector is similar to the previous input vectors, no action is taken. Action is taken only when a novel vector is encountered. Whether a given input vector is regarded as novel depends on the previous vectors; hence, the same input vector could be regarded as familiar or novel, depending on the context of previous input vectors. AEMNs have been proposed as means to enable exploratory robots on remote planets to recognize novel features that could merit closer scientific attention. AEMNs could also be useful for processing data from medical instrumentation for automated monitoring or diagnosis. The primary substructure of an AEMN is called a spindle. In its simplest form, a spindle consists of a central vector (C), a scalar (r), and algorithms for changing C and r. The vector C is constructed from all the vectors in a given continuous stream of inputs, such that it is minimally distant from those vectors. The scalar r is the distance between C and the most remote vector in the same set. The construction of a spindle involves four vital parameters: setup size, spindle-population size, and the radii of two novelty boundaries. The setup size is the number of vectors that are taken into account before computing C. The spindle-population size is the total number of input vectors used in constructing the spindle counting both those that arrive before and those that arrive after the computation of C. The novelty-boundary radii are distances from C that partition the neighborhood around C into three concentric regions (see Figure 1). During construction of the spindle, the changing spindle radius is denoted by h. It is the final value of h, reached before beginning construction on the next spindle, that is denoted by r. During construction of a spindle, if a new vector falls between C and the inner boundary, the vector is regarded as completely familiar and no action is taken. If the new vector falls into the region between the inner and outer boundaries, it is considered unusual enough to warrant the adjustment of C and r by use of the aforementioned algorithms, but not unusual enough to be considered novel. If a vector falls outside the outer boundary, it is considered novel, in which case one of several appropriate responses could be initiation of construction of a new spindle.

Hand, Charles↗
NASA NTRS1993

Asynchronous timing and Doppler recovery in DSP based DPSK modems for fixed and mobile satellite applications

While conventional analog modems employ some kind of clock wave regenerator circuit for synchronous timing recovery, in sampled modem receivers the timing is recovered asynchronously to the incoming data stream, with no adjustment being made to the input sampling rate. All timing corrections are accomplished by digital operations on the sampled data stream, and timing recovery is asynchronous with the uncontrolled, input A/D system. A good timing error measurement algorithm is a zero crossing tracker proposed by Gardner. Digital, speech rate (2400 - 4800 bps) M-PSK modem receivers employing Gardner's zero crossing tracker were implemented and tested and found to achieve BER performance very close to theoretical values on the AWGN channel. Nyguist pulse shaped modem systems with excess bandwidth factors ranging from 100 to 60 percent were considered. We can show that for any symmetric M-PSK signal set Gardner's NDA algorithm is free of pattern jitter for any carrier phase offset for rectangular pulses and for Nyquist pulses having 100 percent excess bandwidth. Also, the Nyquist pulse shaped system is studied on the mobile satellite channel, where Doppler shifts and multipath fading degrade the pi/4-DQPSK signal. Two simple modifications to Gardner's zero crossing tracker enable it to remain useful in the presence of multipath fading.

Koblents, B.↗
NASA NTRS2001

Visualization of CFD Results in Immersive Virtual Environments

An object-oriented event-driven immersive virtual environment (VE) is described for the visualization of computational fluid dynamics (CFD) results. The VE incorporates the following types of primitive software objects: interface objects, support objects, geometric entities, and finite elements. The fluid domain is discretized using either a multi-block structured grid or an unstructured finite element mesh. The VE allows natural 'fly-through' visualization of the model, the CFD grid, and the model's surroundings. In order to help visualize the flow and its effects on the model, the VE incorporates the following objects: stream objects (lines, surface-restricted lines. ribbons. and volumes); colored surfaces; elevation surfaces; surface arrows; global and local iso-surfaces; vortex cores; and separation/attachment surfaces and lines. Most of these objects can be used for dynamically probing the flow. Particles and arrow animations can be displayed on top of stream objects. Primitive response quantities as well as derived quantities can be used. A recursive tree search algorithm is used for real-time point and value search in the CFD grid.

Wasfy, Tamer M.↗
NASA NTRS2002

Program Instrumentation and Trace Analysis

Several attempts have been made recently to apply techniques such as model checking and theorem proving to the analysis of programs. This shall be seen as a current trend to analyze real software systems instead of just their designs. This includes our own effort to develop a model checker for Java, the Java PathFinder 1, one of the very first of its kind in 1998. However, model checking cannot handle very large programs without some kind of abstraction of the program. This paper describes a complementary scalable technique to handle such large programs. Our interest is turned on the observation part of the equation: How much information can be extracted about a program from observing a single execution trace? It is our intention to develop a technology that can be applied automatically and to large full-size applications, with minimal modification to the code. We present a tool, Java PathExplorer (JPaX), for exploring execution traces of Java programs. The tool prioritizes scalability for completeness, and is directed towards detecting errors in programs, not to prove correctness. One core element in JPaX is an instrumentation package that allows to instrument Java byte code files to log various events when executed. The instrumentation is driven by a user provided script that specifies what information to log. Examples of instructions that such a script can contain are: 'report name and arguments of all called methods defined in class C, together with a timestamp'; 'report all updates to all variables'; and 'report all acquisitions and releases of locks'. In more complex instructions one can specify that certain expressions should be evaluated and even that certain code should be executed under various conditions. The instrumentation package can hence be seen as implementing Aspect Oriented Programming for Java in the sense that one can add functionality to a Java program without explicitly changing the code of the original program, but one rather writes an aspect and compiles it into the original program using the instrumentation. Another core element of JPaX is an observation package that supports the analysis of the generated event stream. Two kinds of analysis are currently supported. In temporal analysis the execution trace is evaluated against formulae written in temporal logic. We have implemented a temporal logic evaluator on finite traces using the Maude rewriting system from SRI International, USA. Temporal logic is defined in Maude by giving its syntax as a signature and its semantics as rewrite equations. The resulting semantics is extremely efficient and can handle event streams of hundreds of millions events in few minutes. Furthermore, the implementation is very succinct. The second form of even stream analysis supported is error pattern analysis where an execution trace is analyzed using various error detection algorithms that can identify error-prone programming practices that may potentially lead to errors in some different executions. Two such algorithms focusing on concurrency errors have been implemented in JPaX, one for deadlocks and the other for data races. It is important to note, that a deadlock or data race potential does not need to occur in order for its potential to be detected with these algorithms. This is what makes them very scalable in practice. The data race algorithm implemented is the Eraser algorithm from Compaq, however adopted to Java. The tool is currently being applied to a code base for controlling a spacecraft by the developers of that software in order to evaluate its applicability.

Havelund, Klaus↗
NASA NTRS2011

Technology Readiness Level (TRL) Advancement of the MSPI On-Board Processing Platform for the ACE Decadal Survey Mission

The Xilinx Virtex-5QV is a new Single-event Immune Reconfigurable FPGA (SIRF) device that is targeted as the spaceborne processor for the NASA Decadal Survey Aerosol-Cloud-Ecosystem (ACE) mission's Multiangle SpectroPolarimetric Imager (MSPI) instrument, currently under development at JPL. A key technology needed for MSPI is on-board processing (OBP) to calculate polarimetry data as imaged by each of the 9 cameras forming the instrument. With funding from NASA's ESTO1 AIST2 Program, JPL is demonstrating how signal data at 95 Mbytes/sec over 16 channels for each of the 9 multi-angle cameras can be reduced to 0.45 Mbytes/sec, thereby substantially reducing the image data volume for spacecraft downlink without loss of science information. This is done via a least-squares fitting algorithm implemented on the Virtex-5 FPGA operating in real-time on the raw video data stream.

multiangle-multiwavelength polarimeter↗
DOE OSTI2024

System and method for wave prediction

A method and system for prediction of wave properties include collecting time series data streams from one or more wave measurement devices and processing the data using a wave-prediction algorithm to identify the frequency components of the data and compute wave parameters. The wave-field is propagated in space and time to predict wave height, speed, and velocity at a target location. A sliding window approach is used to continuously update the prediction in real-time.

Previsic, Mirko↗
DOE OSTI2023

An implicit, conservative and asymptotic-preserving electrostatic particle-in-cell algorithm for arbitrarily magnetized plasmas in uniform magnetic fields

Here, we introduce a new electrostatic particle-in-cell algorithm capable of using large timesteps compared to particle gyro-period under a uniform external magnetic field. The algorithm extends earlier electrostatic fully implicit PIC implementations with a new asymptotic-preserving particle-push scheme that allows timesteps much larger than particle gyroperiods. In the large-timestep limit, the integrator preserves all particle drifts, while recovering the full orbit for small timesteps. The scheme allows for a seamless, efficient treatment of particles with coexisting magnetized and unmagnetized species, and conserves energy and charge exactly without spoiling implicit solver performance. We demonstrate by numerical experiment with several problems of variable species magnetization (diocotron instability, modified two-stream instability, and drift instability) that orders of magnitude wall-clock-time speedups vs. the standard fully implicit electrostatic PIC algorithm are possible without sacrificing solution accuracy.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2010

Parallel Processing of Broad-Band PPM Signals

A parallel-processing algorithm and a hardware architecture to implement the algorithm have been devised for timeslot synchronization in the reception of pulse-position-modulated (PPM) optical or radio signals. As in the cases of some prior algorithms and architectures for parallel, discrete-time, digital processing of signals other than PPM, an incoming broadband signal is divided into multiple parallel narrower-band signals by means of sub-sampling and filtering. The number of parallel streams is chosen so that the frequency content of the narrower-band signals is low enough to enable processing by relatively-low speed complementary metal oxide semiconductor (CMOS) electronic circuitry. The algorithm and architecture are intended to satisfy requirements for time-varying time-slot synchronization and post-detection filtering, with correction of timing errors independent of estimation of timing errors. They are also intended to afford flexibility for dynamic reconfiguration and upgrading. The architecture is implemented in a reconfigurable CMOS processor in the form of a field-programmable gate array. The algorithm and its hardware implementation incorporate three separate time-varying filter banks for three distinct functions: correction of sub-sample timing errors, post-detection filtering, and post-detection estimation of timing errors. The design of the filter bank for correction of timing errors, the method of estimating timing errors, and the design of a feedback-loop filter are governed by a host of parameters, the most critical one, with regard to processing very broadband signals with CMOS hardware, being the number of parallel streams (equivalently, the rate-reduction parameter).

Gray, Andrew↗
NASA NTRS1991

Computation of three-dimensional flows using two stream functions

An approach to compute 3-D flows using two stream functions is presented. The method generates a boundary fitted grid as part of its solution. Commonly used two steps for computing the flow fields are combined into a single step in the present approach: (1) boundary fitted grid generation; and (2) solution of Navier-Stokes equations on the generated grid. The presented method can be used to directly compute 3-D viscous flows, or the potential flow approximation of this method can be used to generate grids for other algorithms to compute 3-D viscous flows. The independent variables used are chi, a spatial coordinate, and xi and eta, values of stream functions along two sets of suitably chosen intersecting stream surfaces. The dependent variables used are the streamwise velocity, and two functions that describe the stream surfaces. Since for a 3-D flow there is no unique way to define two sets of intersecting stream surfaces to cover the given flow, different types of two sets of intersecting stream surfaces are considered. First, the metric of the (chi, xi, eta) curvilinear coordinate system associated with each type is presented. Next, equations for the steady state transport of mass, momentum, and energy are presented in terms of the metric of the (chi, xi, eta) coordinate system. Also included are the inviscid and the parabolized approximations to the general transport equations.

Greywall, Mahesh S.↗
DOE OSTI2026

Heterogeneous Multi-Domain Dataset Synthesis to Facilitate Privacy and Risk Assessments in Smart City IoT

The emergence of the Smart Cities paradigm and the rapid expansion and integration of Internet of Things (IoT) technologies within this context have created unprecedented opportunities for high-resolution behavioral analytics, urban optimization, and context-aware services. However, this same proliferation intensifies privacy risks, particularly those arising from cross-modal data linkage across heterogeneous sensing platforms. To address these challenges, this paper introduces a comprehensive, statistically grounded framework for generating synthetic, multimodal IoT datasets tailored to Smart City research. The framework produces behaviorally plausible synthetic data suitable for preliminary privacy risk assessment and as a benchmark for future re-identification studies, as well as for evaluating algorithms in mobility modeling, urban informatics, and privacy-enhancing technologies. As part of our approach, we formalize probabilistic methods for synthesizing three heterogeneous and operationally relevant data streams—cellular mobility traces, payment terminal transaction logs, and Smart Retail nutrition records—capturing the behaviors of a large number of synthetically generated urban residents over a 12-week period. The framework integrates spatially explicit merchant selection using K-Dimensional (KD)-tree nearest-neighbor algorithms, temporally correlated anchor-based mobility simulation reflective of daily urban rhythms, and dietary-constraint filtering to preserve ecological validity in consumption patterns. In total, the system generates approximately 116 million mobility pings, 5.4 million transactions, and 1.9 million itemized purchases, yielding a reproducible benchmark for evaluating multimodal analytics, privacy-preserving computation, and secure IoT data-sharing protocols. To show the validity of this dataset, the underlying distributions of these residents were successfully validated against reported distributions in published research. We present preliminary uniqueness and cross-modal linkage indicators; comprehensive re-identification benchmarking against specific attack algorithms is planned as future work. This framework can be easily adapted to various scenarios of interest in Smart Cities and other IoT applications. By aligning methodological rigor with the operational needs of Smart City ecosystems, this work fills critical gaps in synthetic data generation for privacy-sensitive domains, including intelligent transportation systems, urban health informatics, and next-generation digital commerce infrastructures.

IoT↗
DOE OSTI2022

Online Distribution System State Estimation via Stochastic Gradient Algorithm

Distribution network operation is becoming more challenging because of the growing integration of intermittent and volatile distributed energy resources (DERs). This motivates the development of new distribution system state estimation (DSSE) paradigms that can operate at fast timescale based on real-time data stream of asynchronous measurements enabled by modern information and communications technology. To solve the real-time DSSE with asynchronous measurements effectively and accurately, this paper formulates a weighted least squares DSSE problem and proposes an online stochastic gradient algorithm to solve it. The performance of the proposed scheme is analytically guaranteed and is numerically corroborated with realistic data on IEEE 123-bus feeder.

distribution system state estimation↗
NASA NTRS1983

A simple Lagrangian forecast system with aviation forecast potential

A trajectory forecast procedure is developed which uses geopotential tendency fields obtained from a simple, multiple layer, potential vorticity conservative isentropic model. This model can objectively account for short-term advective changes in the mass field when combined with fine-scale initial analyses. This procedure for producing short-term, upper-tropospheric trajectory forecasts employs a combination of a detailed objective analysis technique, an efficient mass advection model, and a diagnostically proven trajectory algorithm, none of which require extensive computer resources. Results of initial tests are presented, which indicate an exceptionally good agreement for trajectory paths entering the jet stream and passing through an intensifying trough. It is concluded that this technique not only has potential for aiding in route determination, fuel use estimation, and clear air turbulence detection, but also provides an example of the types of short range forecasting procedures which can be applied at local forecast centers using simple algorithms and a minimum of computer resources.

Petersen, R. A.↗
NASA NTRS1977

Vectorization of linear discrete filtering algorithms

Linear filters, including the conventional Kalman filter and versions of square root filters devised by Potter and Carlson, are studied for potential application on streaming computers. The square root filters are known to maintain a positive definite covariance matrix in cases in which the Kalman filter diverges due to ill-conditioning of the matrix. Vectorization of the filters is discussed, and comparisons are made of the number of operations and storage locations required by each filter. The Carlson filter is shown to be the most efficient of the filters on the Control Data STAR-100 computer.

Schiess, J. R.↗
NASA NTRS2006

Spatial and Temporal Distribution of Cloud Properties Observed by MODIS: Preliminary Level-3 Results from the Collection 5 Reprocessing

The Moderate Resolution Imaging Spectroradiometer (MODIS) was developed by NASA and launched onboard the Terra spacecraft on December 18, 1999 and Aqua spacecraft on May 4, 2002. It achieved its final orbit and began Earth observations on February 24, 2000 for Terra and June 24, 2002 for Aqua. A comprehensive set of operational algorithms for the retrieval of cloud physical and optical properties (optical thickness, effective particle radius, water path, thermodynamic phase) have recently been updated and are being used in the new "Collection 5" processing stream being produced by the MODIS Adaptive Processing System (MODAPS) at NASA GSFC. All Terra and Aqua data are undergoing Collection 5 reprocessing with an expected completion date by the end of 2006. The archived products from these algorithms include 1 km pixel-level (Level-2) and global gridded Level-3 products. The cloud products have applications in climate change studies, climate modeling, numerical weather prediction, as well as fundamental atmospheric research. In this talk, we will summarize the available Level-3 cloud properties and their associated statistical data sets, and show preliminary Terra and Aqua results from the available Collection 5 reprocessing effort. Anticipated results include the latitudinal distribution of cloud optical and radiative properties for both liquid water and ice clouds, as well as joint histograms of cloud optical thickness and effective radius for selected geographical locations around the world.

King, Michael D.↗