Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Block-Based Triangle Counting Algorithm on Heterogeneous Environments

Triangle counting is a fundamental building block in graph algorithms. In this article, we propose a block-based triangle counting algorithm to reduce data movement during both sequential and parallel execution. Our block-based formulation makes the algorithm naturally suitable for heterogeneous architectures. The problem of partitioning the adjacency matrix of a graph is well-studied. Our task decomposition goes one step further: it partitions the set of triangles in the graph. By streaming these small tasks to compute resources, we can solve problems that do not fit on a device. We demonstrate the effectiveness of our approach by providing an implementation on a compute node with multiple sockets, cores and GPUs. The current state-of-the-art in triangle enumeration processes the Friendster graph in 2.1 seconds, not including data copy time between CPU and GPU. Using that metric, our approach is 20 percent faster. When copy times are included, our algorithm takes 3.2 seconds. This is 5.6 times faster than the fastest published CPU-only time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Block-Based Triangle Counting Algorithm on Heterogeneous Environments

Triangle counting is a fundamental building block in graph algorithms. In this paper, we propose a block-based triangle counting algorithm to reduce data movement during both sequential and parallel execution. Our block-based formulation makes the algorithm naturally suitable for heterogeneous architectures. The problem of partitioning the adjacency matrix of a graph is well-studied. Our task decomposition goes one step further: it partitions the set of triangles in the graph. By streaming these small tasks to compute resources, we can solve problems that do not fit on a device. We demonstrate the effectiveness of our approach by providing an implementation on a compute node with multiple sockets, cores and GPUs. The current state-of-the-art in triangle enumeration processes the Friendster graph in 2.1 seconds, not including data copy time between CPU and GPU. Using that metric, our approach is 20 percent faster. When copy times are included, our algorithm takes 3.2 seconds. This is 5.6 times faster than the fastest published CPU-only time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Advanced Health Information Technology Analytic Framework and Application to Hazard Detection

Health Information Technology (HIT) aims to improve healthcare outcomes by organizing and analyzing various health-related data. With data accumulating at a staggering rate, the importance of real-time analytics has been increasing dramatically, shifting the focus of informatics from batch processing to streaming analytics. HIT is also facing unprecedented challenges in adapting to this new requirement and leveraging advanced IT technologies. This paper introduces a HIT data and compute platform that supports multi-granularity real-time analytics from heterogeneous data sources. The paper first identifies functional requirements and proposes a framework that satisfies the requirements using state-of-the-art big data technologies including Apache Kafka, Spark Structured Streaming Engine, and Delta Lake. To demonstrate its capability to support data analytics in multiple time granularities analytics, a statistical process control-based hazard detection algorithm has been implemented on top of the framework to detect unexpected hazards from order cancellation data of the Department of US Veterans Affairs (VA) in near real-time.

Kumar, Mohit↗
DOE OSTI2024

Unconventional Quantum Advantages for Computation (U-QuAC)

While quantum computing offers the promise of exponential advantages, limited quantum speedups are known, especially for practical applications. To open new avenues for quantum advantages, we propose Unconventional Quantum Advantages for Computation (U-QuACs), with respect to unconventional resources such as space (number of bits or quantum bits of memory required to solve a problem), accuracy of solution, communication, or energy consumption. We focus on space-efficient quantum algorithms, where we seek to design algorithms that solve a problem using much less space than the total size of the input. A natural setting in which space is critical is the streaming model of computation, where the input data arrives sequentially in pieces that must each be processed individually. Streaming is motivated by a variety of problems including analysis of internet traffic or social networks. We design the first exponential quantum space advantage for a natural streaming problem, which also constitutes the first quantum advantage for approximating a discrete optimization problem, albeit with respect to space.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

Developing a Machine-Learning-Based Processing Framework for Twitter and Other Crowdsourced Data

Crowdsourced data streams such as Twitter and other social media are important sources of real-time and historical global information for Earth science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we have been exploring the Twitter data stream for its potential in augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. To realize this potential, we need to increase the information density and enhance the quality of filtered precipitation tweets. We have implemented various components of a machine learning (ML)-based processing infrastructure for crowdsourced data that outputs, in this instance, useful and usable information derived from precipitation tweets. We have test enriched the Twitter stream with higher quality active tweets from those knowingly contributing to our effort and from existing crowdsourced programs (e.g., mPING, CoCoRaHS). We have experimented with various algorithms for processing tweets, including Naà ve Bayes, Convolutional Neural Network (CNN), Hierarchical Attention Network (HAN), and semi-supervised learning (with tri-training). Our current work focuses on (1) automated review of Earth science-related publications to determine relationships between discipline research needs and ML algorithms; (2) investigating Sequential Generative Adversarial Network (SeqGAN) for processing precipitation tweets for anomaly detection; and (3) managing crowdsourced data in a way that is compatible with existing NASA satellite data archives and using the data for ML applications. Key results include (1) network visualization of NLP-processed publications in various Earth science disciplines; (2) difference between GPM-linked, generated tweets and collected actual tweets that is small for GPM-determined light to moderate rain cases and high for GPM-determined heavy rain cases; and (3) identification of MongoDB for storing raw tweets and Zarr format for gridded tweets (compatible with GPM data). Our results have taken us a step closer to an operational ML-based tweet processing infrastructure and have already demonstrated that tweet-derived precipitation information is potentially useful for validation of Earth science satellite data.

Teng, William↗
NASA NTRS1990

Real-time aerodynamic heating and surface temperature calculations for hypersonic flight simulation

A real-time heating algorithm was derived and installed on the Ames Research Center Dryden Flight Research Facility real-time flight simulator. This program can calculate two- and three-dimensional stagnation point surface heating rates and surface temperatures. The two-dimensional calculations can be made with or without leading-edge sweep. In addition, upper and lower surface heating rates and surface temperatures for flat plates, wedges, and cones can be calculated. Laminar or turbulent heating can be calculated, with boundary-layer transition made a function of free-stream Reynolds number and free-stream Mach number. Real-time heating rates and surface temperatures calculated for a generic hypersonic vehicle are presented and compared with more exact values computed by a batch aeroheating program. As these comparisons show, the heating algorithm used on the flight simulator calculates surface heating rates and temperatures well within the accuracy required to evaluate flight profiles for acceptable heating trajectories.

Quinn, Robert D.↗
DOE OSTI2022

Substructure in the stellar halo near the Sun: II. Characterisation of independent structures

In an accompanying paper, we present a data-driven method for clustering in ‘integrals of motion’ space and apply it to a large sample of nearby halo stars with 6D phase-space information. The algorithm identified a large number of clusters, many of which could tentatively be merged into larger groups. The goal here is to establish the reality of the clusters and groups through a combined study of their stellar populations (average age, metallicity, and chemical and dynamical properties) to gain more insights into the accretion history of the Milky Way. To this end, we developed a procedure that quantifies the similarity of clusters based on the Kolmogorov–Smirnov test using their metallicity distribution functions, and an isochrone fitting method to determine their average age, which is also used to compare the distribution of stars in the colour–absolute magnitude diagram. Also taking into consideration how the clusters are distributed in integrals of motion space allows us to group clusters into substructures and to compare substructures with one another. We find that the 67 clusters identified by our algorithm can be merged into 12 extended substructures and 8 small clusters that remain as such. The large substructures include the previously known Gaia-Enceladus, Helmi streams, Sequoia, and Thamnos 1 and 2. We identify a few over-densities that can be associated with the hot thick disc and host a small metal-poor population. Especially notable is the largest (by number of member stars) substructure in our sample which, although peaking at the metallicity characteristic of the thick disc, has a very well populated metal-poor component, and dynamics intermediate between the hot thick disc and the halo. We also identify additional debris in the region occupied by Sequoia with clearly distinct kinematics, likely remnants of three different accretion events with progenitors of similar masses. Although only a small subset of the stars in our sample have chemical abundance information, we are able to identify different trends of [Mg/Fe] versus [Fe/H] for the various substructures, confirming our dissection of the nearby halo. We find that at least 20% of the halo near the Sun is associated to substructures. When comparing their global properties, we note that those substructures on retrograde orbits are not only more metal-poor on average but are also older. We provide a table summarising the properties of the substructures, as well as a membership list that can be used for follow-up chemical abundance studies for example.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1978

Development of a three-dimensional Navier-Stokes code on CDC star-100 computer

A three-dimensional code in body-fitted coordinates was developed using MacCormack's algorithm. The code is structured to be compatible with any general configuration, provided that the metric coefficients for the transformation are available. The governing equations are developed in primitive variables in order to facilitate the incorporation of physical boundary conditions and turbulence-closure models. MacCormack's two-step, unsplit, time-marching algorithm is used to solve the unsteady Navier-Stokes equations until steady-state solution is achieved. Cases discussed include (1) flat plate in supersonic free stream; (2) supersonic flow along an axial corner; (3) subsonic flow in an axial corner at M infinity = 0.95; and (4) supersonic flow in an axial corner at M infinity 1.5.

Vatsa, V. N.↗
NASA NTRS1988

An upwind parabolized Navier-Stokes code for chemically reacting flows

A new upwind, parabolized Navier-Stokes (PNS) code has been developed to compute the hypersonic, viscous, chemically reacting flow around two-dimensional or axisymmetric bodies. The new code is an extension of the upwind (perfect gas) PNS code of Lawrence et al. (1986). The upwind algorithm is based on Roe's flux-difference splitting scheme which has been modified to account for real gas effects. The algorithm solves the gas dynamic and species continuity equations in a 'loosely' coupled manner. The new code has been validated by computing the laminar flow (at free stream Mach number 25) of chemically reacting air over a wedge and a cone. The results of these computations are compared with the results from a centrally-differenced, fully coupled, nonequilibrium PNS code. The agreement is excellent, except in the vicinity of the shock wave where the present code exhibits superior shock capturing capabilities.

Tannehill, John C.↗
NASA NTRS2021

A High Dynamic-Range Photon-Counting Receiver for Deep Space Optical Communication

The Deep Space Optical Communication (DSOC) project will demonstrate free-space optical communication at almost 3 AU, or 3 orders of magnitude further than any previous attempt. DSOC will utilize the 5m Palomar Hale Telescope to receive the downlink signal, which will couple the downlink light onto an optical table and into a superconducting nanowire single photon detector (SNSPD). The output of the SNSPD is digitized by the Ground Laser Receiver Signal Processing Assembly (GSPA) using a high throughput streaming time to digital converter (TDC). The GSPA is a scalable FPGA-based receiver which demodulates and decodes the DSOC downlink signal through novel signal processing algorithms implemented on Xilinx UltraScale+ FPGAs, as well as Python-based software monitor and control routines. Exploiting the unique TDC-based architecture, the GSPA supports over four orders of magnitude of downlink data rates across multiple orders of magnitude of signal and background powers. In this paper we present an overview of the hardware, firmware and software architectures to implement this system, as well as performance analysis for links ranging from near-Earth to 2.8 AU.

Srinivasan, Meera↗
DOE OSTI2021

Evolutionary NAS in Light of Model Stability for Accurate Continual Learning

Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars and drones. However, continual learning is still facing the catastrophic forgetting problem. Previous work illustrate that model performance on continual learning is not only related to the learning algorithms but also strongly dependent on the inherited model, i.e., the model where continual learning starts. The better stability of the inherited model, the less catastrophic forgetting and thus, the inherited model should be elaborately selected. Inspired by this finding, we develop an evolutionary neural architecture search (ENAS) algorithm that emphasizes the Stability of the inherited model, namely ENAS-S. ENAS-S aims to find optimal architectures for accurate continual learning on edge devices. On CIFAR-10 and CIFAR-100, we present that ENAS-S achieves competitive architectures with lower catastrophic forgetting and smaller model size when learning from a data stream, as compared with handcrafted DNNs.

Du, Xiaocong↗
NASA NTRS2004

Computationally-Efficient Minimum-Time Aircraft Routes in the Presence of Winds

A computationally efficient algorithm for minimizing the flight time of an aircraft in a variable wind field has been invented. The algorithm, referred to as Neighboring Optimal Wind Routing (NOWR), is based upon neighboring-optimal-control (NOC) concepts and achieves minimum-time paths by adjusting aircraft heading according to wind conditions at an arbitrary number of wind measurement points along the flight route. The NOWR algorithm may either be used in a fast-time mode to compute minimum- time routes prior to flight, or may be used in a feedback mode to adjust aircraft heading in real-time. By traveling minimum-time routes instead of direct great-circle (direct) routes, flights across the United States can save an average of about 7 minutes, and as much as one hour of flight time during periods of strong jet-stream winds. The neighboring optimal routes computed via the NOWR technique have been shown to be within 1.5 percent of the absolute minimum-time routes for flights across the continental United States. On a typical 450-MHz Sun Ultra workstation, the NOWR algorithm produces complete minimum-time routes in less than 40 milliseconds. This corresponds to a rate of 25 optimal routes per second. The closest comparable optimization technique runs approximately 10 times slower. Airlines currently use various trial-and-error search techniques to determine which of a set of commonly traveled routes will minimize flight time. These algorithms are too computationally expensive for use in real-time systems, or in systems where many optimal routes need to be computed in a short amount of time. Instead of operating in real-time, airlines will typically plan a trajectory several hours in advance using wind forecasts. If winds change significantly from forecasts, the resulting flights will no longer be minimum-time. The need for a computationally efficient wind-optimal routing algorithm is even greater in the case of new air-traffic-control automation concepts. For air-traffic-control automation, thousands of wind-optimal routes may need to be computed and checked for conflicts in just a few minutes. These factors motivated the need for a more efficient wind-optimal routing algorithm.

Jardin, Matthew R.↗
NASA NTRS2005

Radiative Studies of Planetary Atmospheres

Retrieval algorithms and associated software for application to CIRS infrared spectral data have been developed and coded. A general forward radiative transfer code has been written that runs efficiently on a Macintosh, even at high spectral resolution (0.5 per centimeter). It makes use of the correlated-k approach for representation of the gaseous absorption and can include those gases listed in the HITRAN and GEISA atlases, along with collision-induced absorption. Cloud effects are included as spectrally dependent absorbers. Provision has been made for future extension to include particle scattering in an n-stream approximation. The primary purpose of the code is to produce synthetic data and to serve as the forward calculating element in gas and cloud retrieval programs developed for the Mac as well as other platforms. Initial development of algorithms and production software suitable for application to CIRS data to be obtained from Jupiter, Saturn and Titan has been completed, and production versions of the software for application to the spectral data are in place. This includes temperature, gaseous constituent, and cloud opacity retrieval, algorithms that can be applied to both nadir and limb data. This work has been done as a cooperative effort between Conrath and Matcheva (Cornell), Achterberg (GSFWSSAI), and Flasar (GSFC).

Conrath, Barney J.↗
DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗
NASA NTRS1976

A brief description of the Jameson-Caughey NYU transonic swept-wing computer program: FLO 22

A computer program for analyzing inviscid, isentropic, transonic flow past 3-D swept configurations is presented. Some basic aspects of the program are: (1) the free-stream Mach number is restricted only by the isentropic assumption; (2) weak shock waves are automatically located wherever they occur in the flow; (3) the finite-difference form of the full equation for the velocity potential is solved by the method of relaxation, after the flow exterior to the airfoil is mapped to the upper half plane; (4) the mapping procedure allows exact satisfaction of the boundary conditions and use of supersonic free stream velocities; (5) the finite difference operator is locally rotated in supersonic flow regions so as to properly account for the domain of dependence; and (6) the relaxation algorithm was stabilized using criteria from a time-like analogy.

Jameson, A.↗
NASA NTRS1986

Aspects and applications of patched grid calculations

Patched grid calculations within the framework of an implicit, flux-vector split upwind/relaxation algorithm for the Euler equations are presented. The effect of a metric-discontinuous interface on the convergence rate of the algorithm is discussed along with the spatial accuracy of the solution and the effect of curvature along an interface. Results are presented and discussed for the free-stream problem, shock reflection problem, supersonic inlet with a 5 degree ramp, aerodynamically choked inlet, and three-dimensional analytic forebody.

Walters, R. W.↗
NASA NTRS1990

Hopf bifurcation in the driven cavity

The algorithm employed in the present incompressible two-dimensional calculations of an impulsively-started lid-driven cavity has its basis in the time-dependent stream-function equation. While a Crank-Nicholson differencing scheme is used for the diffusion terms, the Adams-Bashforth scheme is used for the convection terms. The periodic asymptotic solutions obtained for Reynolds numbers of 5000 and 10,000 are found to be precisely periodic; it is demonstrated that they have reached asymptotic states. The indicators of that achievement are discussed.

Goodrich, John W.↗
DOE OSTI2020

Paleo-Megadroughts and Abrupt Climate Changes in the Speleothem Records. Final report

This project is motivated by the speleothem isotope records in Asia, which show regional responses in the hydrologic cycle to different climate forcings. Speleothem isotopic records are typically interpreted in terms of local precipitation variations or monsoon intensity. Our study demonstrates that non-local processes also play an important role. We started this project to understand the regional difference in speleothem isotopic composition between the Last Glacial Maximum (LGM) and the present-day. The record in Southwest China showed greater depletion during the LGM compared to those in East China. Our modeling and analysis showed that speleothems record, in addition, large scale changes in atmospheric circulation and moisture transport and their subsequent impact on precipitation. We developed an algorithm to partition total precipitation according to their formation dynamics, namely into frontal and non-frontal precipitation, and showed that the two have different trends and hence different causal mechanism. We then focused our subsequent attention on circulation impacts on precipitation changes. We applied a machine learning algorithm to detect rainbands in the ERA-Interim reanalysis product, and showed that the seasonal migrations of the rainbands are tied to the seasonal migrations of the jet stream, in particular the northerlies of the jet meanders. These northerlies, in turn, are partly topographic Rossby waves excited as the upstream westerlies impinge on the Tibetan Plateau. The seasonal variations of these upstream westerlies thus contribute to the seasonal movements of the rainbands and regional precipitation changes. Our analysis of the modern precipitation isotope record further confirms the importance of jet stream changes in the isotopic variations and shows that isotope-enriched years have reduced summer seasonality, with less pronounced northward migration of the jet.

54 ENVIRONMENTAL SCIENCES↗