Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “stream processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗
NASA NTRS1995

Valve For Extracting Samples From A Process Stream

Valve for extracting samples from process stream includes cylindrical body bolted to pipe that contains stream. Opening in valve body matched and sealed against opening in pipe. Used to sample process streams in variety of facilities, including cement plants, plants that manufacture and reprocess plastics, oil refineries, and pipelines.

Callahan, Dave↗
DOE OSTI2021

Monitoring Radiochemical Processing Streams for the 238 Pu Supply Program with Process Pulse II

Oak Ridge National Laboratory (ORNL) is developing advanced spectroscopic and real-time monitoring capabilities to improve the timeliness of analytical measurements and process decisions for the 238Pu Supply Program. Reducing the time, resources, and costs associated with each production campaign is critical because overlapping campaigns will be required to meet the production goals of the National Aeronautics and Space Administration. Real-time, in situ analytical measurements in the heavily shielded hot cells at the Radiochemical Engineering Development Center (REDC) will allow for rapid process information feedback and operational benefits that help the 238Pu supply program scale-up production efforts. Noteworthy steps were taken during Campaign 5 to establish the ability to monitor processing streams in real time with spectrophotometry and a commercially available online monitoring software called The Unscrambler X Process Pulse II (PP) multivariate statistical process monitoring system by Camo Analytics (version 5.60). PP automates univariate-type calculations within the software itself and executes multivariate models built using The Unscrambler X (version 10.4 or newer). The Unscrambler is a commercially available data analysis software made by the same company. PP is composed of easy-to-use-tools for all personnel, including data scientists and technicians. The software can be used to plot analyte concentration profiles, spectral data, and other process variables in real time. All process data are represented in a single view with interactive charts useful for viewing how a process evolves over time.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2024

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

Vardan, Gyurjyan↗
DOE OSTI2026

Membrane-based solvent extraction for the recovery of rare earths from phosphate mining process streams

This study reports on the capture of rare earth elements (REEs) from phosphate industry process streams, including phosphoric acid (PA) sludge and phosphogypsum (PG), using a membrane solvent extraction (MSX) process. While MSX has been proven effective for a relatively concentrated feed, its effectiveness for dilute REEs solutions remains unexplored. Investigated PA-sludge and PG particles contain total REEs concentrations of ∼1100 and ∼320 ppm, respectively. Acid leaching, implemented to dissolve the REEs, significantly dilutes the REEs concentration to ∼210 ppm for PA-sludge leachate and ∼60 ppm for PG leachate. These low concentrations, compounded by the higher levels of non-REE ions and radioactive species, uranium (U) and thorium (Th), poses challenges to the MSX process. Here, we demonstrated that N,N,N′,N′-tetraoctyl-diglycolamide (TODGA) selectively binds REEs from a >3 M nitric-acid leachate while effectively rejecting U and Th. Concentrations of light REEs in strip solution were doubled compared to the feed, while heavy REEs were preferentially extracted. Furthermore, >99% purity gypsum, free of U and Th, was precipitated during the acid leaching process, aiding separation by removing significant amounts of non-REEs species (e.g., calcium) prior to the MSX process. Molecular simulations support the experimental data, suggesting preferential separation of heavy over light REEs. Based on these results, a cost-effective integrated process including pretreatment, acid leaching, MSX, and wastewater treatment is proposed for the co-recovery of REEs, phosphoric acid, gypsum, and U. This study shows MSX as a technically and economically feasible process for the recovery of REEs from low-concentration process streams, offering advantages over conventional solvent extraction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Watermarks in stream processing systems: semantics and comparative analysis of Apache Flink and Google cloud dataflow

Streaming data processing is an exercise in taming disorder: from oftentimes huge torrents of information, we hope to extract powerful and timely analyses. But when dealing with streaming data, the unbounded and temporally disordered nature of real-world streams introduces a critical challenge: how does one reason about the completeness of a stream that never ends? In this paper, we present a comprehensive definition and analysis of watermarks, a key tool for reasoning about temporal completeness in infinite streams.First, we describe what watermarks are and why they are important, highlighting how they address a suite of stream processing needs that are poorly served by eventually-consistent approaches:• Computing a single correct answer, as in notifications.• Reasoning about a lack of data, as in dip detection.• Performing non-incremental processing over temporal subsets of an infinite stream, as in statistical anomaly detection with cubic spline models.• Safely and punctually garbage collecting obsolete inputs and intermediate state.• Surfacing a reliable signal of overall pipeline health.Second, we describe, evaluate, and compare the semantically equivalent, but starkly different, watermark implementations in two modern stream processing engines: Apache Flink and Google Cloud Dataflow.

Akidau, Tyler↗
DOE OSTI2025

Rapid Sensing to Facilitate Purification of Rare Earth Element-Containing Process Streams Produced Through Membrane-Assisted Solvent Extraction

The development of an economically competitive domestic supply of rare earth elements and yttrium (REY) is necessary for our nation’s economic growth and national security. The achievement of a secure domestic supply of REY requires not only the development of effective processes for recovery of REY from naturally occurring materials and/or recycled products, but also the development of downstream processes for the ultimate production of high-REY content solids. An impediment to the development of such processes is the scarcity of analytical methods that provide rapid determination of the process stream compositions. In this work, a membrane-based extraction process was used to selectively recover REYs from a dilute solution in the presence of much higher concentrations of Ca and Al. In tandem, the use of a portable spectrometer equipped with an immobilized zinc adeninate benzene tricarboxylate metal-organic framework sensing material makes possible the rapid detection of the presence of ppm concentrations of Tb and Eu in both weakly acidic and strongly acidic process streams within minutes. A solvent extraction processing time of 15-60 min maximized REY selectivity over gangue ions while achieving up to 80% REY and minimal gangue ion recovery. Taken together, these experiments highlight not only an innovative method for REY purification but also the importance of inexpensive, portable characterization methods for near real-time analysis of REY content.

Membrane-assisted solvent extraction↗
DOE OSTI2024

Arroyo Stream Processing Toolset (arroyopy) v0.1.0

Processing event or streaming data presents several technological challenges. A variety of technologies are often used by scientific user facilities. ZMQ is used to stream data and messages in a peer-to-peer fashion. Message brokers like Kafka, Redis Pubsub, EPICS PVA and RabbitMQ are often employed to route and pass messages from instruments to processing workflows. Arroyopy provides an API and structure to flexibly integrate with these tools and incorporate arbitrarily complex processing workflows, letting the hooks to the workflow code be independent of the connection code and hence reusable at a variety of instruments.

Chavez Esparza, Tanny Andrea [Lawrence Berkeley Na↗
DOE OSTI2021

Online Monitoring of Radiochemical Processing Streams for the Plutonium-238 Supply Program

Online monitoring with spectrophotometry is being developed to improve the timeliness of analytical measurements for the Plutonium-238 Supply Program at Oak Ridge National Laboratory. A commercially available online monitoring software was used to calculate and view process data in real time to help identify process deviations and optimize system performance. Monitoring detailed process data will improve processing efficiency and help technicians make decisions during hot-cell operations.

Sadergaski, Luke↗
NASA NTRS1977

Monte Carlo simulation of the jet stream process

A Monte Carlo model is formulated to simulate the orbital evolution of a system of colliding particles. It is found that inelastic collision alone (even if the impact energy dissipation from collision is very large) does not lead to the formation of a narrow ring-like jet stream; instead, a flat disk structure, similar to Saturn's rings, usually results. To produce the radial focusing effect, it is argued that additional dynamical effects, which would strengthen the collisional interaction between the particles in near-circular orbits, is needed.

Ip, W.-H.↗
NASA NTRS1998

Develop Recovery Systems for Separations of Salts from Process Streams for use in Advanced Life Support System

The main objectives of this project were the development of a four-compartment electrolytic cell using high selective membranes to remove nitrate from crop residue leachate and convert it to nitric acid, and the development of an six compartment electrodialysis cell to remove selectively sodium from urine wastes. The recovery of both plant inedible biomass and human wastes nutrients to sustain a biomass production system are important aspects in the development of a controlled ecological life support system (CELSS) to provide the basic human needs required for life support during long term space missions. A four-compartment electrolytic cell has been proposed to remove selectively nitrate from crop residue and to convert it to nitric acid, which is actually used in the NASA-KSC Controlled Ecological Life Support System to control the pH of the aerobic bioreactors and biomass production chamber. Human activities in a closed system require large amount of air, water and minerals to sustain life and also generate wastes. Before using human wastes as nutrients, these must be treated to reduce organic content and to remove some minerals which have adverse effects on plant growth. Of all the minerals present in human urine, sodium chloride (NACl) is the only one that can not be used as nutrient for most plants. Human activities also requires sodium chloride as part of the diet. Therefore, technology to remove and recover sodium chloride from wastes is highly desirable. A six-compartment electrodialysis cell using high selective membranes has been proposed to remove and recover NaCl from human urine.

Colon, Guillermo↗
DOE OSTI2022

Leveraging Multiple Raman Excitation Wavelength Systems for Process Monitoring of Nuclear Waste Streams

Processing nuclear waste from sites such as Hanford is a significant environmental cleanup need while being a significant logistical challenge. Integration of process monitoring tools, that can provide in situ and real-time feedback about the process, can significantly alleviate needs to collect grab samples for process control and product characterization. Raman spectroscopy paired with chemometric analysis is one process monitoring tool that can provide chemical composition information on a large number of chemical targets in nuclear waste streams. However, methods to improve limits of detection as well as drop uncertainty in quantification are needed. Optimizing instrument specifications can achieve this, here this is demonstrated by comparing limits of detection for key analytes when using Raman systems with 671 nm, 532 nm, and 405 nm excitation wavelengths. Generally, limits of detection decease (allowing the measurement of lower salt concentrations) with decreasing wavelength. Similarly, data collection times and averaging were optimized. Finally, multiple chemometric modeling approaches were leveraged, including multiblock methods that combined data from all three Raman systems to simultaneously quantify targets with improved sensitivity.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2001

Defining the cortical visual systems: "what", "where", and "how"

The visual system historically has been defined as consisting of at least two broad subsystems subserving object and spatial vision. These visual processing streams have been organized both structurally as two distinct pathways in the brain, and functionally for the types of tasks that they mediate. The classic definition by Ungerleider and Mishkin labeled a ventral "what" stream to process object information and a dorsal "where" stream to process spatial information. More recently, Goodale and Milner redefined the two visual systems with a focus on the different ways in which visual information is transformed for different goals. They relabeled the dorsal stream as a "how" system for transforming visual information using an egocentric frame of reference in preparation for direct action. This paper reviews recent research from psychophysics, neurophysiology, neuropsychology and neuroimaging to define the roles of the ventral and dorsal visual processing streams. We discuss a possible solution that allows for both "where" and "how" systems that are functionally and structurally organized within the posterior parietal lobe.

NASA Discipline Space Human Factors↗