Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Hydropower Biological Evaluation Toolset Best Practice Guide

Field studies using live fish are necessary for the evaluation of turbine biological performance, but they cannot determine the specific hydraulic conditions or physical stresses experienced by the fish, the locations where deleterious conditions occur, or the specific causes of the biological response. Using the Sensor Fish (SF) sensing technology, this deficiency can be overcome because the SF can be released independently or concurrently with live fish directly into operating infrastructure, and it takes high-frequency measurements of hydraulic conditions such as pressure, acceleration, and rotation acting on a body in situ during downstream passage. The Hydropower Biological Evaluation Tools (HBET; Hou et al. 2018) software package, developed by Pacific Northwest National Laboratory (PNNL), is designed to assemble, organize, and process data collected by the PNNL-developed SF and by live fish. HBET was developed specifically to design SF field studies, process the raw data, and analyze the processed data efficiently and scientifically. Its objectives are to facilitate SF studies focused on characterizing hydraulic conditions and to apply SF data for evaluating the impacts on fish from passage through hydro-structures. HBET allows users to design new studies, analyze data, perform statistical analyses, and evaluate predicted biological responses. It can be used by researchers, turbine designers, hydropower operators, and regulators to evaluate hydro-structures to enhance environmental sustainability in a cost-effective manner.

13 HYDRO ENERGY↗

Linking resource availability to pantropical forest canopy resistance and resilience to cyclone disturbance

Statement of purpose: Tropical cyclones are intensifying and occurring at higher latitudes in recent decades, but the mechanisms underpinning the resistance (ability to withstand disturbance-induced change) and resilience (pace of return to pre-disturbance reference values) of tropical forests to cyclones remains largely unexplored at the pantropical scale. We conducted a meta-analysis to investigate the role of soil resource availability (i.e., total soil phosphorus concentration) in mediating site-level forest canopy resistance and resilience to cyclones pan-tropically. We evaluated cyclone-induced and post-cyclone litterfall mass (g/m2/day), phosphorus (P) and nitrogen (N) fluxes (mg/m2/day), as well as concentrations (mg/g) across 73 case studies in Australia, Guadeloupe, Hawaii, Mexico, Puerto Rico, and Taiwan. The dataset zip file includes three data and two metadata files: - The compiled Litterfall Mass Flux data from tropical forests across the globe prior to and after varying tropical cyclone disturbances are provided in Litterfall_Mass.csv. This data file also includes site location, geographical characteristics, elevation, soil phosphorus concentration, geology, and several variables related to each tropical cyclone disturbance. - The compiled Litterfall Nitrogen and Phosphorus Flux data from tropical forests across the globe prior to and after varying tropical cyclone disturbances are provided in Litterfall_Nutrients.csv. This data file also includes site location, geographical characteristics, elevation, soil phosphorus concentration, geology, and several variables related to each tropical cyclone disturbance. - Tropical cyclone track data compiled from HURDAT2 and IBTrACS databases and used as input in the HURRECON model (https://github.com/hurrecon-model/HurreconR) to generate wind data is provided in hurdat2-1851-2019-052520.txt. - The metadata file (Metadata_Meta-analysis_Litterfall-Mass.pdf) has the complete information on each variable included in the Litterfall_Mass.csv dataset, the data sources, and data processing information. - The metadata file (Metadata_Meta-analysis_Litterfall-Nutrients.pdf) has the complete information on each variable included in the Litterfall_Nutrients.csv dataset, the data sources, and data processing information.

54 ENVIRONMENTAL SCIENCES↗

CVEVOLVE

CVEvolve is an agentic AI system for autonomous algorithm discovery for scientific data processing. It creates workflows where large language model agents freely set up and configure development environments and evaluation harnesses, develop and improve data processing algorithms with designed exploration-exploitation balancing mechanisms, log history and findings in a structured database, and run holdout testing to ensure algorithm generalizability. CVEvolve offers a zero-code interface and does not require users to provide structured data and evaluation scripts.

Cherukara, MatthewJoseph [Argonne National Laborat↗

Real-time GIS Programming and Geocomputation

Streaming data generated continuously from sensor networks, mobile devices, social media platforms and other edge devices have posed significant challenges to existing computing platforms for achieving both high throughput and low latency data processing in addition to scalable computing. This entry introduces a real-time computing and programming platform for time-critical GIS (Geographic Information System) applications. In this platform, advanced streaming data processing software, such as Apache Kafka and Spark Streaming, are integrated to enable data analytics in real-time. This computing platform can also be extended to integrate GeoAI (Geospatial Artificial Intelligence) based machine learning models to leverage both historical and streaming data to achieve real-time prediction and intelligent geospatial analytics. Two real-time geospatial applications in terms of flood simulation and climate data visualization are introduced to demonstrate how real-time programming and computing can help tackle real-world problems with important societal impacts.

Liu, Yan↗

SQuaD: Smart Quantum Detection for Photon Recognition and Dark Count Elimination

Quantum detectors of single photons are an essential component for quantum information processing across computing, communication and networking. Today's quantum detection system, which consists of single photon detectors, timing electronics, control and data processing software, is primarily used for counting the number of single photon detection events. However, it is largely incapable of extracting other rich physical characteristics of the detected photons, such as their wavelengths, polarization states, photon numbers, or temporal waveforms. This work, for the first time, demonstrates a smart quantum detection system, SQuaD, which integrates a field programmable gate array (FPGA) with a neural network model, and is designed to recognize the features of photons and to eliminate detector dark-count. The SQuaD is a fully integrated quantum system with high timing-resolution data acquisition, onboard multi-scale data analysis, intelligent feature recognition and extraction, and feedback-driven system control. Our \name experimentally demonstrates 1) reliable photon counting on par with the state-of-the art commercial systems; 2) high-throughput data processing for each individual detection events; 3) efficient dark count recognition and elimination; 4) up to 100% accurate feature recognition of photon wavelength and polarization. Additionally, we deploy the SQuaD to an atomic (erbium ion) photon emitter source to realize noise-free control and readout of a spin qubit in the telecom band, enabling critical advances in quantum networks and distributed quantum information processing.

Linne, Karl C. [U. Chicago (main)] (ORCID:00090009↗

Tracking Volumetric Units in Modular Factories for Automated Progress Monitoring Using Computer Vision

The construction industry is increasingly adopting off-site and prefabricated methods due to advantages offered in safety, quality, and lead time. Applying industrialized methods for plant management in offsite construction factories requires the collection of large volumes of production process data, which is a tedious task when performed manually. Recent attempts to automate this process have relied on sensor-based data collection methods which are susceptible to noise, expensive, and difficult to validate. Computer vision methods, however, enable process data collection from videos without the limitations of the other sensor-based methods. This technology has not been applied for offsite construction except in very few instances and therefore, this study proposes a novel method to reliably collect the production process data using computer vision method in near real-time from widely used surveillance cameras in offsite construction. The proposed method allows the user to annotate the workstations of interest on the video as ground truths and process these areas throughout the entire video to track the units entering and leaving stations, while continuously updating a near real-time schedule of the production line. This framework was validated by implementing on the surveillance videos of the production process of modular home manufacturing in a factory. The results consistently provided 100% accuracy, after denoising, for all the videos processed including 60 h of work for a station. The developed method enables real-time tracking of station performance, which can enable continuous improvement methods for factory management and resource allocation.

computer vision↗

Numerical Investigation of Desulfurization Kinetics in Gas-Stirred Ladles by a Quick Modeling Analysis Approach

A quick modeling analysis approach for predicting the slag-steel reaction and desulfurization kinetics in argon gas-stirred ladles has been developed in this study. The model consists of two uncoupled components: (i) a computational fluid dynamics (CFD) model for predicting the fluid flow and the characteristics of slag-steel interface, and (ii) a multicomponent reaction kinetics model for calculating the desulfurization evolution. The steel-slag interfacial area and mass transfer coefficients predicted by the CFD simulation are used as the processing data for the reaction model. Since the desulfurization predictions are uncoupled from the CFD simulation, the computational time of this uncoupled predictive approach is decreased by at least 100 times for each case study when compared with the CFD-reaction kinetics fully coupled model. The uncoupled modeling approach was validated by comparing the evolution of steel and slag compositions with the experimentally measured data during ladle metallurgical furnace (LMF) processing at Nucor Steel Tuscaloosa, Inc. Then, the validated approach was applied to investigate the effects of the initial steel and slag compositions, as well as different types of additions during the refining process on the desulfurization efficiency. The results revealed that the sulfur distribution ratio and the desulfurization reaction can be promoted by making Al and CaO additions during the refining process. It was also shown that by increasing the initial Al content in liquid steel, both Al oxidation and desulfurization rates rapidly increase. In addition, it was found that the variation of the initial Si content in steel has no significant influence on the desulfurization rate. Lastly, if the initial CaO content in slag is increased or the initial Al{sub 2}O{sub 3} content is decreased in the fluid-slag compositional range, the desulfurization rate can be improved significantly during the LMF process.

36 MATERIALS SCIENCE↗

Diffraction imaging for seal evaluation using ultra high resolution 3D seismic data

Cap rock is any impermeable or low permeability formation that may trap oil, gas or water, preventing it from migrating to the surface. Evaluation of sealing properties of the cap rock is a critical task for seismic exploration. It is important for oil and gas prospecting as well as for locating reservoirs appropriate for carbon dioxide storage. The latter has been proposed as one solution to global climate change caused by heat-trapping anthropogenic gasses in the atmosphere. A fluid escaped from the reservoir fills voids in the overlying strata thereby causing local changes of acoustic properties. We aim to explore such fluid-saturated areas and thus evaluate sealing properties of underlying cap rocks. We assume that local acoustic impedance changes caused by fluid migration generate seismic diffractions. We isolate diffracted waves from P-Cable seismic data acquired over the area of interest and perform diffraction analysis of the near-surface interval (above 200 m). In the diffraction image obtained, we interpret an extensive system of broad sinuous channels, narrow linear faults, and diffuse fracture networks. In addition, we detect high diffractivity anomalies, which extend vertically and laterally. We note no evidence of these anomalies in a conventional seismic image and associated seismic attribute (similarity and fracture density) volumes. We associate this high diffractivity to a variable pore fluid composition and saturation, which we infer to be due to hydrocarbon migration from the underlying strata. Consequently, we conclude poor seal quality for the underlying cap rocks. During data processing, we test diffraction focusing analysis for velocity model building. In conclusion, this option appears quite valuable for P-Cable data processing because of short offset and limitations of the standard velocity analysis.

58 GEOSCIENCES↗

PATHA: Performance Analysis Tool for HPC Applications

Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Science Capsule: Towards Sharing and Reproducibility of Scientific Workflows

Workflows are increasingly processing large volumes of data from scientific instruments, experiments and sensors. These workflows often consist of complex data processing and analysis steps that might include a diverse ecosystem of tools and also often involve human-in-the-loop steps. Sharing and reproducing these workflows with collaborators and the larger community is critical but hard to do without the entire context of the workflow including user notes and execution environment. In this paper, we describe Science Capsule, which is a framework to capture, share, and reproduce scientific workflows. Science Capsule captures, manages and represents both computational and human elements of a workflow. It automatically captures and processes events associated with the execution and data life cycle of workflows, and lets users add other types and forms of scientific artifacts. Science Capsule also allows users to create `workflow snapshots' that keep track of the different versions of a workflow and their lineage, allowing scientists to incrementally share and extend workflows between users. Our results show that Science Capsule is capable of processing and organizing events in near real-time for high-throughput experimental and data analysis workflows without incurring any significant performance overheads.

Ghoshal, Devarshi↗

Is the Supporting Information the Venue for Reproducibility and Transparency?

Making research data, software, and data processing tools readily available to the public could significantly enhance the impact of scientific publications. Openness and transparency could address reproducibility concerns and accelerate scientific progress.(1, 2) While archives and repositories would be preferable for securing data in the long term, the Supporting Information (SI) already allocates significant space to provide auxiliary files, links, and essential information needed to make scientific findings immediately reproducible as well as data processing protocols and numerical procedures executable. Furthermore, the SI document could ensure that data that might not fit in the tight confines of a journal article lives online even when laboratories move on to other projects, close, and lose track of their data.

Rudshteyn, Benjamin↗

Is the Supporting Information the Venue for Reproducibility and Transparency?

Making research data, software, and data processing tools readily available to the public could significantly enhance the impact of scientific publications. Openness and transparency could address reproducibility concerns and accelerate scientific progress.(1, 2) While archives and repositories would be preferable for securing data in the long term, the Supporting Information (SI) already allocates significant space to provide auxiliary files, links, and essential information needed to make scientific findings immediately reproducible as well as data processing protocols and numerical procedures executable. Furthermore, the SI document could ensure that data that might not fit in the tight confines of a journal article lives online even when laboratories move on to other projects, close, and lose track of their data. The change for openness and transparency might be difficult to embrace, depending on the specific nature of the project. Nevertheless, the trend to make raw research data and open-source software packages available to the public has been building steady momentum. Openness in research data and software sharing is already making transformative contributions to the communication of research findings, critical for ensuring reproducibility as well as training of the next generation of scientists.

Rudshteyn, Benjamin↗

Is the Supporting Information the Venue for Reproducibility and Transparency?

Making research data, software, and data processing tools readily available to the public could significantly enhance the impact of scientific publications. Openness and transparency could address reproducibility concerns and accelerate scientific progress.(1, 2) While archives and repositories would be preferable for securing data in the long term, the Supporting Information (SI) already allocates significant space to provide auxiliary files, links, and essential information needed to make scientific findings immediately reproducible as well as data processing protocols and numerical procedures executable. Furthermore, the SI document could ensure that data that might not fit in the tight confines of a journal article lives online even when laboratories move on to other projects, close, and lose track of their data. The change for openness and transparency might be difficult to embrace, depending on the specific nature of the project. Nevertheless, the trend to make raw research data and open-source software packages available to the public has been building steady momentum. Openness in research data and software sharing is already making transformative contributions to the communication of research findings, critical for ensuring reproducibility as well as training of the next generation of scientists.

Rudshteyn, Benjamin↗

Introducing the Video In Situ Snowfall Sensor (VISSS)

The open-source Video In Situ Snowfall Sensor (VISSS) is introduced as a novel instrument for the characterization of particle shape and size in snowfall. The VISSS consists of two cameras with LED backlights and telecentric lenses that allow accurate sizing and combine a large observation volume with relatively high pixel resolution and a design that limits wind disturbance. VISSS data products include various particle properties such as maximum extent, cross-sectional area, perimeter, complexity, and sedimentation velocity. Initial analysis shows that the VISSS provides robust statistics based on up to 10 000 unique particle observations per minute. Comparison of the VISSS with the collocated PIP (Precipitation Imaging Package) and Parsivel instruments at Hyytiälä, Finland, shows excellent agreement with the Parsivel but reveals some differences for the PIP that are likely related to PIP data processing and limitations of the PIP with respect to observing smaller particles. The open-source nature of the VISSS hardware plans, data acquisition software, and data processing libraries invites the community to contribute to the development of the instrument, which has many potential applications in atmospheric science and beyond.

54 ENVIRONMENTAL SCIENCES↗

Processing Meteorological Data for the CAP-88 PC Model at Los Alamos National Laboratory

The Environmental Protection and Compliance-Compliance Programs (EPC-CP) group at Los Alamos National Laboratory (LANL) uses the Clean Air Act Assessment Package 1988 (CAP-88, Littleton 2020) PC model (Version 4.1) to estimate radiological doses for a set of areal sectors surrounding a release location, in order to satisfy the Environmental Protection Agency (EPA) National Emission Standards for Hazardous Air Pollutants (NESHAP) dose calculation requirement in 40 CFR 61 Subpart H. Among several types of data that must be prepared for CAP-88 input is a text file of meteorological data (“WIND” file), consisting of the joint frequency of wind direction, wind speed, and atmospheric stability categories. EPC-CP produces customized WIND files by running a CAP-88 utility program on a user generated text file of wind data in a different format, known as a STability ARray (STAR) file (Turner, 1964). At LANL, EPC-CP meteorologists prepare customized STAR files with data over desired time periods at selected meteorological towers. A custom program written in Precision Visuals -Workstation Analysis and Visualization Environment (PV-WAVE), a commercial Fortran-like language, is used to read LANL meteorological data and write a STAR file; the executable filename is “Star.out”. However, the outdated PV-WAVE utility program is being phased out by EPC-CP, due to the inefficient process to run it and an inability to modify the code. To preserve the ability to create customized meteorological data for CAP-88 in a way that will be easy to use and maintain, a new replacement utility program, written in the Python programming language, has been developed. The new, improved program reads a data file from any LANL meteorological tower, and at each desired observation time, determines the wind direction, wind speed, and stability categories defined in the CAP-88 documentation. The frequencies of all combinations of the three sets of categories are calculated and written to a file in the STAR format, which can later be converted to a WIND file for input into CAP-88.

54 ENVIRONMENTAL SCIENCES↗

IDC Re-Engineering Phase 2 System Requirements Document Version 1.4

This System Requirements Document (SRD) defines waveform data processing requirements for the International Data Centre (IDC) of the Comprehensive Nuclear Test Ban Treaty Organization (CTBTO). The IDC applies, on a routine basis, automatic processing methods and interactive analysis to raw International Monitoring System (IMS) data in order to produce, archive, and distribute standard IDC products on behalf of all States Parties. The routine processing includes characterization of events with the objective of screening out events considered to be consistent with natural phenomena or non-nuclear, man-made phenomena. This document does not address requirements concerning acquisition, processing and analysis of radionuclide data, but includes requirements for the dissemination of radionuclide data and products.

97 MATHEMATICS AND COMPUTING↗

CHARGE SPECTRUM OF HEAVY AND SUPERHEAVY COMPONENTS OF GALACTIC COSMIC RAYS: RESULTS OF THE OLIMPIYA EXPERIMENT

The aim of the OLIMPIYA experiment is to search for and identify traces of heavy and superheavy nuclei of galactic cosmic rays (GCR) in olivine crystals from stony–iron meteorites serving as nuclear track detectors. The method is based on layer-by-layer grinding and etching of particle tracks in these crystals. Unlike the techniques of other authors, this annealing-free method uses two parameters: the etching rate along the track ( V {sub etch}) and the total track length ( L ), to identify charge Z of a projectile. A series of irradiations with different swift heavy ions at the accelerator facilities of GSI (Darmstadt) and IMP (Lanzhou) were performed in order to determine and calibrate the dependence of projectile charge on V {sub etch} and L . To date, one of the most essential results of the experiment is the obtained charge spectrum of GCR nuclei within the range of Z > 40, based on about 11.6 thousand processed tracks. As the result of data processing, 384 nuclei with charges Z ≥ 75 have been identified, including 10 nuclei identified as actinides (90 < Z < 103). Three tracks were identified to be produced by nuclei with charges 113 < Z < 129. Such nuclei may be part of the Island of Stability of transfermium elements.

79 ASTRONOMY AND ASTROPHYSICS↗

Drone flight visible images, digital elevation maps, and geotiffs, Council, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along each flight path. This package includes data from four flight paths over the NGEE Arctic Council Road Site near mile marker 71 in the Seward Peninsula, Alaska in July 2017. Derived image products for each flight include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗