Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing and logging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

An evaluation of short-wave-infrared (SWIR) data from the AVIRIS and GEOSCAN instruments for mineralogical mapping at Cuprite, Nevada

An evaluation was performed on SWIR (2000-2400 nm) data from two airborne remote sensing systems for discriminating and identifying alteration minerals at Cuprite, Nevada. The data were acquired by the NASA Airborne Visible/InfraRed Imaging Spectrometer (AVIRIS) and the GEOSCAN Mk II multispectral scanner. The evaluation involved comparison of processed imagery and image-derived spectra with existing alteration maps and laboratory spectra of rock samples from Cuprite. Results indicate that both the AVIRIS and GEOSCAN data permit the discrimination of areas of alunite, buddingtonite, kaolinite, and silicification using color composite images formed from three SWIR bands processed with either the decorrelation stretch or a log residual algorithm. The laboratory spectral features alunite, kaolinite and buddingtonite could be seen clearly only in the log residual processed AVIRIS data. However, this does not preclude their identification with the GEOSCAN data.

Hook, Simon J.↗

Hardening of DOE UQ Codes for Ultra-Deep Proactive Geosteering in Oilfield Service

The primary objective of this project was to harden the DOE uncertainty quantification (UQ) code to solve the inverse problem for the geosteering service in the oilfield drilling and exploration. The primary focus is to verify the feasibility of using QUESO to solve geosteering inverse problems through statistical inversion. QUESO is a software library developed by QUEST, a DOE's Scientific Discovery through Advanced Computing (SciDAC) institute. In the process of drilling a borehole, geosteering is the act of adjusting the well trajectory on the fly to reach the geological targets or to keep the wellbore within desired formation. Geosteering inversion is to infer underground formation structure using measurements such as azimuthal resistivity logging data. The geosteering process relies on the real-time reconstruction of the subsurface earth model, and a robust and efficient inversion algorithm is always necessary to satisfy the job requirement. The new generation of azimuthal resistivity geosteering tool is capable of detecting multiple formation boundaries 20-100 ft away, and the corresponding geosteering inversion problem can be very challenging as many parameters need to be inverted within such a large depth of investigation. The success of the project will support US energy security by helping to advance subsurface modeling and characterization technologies in oilfield applications. An effective subsurface inversion algorithm and the implementation as a software package will promote the usage of ultra-deep azimuthal resistivity measurements, which can help well placement in complicated underground environments, reduce costs and risks of drilling, and increase oil and gas production. It will help medium and small size independent service companies and tool manufactures to enter the more advanced and lucrative directional drilling area. In addition, by hardening the QUESO code and building a transparent interface for easy manipulation of the QUESO functions, we provide a user-friendly tool for a larger scientific and industry communities that rely heavily on modeling and simulation tools for forecasting, parameter studies, design, and decision making. These are all activities that can strongly benefit from meaningful representations of modeling uncertainty supported by our product.

58 GEOSCIENCES↗

Global distributed file append using log-structured file system

Techniques are provided for globally appending data from a group of distributed processes to a shared file using a log-structured file system. Data generated by a plurality of processes in a parallel computing system are appended to a shared file by storing the data to the shared file using a log-structured file system (such as a Parallel Log-Structured File System (PLFS)); and generating an index entry for the data, the index entry comprising a logical offset entry and a timestamp entry indicating a time of the storage, wherein the logical offset entry is resolved at read time. The logical offset entry can be populated with an append placeholder that is resolved when the shared file is read. At read time, a plurality of the index entries associated with the shared file can be sorted using the timestamp entry to deliver the requested shared file to a requesting application.

97 MATHEMATICS AND COMPUTING↗

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe- art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.

97 MATHEMATICS AND COMPUTING↗

Orchestrator Telemetry Processing Pipeline

Orchestrator is a software application infrastructure for telemetry monitoring, logging, processing, and distribution. The architecture has been applied to support operations of a variety of planetary rovers. Built in Java with the Eclipse Rich Client Platform, Orchestrator can run on most commonly used operating systems. The pipeline supports configurable parallel processing that can significantly reduce the time needed to process a large volume of data products. Processors in the pipeline implement a simple Java interface and declare their required input from upstream processors. Orchestrator is programmatically constructed by specifying a list of Java processor classes that are initiated at runtime to form the pipeline. Input dependencies are checked at runtime. Fault tolerance can be configured to attempt continuation of processing in the event of an error or failed input dependency if possible, or to abort further processing when an error is detected. This innovation also provides support for Java Message Service broadcasts of telemetry objects to clients and provides a file system and relational database logging of telemetry. Orchestrator supports remote monitoring and control of the pipeline using browser-based JMX controls and provides several integration paths for pre-compiled legacy data processors. At the time of this reporting, the Orchestrator architecture has been used by four NASA customers to build telemetry pipelines to support field operations. Example applications include high-volume stereo image capture and processing, simultaneous data monitoring and logging from multiple vehicles. Example telemetry processors used in field test operations support include vehicle position, attitude, articulation, GPS location, power, and stereo images.

Powell, Mark↗

Automatic Calibration of a Geomechanical Model from Sparse Data for Estimating Stress in Deep Geological Formations

Summary In this study, we demonstrate geomechanical modeling with fully automatic parameter calibration to estimate the full geomechanical stress fields of a prospective US carbon dioxide (CO2) storage site, based on sparse measurement data. The goal is to compute full stress tensor field estimates (principal stresses and orientations) that are maximally compatible with observations within the constraints of the model assumptions, thereby extending pointwise, incomplete partial stress measurement to a simulated full formation stress field, as well as a rough assessment of the associated error. We use the Perch site, located in Otsego County, Michigan, USA, as our case study. The input data consist of partial stress tensor information inferred from in-situ borehole tests, geophysical well logs, and processing of seismic data. A static earth model (SEM) of the site was developed, and geomechanical simulation functionality of the open-source MATLAB Reservoir Simulation Toolbox (MRST) was used to model the stress field. Adjoint-based nonlinear optimization was used to adjust boundary conditions and material properties to calibrate simulated results of observations. Results were interpreted through a Bayesian framework. The focus of this paper is to demonstrate how the fully automatic calibration procedure works and discuss the results obtained; it does not attempt a detailed analysis of the stress field in the context of the proposed CO2 storage initiatives. Our work is part of a larger effort to noninvasively determine in-situ stresses in deep formations considered for CO2 storage. Guided by previously published research on geomechanical model calibration, our work presents a novel calibration approach supporting a potentially large number of linear or nonlinear calibration parameters to produce results optimally agreeing with available measurements and thus extend partial pointwise estimates to full tensor fields compatible with the physics of the site.

Engineering↗

Utah FORGE: Updated FMI Fracture Log from Well 16A(78)-32

This dataset consists of an Excel spreadsheet detailing the fracture picks from a reinterpretation of the formation micro-imaging (FMI) log from Utah FORGE well 16A(78)-32. The provided information details fracture location, geometry, and type. Also included here is a link to the original raw and processed FMI logs, as well as other data from the 2021 well logging.

15 GEOTHERMAL ENERGY↗

On Fast Post-Processing of Global Positioning System Simulator Truth Data and Receiver Measurements and Solutions Data

Post-Processing of data related to a Global Positioning System (GPS) simulation is an important activity in qualification of a GPS receiver for space flight. Because a GPS simulator is a critical resource it is desirable to move off the pertinent simulation data from the simulator as soon as a test is completed. The simulator data files are usually moved to a Personal Computer (PC), where the post-processing of the receiver logged measurements and solutions data and simulated data is performed. Typically post-processing is accomplished using PC-based commercial software languages and tools. Because of commercial software systems generality their general-purpose functions are notoriously slow and more than often are the bottleneck problem even for short duration experiments. For example, it may take 8 hours to post-process data from a 6-hour simulation. There is a need to do post-processing faster, especially in order to use the previous test results as feedback for a next simulation setup. This paper demonstrates that a fast software linear interpolation algorithm is applicable to a large class of engineering problems, like GPS simulation data post-processing, where computational time is a critical resource and is one of the most important considerations. An approach is developed that allows to speed-up post-processing by an order of magnitude. It is based on improving the post-processing bottleneck interpolation algorithm using apriori information that is specific to the GPS simulation application. The presented post-processing scheme was used in support of a few successful space flight missions carrying GPS receivers. A future approach to solving the post-processing performance problem using Field Programmable Gate Array (FPGA) technology is described.

Kizhner, Semion↗

HERO WEC V1 Upgrade - 2023 Laboratory Testing (processed data)

The following submission includes processed laboratory data from NREL's Hydraulic and Electric Reverse Osmosis Wave Energy Converter (HERO WEC), in the form of MATLAB workspaces. This dataset was created using NREL's Large Amplitude Motion Platform (LAMP) and collected between August and September 2023. Included with this submission is a test log of all the processed data "HERO WEC LAMP test run log.xlsx" so that the user can easily find the data of interest. Additionally, more detailed descriptions of the type of data and how it was processed, or calculated, can be found in the document titled "Lamp Data Description.docx". The MATLAB workspaces can be visualized using the file "LAMP_Data_Viewer_Ver2.m/mlx". The user simply needs to upload the workspace of interest and run the file "LAMP_Data_Viewer_Ver2.m/mlx". Both the .m and .mlx file format has been provided depending on the user's preference. The MATLAB workspaces have been separated into zip files corresponding to either Drivetrain, Hydraulic, or Electric configuration runs representing the respective test cases that were run. The drivetrain runs were used to characterize the drivetrain only (no pump or generator). The Hydraulic runs represent the configuration when the seawater pump is installed, and the Electric runs represents the configuration when the generator is installed. The following sub-categories of data are included for each type: - DW - Deep water sine wave profile (not run in drivetrain configuration) - Heave - Heave only sine wave profile - Heave_NoRO (hydraulic configuration only) - Heave_ACC (hydraulic configuration only) - IR - Surge and heave irregular wave profile (not run in drivetrain configuration) - RW - Heave only profile created from real world encoder data (not run in drivetrain configuration) For those interested in the raw, unprocessed, data the authors have created a separate submission, linked below. This submission includes the raw TDMS files and associated files necessary to translate the data into either python or MATLAB formats. This data set has been developed by the National Renewable Energy Laboratory, operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. Funding provided U.S. Department of Energy Office of Energy Efficiency and Renewable Energy Water Power Technologies Office.

16 TIDAL AND WAVE POWER↗

Utah FORGE Well 78B-32 Daily Drilling Reports and Logs

This data set includes the daily drilling reports and Pason data for well 78B-32 and Schlumberger logs acquired after drilling completion. This well was drilled between June 27th and July 31st of 2021. Also included is raw and processed data for a variety of well data metrics including temperature, porosity, density, and sonic data. This data was taken at the Utah FORGE site as part of the Utah FORGE project.

15 GEOTHERMAL ENERGY↗

PATHA: Performance Analysis Tool for HPC Applications

Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Radiation Mapping UAV [Poster]

The NuMI (Neutrinos at the Main Injector) target system delivers a high-energy flux of muon-neutrinos towards a detector more than 800 km away with the goal of better understanding the origin of the universe and its evolution. The target system becomes radioactive because it is irradiated by a sub-Mega-Watt power proton beam. Radiological hazard is one of critical issues for a radiological worker which limits the iractivities near the target system; however, they often need to quantify the radiation level of the target system manually. Thus, the objective of this project is to reduce their exposure to radiation while improving the accuracy of the measurements. A quadcopter UAV fitted with a radiation dosimeter is ideal to traverse the irradiated environment as it can be deployed in any space and requires minimal hardware integration to the test room. Due to the lack of indoor GPS signal, this UAV utilizes a series of ultrasonic beacons that release several varied-frequency pulses simultaneously in order to localize the drone. The flight controller returns a 3D coordinate tied to the radiation data to understand precisely where radiation spikes are being emitted from the horn. The flight controller stores log data that can be processed into an easily interpretable map of radiation intensities. The radiation map serves as one of the primary references for planning the transportation of the NuMI horn.

43 PARTICLE ACCELERATORS↗

System and method for passive assay of holdup deposits in nuclear piping

Disclosed herein are a method, apparatus, and software for passive, non-destructive assay of holdup deposits in nuclear piping by in-pipe apparatus. A detector deployed within a pipe is collimated to observe radiation impinging radially inward from decay of deposits that lie on the pipe wall. A radiation detector is centered in the pipe and collimated by a pair of coaxial shielding discs disposed equidistant from the detector. This arrangement causes radiation from a truncated cylinder of pipe deposit within a field of regard to impinge on the detector, while precluding radiation emanating from pipe walls beyond the field of regard from reaching the detector. Hence, observations are unique to a known cylindrical length of pipe. The detector assembly is translated through pipes by an autonomous mobile robotic apparatus. Computer software controls the robotic apparatus, logs data, and post-processes to assay deposits.

Whittaker, William Lawrence↗

A Performant, Scalable Processing Pipeline for High‐Quality and FAIR Environmental Sensor Data

High-resolution environmental monitoring is necessary to record, understand, and predict biogeochemical and ecological changes particularly in coastal systems but brings significant challenges in processing and making rapidly available the resulting data. The COMPASS-FME project established a network of coastal observational sites across the Chesapeake Bay and western Lake Erie regions extensively instrumented with soil, vegetation, and weather sensors logging data every 15 min. Our data processing framework, written in R and completely open source, prioritizes rapid model-experiment iteration and makes biogeochemical data rapidly available for quality assurance/quality control, analysis, and model ingestion. This pipeline is distinguished by a standardized and modular approach to data curation, extensive metadata and documentation, and its high performance. These attributes combine to make biogeochemical data rapidly accessible across COMPASS-FME and the broader community. Flexible, powerful, and reproducible approaches to handling high-volume environmental data are crucial for accelerating biogeosciences research.

Pennington, Stephanie C. [Pacific Northwest Nation↗

Comparative Assessment of Data-driven Process Models in Health Information Technology

Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

97 MATHEMATICS AND COMPUTING↗

April 2020 Darshan counters from the Summit supercomputer

This dataset is the Darshan counters collected from the Summit supercomputer in a month of April 2020. 1. Description of methods used for collection/generation of data: Job submitted on Summit HPC system when completed successfully and has made I/O calls (captured by Darshan tool) writes a Darshan log file on alpine filesystem. One job can have multiple `jsrun` commands and Darshan will generate separate logs each log corresponding to an `jsrun` command, so a job can have one or more Darshan logs associated with it. 2. Methods for processing the data: To process the data, we first use `darshan-util` tool to parse the Darshan logs. Then we restructure the logs and merge data from multiple Darshan logs if they belong to the same Summit job.

97 MATHEMATICS AND COMPUTING↗

Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants (4th Annual Report)

Nuclear plant sites collect and store large volumes of data collected from various equipment and systems. These datasets typically include plant process parameters, maintenance records, technical logs, online monitoring data, and equipment failure data. The collection of such data affords an opportunity to leverage data-driven machine learning and artificial intelligence technologies to provide diagnostic and prognostic capabilities within the nuclear power industry to reduce operating and maintenance costs. In this way, nuclear energy can become more economically competitive with other energy sources, and premature closures can be avoided. From a maintenance standpoint, savings can be achieved by leveraging machine learning and artificial intelligence technologies to develop data-driven algorithms to better diagnose and predict potential faults within the system. Improved model accuracy can lead to reductions in unnecessary maintenance and more efficient planning of future maintenance, thus lowering the costs associated with parts, labor, and unnecessary planned, forced, or extended outages. From an operations perspective, cost savings can be generated by shifting from route-based monitoring to wireless technologies for online monitoring, and by transitioning from onsite- to cloud-based computing and storage services. Wireless monitoring would reduce the operator manhours required for taking routine measurements, while cloud computing services would generate cost savings by reducing the amount of hardware needing to be purchased and maintained—all while scaling to both computational and storage demands. This report summarizes this project’s effort to shift from costly, labor-intensive preventative maintenance to cheaper predictive maintenance.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

275 C Downhole Microcomputer System

An HC11 controller IC and along with serial SRAM and ROM support ICs chip set were developed to support a data acquisition and control for extreme temperature/harsh environment conditions greater than 275 C. The 68HC11 microprocessor is widely used in well logging tools for control, data acquisition, and signal processing applications and was the logical choice for a downhole controller. This extreme temperature version of the 68HC11 enables new high temperature designs and additionally allows 68HC11-based well logging tools and MWD tools to be upgraded for high temperature operation in deep gas reservoirs, The microcomputer chip consists of the microprocessor ALU, a small boot ROM, 4 kbyte data RAM, counter/timer unit, serial peripheral interface (SPI), asynchronous serial interface (SCI), and the A, B, C, and D parallel ports. The chip is code compatible with the single chip mode commercial 68HC11 except for the absence of the analog to digital converter system. To avoid mask programmed internal ROM, a boot program is used to load the microcomputer program from an external mask SPI ROM. A SPI RAM IC completes the chip set and allows data RAM to be added in 4 kbyte increments. The HC11 controller IC chip set is implemented in the Peregrine Semiconductor 0.5 micron Silicon-on-Sapphire (SOS) process using a custom high temperature cell library developed at Oklahoma State University. Yield data is presented for all, the HC11, SPI-RAM and ROM. The lessons learned in this project were extended to the successful development of two high temperature versions of the LEON3 and a companion 8 Kbyte SRAM, a 200 C version for the Navy and a 275 C version for the gas industry.

03 NATURAL GAS↗