Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing and logging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Multichannel Networked Phasemeter Readout and Analysis

Netmeter software reads a data stream from up to 250 networked phasemeters, synchronizes the data, saves the reduced data to disk (after applying a low-pass filter), and provides a Web server interface for remote control. Unlike older phasemeter software that requires a special, real-time operating system, this program can run on any general-purpose computer. It needs about five percent of the CPU (central processing unit) to process 20 channels because it adds built-in data logging and network-based GUIs (graphical user interfaces) that are implemented in Scalable Vector Graphics (SVG). Netmeter runs on Linux and Windows. It displays the instantaneous displacements measured by several phasemeters at a user-selectable rate, up to 1 kHz. The program monitors the measure and reference channel frequencies. For ease of use, levels of status in Netmeter are color coded: green for normal operation, yellow for network errors, and red for optical misalignment problems. Netmeter includes user-selectable filters up to 4 k samples, and user-selectable averaging windows (after filtering). Before filtering, the program saves raw data to disk using a burst-write technique.

Edmonds, Karina↗

A semi–automatic analytical methodology for characterizing the energy consumption of MRI systems using load duration curves

Background and purpose: Magnetic resonance imaging (MRI) scanners are a major contributor to greenhouse gas emissions from the healthcare sector, and efforts to improve energy efficiency and reduce energy consumption rely on quantification of the characteristics of energy consumption. The purpose of this work was to develop a semi-automatic analytical methodology for the characterization of the energy consumption of MRI systems using only the load duration curve (LDC). LDCs are a fundamental tool used across various fields to analyze and understand the behavior of loads over time. Methods: An electric current transformer sensor and data logger were installed on two 3T MRI scanners from two vendors, termed M1 (outpatient scanner) and M2 (inpatient/emergency scanner). Data was collected for 1 month (7/11/2023 to 8/11/2023). Active power was calculated, assuming a balanced three-phase system, using the average current measured across all three phases, a 480 V reference voltage for both machines, and vendor-provided power factors. An LDC was constructed for each system by sorting the active power values in descending order and computing the cumulative time (in units of percentage) for each data point. The first derivative of the LDC was then computed (LDC’), smoothed by convolution with a window function (sLDC’), and used to detect transitions between different system modes including (in descending power levels): scan, prepared-to-scan, idle, low-power, and off. The final, segmented LDC was used to measure time (% total time), total energy (kWh), and mean power (kW) for each system mode on both scanners. The method was validated by comparing mean power values, computed using the segmented 1-month LDC, for each nonproductive system mode (i.e., prepared-to-scan, idle, lower-power, and off) against power levels measured after a deliberate system shutdown was performed for each scanner (1 day worth of data). Results: The validation revealed differences in mean power values <1.4% for all nonproductive modes and both scanners. In the scan system mode, the mean power values ranged from 29.8 to 37.2 kW and the total energy consumed for 1 month ranged from 11 106 to 14 466 kWh depending on the scanner. Over the course of 1 month, the portion of time the scanners were in nonproductive modes ranged from 76% to 80% across scanners and the nonproductive energy consumption ranged from 8010 to 6722 kWh depending on the scanner. The M1 (outpatient) scanner consumed 99.9 and 183.9 kWh/day in idle mode for weekdays and weekends, respectively, because the scanner spent 23% more time proportionally in idle mode on the weekends. Conclusions: A semi-automatic method for quantifying energy consumption characteristics of MRI scanners was introduced and validated. This method is relatively simple to implement as it requires only power data from the scanners and avoids the technical challenges associated with extracting and processing scanner log files. Finally, the methodology enables quantitative evaluation of the power, time, and energy characteristics of MRI scanners in scan and nonproductive system modes, providing baseline data and the capability of identifying potential opportunities for enhancing the energy efficiency of MRI scanners.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Millimeter-Wave Radar Field Measurements and Inversion of Cloud Parameters for the 1999 Mt. Washington Icing Sensors Project

The Mount Washington Icing Sensors Project (MWISP) was a multi-investigator experiment with participants from Quadrant Engineering, NOAA Environmental Technology Laboratory (NOAA/ETL), the Microwave Remote Sensing Laboratory (MIRSL) of the University of Massachusetts (UMass), and others. Radar systems from UMass and NOAA/ETL were used to measure X-, Ka-, and W-band backscatter data from the base of Mt. Washington, while simultaneous in-situ particle measurements were made from aircraft and from the observatory at the summit. This report presents range and time profiles of liquid water content and particle size parameters derived from range profiles of radar reflectivity as measured at X-, Ka-, and W-band (9.3, 33.1, and 94.9 GHz) using an artificial neural network inversion algorithm. In this report, we provide a brief description of the experiment configuration, radar systems, and a review of the artificial neural network used to extract cloud parameters from the radar data. Time histories of liquid water content (LWC), mean volume diameter (MVD) and mean Z diameter (MZD) are plotted at 300 m range intervals for slant ranges between 1.1 and 4 km. Appendix A provides details on the extraction of radar reflectivity from measured radar power, and Appendix B provides summary logs of the weather conditions for each day in which we processed data.

Pazmany, Andrew L.↗

TEAMER: Twin Ocean Power Wave Energy Converter Comprehensive Overview

These files collectively provide a comprehensive overview of the testing process, data analysis, and validation for the Twin Ocean Power device tested at the O.H. Hinsdale Wave Research Laboratory, supported by TEAMER funding. This resource includes an overview of power results for a series of 7 trials. The files included in this comprehensive overview include a comprehensive log sheet for each trial, a summary of all trials, and processing scripts for the raw data. It includes all raw data in .tsv and MATLAB compatible formats, an average power chart, angular velocity charts for each trial, trial metrics, and power output files. This resource includes images of the Twin Ocean Power Wave Energy Converter device components and movement during testing and video recordings of each trial.

16 TIDAL AND WAVE POWER↗

Leveraging Natural Language Processing and Generative Models in Molecular Chemistry: Property Prediction and Novel Compound Generation

The accurate prediction of molecular properties is important for the rational design and the advancement of green chemistry and sustainable materials research. However, the predictive power of traditional computational chemistry methods is limited due to computational restrictions. Here, in this study, we examine an alternative approach to the accurate prediction of properties of organic compounds: natural language processing (NLP)-based molecular embedding. Using viscosity, partition coefficient (log P), and enthalpy of vaporization as test properties through a survey of comprehensive datasets comprising 5695 data points for viscosity, 25 870 data points for log P, and 2296 data points for enthalpy of vaporization. These are important properties for the design of greener, safer, and sustainable chemical processes. Models were trained using NLP methods such as Mol2vec and fine-tuned ChemBERTa, and results were compared with traditional input featurization techniques such as Morgan fingerprints and quantum chemistry derived sigma profiles and DFT features. Among the various machine learning models, Mol2vec demonstrated superior predictive capabilities, achieving the highest correlation coefficient (R 2 = 0.945) and lowest RMSE (0.106 mPa s) for viscosity, as well as high accuracy for log P and enthalpy of vaporization predictions. These findings establish the Mol2vec featurization technique, graph-convolutional neural networks (GCNN), and fine-tuned ChemBERTa model as powerful tools for predictive modeling of organic compounds properties, offering a significant improvement over previously used featurization techniques and opening up strategies for very-high-throughput computational screening. Finally, we integrated ML models with hybrid language-model-based generative adversarial networks (LM-GAN) to generate novel molecular sequences with desirable properties for different research applications. The ability to computationally design solvents with lower viscosity, lower log P, and lower enthalpy of vaporization offers a data-driven route to accelerating the discovery of sustainable alternatives to traditionally toxic solvents.

ChemBERTa↗

RadTrac : A system for detecting, localizing, and tracking radioactive sources in real time.

Within the homeland security and emergency response communities, there is a need for a low-profile system to detect and locate radioactive sources. RadTrac has been developed at Argonne National Laboratory as an integrated system for the detection, localization, identification, and tracking of radioactive sources in real time. The system is based on a network of radiation detectors and advanced signal-processing algorithms. Features include video surveillance, automated tracking, easy setup, and logging of all data and images. This paper describes the advanced algorithms that were developed and implemented for source detection, localization, and tracking in real time. In the physio-spatial integration approach to source localization, counts from multiple detectors are processed according to the underlying physics linking these counts to obtain the probability that a source is present at any point in space. This information is depicted in a probability density function map. This type of depiction allows the results to be presented in a simple, easy-to-understand manner. It also allows for many different complicated factors to be accounted for in a single image as each factor is computed as a probability density in space. These factors include spatial limitations, variable shielding, directional detectors, moving detectors, and different detector sizes and orientations. The utility and versatility of this approach is described in further detail. Advanced signal-processing algorithms have also been incorporated to improve real-time tracking and to increase signal-to-noise ratios including temporal linking and energy binning. Measurements aimed at demonstrating the sensitivity improvements through the use of advanced signal-processing techniques were performed and are presented. Results of tracking weak sources (<100 {micro}Ci {sup 137}Cs) using four fixed-position detectors are presented.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Toward resilient high performance applications through real time reliability metric generaion and autonomous failure correction

One predominant barrier encountered in furthering research and development efforts aimed at facilitating resilient HPC applications is a substantial lack of existing reliability and performance data originating from extreme-scale computing distributions. In order to develop an understanding of how and why highly scaled HPC applications are encountering increasingly frequent performance interruptions, one must conduct extensive trending and analysis on contemporary machines and their associated programs. However, existing HPC application log files are labyrinthine documents that, even with the assistance of intelligent data mining algorithms, translate poorly to human discern. In addition, conventional log filtering, processes are limited to execution within a post-mortem, reactive time period, as the enormous size of these documents prevents efficient real time interaction. Thus, there exists a strong need within the HPC field for the provision of accurate-yet-concise real time application information. Moreover, the means of reporting this data must be sufficiently lightweight and non-intrusive, as to successfully-yet-discretely attach itself to the multiple processes running on multiple cores within tens (or in some cases, hundreds) of thousands of compute nodes. Furthermore, this information should in turn be used to facilitate the autonomous correction of application-threatening faults, suspensions, and interruptions. This paper describes a dynamic application instrumentation module (utilizing a combination of Open/SpeedShop software and custom scripting) aimed at successfully achieving these goals.

97 MATHEMATICS AND COMPUTING↗

A View from ORNL: Scientific Data Research Opportunities in the Big Data Age

One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1].

Klasky, Scott↗

Citronelle 2013 DAS VSP

The data in this archive are the distributed acoustic sensor (DAS) and well log data used in Daley, et al, 2016. Both raw and processed data are included

borehole↗

The Mobile Agents Integrated Field Test: Mars Desert Research Station April 2003

The Mobile Agents model-based, distributed architecture, which integrates diverse components in a system for lunar and planetary surface operations, was extensively tested in a two-week field "technology retreat" at the Mars Society s Desert Research Station (MDRS) during April 2003. More than twenty scientists and engineers from three NASA centers and two universities refined and tested the system through a series of incremental scenarios. Agent software, implemented in runtime Brahms, processed GPS, health data, and voice commands-monitoring, controlling and logging science data throughout simulated EVAs with two geologists. Predefined EVA plans, modified on the fly by voice command, enabled the Mobile Agents system to provide navigation and timing advice. Communications were maintained over five wireless nodes distributed over hills and into canyons for 5 km; data, including photographs and status was transmitted automatically to the desktop at mission control in Houston. This paper describes the system configurations, communication protocols, scenarios, and test results.

Clancey, William J.↗

SU-C-BRD-03: Analysis of Accelerator Generated Text Logs for Preemptive Maintenance

Purpose: To develop a model to analyze medical accelerator generated parameter and performance data that will provide an early warning of performance degradation and impending component failure. Methods: A robust 6 MV VMAT quality assurance treatment delivery was used to test the constancy of accelerator performance. The generated text log files were decoded and analyzed using statistical process control (SPC) methodology. The text file data is a single snapshot of energy specific and overall systems parameters. A total of 36 system parameters were monitored which include RF generation, electron gun control, energy control, beam uniformity control, DC voltage generation, and cooling systems. The parameters were analyzed using Individual and Moving Range (I/MR) charts. The chart limits were calculated using a hybrid technique that included the use of the standard 3σ limits and the parameter/system specification. Synthetic errors/changes were introduced to determine the initial effectiveness of I/MR charts in detecting relevant changes in operating parameters. The magnitude of the synthetic errors/changes was based on: the value of 1 standard deviation from the mean operating parameter of 483 TB systems, a small fraction (≤ 5%) of the operating range, or a fraction of the minor fault deviation. Results: There were 34 parameters in which synthetic errors were introduced. There were 2 parameters (radial position steering coil, and positive 24V DC) in which the errors did not exceed the limit of the I/MR chart. The I chart limit was exceeded for all of the remaining parameters (94.2%). The MR chart limit was exceeded in 29 of the 32 parameters (85.3%) in which the I chart limit was exceeded. Conclusion: Statistical process control I/MR evaluation of text log file parameters may be effective in providing an early warning of performance degradation or component failure for digital medical accelerator systems. Research is Supported by Varian Medical Systems, Inc.

07 ISOTOPE AND RADIATION SOURCES↗

Reservoir Engineering for Unconventional Gas Reservoirs: What Do We Have to Consider?

The reservoir engineer involved in the development of unconventional gas reservoirs (UGRs) is required to integrate a vast amount of data from disparate sources, and to be familiar with the data collection and assessment. There has been a rapid evolution of technology used to characterize UGR reservoir and hydraulic fracture properties, and there currently are few standardized procedures to be used as guidance. Therefore, more than ever, the reservoir engineer is required to question data sources and have an intimate knowledge of evaluation procedures. We propose a workflow for the optimization of UGR field development to guide discussion of the reservoir engineer's role in the process. Critical issues related to reservoir sample and log analysis, rate-transient and production data analysis, hydraulic and reservoir modeling and economic analysis are raised. Further, we have provided illustrations of each step of the workflow using tight gas examples. Our intent is to provide some guidance for best practices. In addition to reviewing existing methods for reservoir characterization, we introduce new methods for measuring pore size distribution (small-angle neutron scattering), evaluating core-scale heterogeneity, log-core calibration, evaluating core/log data trends to assist with scale-up of core data, and modeling flow-back of reservoir fluids immediately after well stimulation. Our focus in this manuscript is on tight and shale gas reservoirs; reservoir characterization methods for coalbed methane reservoirs have recently been discussed.

01 COAL, LIGNITE, AND PEAT↗

Performance Analysis of Large Scale HPC Workflows for Earth System Models

Climate assessment reports, such as the Intergovernmental Panel on Climate Change’s 5th assesment report [IPCC AR5; IPCC et al., 2014] or the United States National Climate Assessment [U.S. Global Change Research Program, 2014], increasingly rely on many-ensemble climate projections from fully-coupled Earth system models (ESM) to asses possible future climate states. ESM participate in a large set of Coupled Model Intercomparson Projects (CMIP) to better understand their models and the climate system, so that they may provide credible climate projections. However, fully participating in these intercomparison exercises represents an extremely large computational personnel burden for modeling centers [Eyring et al., 2016, Williams et al., 2016a]. However, there has been little attention within the climate modeling community paid to the in situ performance of ESMs up until now. Recently a computational performance model intercomparison project (CPMIP) for CMIP6 has begun [Balaji et al., 2017], but there has not yet been an effort to gather, process, and analyze the historical performance data that has been saved within archived log files from previous modeling efforts, nor the development of performance analysis system capable of providing detailed statistical analyses or visualizations of this performance data. Therefore, we’ve begun a joint effort to analyze the historical and current performance data on both GFDL and OLCF modeling systems. Here, we have developed a preliminary performance analysis system to captue, analyize, and visualize end-to-end in situ performance data and applied it to NOAA/GFDL’s post-processing system. Through the course of the project a number of performance improvements have been made, with the steps necessary to handle the expected volume of data (Big Data) likely to come out of the CMIP6 exercises (TB to PB) provided. We have also begun developing a statistical workload model that will allow modeling centers to determined their expected scientific throughput in the context of expected resource (computational and personnel) availability and accounting for typical workflow disruptions. This will allow them to develop a CMIP participation strategy and maximize their level of participation in the exercises. We expect this infrastructure, and the data it produces, to ultimately allow us to identify performance issues and workflow errors that exist across Earth system modeling centers, and possibly HPC modeling centers in general. Center-independent solutions to workflow or performance issues would help alleviate the computational burden for CMIP6+ modeling projects and should realize both labor and cost savings benefits throughout the CMIP community.

54 ENVIRONMENTAL SCIENCES↗

When Workflow Management Systems and Logging Systems Meet: Analyzing Large-Scale Execution Traces

This poster shows the benefits of integrating a workflow management system with logging and log mining capabilities. By combing two existing, mature technologies: Pegasus-WMS and Netlogger, we are able to efficiently process execution logs of earthquake science workflows consisting of hundreds of thousands to one million tasks. In particular we show results of processing logs of CyberShake, a workflow application running on the TeraGrid. Client-side tools allow scientists to quickly gather statistics about a workflow run and find out which tasks executed, where they were executed, what was their runtime, etc. These statistics can be used to understand the performance characteristics of a workflow and help tune the execution parameters of the workflow management system. This poster shows the scalability of the system presenting results of uploading task execution records into the system and by showing results of querying the system for overall workflow performance information.

97 MATHEMATICS AND COMPUTING↗

Hanford single-shell tanks leak causes and locations - 15509

Historically 67 assumed and known leaking tanks in the single-shell tank (SST) system were identified at the Hanford Site. The US DOE, Washington River Protection Solutions LLC., and the State of Washington, Department of Ecology identified 25 of the 67 tanks as likely having had liner leaks by a comprehensive review of current and historical data. The remaining tanks are those with potential leaks from sources other than liner failures such as overfilling the tanks or the failure of process lines connecting SSTs. The location of the leaks was determined, based largely on logging radiological data from dry wells installed around the tank and/or radiological data from laterals installed under select tanks. The dry wells and laterals provide the best indication of the vicinity of the leak location, but not necessarily the exact location of the leak. Twenty of the 25 liner leaks appear to have occurred at or near the bottom of the tanks. Many of the tanks appeared to have multiple leak locations. Leak locations could not be determined for all tanks with the available data. Three of the tanks showed apparent sidewall leaks. Potential causes for liner leaks that were evaluated included tank design features, tank construction conditions, waste temperature, tank waste chemistry conducive to different types of corrosion (uniform corrosion, nitrate induced stress corrosion cracking, pitting, crevice, and liquid-air interface corrosion), and bulges in the tank bottom. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Informing Plant Asset Reliability and Availability Through AI-Driven Analysis of Operator Logs

The availability and reliability of nuclear power plant (NPP) structures, systems, and components (SSCs) are critical parameters for NPP safety. Tracking these parameters is necessary but costly and labor-intensive, requiring the collection and evaluation of SSC event data such as shutdowns, startups, and failures. To show how these events are needed for the parameters an example is given: one measure of reliability is based on the number of equipment failure events and the number of run hours (i.e., the time from a startup event to a shutdown event). Here, this work investigates using artificial intelligence (AI) to mine NPP operator log entry texts for SSC event data. Four AI approaches were explored for identifying these events, including natural language processing (NLP) methods, generative AI, generative AI combined with NLP, and topic modeling. A key challenge addressed with all four approaches is the brevity of operator log entries. Among these four a neural network–based NLP method was shown to be the most promising for this application, achieving F1 scores of 86.0% for shutdowns, 92.2% for startups, and 80.4% for failures on a subject-matter-expert-curated dataset from NPP operator logs, compared to a baseline of 66.6% for a random classifier. This shows that NLP methods can perform better than generative AI. Additionally, the NLP methods combined with generative AI were shown to perform better than generative AI alone. Generative AI was most successful at providing the background information for the NLP methods to use. This work demonstrates the potential to use AI to automate parameter collection from NPP operator log entries and other records.

97 - MATHEMATICS AND COMPUTING↗

Contrasting Patterns of Damage and Recovery in Logged Amazon Forests From Small Footprint LiDAR Data

Tropical forests ecosystems respond dynamically to climate variability and disturbances on time scales of minutes to millennia. To date, our knowledge of disturbance and recovery processes in tropical forests is derived almost exclusively from networks of forest inventory plots. These plots typically sample small areas (less than or equal to 1 ha) in conservation units that are protected from logging and fire. Amazon forests with frequent disturbances from human activity remain under-studied. Ongoing negotiations on REDD+ (Reducing Emissions from Deforestation and Forest Degradation plus enhancing forest carbon stocks) have placed additional emphasis on identifying degraded forests and quantifying changing carbon stocks in both degraded and intact tropical forests. We evaluated patterns of forest disturbance and recovery at four -1000 ha sites in the Brazilian Amazon using small footprint LiDAR data and coincident field measurements. Large area coverage with airborne LiDAR data in 2011-2012 included logged and unmanaged areas in Cotriguacu (Mato Grosso), Fiona do Jamari (Rondonia), and Floresta Estadual do Antimary (Acre), and unmanaged forest within Reserva Ducke (Amazonas). Logging infrastructure (skid trails, log decks, and roads) was identified using LiDAR returns from understory vegetation and validated based on field data. At each logged site, canopy gaps from logging activity and LiDAR metrics of canopy heights were used to quantify differences in forest structure between logged and unlogged areas. Contrasting patterns of harvesting operations and canopy damages at the three logged sites reflect different levels of pre-harvest planning (i.e., informal logging compared to state or national logging concessions), harvest intensity, and site conditions. Finally, we used multi-temporal LiDAR data from two sites, Reserva Ducke (2009, 2012) and Antimary (2010, 2011), to evaluate gap phase dynamics in unmanaged forest areas. The rates and patterns of canopy gap formation at these sites illustrate potential issues for separating logging damages from natural forest disturbances over longer time scales. Multi-temporal airborne LiDAR data and coincident field measurements provide complementary perspectives on disturbance and recovery processes in intact and degraded Amazon forests. Compared to forest inventory plots, the large size of each individual site permitted analyses of landscape-scale processes that would require extremely high investments to study using traditional forest inventory methods.

Morton, D. C.↗

Summit Darshan Archival Dataset

Summit Darshan Archival Dataset contains 2021 Summit Darshan log data for 25 applications and is grouped into science domains. The dataset is processed, and all the propriety fields are anonymized. The resultant data is converted into a tabular structure and saved in parquet file format. In this notebook, we demonstrate how to access the data. Data Organization: The data is organized into two directories: Darshan total (`darshan_total`): List all the high levels generated by the `darshan-parser --total` command on `.darshan` files. There is one parquet file for each application. Note: `uid` and `exe` field are masked Darshan detail (`darshan_detail`): This data contains detailed job level log information extracted by command `darshan-parser` on the raw `.darshan` files. The data is sorted by directory hierarchy in the order of `year/month/day (2021/12/07)`. For instance, to get the data for a `job_id` 3819766 of application `App11`, which was executed on `2021-12-07`can be accessed as follows. Note:`uid` and `filename` fields are masked

97 MATHEMATICS AND COMPUTING↗