Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing and logging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Secure Data Logging and Processing with Blockchain and Machine Learning (Final Report)

Secure Data Logging and Processing with Blockchain and Machine Learning (ML) research is focused on the development of a platform to securely log and process sensor data in fossil power plants. The platform integrates two emerging technologies, blockchain and ML, and incorporates several innovative mechanisms to ensure the integrity, reliability, and resiliency of power systems. The goal is to protect the power plant from various cyberattacks such as false data injection and denial of service attacks using these technologies. The research goal was enabled by the following Research Project Objectives: 1) Secure authentication and identity verification of sensor nodes, actuators, and other equipment within a network. 2) Development of mechanisms that ensure only data sent by legitimate sensors are accepted and stored in the data repository. 3) Development of data aggregation methodologies using ML / Deep Learning (DL) algorithms to minimize noise / faulty data. 4) Implementation of the blockchain technologies to provide data security using secured IOTA framework & nodes.

20 FOSSIL-FUELED POWER PLANTS↗

Big Data Meets HPC Log Analytics: Scalable Approach to Understanding Systems at Extreme Scale

Today's high-performance computing (HPC) systems are heavily instrumented, generating logs containing information about abnormal events, such as critical conditions, faults, errors and failures, system resource utilization, and about the resource usage of user applications. These logs, once fully analyzed and correlated, can produce detailed information about the system health, root causes of failures, and analyze an application's interactions with the system, providing valuable insights to domain scientists and system administrators. However, processing HPC logs requires a deep understanding of hardware and software components at multiple layers of the system stack. Moreover, most log data is unstructured and voluminous, making it more difficult for system users and administrators to manually inspect the data. With rapid increases in the scale and complexity of HPC systems, log data processing is becoming a big data challenge. This paper introduces a HPC log data analytics framework that is based on a distributed NoSQL database technology, which provides scalability and high availability, and the Apache Spark framework for rapid in-memory processing of the log data. The analytics framework enables the extraction of a range of information about the system so that system administrators and end users alike can obtain necessary insights for their specific needs. We describe our experience with using this framework to glean insights from the log data about system behavior from the Titan supercomputer at the Oak Ridge National Laboratory.

Park, Byung↗

Deactivation and decommissioning web log analysis using big data technology - 15710

The D and D Knowledge Management Information Tool (D and D KM- IT) is a web-based knowledge management information tool built for the D and D user community. Big data is a massive volume of structured and unstructured datasets which is so large that it's difficult to process using traditional database techniques. Web logs are the repository of files which are generated automatically for any operation on the web site. Web log files generated from the D and D KM-IT will be processed using the Apache Hadoop Framework to extract meaningful data. Hadoop is an open-source software framework for storing and processing big data in a distributed environment on large clusters of commodity hardware. Hadoop framework consists of two main layers: Hadoop distributed file system (HDFS) and the Execution engine (MapReduce). Automatic parallelization and distribution; clean and simple programming abstraction. Implementation: Web log files from the D and D KM-IT server are fetched using Apache Flume and loaded into the Hadoop distributed file system. Using the Apache Hcatalog tool, the web log data is parsed and converted into a structured table format. A Pig program is developed to query the tables to retrieve and store the significant information onto the HDFS. The extracted data is fed to business visualization tools such as Microsoft Excel for analysis and reporting. Results The reports generated through the web log data processing will be analyzed to improve the usability and performance of the D and D KM-IT. Conclusion/Future Work: User keyword information can be explored to add new trending topics to the D and D KM-IT application. Statistics generated can be compared to analysis reports from existing analytical tools.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Approach to qualification using E-PBF in-situ process monitoring in Ti-6Al-4V

Traditional design and qualification methodologies for parts manufactured by traditional methods are being applied to Additive Manufacturing (AM) without understanding the nuances of the machines. While mapping process variables and tracking build data is helpful, some variables such as build geometry, support structure, and part melt order have not been researched in depth. Changing these variables can result in significant variations in material properties and defect structure such that the process appears to be unreliable compared to traditional manufacturing. Hence, this research focuses on the need to understand the effects of overlooked variables such as melt order and nested geometry on the distribution of defects and bulk material properties in Ti-6Al-4 V alloy builds manufactured using the Arcam AB ® electron beam powder bed fusion process 1 . This work collected and analyzed process log data and near infrared (NIR) images for every layer to correlate trends in porosity formation and mechanical performance. The location of pores, while naturally stochastic, is greatly influenced by the cross-sectional area as detected by NIR images and correlates with the failure sites from uniaxial testing.

36 MATERIALS SCIENCE↗

Poisson Log-Normal Process for Count Data Prediction

Modeling count data is important in physics and other scientific disciplines, where measurements often involve discrete, non-negative quantities such as photon or neutrino detection events. Traditional parametric approaches can be trained to generate integer-count predictions but may struggle with capturing complex, non-linear dependencies often observed in the data. Gaussian process (GP) regression provides a robust non-parametric alternative to modeling continuous data; however, it cannot generate integer outputs. We propose the Poisson Log-Normal (PoLoN) process, a framework that employs GP to model Poisson log-rates. As in GP regression, our approach relies on the correlations between data points captured via GP kernel structure rather than explicit functional parameterizations. We demonstrate that the PoLoN predictive distribution is Poisson-LogNormal and provide an algorithm for optimizing kernel hyperparameters. Furthermore, we adapt the PoLoN approach to the problem of detecting weak localized signals superimposed on a smoothly varying background - a task of considerable interest in many areas of science and engineering. Our framework allows us to predict the strength, location and width of the detected signals. We evaluate PoLoN's performance using both synthetic and real-world datasets, including the open dataset from CERN which was used to detect the Higgs boson at the Large Hadron Collider. Our results indicate that the PoLoN process can be used as a non-parametric alternative for analyzing, predicting, and extracting signals from integer-valued data.

Saha, Anushka [Rutgers U., Piscataway]↗

MCPC Friction Stir Welding (FSW) Process Data

Processing parameters and machine log data for the MCPC LDRD Agile investment is collected material samples processed. This dataset captures the selected processing parameters, machine logs captured during material processing, and descriptions of how characterization samples were extracted from processed plates of material. The collect characterization data is captured in other datasets.

316 Stainless Steel↗

Utah FORGE: Injection and Production Test results and Reports from August 2024

This dataset includes results and reports from the injection/production test for wells 16A(78)-32 and 16B(78)-32 in August, 2024 at Utah FORGE. Materials include injection, post stimulation production, flow rate, gamma, temperature and pressure survey results. For the respective wells, injection and production profile results and interpretations are provided in .xlsx and .las file formats. Additionally, for both wells, preliminary and final reports are included and provide logging procedures and visualizations of the results.

15 GEOTHERMAL ENERGY↗

One Way of Testing a Distributed Processor

Launch processing for Space Shuttle is checked out, controlled, and monitored with new system. Entire system can be exercised by two computer programs--one in master console and other in each of operations consoles. Control program in each operations console detects change in status and begins task initiation. All of front-end processors are exercised from consoles through common data buffer, and all data are logged to processed-data recorder for posttest analysis.

Edstrom, R.↗

EGS Collab Experiment 3: 4100 Tensile Stimulation and Thermal Circulation Testing

These data and test descriptions are from a set of primarily tensile hydraulic-fracture stimulations in wells E2-TC and E2-TU and a subsequent chilled water circulation test conducted by injecting in well E2-TU on the 4100 level of the Sandford Underground Research Facility (SURF). Stimulations were carried out between April and May of 2022. The thermal circulation test ran semi-continuously from May 19 through August 26, 2022, though chilled water injection began on June 3. More information about the test, rationale, and processing of data is available on the EGS Collab project page, which is linked below.

15 GEOTHERMAL ENERGY↗

Idaho National Laboratory Data Acquisition And Processing System

The INLDAS data acquisition and processing system is designed to develop prototype measurements and real-time processing techniques. The INLDAS hardware and firmware currently consists of National Instruments • NI-DAQmx 14.1 • cDAQ-9184 • 9205 • 9211 INLDAS can perform of standard data acquisition functions as well as novel functions and real-time processing algorithms. There are 3 acquisition modes to choose from: • Continuous mode runs when the user hits start, processing data and logging it to file • SWTrigger mode waits for one or more predefined triggers before acquiring data. It will buffer data as well, so it can record data that happened shortly before the trigger • Wakeup mode waits on predefined timers. When a time goes off, it acquires a preset amount of data There are also 3 Data Processing Modes • Normal mode does no processing besides the rolling average • FFT Mode Performs an FFT on incoming data every time the time window has elapsed • STFFT mode

Smith, JamesA↗

On Road Testing Data

This dataset provides the following on road testing data: - Videos - In-vehicle dash camera videos during different testing scenarios. - Signal controller data - NTCIP log data and processed signal timing data from the corresponding signal controllers - Vehicle data - Vehicle data recorded during the testing, including GNSS, communication, CAN signals.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Dunham Pad A 4H Well

Medical CT scans, raw and processed core logging data (including XRF) and core images of the Dunham Pad A 4H well, obtained by Triana Energy, as supplemental information to the NETL Technical Report Series Crandall, D.; Brown, S.; Moore, J.; Martin, K.; Mackey, P.; Carr, T. CT Scanning and Geophysical Measurements of the Marcellus Formation from the Dunham Pad A 4H Well NETL-TRS-X-2018; NETL Technical Report Series; U.S. Department of Energy, National Energy Technology Laboratory: Morgantown, WV, 2018; p 48.

Core Logs↗

Utah FORGE: Well 16A(78)-32 and 16B(78)-32 Stimulation Production Logging Tests

These are production logging tests (PLT) results from Halliburton collected during the nine hour circulation test performed at Utah FORGE on April 27th, 2024. The circulation test involved injecting into well 16A(78)-32 and producing out of well 16B(78)-32) and was carried out after the fracturing of both of those wells. Data attached here includes raw and processed PLT logs from both wells, as well as technical reports summarizing the data collection and results.

15 GEOTHERMAL ENERGY↗

LANL HPC Data Center Monitoring (Summer 2022 Talk) [Slides]

HPC Monitoring is tasked with the understanding of everything that happens in the Data Center. They get to that understanding by: collecting logs and metrics, processing data, analyzing data, displaying information, and receiving alerts on events. Their ultimate (and probably unreachable) goal is a holistic view of the Data Center and an understanding of how everything is correlated.

97 MATHEMATICS AND COMPUTING↗

ORPC RivGen 2.0 at Igiugig, AK Operational Data 2019 and 2020

Information enclosed includes operational data from the ORPC RivGen 2.0 deployed at Igiugig, AK in 2019 and 2020. Flow velocities and device performance metrics are recorded along with a report detailing flow speed characterization where Igiugig river flow data is aggregated into a single flow velocity.

16 TIDAL AND WAVE POWER↗

Process mining for healthcare: Characteristics and challenges

Process mining techniques can be used to analyse business processes using the data logged during their execution. These techniques are leveraged in a wide range of domains, including healthcare, where it focuses mainly on the analysis of diagnostic, treatment, and organisational processes. Despite the huge amount of data generated in hospitals by staff and machinery involved in healthcare processes, there is no evidence of a systematic uptake of process mining beyond targeted case studies in a research context. When developing and using process mining in healthcare, distinguishing characteristics of healthcare processes such as their variability and patient-centred focus require targeted attention. Against this background, the Process-Oriented Data Science in Healthcare Alliance has been established to propagate the research and application of techniques targeting the data-driven improvement of healthcare processes. This paper, an initiative of the alliance, presents the distinguishing characteristics of the healthcare domain that need to be considered to successfully use process mining, as well as open challenges that need to be addressed by the community in the future.

59 BASIC BIOLOGICAL SCIENCES↗