SEARCH · Engineering Papers
Results for “Data analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT): Semi-Annual Progress Report
This report summarizes work carried out by the Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) Team for the period of July 1, 2011 through December 31, 2011. It discusses highlights, overall progress, period goals, and collaborations and lists papers and presentations. The UV-CDAT team is positioned to address the following high-level visualization requirements: (1) Alternative parallel streaming statistics and analysis pipelines - Data parallelism, Task parallelism, Visualization parallelism; (2) Optimized parallel input/output (I/O); (3) Remote interactive execution; (4) Advanced intercomparison visualization; (5) Data provenance processing and capture; and (6) Interfaces for scientists - Workflow data analysis and visualization construction tools, Visualization interfaces.
Integrating HPC, AI, and Workflows for Scientific Data Analysis: Report from Dagstuhl Seminar 23352
The Dagstuhl Seminar 23352, titled “Integrating HPC, AI, and Workflows for Scientific Data Analysis,” held from August 27 to September 1, 2023, was a significant event focusing on the synergy between High-Performance Computing (HPC), Artificial Intelligence (AI), and scientific workflow technologies. The seminar recognized that modern Big Data analysis in science rests on three pillars: workflow technologies for reproducibility and steering, AI and Machine Learning (ML) for versatile analysis, and HPC for handling large data sets. These elements, while crucial, have traditionally been researched separately, leading to gaps in their integration. The seminar aimed to bridge these gaps, acknowledging the challenges and opportunities at the intersection of these technologies. The event highlighted the complex interplay between HPC, workflows, and ML, noting how ML has increasingly been integrated into scientific workflows, thereby enhancing resource demands and bringing new requirements to HPC architectures, like support for GPUs and iterative computations. The seminar also addressed the challenges in adapting HPC for large-scale ML tasks, including in areas like deep learning, and the need for workflow systems to evolve to leverage ML in data analysis fully. Moreover, the seminar explored how ML could optimize scientific workflow systems and HPC operations, such as through improved scheduling and fault tolerance. A key focus was on identifying prestigious use cases of ML in HPC and understanding their unique, unmet requirements. The stochastic nature of ML and its impact on the reproducibility of data analysis on HPC systems was also a topic of discussion.
ARCHIE: Data Analysis Acceleration with Array Caching in Hierarchical Storage
© 2018 IEEE. Scientific data analysis typically involves reading massive amounts of data that was generated by simulations, experiments, and observations. Performance of reading such large volumes of data from disk-based file systems is often poor because of the slow and mechanical components in the disks. Recent supercomputing systems are adding non-volatile storage layers in a hierarchy to handle the performance gap between fast main memory and slow disk-based storage. Software libraries for managing this hierarchy not only need efficient reading of data but also reduce user-involvement for cross-layer data movement. Furthermore, these libraries need to support array data access patterns into hierarchical storage management as scientific data is often organized in array-based data structures. Existing software typically manage individual storage layers requiring significant manual process in moving data among them. In this paper, we introduce a new array caching in hierarchical storage (ARCHIE) to accelerate array data analysis in a seamless fashion. ARCHIE evaluates array access patterns and prefetches data with array semantics between storage layers. Our evaluation shows that ARCHIE outperforms state-of-the-art file systems, i.e., Lustre and DataWarp, on a production supercomputing system by up to 5.8× in accessing data by scientific analysis applications.
Earthquake response of storey building in Jakarta using accelerographs data analysis
As seismotectonic, the Jakarta city will be greatly affected by the earthquake which originated from the subduction zone of the Sunda Strait and south of Java. Some occurrences of earthquakes in these location are often perceived by the occupants in the upper floors of multi-storey buildings in Jakarta but was not perceived by the occupants on the ground floor. The case shows the difference in ground-motion parameters on each floor height. The analysis of the earthquake data recorded by accelerographs on different floors need to be done to know the differences in ground-motion parameters. Data used in this research is accelerograph data installed on several floors in the main building of Meteorology Climatology and Geophysics Agency with a case study of Kebumen earthquake on January 25{sup th} 2014. Parameters analyzed include the Peak Ground Acceleration (PGA), Peak Ground Displacement (PGD), Peak Spectral Acceleration (PSA), Amplification (Ag), and the Effective Duration of earthquake (t{sub e}). Research stages include accelerographs data acquisition in three (3) different floors, conversion and data partition for each component, conversion to units of acceleration, determination of PGA, PGD, PSA, Ag and t{sub e} as well as data analysis. The study shows the value of PGA on the ground floor, 7{sup th} floor and 15{sup th} floors, respectively are 0.016 g, 0.053 g and 0.116 g. PGD on the ground floor, 7{sup th} floor and 15{sup th} floor respectively are 2.15 cm, 2.98 cm and 4.92 cm. PSA on the ground floor, 7{sup th} floor and 15{sup th} floor respectively are 0.067 g, 0.308 g and 0.836 g. Amplification of the peak acceleration value on the ground floor, 7{sup th} floor and 15{sup th} floor to the surface rock are 4.37, 6.07 and 7.30. Effective duration of the earthquake on the ground floor, 7{sup th} floor and 15{sup th} floor respectively are 222.28 s, 202.28 s and 91.58 s. In general, with increasing floor of the building, the value of the peak ground acceleration, peak ground displacement, peak spectral acceleration and amplification growing, contrary to the value of the effective duration of earthquake decreases. The difference in this parameter is strongly influenced by local soil conditions and building construction.
TA-DA: A TOOL FOR ASTROPHYSICAL DATA ANALYSIS
We present the Tool for Astrophysical Data Analysis (TA-DA), a new software aimed to greatly simplify and improve the analysis of stellar photometric data in comparison with theoretical models, and allow the derivation of stellar parameters from multi-band photometry. Its flexibility allows one to address a number of such problems: from the interpolation of stellar models, or sets of stellar physical parameters in general, to the computation of synthetic photometry in arbitrary filters or units; from the analysis of observed color-magnitude diagrams to a Bayesian derivation of stellar parameters (and extinction) based on multi-band data. TA-DA is available as a pre-compiled Interactive Data Language widget-based application; its graphical user interface makes it considerably user-friendly. In this paper, we describe the software and its functionalities.
Study of movement of the western and central belts of Peninsular Malaysia using GPS data analysis
Since the large earthquakes in Sumatera and Nias, there were some tremors incidents at Bukit Tinggi. Therefore, a study on the earth’s crust movement and the effects of the earthquake in Indonesia on the tectonic blocks of Peninsular Malaysia have been carried out using GPS data analysis. GPS data from five MyRTKnet stations within Peninsular Malaysia have been analyzed to monitor the movement of two major tectonic blocks of Peninsular Malaysia which are the western belt represented by the Behrang (BEHR) and UPM Serdang (UPMS) stations and the central belt represented by Bentong (BENT), Jerantut (JRNT) and Temerloh (TLOH) stations. GPS data recorded from 2005 to 2010 were analysed based on horizontal and vertical displacements of the respective stations by using Trimble Business Centre (TBC) software. Based on the results of accumulated displacements of recorded GPS data from January 2006 to December 2013, it shows that the western belt which represented by UPMS has shifted 0.096m towards northwest with changes of ellipsoidal height of +0.030m while the central belt which represented by TLOH has shifted 0.080m towards northwest with changes of ellipsoidal height of −0.015m. Meanwhile, BENT station which is located on the Bentong-Raub suture zone turns to its original position as well as JRNT station. However, BEHR station which are located in western belt do not show any movements. All of these movements may be due to the influence of reactive faults in the stations area stimulated by several large earthquakes that occurred in 2005 to 2010. Study on using the GPS data analysis and combine with integrated geophysical methods are necessary to understand in detail about the tectonic evolution of Peninsular Malaysia.
Xarray Climate Data Analysis Tools
xCDAT is an extension of xarray for climate data analysis on structured grids. It serves as a modern successor to the Community Data Analysis Tools (CDAT) library. Xarray is an "open source project and Python package that introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, which allows for more intuitive, more concise, and less error-prone user experience. Xarray includes a large and growing library of domain-agnostic functions for advanced analytics and visualization with these data structures" (source: https://xarray.dev/). The goal of xCDAT is to provide generalizable features and utilities for simple and robust analysis of climate data. xCDAT's design philosophy is focused on reducing the overhead required to accomplish certain tasks in xarray. Some key xCDAT features are inspired by or ported from the core CDAT library, while others leverage powerful libraries in the xarray ecosystem (e.g., xESMF and cf_xarray) to deliver robust APIs.
Validation of finite-element models using full-field experimental data: Levelling finite-element analysis data through a digital image correlation engine
Full-field data from digital image correlation (DIC) provide rich information for finite-element analysis (FEA) validation. However, there are several inherent inconsistencies between FEA and DIC data that must be rectified before meaningful, quantitative comparisons can be made, including strain formulations, coordinate systems, data locations, strain calculation algorithms, spatial resolutions and data filtering. As such, in this paper, we investigate two full-field validation approaches: (1) the direct interpolation approach, which addresses the first three inconsistencies by interpolating the quantity of interest from one mesh to the other, and (2) the proposed DIC-levelling approach, which addresses all six inconsistencies simultaneously by processing the FEA data through a stereo-DIC simulator to ‘level’ the FEA data to the DIC data in a regularisation sense. Synthetic ‘experimental’ DIC data were generated based on a reference FEA of an exemplar test specimen. The direct interpolation approach was applied, and significant strain errorswere computed, even though therewas no model form error, because the filtering effect of theDIC enginewas neglected. In contrast, the levelling approach provided accurate validation results, with no strain error when no model form error was present. Next, model form error was purposefully introduced via a mismatch of boundary conditions. With the direct interpolation approach, the mismatch in boundary conditions was completely obfuscated, while with the levelling approach, it was clearly observed. Finally, the ‘experimental’ DIC datawere purposefully misaligned slightly fromthe FEA data. Both validation techniques suffered from the misalignment, thus motivating continued efforts to develop a robust alignment process. In conclusion, direct interpolation is insufficient, and the proposed levelling approach is required to ensure that the FEA and the DIC data have the same spatial resolution and data filtering. Only after the FEA data have been ‘levelled’ to the DIC data can meaningful, quantitative error maps be computed.
In-situ Condition Monitoring of Components in Small Modular Reactors Using Process and Electrical Signature Analysis. Final report, volume 1. Development of experimental flow control loop, data analysis and plant monitoring
The research and development under this project was focused on the following three major objectives: Objective 1: Identification of critical in-vessel SMR components for remote monitoring and development of their low-order dynamic models, along with a simulation model of an integral pressurized water reactor (iPWR). Objective 2: Development of an experimental flow control loop with motor-driven valves and pumps, incorporating data acquisition and on-line monitoring interface. Objective 3: Development of stationary and transient signal processing methods for electrical signatures, machinery vibration, and for characterizing process variables for equipment monitoring. This objective includes the development of a data analysis toolbox. The following is a summary of the technical accomplishments under this project: - A detailed literature review of various SMR types and electrical signature analysis of motor-driven systems was completed. A bibliography of literature is provided at the end of this report. Assistance was provided by ORNL in identifying some key references. - A review of literature on pump-motor modeling and digital signal processing methods was performed. - An existing flow control loop was upgraded with new instrumentation, data acquisition hardware and software. The upgrading of the experimental loop included the installation of a new submersible pump driven by a three-phase induction motor. All the sensors were calibrated before full-scale experimental runs were performed. - MATLAB-Simulink model of a three-phase induction motor and pump system was completed. The model was used to simulate normal operation and fault conditions in the motor-pump system, and to identify changes in the electrical signatures. - A simulation model of an integral PWR (iPWR) was updated and the MATLAB-Simulink model was validated for known transients. The pump-motor model was interfaced with the iPWR model for testing the impact of primary flow perturbations (upsets) on plant parameters and the pump electrical signatures. Additionally, the reactor simulation is being used to generate normal operation data and data with instrumentation faults and process anomalies. A frequency controller was interfaced with the motor power supply in order to vary the electrical supply frequency. The experimental flow control loop was used to generate operational data under varying motor performance characteristics. Coolant leakage events were simulated by varying the bypass loop flow rate. The accuracy of motor power calculation was improved by incorporating the power factor, computed from motor current and voltage in each phase of the induction motor.- A variety of experimental runs were made for steady-state and transient pump operating conditions. Process, vibration, and electrical signatures were measured using a submersible pump with variable supply frequency. High correlation was seen between motor current and pump discharge pressure signal; similar high correlation was exhibited between pump motor power and flow rate. Wide-band analysis indicated high coherence (in the frequency domain) between motor current and vibration signals. - Wide-band operational data from a PWR were acquired from AMS Corporation and used to develop time-series models, and to estimate signal spectrum and sensor time constant. All the data were from different pressure transmitters in the system, including primary and secondary loops. These signals were pre-processed using the wavelet transform for filtering both low-frequency and high-frequency bands. This technique of signal pre-processing provides minimum distortion of the data, and results in a more optimal estimation of time constants of plant sensors using time-series modeling techniques.
The CMS data analysis school experience
The CMS Data Analysis School is an official event organized by the CMS Collaboration to teach students and post-docs how to perform a physics analysis. The school is coordinated by the CMS schools committee and was first implemented at the LHC Physics Center at Fermilab in 2010. As part of the training, there are a number of “short” exercises on physics object reconstruction and identification, Monte Carlo simulation, and statistical analysis, which are followed by “long” exercises based on physics analyses. Some of the long exercises go beyond the current state of the art of the corresponding CMS analyses. This paper describes the goals of the school, the preparations for a school, the structure of the training, and student satisfaction with the experience as measured by surveys.
2014 Earth System Grid Federation and Ultrascale Visualization Climate Data Analysis Tools Conference Report
The climate and weather data science community met December 9–11, 2014, in Livermore, California, for the fourth annual Earth System Grid Federation (ESGF) and Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT) Face-to-Face (F2F) Conference, hosted by the Department of Energy, National Aeronautics and Space Administration, National Oceanic and Atmospheric Administration, the European Infrastructure for the European Network of Earth System Modelling, and the Australian Department of Education. Both ESGF and UVCDATremain global collaborations committed to developing a new generation of open-source software infrastructure that provides distributed access and analysis to simulated and observed data from the climate and weather communities. The tools and infrastructure created under these international multi-agency collaborations are critical to understanding extreme weather conditions and long-term climate change. In addition, the F2F conference fosters a stronger climate and weather data science community and facilitates a stronger federated software infrastructure. The 2014 F2F conference detailed the progress of ESGF, UV-CDAT, and other community efforts over the year and sets new priorities and requirements for existing and impending national and international community projects, such as the Coupled Model Intercomparison Project Phase Six. Specifically discussed at the conference were project capabilities and enhancements needs for data distribution, analysis, visualization, hardware and network infrastructure, standards, and resources.
Machine Learning Based Software Package for Reproducible Data Analysis for Electron Microscopy
Advances in hardware and computation technologies are enabling large hyperspectral imaging during Electron Microscopy (EM), resulting in datasets being produced at rapidly increasing spatial and temporal resolutions. This presents exciting opportunities for microstructure analysis, but also poses great challenges in data analysis. Under this STTR program, QuesTek Innovations LLC, a leader in the field of integrated computational materials engineering (ICME), collaborated with Lawrence Livermore National Laboratory and Argonne National Laboratory. The team developed an open-source machine learning (ML) powered tool for EM data analysis of multiple systems and data types, and collected EM data for ML model development using the cutting-edge PicoProbe equipment at Argonne National Laboratory.
A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure
This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.
Measurement Uncertainty in One-Of-A-Kind Event Data Analysis
A golden standard in science is to repeat an experiment a statistically significant number of times, recording data using the same set of detectors and the same data analysis methodology. In such case experimental error includes both the range of true values generated by repetitions of the experiment, and measurement uncertainty caused by the detector. They are independent. It is a huge and too frequently used simplification, to assume that one can measure multiple repetitions of an identical experiment, resulting in identical true experimental value. Repetitions, as similar is it is experimentally achievable, have unavoidable built-in differences resulting in a range of the true values rather than in a single value. When modern, very sensitive and well calibrated measurement systems are used, this range is not negligible, and sometimes dominates over the measurement uncertainty. Range of true values depends on built-in differences in physics of the experiment. Stochastic physical processes result typically in a broader range of true values than non-stochastic processes do. Measurement uncertainty depends on a measurement method (properties of the detector not of the experiment). Modern measurement methods, including digital ones, frequently make the measurement uncertainty very small. When data from one–of –a kind experiment are analyzed, only the measurement uncertainty is reported. It provides no information about the range of true experimental values, neither about reliability of a reported data point. Reliability of a data point is in general independent from its measurement uncertainty. However, in practice reliable measurement methods frequently have high measurement uncertainty, while low reliability methods are applied to limit measurement uncertainty. Comparison of reliable data with high measurement uncertainty to not so reliable data measured with low uncertainty is discussed – in different scenarios different data analysis methods are applicable. Methods for data analysis from an experiment repeated statistically significant number of times are very well developed. They do not require a detailed expertise in physics of an experiment, nor in the properties of the measurement system used, and meaning of the reported uncertainty is well understood in any scientific community. It all changes when data from one-of-a-kind experiment is analyzed. Analyst’s expertise is required both in the physics of the experiment and in all aspects of the measurement system, all possible malfunctions. Data users must remember that only measurement uncertainty is reported from any one-of-a-kind experiment. Theory with simulations may provide estimation of expected built-in differences in the experiment, and by this of expected range of true values for a given experiment; yet measurement uncertainty can never be used in place of the range of true experimental values.
Machine learning enabled electron microscopy data analysis tool
This software is for materials image and spectral data analysis. It consists machine learning models that can segment a large amount of image data in a high-throughput way, and an interface for image data and spectral data analysis.
Using integrated data analysis to extend measurement capability (invited)
The analysis approach called integrated data analysis (IDA) provides a means to exploit all information present in multiple streams of raw data to produce the best inference of a plasma parameter. This contrasts with the typical approach in which information (data) from a single diagnostic is used to measure a given parameter, e.g., visible bremsstrahlung → Z eff . Data from a given diagnostic usually contain information on many parameters. For example, a Thomson scattering diagnostic is sensitive to bremsstrahlung and line emission in addition to electron temperature. This background light is typically subtracted off and discarded but could be used to improve knowledge of Z eff . IDA encourages explicit awareness of such information and provides the quantitative framework to exploit it. This gives IDA the ability to increase spatial and temporal resolution, increase precision and accuracy of inferences, and measure plasma parameters that are difficult or impossible to measure using single diagnostic techniques. One example is the measurement of Z eff on Madison symmetric torus using IDA since no single diagnostic can provide a robust measurement. Thus, as we enter the burning plasma era, application of IDA will be critical to the measurement of certain parameters, as diagnostic access in the harsh fusion environment will be extremely limited.
EDA and ML - A Perfect Pair for Large-Scale Data Analysis
This short position paper presents our view of how exploratory data analysis techiniques and machine learning methods can be applied together for large-scale data analysis.