FITNESS FOR PURPOSE: OVERVIEW OF ASTM GUIDES FOR QUALITY ASSURANCE OF DATA FROM NUCLEAR ANALYTICAL LABORATORIES
Explore the source record for details and available documents.
Brisson, Mike↗
SEARCH · Engineering Papers
Results for “data analytics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data Management, Organization & Analytics
Presented by LRST at the eXtremeMAT Stakeholder Advisory Board Review Meeting, 10/15
Wenzlick, Madison Z.↗
The power of event-driven analytics in Large Scale Data Processing
FeedZai is a software company specialized in creating high-‐throughput low-‐latency data processing solutions. FeedZai develops a product called "FeedZai Pulse" for continuous event-‐driven analytics that makes application development easier for end users. It automatically calculates key performance indicators and baselines, showing how current performance differ from previous history, creating timely business intelligence updated to the second. The tool does predictive analytics and trend analysis, displaying data on real-‐time web-‐based graphics. In 2010 FeedZai won the European EBN Smart Entrepreneurship Competition, in the Digital Models category, being considered one of the "top-‐20 smart companies in Europe". The main objective of this seminar/workshop is to explore the topic for large-‐scale data processing using Complex Event Processing and, in particular, the possible uses of Pulse in the scope of the data processing needs of CERN. Pulse is available as open-‐source and can be licensed both for non-‐commercial and commercial applications. FeedZai is interested in exploring possible synergies with CERN in high-‐volume low-‐latency data processing applications. The seminar will be structured in two sessions, the first one being aimed to expose the general scope of FeedZai's activities, and the second focused on Pulse itself: 10:00-11:00 FeedZai and Large Scale Data Processing; Introduction to FeedZai; FeedZai Pulse and Complex Event Processing; Demonstration; Use-Cases and Applications; Conclusion and Q&A. 11:00-11:15 Coffee break 11:15-12:30 FeedZai Pulse Under the Hood; A First FeedZai Pulse Application; PulseQL overview; Defining KPIs and Baselines; Conclusion and Q&A. About the speakers Nuno Sebastião is the CEO of FeedZai. Having worked for many years for the European Space Agency (ESA), he was responsible the overall design and development of Satellite Simulation Infrastructure of the agency. Having left ESA to found FeedZai, Nuno is currently responsible for the whole operations of the company. Nuno holds an M.Eng. in Informatics Engineering for the University of Coimbra, and an MBA from the London Business School. Paulo Marques is the CTO of FeedZai, being responsible for product development. Paulo is an Assistant Professor at the University of Coimbra, in the area of Distributed Data Processing, and an Adjunct Associated Professor at Carnegie Mellon, in the US. In the past Paulo lead a large number of projects for institutions like the ESA, Microsoft Research, SciSys, Siemens, among others, being now fully dedicated to FeedZai. Paulo holds a Ph.D. in Distributed Systems from the University of Coimbra.
97 MATHEMATICS AND COMPUTING↗
Open and Extensible Control & Analytics Platform for Synchrophasor Data (Final Report)
The objective of the Open and Extensible Control and Analytics (openECA) Platform for Phasor Data project was to develop an open source software platform that significantly accelerates the production, use, and ongoing development of real-time decision support tools, automated control systems, and off-line planning systems that (1) incorporate synchrophasor data and (2) enhance system reliability while enabling the North American Electric Reliability Corporation (NERC) operating functions of reliability coordinator, transmission operator, and/or balancing authority to be executed more effectively.
99 GENERAL AND MISCELLANEOUS↗
Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART)
This report contains key findings from a project titled Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART), which was carried out through a collaborative effort of a team of researchers from Texas A&M Engineering Experiment Station, Temple University, and Quanta Technology, LLC. The in-kind support came from OSIsoft (acquired by AVEVA), which provided their PI Historian software to demonstrate the use case of streaming PMU data. The first section of the report describes the project goals and objectives related to the development of Machine Learning (ML) models capable of detecting and classifying events by processing phasor measurements captured in the field by Phasor Measurement Units (PMUs). The data for this study was contributed by the utilities/ISOs from the Western and Eastern interconnects and ERCOT, further referred to as Interconnect B (IC B), Interconnect A (IC A), and Interconnect C (IC C), respectively. The approach that the BDSMART Research Team proposed and the key research tasks defined by the team are outlined in this section. The next section describes the technical approach. We first discuss the data constraints related to the PMU measurements and data interpretation constraints imposed by the data contributors. They provided neither the topological information of the grid nor PMU placement locations and captured recorded data at very few locations in the system with the reporting rate of either 30 or 60 fps. The recordings are mostly positive sequence voltage, frequency, and ROCOF, and in some limited cases, three-phase voltages and currents. We then reflect on the bad data issues that stem from poor recording practices and vague definitions of the PMU status bits to supposedly be used for bad data identification. Finally, the data discovery points to imprecise time stamps with incomplete event start/end time, as well as inconsistent and incomplete event labeling, which combined make the implementation of the data models using supervising learning quite challenging. Following the data discovery study, we hypothesize that because the IC B data has the most complete labels, we should focus our model development on that data and then test it on data from other interconnects. We also define the common metrics used to evaluate the results from the ML algorithm tests. We concluded this section by summarizing the common ML models we used and explaining how we implemented and tested them. The issues from this section are expanded in the Training Dataset Report from this project. The final section of this report deals with the accomplishments and conclusions. As the accomplishments, we formulate the problem we are solving and what is achieved by solving the problem. We then reflect on each of the analytics tools we developed and point out the performance of each tool when applied to solving the mentioned problems. We reference this work for further details to the papers we published on each tool. In the conclusions, we give recommendations on how to improve future PMU recording practices to facilitate the ML algorithm implementation and guidance for the future standardization work aimed at clarifying the ambiguities associated with the PMU status bits. We finally list future tasks that can bring about further improvements in the proposed algorithms. The issues from this section are expanded in the Training, and Test Dataset Report filed at the project completion date.
97 MATHEMATICS AND COMPUTING↗
Data-driven occupant-behavior analytics for residential buildings
Many advances have been made in building technology to help save energy, but influencing the behavior of the occupants is still necessary to achieve low-energy use targets. One of the most practical ways to influence and change occupant behaviors is through incentives. Developing incentives for energy-saving and quantifying the impact of occupant behaviors are both active areas of research. Here, we propose a data analytics framework for detecting changes in occupant behaviors, which will help build an analytics feedback loop from behavior impact to incentive design. The framework has two major parts. The first forecasts energy consumption for each occupant, while the second determines a probability distribution for changes in energy consumption. The parts are interchangeable with other existing machine learning and statistical methods. A specific instantiation of the framework, using kernel ridge-regression for forecasting and k-means to find an empirical behavior distribution, is described in detail. An HVAC use-case with 5 different incentivized behaviors is used as an example to show that the framework can detect behavior changes induced by incentives. Furthermore, we show that some simpler behavior-change detection methods do not work, further justifying the use of advanced analytics.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
CO 2 Storage Site Screening Platform Development and CO 2 Storage Resource Analysis in SECARB Offshore Reservoirs Using SAS Viya
A major goal of the SECARB Offshore Partnership (DE-FE0031557) is to screen deep saline aquifers and hydrocarbon reservoirs in the central Gulf of Mexico for CO 2 sequestration and CO 2 -enhanced oil and gas recovery (EOR/EGR) and estimate the corresponding CO 2 storage resources for select reservoirs. CO 2 storage potential associated with offshore CO 2 -EOR is considerable and likely represents “low hanging fruit” for near-term CO 2 storage given the in-place infrastructure in the region. It is for these reasons that this assessment focuses on oil and gas fields. To this end, three major objectives have been completed and include (1) managing geological data derived from different sources, (2) building a reservoir screening platform for CO 2 storage, and (3) ranking the reservoirs based on the estimated CO 2 storage resources. The SAS ® Viya platform was used for data management and analytics. The Viya platform is a cloud service platform that provides data integration, data management, quick analytics, data visualization, machine learning functions, and application programming interfaces (APIs) for multi-programming languages. Different sources of data containing geologic information, reservoir properties, and EOR/EGR information were collected, cleaned, formatted, and loaded into the SAS ® Viya platform for evaluation. The major geological characteristics of both shelf and deep-water areas of the central Gulf were examined and compared to define the appropriate reservoir screening criteria. Next, a CO 2 storage site screening system was built in the SAS ® Viya platform with the pre-defined criteria. Finally, the CO 2 storage resources of the screened reservoirs were calculated and reported at the BOEM field level to identify fields with the highest estimated CO 2 storage resource. The fields with the largest total estimated CO 2 storage resource are located in the Mississippi Canyon protraction area. Due to proximity to the Mississippi Delta (indicative of less infrastructure) and large estimated CO 2 storage resources, future development activities may wish to focus efforts in the Mississippi Canyon protraction area.
02 PETROLEUM↗
Materials data science using CRADLE: A distributed, data-centric approach
Abstract There is a paradigm shift towards data-centric AI, where model efficacy relies on quality, unified data. The common research analytics and data lifecycle environment (CRADLE™) is an infrastructure and framework that supports a data-centric paradigm and materials data science at scale through heterogeneous data management, elastic scaling, and accessible interfaces. We demonstrate CRADLE’s capabilities through five materials science studies: phase identification in X-ray diffraction, defect segmentation in X-ray computed tomography, polymer crystallization analysis in atomic force microscopy, feature extraction from additive manufacturing, and geospatial data fusion. CRADLE catalyzes scalable, reproducible insights to transform how data is captured, stored, and analyzed. Graphical abstract
97 MATHEMATICS AND COMPUTING↗
Nondestructive Assay Technical Infrastructure Program Mission and Vision
The Nondestructive Assay Program (NDAP) mission and vision is achieved by identifying and accomplishing a set of programmatic goals that correspond with eight broad technical program elements. The NDAP was created as a result of Defense Nuclear Facility Safety Board Recommendation 2007-1, entitled “Safety-Related In Situ Nondestructive Assay of Radioactive Materials,” which emphasized the need to improve in situ measurements of radioactive material at US Department of Energy (DOE) defense nuclear facilities. An NDAP five-year plan will define tasks to accomplish specific goals identified in the NDAP mission and vision. This mission and vision is applicable for eight technical program elements: Hardware/Software Development, Algorithm Development and Nuclear Data, Uncertainty Quantification, Nuclear Materials, Staffing, Personnel, And Training, Data Management, Requirement and Standards, and Information Preservation and Dissemination. An additional program element is technical support, which provides daily execution management support for the NDAP, site scope, and deliverable tracking via site task managers, along with technical advisement and support from the technical support group. The NDAP mission and vision provides attributes with specific goals for each program element, and some goals benefit multiple program elements. The goals and attributes defined herein are implemented via an NDAP five-year execution plan that defines site work scope, budget, and deliverables, all of which are updated annually. The NDAP is designed to benefit nondestructive assay (NDA) needs to support DOE nuclear criticality safety programs, ensuring that NDA technology is sufficiently capable of guaranteeing the safety of those who handle, store, process, or transport fissionable materials in the complex. Especially important for the NDAP is to maximize capabilities to identify, characterize, and manage in situ fissile material deposits in process equipment to ensure nuclear criticality safety at processing facilities.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE SBIR Phase I Final Technical Report - A Cloud-enabled Mobile System For Integrated Nuclear Forensic Analysis
Explore the source record for details and available documents.
98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants
Analytics-at-scale of Sensor Data for Digital Monitoring in Nuclear Plants Project Summary
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Analytically Derived Piezoelectric Ejecta Data Transformations, with Implications for Claims of Experimentally Observed Self-Similarity
Building upon a previously documented analytic formalism, we derive a set of data transformations that relate piezoelectrically inferred time-dependent ejecta masses, dynamical ejecta cloud densities, and "radiographic" ejecta cloud density distributions. These transformation rules are derived rigorously, with careful attention to the assumption of instant ejecta production which is inherent in the piezoelectric mass inference procedure. Some of these relationships have not appeared previously in the literature, including solutions for the unique ejecta velocity distribution and radiographic ejecta cloud density distribution consistent with the instant-production assumption for a given piezoelectrically inferred mass dataset. In deriving the latter, we prove that—again subject to the ubiquitous instant-creation assumption—measurements at a single sensor are sufficient to determine properties of the ejecta cloud everywhere at all times. Other data transformation relationships have appeared in the literature without a rigorous derivation, or have been stated in forms that obscure the key mathematical relationships. We apply these results to arguments for self-similar ejecta cloud evolution, thereby deriving mathematical expressions by which claims of self-similarity may be examined. We prove that one analysis used in the literature to claim experimental evidence of a short ejecta production timescale—and consequently of self-similar cloud evolution—in fact places no constraint the ejecta production interval and therefore does not support the associated conclusion. We also provide a potentially useful uniqueness proof.
30 DIRECT ENERGY CONVERSION↗
PlanetSense: A Real-time Streaming and Spatio-temporal Analytics Platform for Gathering Geo-spatial Intelligence from Open Source Data
Geospatial intelligence has traditionally relied on the use of archived and unvarying data for planning and exploration purposes. In consequence, the tools and methods that are architected to provide insight and generate projections only rely on such datasets. Albeit, if this approach has proven effective in several cases, such as land use identification and route mapping, it has severely restricted the ability of researchers to inculcate current information in their work. This approach is inadequate in scenarios requiring real-time information to act and to adjust in ever changing dynamic environments, such as evacuation and rescue missions. In this work, we propose PlanetSense, a platform for geospatial intelligence that is built to harness the existing power of archived data and add to that, the dynamics of real-time streams, seamlessly integrated with sophisticated data mining algorithms and analytics tools for generating operational intelligence on the fly. The platform has four main components – i) GeoData Cloud – a data architecture for storing and managing disparate datasets; ii) Mechanism to harvest real-time streaming data; iii) Data analytics framework; iv) Presentation and visualization through web interface and RESTful services. Using two case studies, we underpin the necessity of our platform in modeling ambient population and building occupancy at scale.
big data architecture↗
UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability
The emerging distribution-level phasor measurement unit (D-PMU) is expected to play an important role in enhancing distribution system observability and situational-awareness. It is a demanding yet challenging task to develop advanced D-PMU data management and analytics tools to improve D-PMU data usability and further promote D-PMU projects deployment. This paper focuses on D-PMU data processing and storage. It presents a brief review of existing D-PMU data storage systems and points out their limitations on high-performance, flexibility, and scalability. To overcome the limitations, a unified PMU-data storage system (UPS) is proposed. Specifically, a unified I/O interface between storage servers and computing jobs is developed to effectively reduce the overhead of managing various computing jobs and data analytics over multiple storage infrastructures; and PMUCache with PMUCache partitioning and PMUCache replacement algorithms are designed to support in-situ data processing and shared distributed data storage and further serve the computing jobs/queries with different Quality of Service (QoS) requirements. Through a series of experiments, it is demonstrated that UPS achieves high performance on fast and big data processing and storage, and efficiently increases the flexibility and scalability of PMU data management systems.
42 ENGINEERING↗
Data processing and quality verification for improved photovoltaic performance and reliability analytics
Not Available
14 SOLAR ENERGY↗
PARLO: PArallel Run-Time Layout Optimization for Scientific Data Explorations with Heterogeneous Access Patterns
The size and scope of cutting-edge scientific simulations are growing much faster than the I/O and storage capabilities of their run-time environments. The growing gap is exacerbated by exploratory, data-intensive analytics, such as querying simulation data with multivariate, spatio-temporal constraints, which induces heterogeneous access patterns that stress the performance of the underlying storage system. Previous work addresses data layout and indexing techniques to improve query performance for a single access pattern, which is not sufficient for complex analytics jobs. We present PARLO a parallel run-time layout optimization framework, to achieve multi-level data layout optimization for scientific applications at run-time before data is written to storage. The layout schemes optimize for heterogeneous access patterns with user-specified priorities. PARLO is integrated with ADIOS, a high-performance parallel I/O middleware for large-scale HPC applications, to achieve user-transparent, light-weight layout optimization for scientific datasets. It offers simple XML-based configuration for users to achieve flexible layout optimization without the need to modify or recompile application codes. Experiments show that PARLO improves performance by 2 to 26 times for queries with heterogeneous access patterns compared to state-of-the-art scientific database management systems. Compared to traditional post-processing approaches, its underlying run-time layout optimization achieves a 56% savings in processing time and a reduction in storage overhead of up to 50%. PARLO also exhibits a low run-time resource requirement, while also limiting the performance impact on running applications to a reasonable level.
Computer Science↗
PARLO: PArallel Run-Time Layout Optimization for Scientific Data Explorations with Heterogeneous Access Pattern
Download Citation Email Print Request Permissions Save to Project The size and scope of cutting-edge scientific simulations are growing much faster than the I/O and storage capabilities of their run-time environments. The growing gap is exacerbated by exploratory, data-intensive analytics, such as querying simulation data with multivariate, spatio-temporal constraints, which induces heterogeneous access patterns that stress the performance of the underlying storage system. Previous work addresses data layout and indexing techniques to improve query performance for a single access pattern, which is not sufficient for complex analytics jobs. We present PARLO a parallel run-time layout optimization framework, to achieve multi-level data layout optimization for scientific applications at run-time before data is written to storage. The layout schemes optimize for heterogeneous access patterns with user-specified priorities. PARLO is integrated with ADIOS, a high-performance parallel I/O middleware for large-scale HPC applications, to achieve user-transparent, light-weight layout optimization for scientific datasets. It offers simple XML-based configuration for users to achieve flexible layout optimization without the need to modify or recompile application codes. Experiments show that PARLO improves performance by 2 to 26 times for queries with heterogeneous access patterns compared to state-of-the-art scientific database management systems. Compared to traditional post-processing approaches, its underlying run-time layout optimization achieves a 56% savings in processing time and a reduction in storage overhead of up to 50%. PARLO also exhibits a low run-time resource requirement, while also limiting the performance impact on running applications to a reasonable level.
Gong, Zhenhuan↗