Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data analytic”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Air Traffic Management TestBed Simulation Architect: User's Guide

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform provides cloud services including back-end big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various concepts such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. The Simulation Architect application provides a graphical user interface tool for designing traffic scenarios and simulations using blocks representing components and links representing message channels linking them. This guide describes a high-level user interface design of Simulation Architect and provides information for a new user to compose traffic scenarios and simulations.

Software User Guide↗

Air Traffic Management TestBed Traffic Viewer: Developer's Guide

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform is designed to provide cloud services including back-end, big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various methods such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. The Traffic Viewer application provides a graphical user interface tool for visualizing real and simulated air traffic as well as airspace definition in two-dimensional space. This guide describes a high-level design and implementation of Traffic Viewer and provides information for a new developer or a user to add new capabilities by following the software design and leveraging existing capabilities.

Lai, Chok Fung↗

Air Traffic Management TestBed Data Exchange Model

The Air Traffic Management (ATM) TestBed is a Platform as a Service that is being developed by the National Aeronautics and Space Administration (NASA) to help design, configure, integrate, run, and monitor air traffic simulations. The platform is designed to provide cloud services including back-end, big-data analytics tools, on-demand computing resource management, data storage, and communication middleware. The ATM TestBed reduces the time to test concepts and technologies, supports interactions among various methods such as human-in-the-loop and automation-in-the-loop simulations, and enables collaborative simulations by sharing technologies and tools in the ATM community. In order to allow easier access to simulation components, TestBed provides a messaging support layer for connectivity using a consistent set of input/output interfaces. In addition, a standard data format is introduced to facilitate communication between the components. The data exchange model, supported in the messaging support layer, standardizes the format of the information to be exchanged among the components. This document describes the messaging data model currently developed in TestBed and provides data dictionaries for references to component developers as well as simulation engineers.

air traffic simulation↗

Web-based Visualization and Analytics of Petascale Data: Equity as a Tide that Lifts All Boats

Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Data visualization↗

Visualizing the NIOSH Pocket Guide: Open-source web application for accessing and exploring the NIOSH Pocket Guide to Chemical Hazards

The NIOSH Pocket Guide to Chemical Hazards is a trusted resource that displays key information for a collection of chemicals commonly encountered in the workplace. Entries contain chemical structures—occupational exposure limit information ranging from limits based on full-shift time-weighted averages to acute limits such as short-term exposure limits and immediately dangerous to life or health values, as well as a variety of other data such as chemical-physical properties and symptoms of exposure. The NIOSH Pocket Guide (NPG) is available as a printed, hardcopy book, a PDF version, an electronic database, and a downloadable application for mobile phones. All formats of the NIOSH Pocket Guide allow users to access the data for each chemical separately, however, the guide does not support data analytics or visualization across chemicals. This project reformatted existing data in the NPG to make it searchable and compatible with exploration and analysis using a web application. The resulting application allows users to investigate the relationships between occupational exposure limits, the range and distribution of occupational exposure limits, and the specialized sorting of chemicals by health endpoint or to summarize information of particular interest. These tasks would have previously required manual extraction of the data and analysis. The usability of this application was evaluated among industrial hygienists and researchers and while the existing application seems most relevant to researchers, the open-source code and data are amenable to modification by users to increase customization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA’s Advance Information Systems Technology (AIST) Program

NASA’s Earth Science Technology Office (ESTO) develops early-stage technologies to enhance scientific understanding. ESTO’s technologies can be reliably and confidently applied to a broad range of science measurements and missions, as well as facilitate practical applications to benefit society at large. As the lead technology office within the Earth Science Division of the NASA Science Mission Directorate, ESTO is focused on the technological challenges inherent to space-based investigations of our planet's dynamic, interrelated systems. Through flexible, science-driven technology strategies and a competitive selection process, ESTO-funded technologies have supported numerous Earth science missions as well as commercial applications. From next generation sensors and instruments to communication and information systems, ESTO technologies enable many NASA missions and data services. The Advanced Information Systems Technology (AIST) Program is one focal area of ESTO. AIST innovates information system technologies that enable the development of new observing systems as well as agile science investigations through data analytics and artificial intelligence tools and algorithms. AIST utilizes an end-to-end development approach with the goal of infusing mature technologies into future missions, measurements, and analysis. One thrust of AIST is data exploitation and analysis using an analytic center framework to make observational data and model output more accessible and usable to scientists conducting specific investigations as well as to extract higher-level science content and information from the data. The framework accelerates scientific discovery by harmonizing the data, tools, and computational resources. NASA’s upcoming Surface, Biology and Geology (SBG) mission is one such mission whose data architecture will leverage many of the technologies developed through the AIST program. This talk will highlight current and future program development goals.

Laura Rogers↗

A path to intelligent watersheds: coordinating the data to decision pipeline

Operations of multi-reservoir systems are challenged in-part by the interplay of complex physical processes functioning within the watershed. The employment of intelligent systems can be of aid by linking environmental sensing, information technology, data analytics, simulation and decision support to achieve a data-to-decision flow of information. A further challenge is that watershed resources are managed for multiple purposes requiring some level of coordination among numerous resource managers, asset operators and users. System intelligence in this context relies on shared community platforms (data portals, community models), and coordinated communication between decision makers. Opportunities to enrich watershed intelligence has been the subject of a roadmapping exercise for the Department of Energy’s Water Power Technologies Office which has relied on broad stakeholder engagement. Initial phases of engagement involved personal interviews and a series of virtual group meetings, which focused on identifying opportunities to improve the intelligence of the physical infrastructure within our watersheds—examples of feedback include improved sensing of snowpack and runoff, data standards for facilitated data sharing, and better forecasting tools. The latter phase of engagement involved the conduct of a case study in the Upper Colorado River basin where key stakeholders were interviewed to map how their decisions are informed by intelligence from other basin stakeholders. Our presentation will highlight the interdisciplinary flow of information in complex watershed systems and identify physical and institutional opportunities toward the strategic operation of water infrastructure.

Colorado River↗

Singleton Sieving: Overcoming the Memory/Speed Trade-Off in Exascale k-mer Analysis

Traditional filter data structures, such as Bloom filters, do not offer necessary features that modern high-performance data analytics applications need in order to efficiently perform complex data analysis tasks. For example, MetaHipMer, a de novo metagenome assembler, can use filters to weed out singleton k-mers and reduce memory usage by 30%-70%. However, the filter needs the ability to associate values with k-mers in order to perform the analysis in a single communication pass. Bloom filters do not support value associations and cause the application to perform an extra communication pass, thereby increasing the run time. Therefore, MetaHipMer faces a trade off between memory and speed due to the limited capabilities of traditional filters. In this paper, we overcome the memory and speed trade off in MetaHipMer by integrating a GPU-based feature-rich filter, the Two-Choice filter (TCF), in the MetaHipMer pipeline. The TCF uses key-value association to approximately store k-mers with extensions. This allows MetaHipMer to perform k-mer analysis on the GPUs in a single communication pass. Our empirical analysis shows a 50% reduction in memory usage in k-mer analysis on each node in MetaHipMer without any effect on the overall run time or assembly quality. The memory reduction in turn results in a 43% reduction in the number of nodes required to assemble datasets and enables MetaHipMer to scale to much larger datasets.

McCoy, Hunter↗

Data review processes in the analytical laboratory

These documents contain information on the Analytical laboratory data review processes for the following analytical techniques: - alpha spectrometry - light elements analyses - gamma spectrometry - gas proportional flow counting - HR-ICP-MS - Q-ICP-MS - MC-ICP-MS - ICP OES - liquid scintillation counting

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Advances in the electron diffraction characterization of atomic clusters and nanoparticles

Nanoparticles and metallic clusters continue to make a remarkable impact on novel and emerging technologies. In recent years, there have been impressive advances in the controlled synthesis of clusters and their advanced characterization. One of the most common ways to determine the structures of nanoparticles and clusters is by means of X-ray diffraction methods. However, this requires the clusters to crystallize in a similar way to those used in protein studies, which is not possible in many cases. Novel methods based on electron diffraction have been used to efficiently study individual nanoparticles and clusters and these can overcome the obstacles commonly encountered during X-ray diffraction methods without the need for large crystals. These novel methodologies have improved with advances in electron microscopy instrumentation and electron detection. Here, we review advanced methodologies for characterizing metallic nanoparticles and clusters using a variety of electron diffraction procedures. These include selected area electron diffraction, nanobeam diffraction, coherent electron diffraction, precession electron diffraction, scanning transmission electron microcopy diffraction, and high throughput data analytics, which leverage deep learning to reduce the propensity for data errors and translate nanometer and atomic scale measurements into material data.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Big Data Analysis of Massive PMU Datasets: A Data Platform Perspective

The discovery of `event signatures' and useful insights from very large historical Phasor Measurement Unit (PMU) datasets is predicated on offline Big Data analysis approaches that rely on the generation of predictive features on a massive scale. This paper presents lessons learned from a data platform perspective towards reducing barriers to adoption of Big Data analytics against a real dataset of almost half a trillion data points drawn from over 400 PMUs distributed across the North American power grid. We demonstrate software abstractions and targeted performance optimizations that can lead to significant productivity gains for power systems researchers seeking to perform offline exploratory temporal analysis and modeling tasks, with a focus on feature generation. We describe how our optimized approach goes beyond a naive application of mainstream Big Data technologies, enabling feature generation tasks, that previously took days or even weeks, to now be completed in just a few hours.

Kumar, Vijay S.↗

AI for Science: Report on the Department of Energy (DOE) Town Halls on Artificial Intelligence (AI) for Science

The report documents the DOE Town Halls held during 2019 at Argonne National Laboratory, Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, and in Washington, DC. From July to October 2019, the Argonne, Oak Ridge, and Berkeley National Laboratories hosted a series of four town hall meetings attended by more than 1,000 U.S. scientists and engineers. The goal of the town hall series was to examine scientific opportunities in the areas of artificial intelligence (AI), Big Data, and high-performance computing (HPC) in the next decade, and to capture the big ideas, grand challenges, and next steps to realizing these opportunities. In this report and in the Department of Energy (DOE) laboratory community, we use the term “AI for Science” to broadly represent the next generation of methods and scientific opportunities in computing, including the development and application of AI methods (e.g., machine learning, deep learning, statistical methods, data analytics, automated control, and related areas) to build models from data and to use these models alone or in conjunction with simulation and scalable computing to advance scientific research. The AI for Science town hall discussions focused on capturing the transformational uses of AI that employ HPC and/or data analysis, leveraging data sets from HPC simulations or instruments and user facilities, and addressing scientific challenges unique to DOE user facilities and the agency’s wide-ranging fundamental and applied science enterprise.

36 MATERIALS SCIENCE↗

Machine Learning Approaches for Nuclear Material Accounting Data from Irradiation and Reprocessing

We are currently exploring data analysis methods for their ability to strengthen the synthesis and evaluation of information generated within domestic and international safeguard regimes. Safeguard data typically includes rich heterogenous datasets amenable to advanced data analytics methods, such as machine learning. We have converted transactional data containing both numerical and categorical attributes from a domestic nuclear material control and accountability (NMC&A) system into a low-dimensional numerical structure via linear principal component analysis. This data representation allows for global structure discovery via cluster analysis, which can characterize the typical behavior of each of the primary types of transaction events. Furthermore, the structure of the top principal components captures the “typical behavior” of the data and is thus amenable to anomaly detection through statistical hypothesis testing. We explored this capability by generating erroneous permutations of the data and computing the Q-residual quantity increase associated with the information loss when these data are projected into the low-dimensional principal component analysis space representative of typical transactions.Future work will focus on identifying data transformations (e.g., graph networks) that more closely align with the inherent structure of these transactions to explain more salient information and disambiguate the underlying nuclear process—in this case, irradiation and reprocessing—from artifacts of NMC&A system transactional record keeping data.

Drescher, Adam↗

Opportunities for an Integrated Web-Based Workbench for Data Access and Analysis - 20244

Management of environmental issues can require integration of multiple types of data and information, conducting data analysis and interpretation, and providing data visualization for effective communications. These data elements are important for site management to support regulator interactions and provide defensibility for remedial decisions. Databases and information repositories are core elements of managing data; however, efficient data access and analysis also enable effective site management. The U.S. Department of Energy (DOE) Hanford Site is an example of a complex site with a voluminous quantity of environmental data and a need for efficient site management. Different tiers of data and information tools have been developed and deployed to address site needs. These tools are configured for ready access via the web site interfaces and meet the rigorous quality requirements for environmental site management. Evolving efforts are focused on an integrated platform to meet site environmental management needs. In this platform, users can access site information at multiple levels of detail based on their need and permissions, so that data and associated analyses are presented within the context of the site mission and the user's management or technical needs. This concept is not only applicable at individual sites like Hanford but also applicable at other sites within the DOE complex. An integrated web-based architecture that links data visualization, data analytics, and management tools can provide holistic access to large data sets, minimize complexity, and maximize interactivity and technical communication. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

EDX ClaiMM

EDX ClaiMM is a centralized data & analytical platform designed to revolutionize U.S. critical minerals and materials (CMM) activities. By providing a robust digital infrastructure, ClaiMM will accelerate the combination, leveraging, and rapid utilization of vital data, advanced tools, and cutting-edge research advancements in CMM. This adaptive digital research hub connects the CMM community to essential knowledge products and offers access to interoperable datasets, databases, models, software, and tools from the National Energy Technology’s (NETL’s) Energy Data eXchange (EDX) and other authoritative sources, serving both public and private sectors. EDX ClaiMM delivers AI-informed solutions to address fundamental knowledge gaps and fosters the innovation of new techniques for enhanced characterization and recovery of CMMs within the U.S. By leveraging cloud-hosted, scalable digital infrastructure, ClaiMM meets public–private applied energy needs. It equips the CMM community with priority digital resources that harness on-site and cloud compute capabilities, enabling big data storage, advanced processing, analytics, and visualization.

Critical Materials; Critical Minerals; Rare Earth ↗

Event Definition for the Automated Detection of Nuclear Proliferation Activities

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Discovery Analytics Center (DAC) at Virginia Polytechnic Institute and State University (VT) began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on largescale open data sources to identify new, developing, or undeclared nuclear programs. One of the most challenging aspects of applying machine learning techniques to such a problem is the high likelihood of extremely sparse data from disparate sources. To overcome this challenge, the current work will use a strategic combination of supervised, semi-supervised, and unsupervised learning techniques to ingest and fuse data streams to make a forecast of nuclear activities in a targeted geospatial location. Identifying potential data sources and training supervised learning algorithms is dependent upon the development of a robust foundation of targeted event domains that fundamentally define the nuclear activities of interest. This report documents the definition of a hierarchical structure for both nuclear activity and event domains that will be used to guide the research team in development or use of existing semantic dictionaries that are instrumental to searching, parsing, and categorizing events for the forecasting system’s use.

97 MATHEMATICS AND COMPUTING↗

LANL Regular Employee Population and Demographics, FY15-21

This document summarizes an effort to compile and analyze the LANL regular employee population’s demographic attributes and trends between FY15-FY21 using MicroStrategy, a web-based data analytics and visualization tool. LANL’s HR Division collected and prepared data for the Weapons Program’s annual reporting activities for the 2017 through 2023 NNSA Stockpile Stewardship and Management Plan (SSMP). Los Alamos workforce data represent the fiscal year-end snapshot of the permanent employee population who are categorized by the Common Occupational Classification System (COCS). This work enables further analyses, including trends for all workforce attributes presented in the SSMPs as well as a definition of a crosswalk between COCS categorization and the internal laboratory job classification hierarchy. Data for other sites and the federal workforce, as shown in the SSMPs, are included and corresponding figures can be visualized within the MicroStrategy tool. A portable, Excel-based version of the tool was developed and shared with the multi-site workforce working group. This summary highlights selected aspects of the LANL regular workforce. The data analyses and visualizations communicate insights within three major themes regarding permanent LANL employees: 1) demographic transformation, 2) attrition, and 3) skills. Regarding demographic transformation, and consistent with the general trend across the enterprise, early career regular employees are the fastest growing group, with growth ranging between 12-34% year-to-year over the seven-year period analyzed. The data also reflect effects of internal and external factors on attrition; the large spike in separations in FY18 aligns with the contract transition from LANS to Triad and the dip in attrition between FY20-FY21 is likely a result of the global pandemic. The population of regular employees in general management, engineer, and operator COCS categories have the fastest growth rates amongst all the occupational categories. The MicroStrategy and Excel tools provide additional details and trends.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗