Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Bloom Filter framework for Web Archives

The software provides a framework to build a fast look up layer that reflects the holdings of an archive or database and operates between search/discovery system and disk storage. The software utilizes Bloom filter (BF) data structure. The discovery service powered by the Bloom filter layer comes with a very high level of accuracy yet space-saving, since BF compresses data to fit into RAM.

Balakireva, Lyudmila↗

Data from Rossi-alpha and pulsed neutron prompt neutron time decay measurements at the Oak Ridge Critical Experiments Facility

This report briefly describes the Rossi-alpha and pulsed neutron prompt neutron time decay measurements performed at the Oak Ridge Critical Experiments Facility. This description includes the measurement equipment, type of data, and how to access the actual data from Records Management Services at Oak Ridge National Laboratory and the Nuclear Criticality Safety Benchmark Program (NCSBEP) of the Idaho National Laboratory. Most of the data were acquired between 1960 and 1975 on weapons-grade enriched uranium metal assemblies. Also included are measurements of the same type performed at Los Alamos National Laboratory by Oak Ridge National Laboratory with the JEZEBEL and FLATTOP criticality assemblies and other plutonium metal parts. The purpose of this report is to allow future mining of the data for publication of additional information that can be used to verify calculational methods and nuclear cross sections. These data can be used to create reactor physics benchmarks for the Nuclear Energy Agency’s International Handbook of Evaluated Reactor Physics Experiments. Present calculational methods can be used to directly calculate the measured data. The prompt neutron decay is more neutron spectrum sensitive than other measured quantities, particularly for interacting fissile metal systems where the flight time of neutrons between interacting sections is a large part of the neutron lifetime.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Cross-Layered Distributed Data-Driven Framework for Enhanced Smart Grid Cyber-Physical Security

Smart Grid (SG) research and development has drawn much attention from academia, industry and government due to the great impact it will have on society, economics and the environment. Securing the SG is a considerably significant challenge due the increased dependency on communication networks to assist in physical process control, exposing them to various cyber-threats. In addition to attacks that change measurement values using False Data Injection (FDI) techniques, attacks on the communication network may disrupt the power system's real-time operation by intercepting messages, or by flooding the communication channels with unnecessary data. Addressing these attacks requires a cross-layer approach. In this paper a cross-layered strategy is presented, called Cross-Layer Ensemble CorrDet with Adaptive Statistics(CECD-AS), which integrates the detection of faulty SG measurement data as well as inconsistent network inter-arrival times and transmission delays for more reliable and accurate anomaly detection and attack interpretation. Numerical results show that CECD-AS can detect multiple False Data Injections, Denial of Service (DoS) and Man In The Middle (MITM) attacks with a high F1-score compared to current approaches that only use SG measurement data for detection such as the traditional physics-based State Estimation, Ensemble CorrDet with Adaptive Statistics strategy and other machine learning classification-based detection schemes.

cyber-physical security↗

Coordinating an operational data distribution network for CMIP6 data

The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

CLARA: CLAS12 Reconstruction and Analysis Framework

In this paper we present SOA based CLAS12 event Reconstruction and Analyses (CLARA) framework. CLARA design focus is on two main traits: real-time data stream processing, and service-oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, Vardan↗

Component based dataflow processing framework

In this paper we present SOA based CLAs12 event Reconstruction and Analyses (CLARA) framework used to develop Earth Science multi-sensor data fusion, processing, and analytics applications (NAIADS: NASA JLAB collaboration). CLARA design focus is on two main traits: a) real-time data stream processing, and b) service oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions, capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, V.↗

Climate Dynamics Preceding Summer Forest Fires in California and the Extreme Case of 2018

Recent record-breaking wildfire seasons in California prompt an investigation into the climate patterns that typically precede anomalous summer burned forest area. Using burned-area data from the U.S. Forest Service’s Monitoring Trends in Burn Severity (MTBS) product and climate data from the fifth major global reanalysis produced by the European Centre for Medium-Range Weather Forecasts (ERA5) over 1984–2018, relationships between the interannual variability of antecedent climate anomalies and July California burned area are spatially and temporally characterized. Lag correlations show that antecedent high vapor pressure deficit (VPD), high temperatures, frequent extreme high temperature days, low precipitation, high subsidence, high geopotential height, low soil moisture, and low snowpack and snowmelt anomalies all correlate significantly with July California burned area as far back as the January before the fire season. Seasonal regression maps indicate that a global midlatitude atmospheric wave train in late winter is associated with anomalous July California burned area. July 2018, a year with especially high burned area, was to some extent consistent with the general patterns revealed by the regressions: low winter precipitation and high spring VPD preceded the extreme burned area. However, geopotential height anomaly patterns were distinct from those in the regressions. Extreme July heat likely contributed to the extent of the fires ignited that month, even though extreme July temperatures do not historically significantly correlate with July burned area. While the 2018 antecedent climate conditions were typical of a high-burned-area year, they were not extreme, demonstrating the likely limits of statistical prediction of extreme fire seasons and the need for individual case studies of extreme years.

54 ENVIRONMENTAL SCIENCES↗

Discover, Visualize, and Deliver Geospatial Data through OGC Standards-based WebGIS System

Geospatial data are important to understand the Earth - ecosystem dynamics, land cover changes, resource management, and human interactions with the Earth to name a few. One of the biggest difficulties users face is to discover, access, and assemble distributed, large volume, heterogeneous geospatial data to conduct geo-analysis. Traditional methods of geospatial data discovery, visualization, and delivery lack the capabilities of resource sharing and automation across systems or organizational boundaries. They require users to download the data ldquoas-isrdquo in their original file format, projection, and extent. Also, discovering data served by traditional methods requires prior knowledge of data location, and processing requires specialized expertise. These drawbacks of traditional methods create additional burden to users, introduce too much overhead to research, and also reduce the potential usage of the data. At the Oak Ridge National Laboratory (ORNL), researchers working on NASA-sponsored projects: Distributed Active Archive Center (DAAC) and Modeling and Synthesis Thematic Data Center (MAST-DC) have tapped into the benefits of Open Geospatial Consortium (OGC) standards to overcome the drawbacks of traditional methods of geospatial data discovery, visualization, and delivery. The OGC standards-based approach facilitates data sharing and interoperability across network, organizational, and geopolitical boundaries. Tools and services based on OGC standards deliver the data in many user defined formats and allow users to visualize the data prior to download. This paper introduces an approach taken to visualize and deliver ORNL DAAC, MAST-DC, and other relevant geospatial data through OGC standards-based Web Services, including Web Map Service (WMS), Web Coverage Service (WCS), and Web Feature Service (WFS). It also introduces a WebGIS system built on top of OGC services that helps users discover, visualize, and access geospatial data.

99 GENERAL AND MISCELLANEOUS↗

A message-queuing framework for STAR's online monitoring and metadata collection

We report our experience on migrating STARs Online Services (Run Control System, Data Acquisition System, Slow Control System and Subsystem Monitoring) from direct read/write database accesses to a modern non-blocking message-oriented infrastructure. Based on the Advanced Messaging Queuing Protocol (AMQP) and standards, this novel approach does not specify the message data structure, allowing great flexibility in its use. After careful consideration, we chose Google Protocol Buffers as our primary (de)serialization format for structured data exchange. This migration allows us to reduce the overall system complexity and greatly improve the reliability of the metadata collection and the performance of our online services in general. We will present this new framework through its software architecture overview, providing details about our staged and non-disruptive migration process as well as details of the implementation of pluggable components to provide future improvements without compromising stability and availability of services.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Dissolved Oxygen and Temperature Data from the Hyporheic Zone of the East River Watershed July 2017 to October 2018

Dissolved oxygen (DO) is critical for aquatic ecosystems. Our focus is on the long-term DO dynamics in hyporheic zone of rivers, which are a function of both transport (hydrologic exchange between river and hyporheic zone) and uptake by biogeochemical reactions or respiration. The study site is the alpine East River watershed in Colorado, USA, meander A downstream from the pump house. Opti O2 probes were deployed in the water column and directly within the river-bed at 10, 20, and 35 cm depth (38°55'23.38"N, 106°57'4.21"W, 9046.88m) to monitor DO and temperature. A continuous data stream from July 24, 2017 to Oct 24, 2018 was autonomously telemetered to the cloud. This 14-month DO and temperature time series were obtained without any servicing for maintenance or data downloads; additionally the ability to remotely verify probe performance during field deployment was essential to confirm data validity during winter freeze-in and hydrological/weather events, such as spring melt and summer monsoons. We investigate the variations in dissolved oxygen dynamics of this snow-pack dominated watershed during a comparatively low flow water year (2018) and a relatively normal water year (2017), enabled by distinctive, in-situ, high frequency (∆t = 5min) sensors that provided a continuous time-series from the undisturbed study site over multiple seasons.

54 ENVIRONMENTAL SCIENCES↗

Developing Poultry Facility Type Information from USDA Agricultural Census Data for Use in Epidemiological and Economic Models

The epidemiological and economic modeling of poultry diseases requires knowing the size, location, and operational type of each poultry type operation within the US. At the present time, the only national database of poultry operations that is available to the general public is the USDA's 2002 Agricultural Census data, published by the National Agricultural Statistics Service, herein referred to as the 'NASS data'. The NASS data provides census data at the county level on poultry operations for various operation types (i.e., layers, broilers, turkeys, ducks, geese). However, the number of farms and sizes of farms for the various types are not independent since some facilities have more than one type of operation. Furthermore, some data on the number of birds represents the number sold, which does not represent the number of birds present at any given time. In addition, any data tabulated by NASS that could identify numbers of birds or other data reported by an individual respondent is suppressed by NASS and coded with a 'D'. To be useful for epidemiological and economic modeling, the NASS data must be converted into a unique set of facility types (farms having similar operational characteristics). The unique set must not double count facilities or birds. At the same time, it must account for all the birds, including those for which the data has been suppressed. Therefore, several data processing steps are required to work back from the published NASS data to obtain a consistent database for individual poultry operations. This technical report documents data processing steps that were used to convert the NASS data into a national poultry facility database with twenty-six facility types (7 egg-laying, 6 broiler, 1 backyard, 3 turkey, and 9 others, representing ducks, geese, ostriches, emus, pigeons, pheasants, quail, game fowl breeders and 'other'). The process involves two major steps. The first step defines the rules used to estimate the data that is suppressed within the NASS database. The first step is similar to the first step used to estimate suppressed data for livestock [Melius et al (2006)]. The second step converts the NASS poultry types into the operational facility types used by the epidemiological and economic model. We also define two additional facility types for high and low risk poultry backyards, and an additional two facility types for live bird markets and swap meets. The distribution of these additional facility types among counties is based on US population census data. The algorithm defining the number of premises and the corresponding distribution among counties and the resulting premises density plots for the continental US are provided.

60 APPLIED LIFE SCIENCES↗

Macroinvertebrate Abundance and Biomass: 2007 Data, BPA-51; Preliminary Report, February 10, 2009..

Four Excel files containing information on the 2007 macroinvertebrate data were initially provided to Statistical Consulting Services (SCS) by EcoAnalysts on 1/27/2009. These data files contained information on abundance and biomass data at the level of taxonomic groups. The data were subsequently reformatted and compiled, and aggregated for analysis by SCS. All descriptions and analyses below relate to this compiled data. Computations were carried out separately for each site over all sample periods. Basic summary information for both the abundance and biomass data is presented in Print Out No.2. The 14 sites varied widely in their minimum, mean, maximum and variance values. The number of observations ranged from 10 to 18. Some large abundance values (abundance > 40,000) were noted for sites KR6 and KR13. A more detailed summary of each site is given in Print Out No.3. Site KR3, for example, had a mean abundance of 6914 with a sample size of 17. The variance was 4591991 and the standard error of the mean was 1643. The skewness value, a measure of symmetry for the frequency distribution, was moderately large at 1.29 indicating an asymmetric distribution. Biomass for KR3 had a mean value of 0.87 g/m{sup 2} with 17 observations. The variance was 0.8872 and the standard error was 0.228 g/m{sup 2}. Skewness for biomass was also high at 1.29. Further examination of the quantiles and frequency plots for abundance and biomass also indicate considerable skewness. The stem and leaf diagram (frequency plot) for abundance in KR3 shows most of the data centered on smaller values with a few very large counts. The distribution for biomass has a similar pattern. Statistical tests for normality are significant for both response variables in KR3, thus, the hypothesis that the data originates from a symmetric normal distribution is rejected. Because sample size estimation and statistical inference assume normally distributed data, a transformation of the data is required prior to further analysis. As was the case for previous years, the natural logarithm was chosen as a transformation to mitigate distributional skewness. Abundance and biomass for the remaining sites were also notably skewed, therefore, these data were also log transformed prior to analysis. Summary information for the transformed data (referred to as L-abun and L-bio for abundance and biomass, respectively) are given in Print Out No.4. For site KR3, the logarithmic transformation reduced skewness value for biomass to -0.66. The distributions of abundance and biomass in the other sites also generally showed improvement as well. Hence, all subsequent statistical analyses reported here will be based on the log transformed data.

13 HYDRO ENERGY↗

Detecting insider activity using enhanced directory virtualization.

Insider threats often target authentication and access control systems, which are frequently based on directory services. Detecting these threats is challenging, because malicious users with the technical ability to modify these structures often have sufficient knowledge and expertise to conceal unauthorized activity. The use of directory virtualization to monitor various systems across an enterprise can be a valuable tool for detecting insider activity. The addition of a policy engine to directory virtualization services enhances monitoring capabilities by allowing greater flexibility in analyzing changes for malicious intent. The resulting architecture is a system-based approach, where the relationships and dependencies between data sources and directory services are used to detect an insider threat, rather than simply relying on point solutions. This paper presents such an architecture in detail, including a description of implementation results.

99 GENERAL AND MISCELLANEOUS↗

The CMS DBS Query Language

The CMS experiment has implemented a flexible and powerful system enabling users to find data within the CMS physics data catalog. The Dataset Bookkeeping Service (DBS) comprises a database and the services used to store and access metadata relate d to CMS physics data. To this, we have added a generalized query system in addition to the existing web and programmatic interfaces to the DBS. This query system is based on a query language that hides the complexity of the underlying database structure by discovering the join conditions between database tables. This provides a way of querying the system that is simple and straightforward for CMS data managers and physicists to use without requiring knowledge of the database tables or keys. The DBS Query Language uses the ANTLR tool to build the input query parser and tokenizer, followed by a query builder that uses a graph representation of the DBS schema to construct the SQL query sent to underlying database. We will describe the design of the query sy stem, provide details of the language components and overview of how this component fits into the overall data discovery system architecture.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Data Management Facility Operations Plan

The Data Management Facility (DMF) is the data center that houses several critical Atmospheric Radiation Measurement (ARM) Climate Research Facility services, including first-level data processing for the ARM Mobile Facilities (AMFs), Eastern North Atlantic (ENA), North Slope of Alaska (NSA), Southern Great Plains (SGP), and Tropical Western Pacific (TWP) sites, as well as Value-Added Product (VAP) processing, development systems, and other network services.

Keck, Nicole N↗

Commercial Building Energy Asset Score

This software (Asset Scoring Tool) is designed to help building owners and managers to gain insight into the as-built efficiency of their buildings. It is a web tool where users can enter their building information and obtain an asset score report. The asset score report consists of modeled building energy use (by end use and by fuel type), building systems (envelope, lighting, heating, cooling, service hot water) evaluations, and recommended energy efficiency measures. The intended users are building owners and operators who have limited knowledge of building energy efficiency. The scoring tool collects minimum building data (~20 data entries) from users and build a full-scale energy model using the inference functionalities from Facility Energy Decision System (FEDS). The scoring tool runs real-time building energy simulation using EnergyPlus and performs life-cycle cost analysis using FEDS. An API is also under development to allow the third-party applications to exchange data with the web service of the scoring tool.

Source record↗

Estimating a Service-Life Distribution Based on Production Counts and a Failure Database

A manufacturer wanted to compare the service-life distributions of two similar products. These concern product lifetimes after installation (not manufacture). For each product, there were available production counts and an imperfect database providing information on failing units. In the real case, these units were expensive repairable units warrantied against repairs. Failure (of interest here) was relatively rare and driven by a different mode/mechanism than ordinary repair events (not of interest here). Approach: Data models for the service life based on a standard parametric lifetime distribution and a related limited failure population were developed. These models were used to develop expressions for the likelihood of the available data that properly accounts for information missing in the failure database. Results: A Bayesian approach was employed to obtain estimates of model parameters (with associated uncertainty) in order to investigate characteristics of the service-life distribution. Custom software was developed and is included as Supplemental Material to this case study. One part of a responsible approach to the original case was a simulation experiment used to validate the correctness of the software and the behavior of the statistical methodology before using its results in the application, and an example of such an experiment is included here. Because of confidentiality issues that prevent use of the original data, simulated data with characteristics like the manufacturer’s proprietary data are used to illustrate some aspects of our real analyses. Lastly, we also note that, although this case focuses on rare and complete product failure, the statistical methodology provided is directly applicable to more standard warranty data problems involving typically much larger warranty databases where entries are warranty claims (often for repairs) rather than reports of complete failures.

97 MATHEMATICS AND COMPUTING↗

HPC s Pivot to Data

Computer centers such as NERSC and OLCF have traditionally focused on delivering computational capability that enables breakthrough innovation in a wide range of science domains. Accessing that computational power has required services and tools to move the data from input and output to computation and storage. A ''pivot to data'' is occurring in HPC. Data transfer tools and services that were previously peripheral are becoming integral to scientific workflows. Emerging requirements from high-bandwidth detectors, high-throughput screening techniques, highly concur- rent simulations, increased focus on uncertainty quantification, and an emerging open-data policy posture toward published research are among the data-drivers shaping the networks, file systems, databases, and overall compute and data environment. In this paper we explain the pivot to data in HPC through user requirements and the changing resources provided by HPC with particular focus on data movement. For WAN data transfers we present the results of a study of network performance between centers.

97 MATHEMATICS AND COMPUTING↗