Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Archive”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗

A web-based subsetting service for regional scale MODIS land products

The Moderate Resolution Imaging Spectroradiometer (MODIS) sensor has provided valuable information on various aspects of the Earth System since March 2000. The spectral, spatial, and temporal characteristics of MODIS products have made them an important data source for analyzing key science questions relating to Earth System processes at regional, continental, and global scales. The size of the MODIS product and native HDF-EOS format are not optimal for use in field investigations at individual sites (100 - 100 km or smaller). In order to make MODIS data readily accessible for field investigations, the NASA-funded Distributed Active Archive Center (DAAC) for Biogeochemical Dynamics at Oak Ridge National Laboratory (ORNL) has developed an online system that provides MODIS land products in an easy-to-use format and in file sizes more appropriate to field research. This system provides MODIS land products data in a nonproprietary comma delimited ASCII format and in GIS compatible formats (GeoTIFF and ASCII grid). Web-based visualization tools are also available as part of this system and these tools provide a quick snapshot of the data. Quality control tools and a multitude of data delivery options are available to meet the demands of various user communities. This paper describes the important features and design goals for the system, particularly in the context of data archive and distribution for regional scale analysis. The paper also discusses the ways in which data from this system can be used for validation, data intercomparison, and modeling efforts.

54 ENVIRONMENTAL SCIENCES↗

A century of genetic variation inferred from a persistent soil‐stored seed bank

Abstract Stratigraphic accretion of dormant propagules in soil can result in natural archives useful for studying ecological and evolutionary responses to environmental change. Few attempts have been made, however, to use soil‐stored seed banks as natural archives, in part because of concerns over nonrandom attrition and mixed stratification. Here, we examine the persistent seed bank of Schoenoplectus americanus , a foundational brackish marsh sedge, to determine whether it can serve as a resource for reconstructing historical records of demographic and population genetic variation. After assembling profiles of the seed bank from radionuclide‐dated soil cores, we germinated seeds to “resurrect” cohorts spanning the 20th century. Using microsatellite markers, we assessed genetic diversity and differentiation among depth cohorts, drawing comparisons to extant plants at the study site and in nearby and more distant marshes. We found that seed density peaked at intermediate soil depths. We also detected genotypic differences among cohorts as well as between cohorts and extant plants. Genetic diversity did not decline with depth, indicating that the observed pattern of differentiation is not due to attrition. Patterns of differentiation within and among extant marshes also suggest that local populations persist as aggregates of small clones, likely reflecting repeated seedling recruitment and low immigration from admixed regional gene pools. These findings indicate that persistent and stratified soil‐stored seed banks merit further consideration as resources for reconstructing decadal‐ to century‐long records that can lend insight into the tempo and nature of ecological and evolutionary processes that shape populations over time.

54 ENVIRONMENTAL SCIENCES↗

Modelling the flaring activity of the high-z, hard X-ray-selected blazar IGR J22517+2217: Flaring activity of IGR J22517+2217

We present new Suzaku and Fermi data and re-analysed archival hard X-ray data from the INTErnational Gamma-Ray Astrophysics Laboratory (INTEGRAL) and Swift–Burst Alert Telescope (BAT) surveys to investigate the physical properties of the luminous, high-redshift, hard X-ray-selected blazar IGR J22517+2217, through the modelling of its broad-band spectral energy distribution (SED) in two different activity states. Through analysis of new Suzaku data and flux-selected data from archival hard X-ray observations, we build the source SED in two different states, one for the newly discovered flare that occurred in 2005 and one for the following quiescent period. Both SEDs are strongly dominated by the high-energy hump peaked at 10 20 –10 22 Hz, which is at least two orders of magnitude higher than the low-energy (synchrotron) one at 10 11 –10 14 Hz and varies by a factor of 10 between the two states. In both states the high-energy hump is modelled as inverse Compton emission between relativistic electrons and seed photons produced externally to the jet, while the synchrotron self-Compton component is found to be negligible. In our model the observed variability can be accounted for by a variation of the total number of emitting electrons and by a dissipation region radius changing from inside to outside the broad-line region as the luminosity increases. In its flaring activity, IGR J22517+2217 is revealed as one of the most powerful jets among the population of extreme, hard X-ray-selected, high-redshift blazars observed so far.

79 ASTRONOMY AND ASTROPHYSICS↗

Offline multiple adaptive planning strategy for concurrent irradiation of the prostate and pelvic lymph nodes

Purpose: Concurrent irradiation of the prostate and pelvic lymph nodes (PLNs) can be challenging due to the independent motion of the two target volumes. To address this challenge, the authors have proposed a strategy referred to as Multiple Adaptive Planning (MAP). To minimize the number of MAP plans, the authors’ previous work only considered the prostate motion in one major direction. After analyzing the pattern of the prostate motion, the authors investigated a practical number of intensity-modulated radiotherapy (IMRT) plans needed to accommodate the prostate motion in two major directions simultaneously. Methods: Six patients, who received concurrent irradiation of the prostate and PLNs, were selected for this study. Nine MAP-IMRT plans were created for each patient with nine prostate contours that represented the prostate at nine locations with respect to the PLNs, including the original prostate contour and eight contours shifted either 5 mm in a single anterior-posterior (A-P), or superior-inferior (S-I) direction, or 5 mm in both A-P and S-I directions simultaneously. From archived megavoltage cone beam CT (MV-CBCT) and a dual imaging registration, 17 MV-CBCTs from 33 available MV-CBCT from these patients showed large prostate displacements (>3 mm in any direction) with respect to the pelvic bones. For each of these 17 fractions, one of nine MAP-IMRT plans was retrospectively selected and applied to the MV-CBCT for dose calculation. For comparison, a simulated isocenter-shifting plan and a reoptimized plan were also created for each of these 17 fractions. The doses to 95% (D95) of the prostate and PLNs, and the doses to 5% (D5) of the rectum and bladder were calculated and analyzed. Results: For the prostate, D95 > 97% of the prescription dose was observed in 16, 16, and 17 of 17 fractions for the MAP, isocenter-shifted, and reoptimized plans, respectively. For PLNs, D95 > 97% of the prescription doses was observed in 10, 3, and 17 of 17 fractions for the three types of verification plans, respectively. The D5 (mean ± SD) of the rectum was 45.78 ± 5.75, 45.44 ± 4.64, and 44.64 ± 2.71 Gy, and the D5 (mean ± SD) of the bladder was 45.18 ± 2.70, 46.91 ± 3.04, and 45.67 ± 3.61 Gy for three types of verification plans, respectively. Conclusions: The MAP strategy with nine IMRT plans to accommodate the prostate motions in two major directions achieved good dose coverage to the prostate and PLNs. The MAP approach can be immediately used in clinical practice without requiring extra hardware and software.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

The Bee Microbiome: Impact on Bee Health and Model for Evolution and Ecology of Host-Microbe Interactions

ABSTRACT As pollinators, bees are cornerstones for terrestrial ecosystem stability and key components in agricultural productivity. All animals, including bees, are associated with a diverse community of microbes, commonly referred to as the microbiome. The bee microbiome is likely to be a crucial factor affecting host health. However, with the exception of a few pathogens, the impacts of most members of the bee microbiome on host health are poorly understood. Further, the evolutionary and ecological forces that shape and change the microbiome are unclear. Here, we discuss recent progress in our understanding of the bee microbiome, and we present challenges associated with its investigation. We conclude that global coordination of research efforts is needed to fully understand the complex and highly dynamic nature of the interplay between the bee microbiome, its host, and the environment. High-throughput sequencing technologies are ideal for exploring complex biological systems, including host-microbe interactions. To maximize their value and to improve assessment of the factors affecting bee health, sequence data should be archived, curated, and analyzed in ways that promote the synthesis of different studies. To this end, the BeeBiome consortium aims to develop an online database which would provide reference sequences, archive metadata, and host analytical resources. The goal would be to support applied and fundamental research on bees and their associated microbes and to provide a collaborative framework for sharing primary data from different research programs, thus furthering our understanding of the bee microbiome and its impact on pollinator health.

Microbiology↗

PlanetSense: A Real-time Streaming and Spatio-temporal Analytics Platform for Gathering Geo-spatial Intelligence from Open Source Data

Geospatial intelligence has traditionally relied on the use of archived and unvarying data for planning and exploration purposes. In consequence, the tools and methods that are architected to provide insight and generate projections only rely on such datasets. Albeit, if this approach has proven effective in several cases, such as land use identification and route mapping, it has severely restricted the ability of researchers to inculcate current information in their work. This approach is inadequate in scenarios requiring real-time information to act and to adjust in ever changing dynamic environments, such as evacuation and rescue missions. In this work, we propose PlanetSense, a platform for geospatial intelligence that is built to harness the existing power of archived data and add to that, the dynamics of real-time streams, seamlessly integrated with sophisticated data mining algorithms and analytics tools for generating operational intelligence on the fly. The platform has four main components – i) GeoData Cloud – a data architecture for storing and managing disparate datasets; ii) Mechanism to harvest real-time streaming data; iii) Data analytics framework; iv) Presentation and visualization through web interface and RESTful services. Using two case studies, we underpin the necessity of our platform in modeling ambient population and building occupancy at scale.

big data architecture↗

ramdisk, Version 0.x

The intent of the software is eventually to become a middleware library to provide a cross platform interface for creating and managing a ramdisk. Eventually it will cover many platforms (*nix, Windows, Mac), and have the library written in several languages. At this point, the library is a functional python library for managing a Mac ramdisk. Some code is also included in how to use the library. Initial intended uses for the library: archive deleteone.sql elink_2017_10_20.json file.json fix_on_records.sh just-doi-it.sh onrecords postfile.sh postit.sh solr-delete testrepos.sh wipe.postgresql Provide a throw-away filesystem for unit testing filesystem related functionality. archive deleteone.sql elink_2017_10_20.json file.json fix_on_records.sh just-doi-it.sh onrecords postfile.sh postit.sh solr-delete testrepos.sh wipe.postgresql Improve compile time when building software.

Nielsen, Roy↗

ARM Data Integrator

The Atmospheric Radiation Measurement (ARM) Data Integrator (ADI) streamlines the development of scientific algorithms and analysis of time-series NetCDF data, and improves the content and consistency of the output data products produced by these algorithms. The framework automates the process of retrieving and preparing data for analysis, and allows users to design output data products through a graphical interface. It also provides a modular, flexible software development architecture that scientists can use to generate C, Python, and IDL source code templates that embed the pre and post processing logic allowing the scientist to focus on only their science. The input data, preprocessing, and output data specifications of algorithms are defined through a graphical interface and stored in a database. ADI implements workflow for data integration and supports user access to data through a library of software modules. Data preprocess capabilities supported include automated retrieval of data from input files, merging the retrieved data into appropriately sized chunks, and transformation of the data onto a common coordinate system grid. Through the graphical interface, users can view the details of both their data products and those in the ARM catalog and allows developers to use existing data product to build new data products. Views of the output data products include an overlay of how the design meets ARM archive's data standards providing the user with a visual cue indicating where their output violates an archive standard. The ADI libraries access the information provided through the GUI via a Postgres database. The ADI framework and its supporting components can significantly decrease the time and cost of implementing scientific algorithms while improving the ability of scientists to disseminate their results.

Source record↗

pnnl/archive-sprinter

The Archive Sprinter tool is designed to efficiently process synchrophasor measurements from electric power grids and export data signatures that summarize the grid's behavior. Parallel processing will allow data to be processed quickly to enable practical analyses of archives spanning years. The grid's behavior will be summarized using a wide-array of signatures calculated from the input data.

Follum, Jim↗

The earth system Grid : enabling access to multi-model climate simulation data.

Climate scientists face a wide variety of practical problems, but there is an overarching need to efficiently access and manipulate climate model data. Increasingly, for example, researchers must assemble and analyze large datasets that are archived in different formats on disparate platforms, and extract portions of datasets to compute statistical or diagnostic metrics 'in place'. The need for a common virtual environment in which to access both climate model datasets and analysis tools is therefore keenly felt. The software infrastructure to support such an environment must therefore not only provide ready access to climate data, but also must facilitate the use of visualization software, diagnostic algorithms, and related resources. To this end, the Earth System Grid Center for Enabling Technologies (ESG-CET) was established in 2006 by the Scientific Discovery through Advanced Computing (SciDAC)-2 program of the U.S. Department of Energy (DOE) through the Office of Advanced Scientific Computing Research (OASCR) and the Office Biological and Environmental Research (OBER) within the Office of Science (Fig. 1). ESG-CET is working to advance climate science by developing computational resources for accessing and managing model data that are physically located in distributed multiplatform archives.

54 ENVIRONMENTAL SCIENCES↗

The Midlatitude Continental Convective Clouds Experiment (MC3E)

The Midlatitude Continental Convective Clouds Experiment (MC3E), a field program jointly led by the U.S. Department of Energy’s Atmospheric Radiation Measurement program and the NASA Global Precipitation Measurement (GPM) Mission, was conducted in south-central Oklahoma during April – May 2011. MC3E science objectives were motivated by the need to improve understanding of midlatitude continental convective cloud system lifecycles, microphysics, and GPM precipitation retrieval algorithms. To achieve these objectives a multi-scale surface- and aircraft-based in situ and remote sensing observing strategy was employed. A variety of cloud and precipitation events were sampled during the MC3E, of which results from three deep convective events are highlighted. Vertical structure, air motions, precipitation drop-size distributions and ice properties were retrieved from multi-wavelength radar, profiler, and aircraft observations for an MCS on 11 May. Aircraft observations for another MCS observed on 20 May were used to test agreement between observed radar reflectivities and those calculated with forward-modeled reflectivity and microwave brightness temperatures using in situ particle size distributions and ice water content. Multi-platform observations of a supercell that occurred on 23 May allowed for an integrated analysis of kinematic and microphysical interactions. A core updraft of 25 ms-1 supported growth of hail and large rain drops. Data collected during the MC3E campaign is being used in a number of current and ongoing research projects and is available through the DOE ARM and NASA data archives.

Jensen, Mark P.↗

Assessing state-of-the-art capabilities for probing the atmospheric boundary layer: The XPIA field campaign

To assess current capabilities for measuring flow within the atmospheric boundary layer, including within wind farms, the U.S. Dept. of Energy sponsored the eXperimental Planetary boundary layer Instrumentation Assessment (XPIA) campaign at the Boulder Atmospheric Observatory (BAO) in spring 2015. Herein, we summarize the XPIA field experiment, highlight novel measurement approaches, and quantify uncertainties associated with these measurement methods. Line-of-sight velocities measured by scanning lidars and radars exhibit close agreement with tower measurements, despite differences in measurement volumes. Virtual towers of wind measurements, from multiple lidars or radars, also agree well with tower and profiling lidar measurements. Estimates of winds over volumes from scanning lidars and radars are in close agreement, enabling assessment of spatial variability. Strengths of the radar systems used here include high scan rates, large domain coverage, and availability during most precipitation events, but they struggle at times to provide data during periods with limited atmospheric scatterers. In contrast, for the deployment geometry tested here, the lidars have slower scan rates and less range, but provide more data during non-precipitating atmospheric conditions. Microwave radiometers provide temperature profiles with approximately the same uncertainty as Radio-Acoustic Sounding Systems (RASS). Using a motion platform, we assess motion-compensation algorithms for lidars to be mounted on offshore platforms. As a result, we highlight cases for validation of mesoscale or large-eddy simulations, providing information on accessing the archived dataset. We conclude that modern remote sensing systems provide a generational improvement in observational capabilities, enabling resolution of fine-scale processes critical to understanding inhomogeneous boundary-layer flows.

17 WIND ENERGY↗

The Second Wind Forecast Improvement Project (WFIP 2): General Overview

In 2015 the U.S. Department of Energy initiated a four-year study, the second Wind Forecast Improvement Project (WFIP 2), to improve the representation of boundary-layer physics and related processes in mesoscale models for better wind and wind power forecasts. This goal challenges numerical weather prediction models in complex terrain due to inherent assumptions underlying their boundary-layer parameterizations. The WFIP 2 effort involved the wind industry, universities, NOAA, and DOE's national laboratories in an integrated observational and modeling study. Observations spanned 18-months to assure a full annual cycle of continuously recorded observations from remote-sensing and in situ measurement systems. The study area comprised the Columbia Basin of eastern Washington and Oregon, containing more than 4 GW of installed wind capacity; nests of observational systems captured important atmospheric scales from mesoscale to NWP sub-grid scale. Model improvements targeted NOAA's High-Resolution Rapid Refresh (HRRR) model to facilitate transfer of improvements to NWS operational forecast models. This paper describes the general WFIP 2 scope and objectives, the particular scientific challenges of improving wind forecasts in complex terrain, early successes of the project, and an integrated approach to archiving observations and model output. It provides an introduction for a set of more detailed BAMS papers addressing WFIP 2 observational science, modeling challenges and solutions, practical transfer of boundary layer advances to industry use, and advances in coupling improved mesoscale models to microscale models that can represent interactions between wind plants and the atmosphere.

WFIP2, boundary layer, complex terrain, forecastin↗

Dominating Controls for Wetter South Asian Summer Monsoon in the Twenty-First Century

This study analyzes a suite of global climate models from phase 5 of the Coupled Model Intercomparison Project (CMIP5) archives to understand the mechanisms behind a net increase in the South Asian summer monsoon precipitation in response to enhanced radiative forcing during the twenty-first century. An increase in radiative forcing fuels an increase in the atmospheric moisture content through warmer temperatures, which overwhelms the weakening of monsoon circulation and results in an increase of moisture convergence and therefore summer monsoon precipitation over South Asia. Moisture source analysis suggests that both regional (local recycling, the Arabian Sea, the Bay of Bengal) and remote (including the south Indian Ocean) sources contribute to the moisture supply for precipitation over South Asia during the summer season that is facilitated by the monsoon dynamics. For regional moisture sources, the effect of excessive atmospheric moisture is offset by weaker monsoon circulation and uncertainty in the response of the evapotranspiration over land, so anomalies in their contribution to the total moisture supply are either mixed or muted. In contrast, weakening of the monsoon dynamics has less influence on the moisture supply from remote sources that not only is a dominant moisture contributor in the historical period but is also the net driver of the positive summer monsoon precipitation response in the twenty-first century. Finally, the results also indicate that historic measures of the monsoon dynamics may not be well suited to predict the nonstationary moisture-driven South Asian summer monsoon precipitation response in the twenty-first century.

54 ENVIRONMENTAL SCIENCES↗

Bacillus anthracis in China and its relationship to worldwide lineages

Background: The global pattern of distribution of 1033 B. anthracis isolates has previously been defined by a set of 12 conserved canonical single nucleotide polymorphisms (canSNP). These studies reinforced the presence of three major lineages and 12 sub-lineages and sub-groups of this anthrax-causing pathogen. Isolates that form the A lineage (unlike the B and C lineages) have become widely dispersed throughout the world and form the basis for the geographical disposition of "modern" anthrax. An archival collection of 191 different B. anthracis isolates from China provides a glimpse into the possible role of Chinese trade and commerce in the spread of certain sub-lineages of this pathogen. Canonical single nucleotide polymorphism (canSNP) and multiple locus VNTR analysis (MLVA) typing has been used to examine this archival collection of isolates. Results: The canSNP study indicates that there are 5 different sub-lineages/sub-groups in China out of 12 previously described world-wide canSNP genotypes. Three of these canSNP genotypes were only found in the western-most province of China, Xinjiang. These genotypes were A.Br.008/ 009, a sub-group that is spread across most of Europe and Asia; A.Br.Aust 94, a sub-lineage that is present in Europe and India, and A.Br.Vollum, a lineage that is also present in Europe. The remaining two canSNP genotypes are spread across the whole of China and belong to sub-group A.Br.001/002 and the A.Br.Ames sub-lineage, two closely related genotypes. MLVA typing adds resolution to the isolates in each canSNP genotype and diversity indices for the A.Br.008/009 and A.Br.001/002 sub-groups suggest that these represent older and established clades in China. Conclusion: B. anthracis isolates were recovered from three canSNP sub-groups (A.Br.008/009, A.Br.Aust94, and A.Br.Vollum) in the western most portion of the large Chinese province of Xinjiang. The city of Kashi in this province appears to have served as a crossroads for not only trade but the movement of diseases such as anthrax along the ancient "silk road". Phylogenetic inference also suggests that the A.Br.Ames sub-lineage, first identified in the original Ames strain isolated from Jim Hogg County, TX, is descended from the A.Br.001/002 sub-group that has a major presence in most of China. These results suggest a genetic discontinuity between the younger Ames sub-lineage in Texas and the large Western North American sub-lineage spread across central Canada and the Dakotas.

59 BASIC BIOLOGICAL SCIENCES↗

Model-Data for Joint Estimation of Biogeochemical Model Parameters from Multiple Experiments: A Bayesian Approach Applied to Mercury Methylation

This modeling archive supports the manuscript submitted for publication in the Environmental Modeling and Software. This study is supported by ORNL-SFA and IDEAS-Watershed. This study aims to improve calibration of complex biogeochemical models using datasets from multiple experiments targeting specific subprocesses. The proposed Bayesian joint-fitting scheme calibrates the entire biogeochemical model in one go using all the available datasets and estimate parameter uncertainties using Markov Chain Monte Carlo (MCMC). This allows for complete propagation of uncertainties and utilization of the information shared between different datasets. Mapping joint distribution of parameters guides model improvement by identifying null spaces in the parameter space. This archive contains files used to perform MCMC, post-process outputs and visualize results.

East Fork Poplar Creek↗