Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68

Data from: "A guide to using GitHub for developing and versioning data standards and reporting formats"

These data are the results of a systematic review that investigated how data standards and reporting formats are documented on the version control platform GitHub. Our systematic review identified 32 data standards in earth science, environmental science, and ecology that use GitHub for version control of data standard documents. In our analysis, we characterized the documents and content within each of the 32 GitHub repositories to identify common practices for groups that version control their documents on GitHub.In this data package, there are 8 CSV files that contain data that we characterized from each repository, according to the location within the repository. For example, in 'readme_pages.csv' we characterize the content that appears across the 32 GitHub repositories included in our systematic review. Each of the 8 CSV files has an associated data dictionary file (names appended with '_dd.csv' and here we describe each content category within CSV files.There is one file-level metadata file (flmd.csv) that provides a description of each file within the data package.

54 ENVIRONMENTAL SCIENCES↗

DMTN-107: Options for Alert Production in LSST Operations Year 1

This document reviews five options for alert production in LSST Operations Year 1 (LOY1), taking into account any implications on LSST formal requirements including up-scopes, down-scopes or explicit violations. The Data Management System Science Team's preferred option for maximizing LSST science is to generate template images from as much of the data from the commissioning and/or science verification phases as possible and use them to run Difference Image Analysis and alert production during LOY1. A proposal to increase the sky area covered during commissioning via a "filler" scheduler program is also presented. As a potential moderate up-scope, this study presents an option to build interim templates on a $\sim$monthly basis during LOY1, which could increase the accessible sky area by ~1000-2000 deg**2 per month, and should be reconsidered closer to the start of Operations.

79 ASTRONOMY AND ASTROPHYSICS↗

Climate Science for a Sustainable Energy Future Test Bed and Data Infrastructure Final Report

The collaborative Climate Science for a Sustainable Energy Future (CSSEF) project started in July 2011 with the goal of accelerating the development of climate model components (i.e., atmosphere, ocean and sea ice, and land surface) and enhancing their predictive capabilities while incorporating uncertainty quantification (UQ). This effort required accessing and converting observational data sets into specialized model testing and verification data sets and building a model development test bed, where model components and sub-models can be rapidly evaluated. CSSEF’s prototype test bed demonstrated, how an integrated testbed could eliminate tedious activities associated with model development and evaluation, by providing the capability to constantly compare model output—where scientists store, acquire, reformat, regrid, and analyze data sets one-by-one—to observational measurements in a controlled test bed.

54 ENVIRONMENTAL SCIENCES↗

Sap Velocity, Leaf-level Measurements, and Meteorological Data in Manaus and Santarem, Plant Science, Brazil: 2015-2017

These datasets were used in the paper Gimenez et al., 2019 (Species-Specific Shifts in Diurnal Sap Velocity Dynamics and Hysteretic Behavior of Ecophysiological Variables During the 2015–2016 El Niño Event in the Amazon Forest) published in Frontiers in Plant Science, special issue: Water-Use Efficiency: Advances and Challenges in a Changing Climate. This data package contains data from ICT sap flow sensors, leaf temperature (Infrared Sensors), leaf gas exchange, leaf water potential and leaf thermocouple data used for some experiments of the NGEE-Tropics project in the Amazon Forest (sites: BR-Ma2 and BR-Sa1). The data package also includes environmental data (Relative humidity, air temperature, and VPD) collected using thermo-hygrometers installed in K-34 tower vertical profile (Manaus - BR-Ma2).

54 ENVIRONMENTAL SCIENCES↗

Plenario: An Open Data Discovery and Exploration Platform for Urban Science

The past decade has seen the widespread release of open data concerning city services, conditions, and activities by government bodies and public institutions of all sizes. Hundreds of open data portals now host thousands of datasets of many different types. These new data sources represent enormous po- tential for improved understanding of urban dynamics and processes—and, ultimately, for more livable, efficient, and prosperous communities. However, those who seek to realize this potential quickly discover that discovering and applying those data relevant to any particular question can be extraordinarily dif- ficult, due to decentralized storage, heterogeneous formats, and poor documentation. In this context, we introduce Plenario, a platform designed to automating time-consuming tasks associated with the discovery, exploration, and application of open city data—and, in so doing, reduce barriers to data use for researchers, policymakers, service providers, journalists, and members of the general public. Key innovations include a geospatial data warehouse that allows data from many sources to be registered into a common spatial and temporal frame; simple and intuitive interfaces that permit rapid discovery and exploration of data subsets pertaining to a particular area and time, regardless of type and source; easy export of such data subsets for further analysis; a user-configurable data ingest framework for automated importing and periodic updating of new datasets into the data warehouse; cloud hosting for elastic scaling and rapid creation of new Plenario instances; and an open source implementation to enable community contributions. We describe here the architecture and implementation of the Plenario platform, discuss lessons learned from its use by several communities, and outline plans for future work.

Catlett, Charlie↗

An open science exploration of global 1-km simulations of the earth's atmosphere

We are making available a multi-petabyte data set for open science via a data hackathon. The participants bring their science to the computing and data facility, alleviating the need to transfer and manage the large volume of data. This accelerates the process of discovery science while increasing scientific throughput.

Anantharaj, Valentine↗

Open Source Scalable Data Services and Data Fusion for Biological and Environmental Sciences (SBIR Phase I Final Scientific/ Technical Report)

The overarching goal of the project is to develop an integrated open-source scientific data management system (Apache V2 license), ResonantEco, that meets the need of biological and environmental researchers and developers for data management, curation, and data processing for analyses with a wide range of scale and complexity. ResonantEco will provide web enabled data services with features such as unified data interfaces and federated views of data and metadata for heterogeneous data sources with an interactive web client for data exploration. Our use of the term fusion is taken from geospatial (GIS) domain where data fusion is often synonymous with data integration. In particular, data integration in ResonantEco involves combining data residing in different sources and providing users with a unified view of them.

99 GENERAL AND MISCELLANEOUS↗

Data analysis and modeling pipelines for controlled networked social science experiments

There is large interest in networked social science experiments for understanding human behavior at-scale. Significant effort is required to perform data analytics on experimental outputs and for computational modeling of custom experiments. Moreover, experiments and modeling are often performed in a cycle, enabling iterative experimental refinement and data modeling to uncover interesting insights and to generate/refute hypotheses about social behaviors. The current practice for social analysts is to develop tailor-made computer programs and analytical scripts for experiments and modeling. This often leads to inefficiencies and duplication of effort. In this work, we propose a pipeline framework to take a significant step towards overcoming these challenges. Our contribution is to describe the design and implementation of a software system to automate many of the steps involved in analyzing social science experimental data, building models to capture the behavior of human subjects, and providing data to test hypotheses. The proposed pipeline framework consists of formal models, formal algorithms, and theoretical models as the basis for the design and implementation. We propose a formal data model, such that if an experiment can be described in terms of this model, then our pipeline software can be used to analyze data efficiently. The merits of the proposed pipeline framework is elaborated by several case studies of networked social science experiments.

97 MATHEMATICS AND COMPUTING↗

SITCOMTN-154: Initial studies of photometric redshifts with LSSTComCam from DP1

This technote holds reports based on the first analyses of the Data Preview 1 (DP1) data by the Science Unit for photometric redshifts. Although photometric redshifts are not an official DP1 data product, the "Photo-z Science Unit" generated photo-z estimates for every galaxy in DP1 using the available multi-band imaging on a best-effort basis. This work included developing training and test datasets by matching DP1 data to high-quality reference redshifts obtained with spectroscopy, Grism data, and multi-band photometry. The Science Unit used the RAIL software package to make photometric redshift estimates using eight different algorithms, developed simple scientific performance metrics, used those metrics to explore how the performance of the algorithms varied with configuration changes, derived more optimized configurations of the algorithms and tested the performance of those configurations. This work, the resulting data products and expected data distribution mechanism are all described there.

79 ASTRONOMY AND ASTROPHYSICS↗

Atmospheric PSF Interpolation for Weak Lensing in Short Exposure Imaging Data

A main science goal for the Large Synoptic Survey Telescope (LSST) is to measure the cosmic shear signal from weak lensing to extreme accuracy. One difficulty, however, is that with the short exposure time ({approx_equal}15 seconds) proposed, the spatial variation of the Point Spread Function (PSF) shapes may be dominated by the atmosphere, in addition to optics errors. While optics errors mainly cause the PSF to vary on angular scales similar or larger than a single CCD sensor, the atmosphere generates stochastic structures on a wide range of angular scales. It thus becomes a challenge to infer the multi-scale, complex atmospheric PSF patterns by interpolating the sparsely sampled stars in the field. In this paper we present a new method, psfent, for interpolating the PSF shape parameters, based on reconstructing underlying shape parameter maps with a multi-scale maximum entropy algorithm. We demonstrate, using images from the LSST Photon Simulator, the performance of our approach relative to a 5th-order polynomial fit (representing the current standard) and a simple boxcar smoothing technique. Quantitatively, psfent predicts more accurate PSF models in all scenarios and the residual PSF errors are spatially less correlated. This improvement in PSF interpolation leads to a factor of 3.5 lower systematic errors in the shear power spectrum on scales smaller than {approx} 13, compared to polynomial fitting. We estimate that with psfent and for stellar densities greater than {approx_equal}1/arcmin{sup 2}, the spurious shear correlation from PSF interpolation, after combining a complete 10-year dataset from LSST, is lower than the corresponding statistical uncertainties on the cosmic shear power spectrum, even under a conservative scenario.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

User Access to Scientific Facilities via 5G: A Cyber Security Thought Experiment

5G is more than an over-the-air radio technology upgrade. It is a strategy to extend Mobile Network Operator service offerings beyond traditional voice, instant messaging and Internet access. 5G Mobile Network Operators will offer new telecommunication services that include enhanced guarantees of confidentiality, integrity and availability. How could such services change the way Science collaborations connect scientists to supercomputers and other scientific facilities? Current scientific collaborations implicitly trust cloud service providers to securely store and process data. The perceived risks of outsourcing Science data security are counterbalanced by assurances that cloud providers operate at a scale that allows them to implement security measures impractical for Science collaborations (e.g. continuous system administrator behavioral monitoring and strict individual separation of duties). If that is true for a cloud service provider like Amazon Web Services (2018 revenue: $25.7 billion), could it also be true for Mobile Network Operators like Verizon Wireless (2018 revenue: $91.7 billion) or AT&T Mobility (2018 revenue: $71.3 billion)? DOE Leadership Class supercomputer facility users currently access them from the public Internet via Secure Shell. The sponsors and operators of the supercomputer facilities have determined that the public Internet path between the Scientist’s Device and the Login Node does not natively provide enough confidentiality or integrity to protect those communications. Therefore, the facilities achieve additional confidentiality and integrity by requiring Secure Shell encryption across those untrusted network paths. Using 5G Network Slice technology, a Mobile Network Operator may offer communication services between supercomputer users and facilities that natively provide confidentiality and integrity guarantees. Sponsors and operators of supercomputer facilities may determine that these guarantees provide enough confidentiality and integrity to protect those communications. If so, a 5G Network Slice could replace an SSH session running over the public Internet. Finally, this use case could be extended to other Office of Science user facility access requirements. Consider microscopy instruments at (e.g.) the Center for Nanoscale Materials or the Environmental Molecular Sciences Laboratory. The embedded systems controlling such instruments may not always support encrypted network access technologies like SSH. 5G Network Slices may offer an alternative to current VPN or SSH tunneling techniques, with additional benefits like guaranteed minimum bandwidth.

5G↗

Measuring the impact of burst buffers on data-intensive scientific workflows

Science applications frequently produce and consume large volumes of data, but delivering this data to and from compute resources can be concerning, as parallel file system performance is not keeping up with compute and memory performance. To alter this I/O bottleneck, some systems have deployed burst buffers, but their impact on performance for real-world scientific workflow applications is still not clear. In this paper, we examine the impact of burst buffers through the remote-shared, allocatable burst buffers on the Cori system at NERSC. By running two data-intensive workflows, a high-throughput genome analysis workflow, and a subset of the SCEC high-performance CyberShake workflow, a production seismic hazard analysis workflow, we find that using burst buffers offers read and write improvements of an order of magnitude, and these improvements lead to increased job performance, and thereby increased overall workflow performance, even for long-running CPU-bound jobs.

42 ENGINEERING↗

Hybrid Analysis of Fusion Data for Online Understanding of Complex Science on Extreme Scale Computers

The current practice for fusion scientists running first principle simulations on high performance computing plat-forms is to either run their simulations and output their data for post-hoc analysis, or to place in situ analytics into their code. In this paper we examine a complex workflow using XGC fusions simulation run on the Oak Ridge Leadership Computing Facility's supercomputer Summit, which also involve three anal-yses as part of the results necessary for scientific discovery. We discuss the challenges faced when implementing these algorithms and present an original hybrid staging technique to help enable the physicists to make discoveries during the execution of the simulation. By creating this infrastructure, we can examine complicated physics results, which may not have been possible without the infrastructure. For example, our work enables the online visualization of turbulent homoclinic tangle around the magnetic X-point, breaking the last confinement surface. This visualization could help fusion scientists to better understand and improve the turbulence spread of plasma exhaust heat, which is crucial toward realizing plasmas beyond the currently accessible physics regimes of present-day tokamak reactors. The physics of turbulent homoclinic tangle will be reported in a future physics publication, by utilizing the original online analysis/visualization framework presented in this paper.

Suchyta, Eric↗

When physics-informed data analytics outperforms black-box machine learning: A case study in thickness control for additive manufacturing

Aerosol jet printing (AJP) has emerged as a promising noncontact additive manufacturing method for high-resolution printing for a wide range of material systems. A key challenge limiting the broader adoption of AJP in the material science community is the lack of methods to precisely control thickness. Herein, we develop a model-based design of experiment (MBDoE) framework that integrates physics-informed models, nonlinear regression, and information criteria to postulate, select and calibrate the best model to describe and optimize the AJP manufacturing process. Starting with already available data from system commissioning (e.g., prior single variable sensitivity analysis), four candidate physics-informed models are postulated and trained. MBDoE identifies a single additional optimal experiment to validate these predictive models with quantified uncertainties, which are then used to determine the best experimental conditions to control printed film thickness. As a comparative benchmark, the analysis is repeated using the same dataset with nonparametric Gaussian process regression (GPR) model that does not incorporate physical information. Using MBDoE principles, we find that only five experiments are necessary to calibrate the nonlinear physics-informed parametric model, and with said limited data, this model outperforms the black-box machine learning GPR model. This key result underscores an emerging trend in the data science community: incorporating physical information into predictive models often drastically reduces the data requirements. Leveraging MBDoE further increased the data efficiency. By design, the proposed data science framework is general in nature and can be easily extended to other experimental and additive manufacturing systems beyond AJP.

Aerosol jet printing↗

Completeness and coverage of open-access freshwater fish distribution data in the United States

Aim: Primary biodiversity data constitute observations of particular species at given points in space and time. Open-access databases provide unprecedented access to these data, but their utility depends on how complete species inventories are at a given surveying resolution and how uniformly distributed surveys are in space and time. Our aim was to characterize the completeness of freshwater fish data across spatial scales and among habitat types (lotic stream reaches and lentic waterbodies), and assess coverage bias in space and time. Location: Freshwater habitats of the contiguous United States. Methods: We compiled fish occurrence records from the Global Biodiversity Information Facility (GBIF), the Multi-state Aquatic Resources Information Server (MARIS), and three federally-administered surveys (EMAP, NAQWA, NRS). Next, we evaluated survey completeness across seven spatial resolutions representing nested stream catchments, and separately for lotic and lentic habitats. Lastly, we tested for bias in survey coverage along environmental and temporal gradients for each habitat type. Results: A total of 39,855,157 records representing 1925 species were available from 1800 to 2014. The percent of catchments surveyed increased as spatial resolution decreased while the density of occurrence records within surveyed catchments decreased. Species richness also increased with catchment size consistent with putative species–area relationships. Occurrence records were distributed among 72,114 lotic reaches and 3,325 lentic waterbodies throughout the contiguous US. Seven percent (N=5,055) of surveyed lotic reaches and 23% (N=771) of surveyed lentic waterbodies were well-surveyed based on the most conservative thresholds for two completeness indices. Most well-surveyed lotic reaches (95%) and undammed lentic waterbodies (97%) were completed since 1980, and most were surveyed during only one 20-year time interval (83% and 56% for lotic and lentic waterbodies, respectively). Coverage analyses indicated that contemporary and future climates corresponding to regions along the west coast and gulf coast are the most under-represented for both lotic and lentic habitats. Small lentic waterbodies (surface area < 0.1 km2) are also under-represented. Main Conclusions: Contemporary surveys of lotic reaches and, to a lesser extent, lentic waterbodies are plentiful and generally representative of climatic and landscape characteristics. Data compilation efforts should focus on adding historical data to public repositories. Clustering of well-surveyed waterbodies in MARIS-participating states highlights the value of state-agency data and demonstrates the need for dissemination of these data by more state agencies.

60 APPLIED LIFE SCIENCES↗

BEAM: A Computational Workflow System for Managing and Modeling Material Characterization Data in HPC Environments

Improvements in scientific instrumentation allow imaging at mesoscopic to atomic length scales, many spectroscopic modes, and now—with the rise of multimodal acquisition systems and the associated processing capability—the era of multidimensional, informationally dense data sets has arrived. Technical issues in these combinatorial scientific fields are exacerbated by computational challenges best summarized as a necessity for drastic improvement in the capability to transfer, store, and analyze large volumes of data. The Bellerophon Environment for Analysis of Materials (BEAM) platform provides material scientists the capability to directly leverage the integrated computational and analytical power of High Performance Computing (HPC) to perform scalable data analysis and simulation via an intuitive, cross-platform client user interface. This framework delivers authenticated, “push-button” execution of complex user workflows that deploy data analysis algorithms and computational simulations utilizing the converged compute-and-data infrastructure at Oak Ridge National Laboratory's (ORNL) Compute and Data Environment for Science (CADES) and HPC environments like Titan at the Oak Ridge Leadership Computing Facility (OLCF). In this work we address the underlying HPC needs for characterization in the material science community, elaborate how BEAM's design and infrastructure tackle those needs, and present a small sub-set of user cases where scientists utilized BEAM across a broad range of analytical techniques and analysis modes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Interpretation of biomonitoring data in clinical medicine and the exposure sciences

Biomonitoring has become a fundamental tool in both exposure science and clinical medicine. Despite significant analytical advances, the clinical use of environmental biomarkers remains in its infancy. Clinical use of environmental biomarkers poses some complex scientific and ethical challenges. The purpose of this paper is compare how the clinical and exposure sciences differ with respect to their interpretation and use of biological data. Additionally, the clinical use of environmental biomonitoring data is discussed. A case study is used to illustrate the complexities of conducting biomonitoring research on highly vulnerable populations in a clinical setting.

60 APPLIED LIFE SCIENCES↗