Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “resource data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69

Building a Wide-Area File Transfer Performance Predictor: An Empirical Study

Wide-area data transfer is central to geographically distributed scientific workflows. Faster delivery of data is important for these workflows. Predictability is equally (or even more) important. With the goal of providing a reasonably accurate estimate of data transfer time to improve resource allocation & scheduling for workflows and enable end-to-end data transfer optimization, we apply machine learning methods to develop predictive models for data transfer times over a variety of wide area networks. To build and evaluate these models, we use 201,388 transfers, involving 759 million files totaling 9 PB transferred, over 115 heavily used source-destination pairs (“edges”) between 135 unique endpoints. We evaluate the models for different retraining frequencies and different window size of history data. In the best case, the resulting models have a median prediction error of ≤21% for 50% of the edges, and ≤32% for 75% of the edges. We present a detailed analysis of these results that provides insights into the cause of some of the high errors. We envision that the performance predictor will be informative for scheduling geo-distributed workflows. The insights also suggest obvious directions for both further analysis and transfer service optimization.

Liu, Zhengchun↗

Application of Reconfigurable Computing Technology to Multi-KiloHertz Micro-Laser Altimeter (MMLA) Data Processing

The Multi-KiloHertz Micro-Laser Altimeter (MMLA) is an aircraft based instrument developed by NASA Goddard Space Flight Center with several potential spaceflight applications. This presentation describes how reconfigurable computing technology was employed to perform MMLA signal extraction in real-time under realistic operating constraints. The MMLA is a "single-photon-counting" airborne laser altimeter that is used to measure land surface features such as topography and vegetation canopy height. This instrument has to date flown a number of times aboard the NASA P3 aircraft acquiring data at a number of sites in the Mid-Atlantic region. This instrument pulses a relatively low-powered laser at a very high rate (10 kHz) and then measures the time-of-flight of discrete returns from the target surface. The instrument then bins these measurements into a two-dimensional array (vertical height vs. horizontal ground track) and selects the most likely signal path through the array. Return data that does not correspond to the selected signal path are classified as noise returns and are then discarded. The MMLA signal extraction algorithm is very compute intensive in that a score must be computed for every possible path through the two dimensional array in order to select the most likely signal path. Given a typical array size with 50 x 6, up to 33 arrays must be processed per second. And for each of these arrays, roughly 12,000 individual paths must be scored. Furthermore, the number of paths increases exponentially with the horizontal size of the array, and linearly with the vertical size. Yet, increasing the horizontal and vertical sizes of the array offer science advantages such as improved range, resolution, and noise rejection. Due to the volume of return data and the compute intensive signal extraction algorithm, the existing PC-based MMLA data system has been unable to perform signal extraction in real-time unless the array is limited in size to one column, This limits the ability of the MMLA to operate in environments with sparse signal returns and a high number of noise return. However, under an IR&D project, an FPGA-based, reconfigurable computing data system has been developed that has been demonstrated to perform real-time signal extraction under realistic operating constraints. This reconfigurable data system is based on the commercially available Firebird Board from Annapolis Microsystems. This PCI board consists of a Xilinx Virtex 2000E FPGA along with 36 MB of SRAM arranged in five separately addressable banks. This board is housed in a rackmount PC with dual 850MHz Pentium processors running the Windows 2000 operating system. This data system performs all signal extraction in hardware on the Firebird, but also runs the existing "software based" signal extraction in tandem for comparison purposes. Using a relatively small amount of the Virtex XCV2000E resources, the reconfigurable data system has demonstrated to improve performance improvement over the existing software based data system by an order of magnitude. Performance could be further improved by employing parallelism. Ground testing and a preliminary engineering test flight aboard the NASA P3 has been performed, during which the reconfigurable data system has been demonstrated to match the results of the existing data system.

Powell, Wesley↗

North and South Dakota High Resolution 50m Wind Resource

Annual average wind resource potential for North and South Dakota at a 50 meter height in a GIS shapefile and links to US national wind resource information tools. This data set was produced and validated by NREL using their WRAM model.

50m↗

Moving from Descriptive to Causal Analytics: Case Study of the Health Indicators Warehouse

The KDD community has described a multitude of methods for knowledge discovery on large datasets. We consider some of these methods and integrate them into an analyst s workflow that proceeds from the data-centric descriptive level to the model-centric causal level. Examples of the workflow are shown for the Health Indicators Warehouse, which is a public database for community health information that is a potent resource for conducting data science on a medium scale. We demonstrate the potential of HIW as a source of serious visual analytics efforts by showing correlation matrix visualizations, multivariate outlier analysis, multiple linear regression of Medicare costs, and scatterplot matrices for a broad set of health indicators. We conclude by sketching the first steps toward a causal dependence hypothesis.

Schryver, Jack C.↗

Skylab EREP Investigations Summary

The problems in the areas of agriculture, range and forestry; land use and cartography; geology and hydrology; oceans atmosphere, and data analysis techniques were investigated and summarized using Earth Resources Experiment Package (EREP) data.

Pierson, W. J.↗

Global Surface Solar Energy Anomalies Including El Nino and La Nina Years

This paper synthesizes past events in an attempt to define the general magnitude, duration, and location of large surface solar anomalies over the globe. Surface solar energy values are mostly a function of solar zenith angle, cloud conditions, column atmospheric water vapor, aerosols, and surface albedo. For this study, solar and meteorological parameters for the 10-yr period July 1983 through June 1993 are used. These data were generated as part of the Release 3 Surface meteorology and Solar Energy (SSE) activity under the NASA Earth Science Enterprise (ESE) effort. Release 3 SSE uses upgraded input data and methods relative to previous releases. Cloud conditions are based on recent NASA Version-D International Satellite Cloud Climatology Project (ISCCP) global satellite radiation and cloud data. Meteorological inputs are from Version-I Goddard Earth Observing System (GEOS) reanalysis data that uses both weather station and satellite information. Aerosol transmission for different regions and seasons are for an 'average' year based on historic solar energy data from over 1000 ground sites courtesy of Natural Resources Canada (NRCan). These data are input to a new Langley Parameterized Shortwave Algorithm (LPSA) that calculates surface albedo and surface solar energy. That algorithm is an upgraded version of the 'Staylor' algorithm. Calculations are performed for a 280X280 km equal-area grid system over the globe based on 3-hourly input data. A bi-linear interpolation process is used to estimate data output values on a 1 X 1 degree grid system over the globe. Maximum anomalies are examined relative to El Nino and La Nina events in the tropical Pacific Ocean. Maximum year-to-year anomalies over the globe are provided for a 10-year period. The data may assist in the design of systems with increased reliability. It may also allow for better planning for emergency assistance during some atypical events.

Whitlock, C. H.↗

The Virtual Space Physics Observatory: Quick Access to Data and Tools

The Virtual Space Physics Observatory (VSPO; see http://vspo.gsfc.nasa.gov) has grown to provide a way to find and access about 375 data products and services from over 100 spacecraft/observatories in space and solar physics. The datasets are mainly chosen to be the most requested, and include most of the publicly available data products from operating NASA Heliophysics spacecraft as well as from solar observatories measuring across the frequency spectrum. Service links include a "quick orbits" page that uses SSCWeb Web Services to provide a rapid answer to questions such as "What spacecraft were in orbit in July 1992?" and "Where were Geotail, Cluster, and Polar on 2 June 2001?" These queries are linked back to the data search page. The VSPO interface provides many ways of looking for data based on terms used in a registry of resources using the SPASE Data Model that will be the standard for Heliophysics Virtual Observatories. VSPO itself is accessible via an API that allows other applications to use it as a Web Service; this has been implemented in one instance using the ViSBARD visualization program. The VSPO will become part of the Space Physics Data Facility, and will continue to expand its access to data. A challenge for all VOs will be to provide uniform access to data at the variable level, and we will be addressing this question in a number of ways.

Cornwell, Carl↗

Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Earth Observation data are a vital resource for studying long term changes, but the large data volumes can be challenging to analyze. Time series analysis in particular is hampered by the typical thin-time-slice file organization. We examine several potential solutions inspired in large part by the data-parallel methods that have arisen with cloud computing. These solutions include various combinations of data re-organization, spatial indexing, distributed storage and pre-computation that we term "Analytics Optimized Data Stores" (AODS). We find that even simple solutions (such as a data cube) produce more than an order of magnitude improvement; the best provide two to three orders of magnitude improvement. The most performant solutions have tradeoffs in terms of generality or storage footprint, but may nonetheless be useful components in data analytics frameworks where performance is critical.

parallel processing (computers)↗

Nimbus earth resources observations

The potential for utilizing data gathered by Nimbus satellites to study the earth surface and its physical properties is illustrated. The Nimbus data applicable to investigations of the earth and its resources, and to the problems of resolution and cloud cover are described. Geological, hydrological, and oceanographic applications are discussed. Applications of the data to other fields, such as cartography, agriculture, forestry, and urban analysis are presented. Relevant information is also given on the Nimbus orbit and experiments; surface and atmospheric effects on HRIR and THIR radiation measurements; and noise problems in the AVCS, IDCS, HRIR, and THIR data.

Sabatini, R. R.↗

A Multi-scale, Geo-data Science Method for Assessing Unconventional Critical Mineral Resources

Critical minerals (CM) supply raw materials that constitute many of our essential infrastructure, defense, technology, and electrification needs. Currently, production and refinement of these materials from conventional sources is limited to few regions globally, which makes supply of these resources particularly venerable to disruption. To help overcome these risks and meet growing demand, attention has focused on identifying and developing resource potential of unconventional geologic CM sources, such as rare-earth elements in sedimentary systems. However, the unconventional nature of these types of sources means they are often poorly characterized and/or under-explored with respect to conventional counterparts. We present a regional case study for an Unconventional Rare-earth and Critical minerals (URC) assessment method for predicting and identifying REE resource potential and occurrence in unconventional systems in the Central Appalachian Basin (CAB). The method utilizes a geologic and geospatial data-driven approach, informed and guided by knowledge of REE enrichment processes, to systematically predict and identify areas of higher enrichment. Results from the test case indicate locations with potential for different types of coal-REE deposits, demonstrating its utility for reducing the area of exploration and identifying sites for more detailed investigation. Building upon the regional scale assessment capability, ongoing science-based enhancements to the method will allow for finer-scale (e.g., mine-scale) predictions required to support technical and economic assessments.

Creason, Christopher↗

An Integrated Platform for Mission Performance: Goddard Space Flight Center's Meta Information System

Successful management of NASA Goddard Space Flight Centers portfolio of mission projects (i.e. spacecraft, spacecraft instruments / subsystems, and ground systems) depends upon numerous processes, data and information resources coming together throughout the life-cycle of the projects to enable sound decision-making, risk reduction, conformance with requirements and continual improvement. In the summer of 2012, GSFC SMA management recognized that the set of legacy data systems used to support the SMA mission and responsibilities at GSFC were not effective, integrated nor on par with best practices. Accordingly, GSFC SMA management launched a project to transform the GSFC electronic management system to create and deploy an integrated, state-of-the-market information system for process performance, data management and analytics that is currently operating in support of GSFC overall mission performance. The information system, known as Meta, provides controlled, process-based applications, integrated data management, and analytics for a robust set of integrated applications. Since the inception of the project, a lean, award-winning SMA project team has thus far designed, configured and managed the system to serve as an integrated platform for 18 applications that support key processes for GSFC and the Science Mission Directorate at NASA headquarters, such as internal Management System assessments, Supplier Assessments, Supplier Research & Analysis, Problem Reporting for in-house mission projects, On-orbit Anomaly Reporting for GSFC-operated spacecraft, GSFC Project Life-Cycle Reviews, Risk Management and Reporting, and SMA Project Team Reporting. The system has experienced steady growth in utilization with 1,917 users (NASA employees and support contractor personnel) and 44 gigabytes of data and related files managed by the system as of the end of April 2018. Overall, Meta has achieved increasing levels of process management and real-time data / information integration to enable improved process performance, richer analytics and informed decision-making. This paper will present and examine the need, goals, approach and design driving the management and operation of the Meta information system and its ongoing growth as an integrated platform for advancing NASA mission performance.

Integration↗

NASA's Fire Information for Resource Management System (FIRMS): Near Real-Time Global Fire Monitoring Using Data from MODIS and VIIRS

NASA's Fire Information for Resource Management System (FIRMS) provides near-real time active fire / hotspot products from MODIS and VIIRS to users in over 160 countries. The goal of FIRMS is to meet the needs of natural resource and protected area managers that face considerable challenges in obtaining timely satellite-derived information on fires burning within and around their management area. FIRMS has been reliably providing active fire / hotspot data in easy to use formats since its inception in 2006. Fire information is provided through a web map interface, email alerts, a web mapping service, and a range of downloadable files (SHP, CSV, KML and JSON). FIRMS data are used directly by end users and by brokers who take the data and add value to it before re-distributing it.FIRMS was initially developed by the University of Maryland in 2006; it was funded by the United Nations FAO and NASA's Applied sciences program under a NASA ROSES call. In 2012 FIRMS became integrated in to NASA's Land Atmosphere Near real-time Capability for EOS (LANCE); a virtual system that leverages NASAs existing science processing capabilities to deliver NRT data from ten instruments within 3 hours of satellite overpass. This presentation will describe the FIRMS system, provide an overview of the system, briefly describe some of the known applications and describe plans to further integrate the data in to NASA's Global Imagery Browse Services (GIBS) public mapping services and Worldview website.

active fire/hotspots↗

Scientific Synergy between LSST and Euclid

We report that Euclid and the Large Synoptic Survey Telescope (LSST) are poised to dramatically change the astronomy landscape early in the next decade. The combination of high-cadence, deep, wide-field optical photometry from LSST with high-resolution, wide-field optical photometry, and near-infrared photometry and spectroscopy from Euclid will be powerful for addressing a wide range of astrophysical questions. We explore Euclid/LSST synergy, ignoring the political issues associated with data access to focus on the scientific, technical, and financial benefits of coordination. We focus primarily on dark energy cosmology, but also discuss galaxy evolution, transient objects, solar system science, and galaxy cluster studies. We concentrate on synergies that require coordination in cadence or survey overlap, or would benefit from pixel-level co-processing that is beyond the scope of what is currently planned, rather than scientific programs that could be accomplished only at the catalog level without coordination in data processing or survey strategies. Also, we provide two quantitative examples of scientific synergies: the decrease in photo-z errors (benefiting many science cases) when high-resolution Euclid data are used for LSST photo-z determination, and the resulting increase in weak-lensing signal-to-noise ratio from smaller photo-z errors. We briefly discuss other areas of coordination, including high-performance computing resources and calibration data. Finally, we address concerns about the loss of independence and potential cross-checks between the two missions and the potential consequences of not collaborating.

79 ASTRONOMY AND ASTROPHYSICS↗

Future SDN-HPON Control Plane Architecture and Protocol for On-Demand Terabit End-to-End Extreme-Scale Science Applications

The UC Davis team conducted architecture, algorithm and experimental studies for next generation ultra-high-bandwidth optical networks in support of extreme-scale science applications. In particular, over the course of this 3-year project, the UC Davis team achieved the following main accomplishments: Designed a network architecture for automated multi-domain service provisioning: Designed information exchange schemes between autonomous systems (domains) Designed application programming interfaces to orchestrate resource reservation in data centers, high performance computation facilities, and communication networks belonging to different autonomous systems Experimentally assessed the multi-domain network architecture in a distributed field trial set-up connecting premises in three continents Designed a broker-based orchestrated multi-domain optical networking framework supporting observe-analyze-act cycle based cognitive inter-domain service provisioning: Designed an inter-domain operation mechanism, e.g., topology abstraction, for the broker plane and domain managers to collaboratively accomplish inter-domain service provisioning while respecting the autonomy of each administrative domain Designed an alien wavelength monitoring scheme and an artificial neural network based optical signal-to-noise ratio estimator for quality-of-transmission aware inter-domain service provisioning Developed a deep neural network based multi-domain traffic predictor and a cognitive routing and spectrum assignment algorithm to realize knowledge-based autonomous traffic engineering in multi-domain networks Implemented an optical signal-to-noise ratio estimator and traffic predictor with Google’s Tensor flow and assessed their performance using experimental and synthetic data Demonstrated the benefits of the proposed cognitive schemes with experiments ona two-domain elastic optical network testbed and evaluations on a six-domain simulation framework Built a two-domain software-defined networking control plane testbed using the ONOS platform and demonstrated the workflow of broker and domain manager cooperation, including hierarchical performance monitoring, topology abstraction and cognitive resource allocation. Designed self-learning cognitive service provisioning schemes using advanced machine learning technologies: Designed a self-taught anomaly detection framework for optical networks, including an unsupervised data clustering module for pattern recognition, a supervised data regression and classification module for online anomaly detection, and the system operation principle Experimentally validated the effectiveness of the self-taught anomaly detection framework with various failure/anomaly scenarios. Designed a self-learning autonomic routing, modulation and spectrum assignment (RMSA) provisioning framework for elastic optical networks with deep reinforcement learning. Designed effective modeling and training mechanisms for the self-learning RMSA framework Conducted extensive numerical simulations with various traffic patterns and topologies to verify the advantage of the self-learning RMSA framework.

42 ENGINEERING↗

ECO: the Evidence and Conclusion Ontology, an update for 2022

The Evidence and Conclusion Ontology (ECO) is a community resource that provides an ontology of terms used to capture the type of evidence that supports biomedical annotations and assertions. Consistent capture of evidence information with ECO allows tracking of annotation provenance, establishment of quality control measures, and evidence-based data mining. ECO is in use by dozens of data repositories and resources with both specific and general areas of focus. ECO is continually being expanded and enhanced in response to user requests as well as our aim to adhere to community best-practices for ontology development. The ECO support team engages in multiple collaborations with other ontologies and annotating groups. Here we report on recent updates to the ECO ontology itself as well as associated resources that are available through this project. ECO project products are freely available for download from the project website (https://evidenceontology.org/) and GitHub (https://github.com/evidenceontology/evidenceontology). ECO is released into the public domain under a CC0 1.0 Universal license.

59 BASIC BIOLOGICAL SCIENCES↗

MERRA Analytic Services: Meeting the Big Data Challenges of Climate Science Through Cloud-enabled Climate Analytics-as-a-service

Climate science is a Big Data domain that is experiencing unprecedented growth. In our efforts to address the Big Data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). We focus on analytics, because it is the knowledge gained from our interactions with Big Data that ultimately produce societal benefits. We focus on CAaaS because we believe it provides a useful way of thinking about the problem: a specialization of the concept of business process-as-a-service, which is an evolving extension of IaaS, PaaS, and SaaS enabled by Cloud Computing. Within this framework, Cloud Computing plays an important role; however, we it see it as only one element in a constellation of capabilities that are essential to delivering climate analytics as a service. These elements are essential because in the aggregate they lead to generativity, a capacity for self-assembly that we feel is the key to solving many of the Big Data challenges in this domain. MERRA Analytic Services (MERRAAS) is an example of cloud-enabled CAaaS built on this principle. MERRAAS enables MapReduce analytics over NASAs Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of 26 key climate variables. It represents a type of data product that is of growing importance to scientists doing climate change research and a wide range of decision support applications. MERRAAS brings together the following generative elements in a full, end-to-end demonstration of CAaaS capabilities: (1) high-performance, data proximal analytics, (2) scalable data management, (3) software appliance virtualization, (4) adaptive analytics, and (5) a domain-harmonized API. The effectiveness of MERRAAS has been demonstrated in several applications. In our experience, Cloud Computing lowers the barriers and risk to organizational change, fosters innovation and experimentation, facilitates technology transfer, and provides the agility required to meet our customers' increasing and changing needs. Cloud Computing is providing a new tier in the data services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. For climate science, Cloud Computing's capacity to engage communities in the construction of new capabilies is perhaps the most important link between Cloud Computing and Big Data.

Data Analytics↗

Plant single-cell solutions for energy and the environment

Progress in sequencing, microfluidics, and analysis strategies has revolutionized the granularity at which multicellular organisms can be studied. In particular, single-cell transcriptomics has led to fundamental new insights into animal biology, such as the discovery of new cell types and cell type-specific disease processes. However, the application of single-cell approaches to plants, fungi, algae, or bacteria (environmental organisms) has been far more limited, largely due to the challenges posed by polysaccharide walls surrounding these species’ cells. In this perspective, we discuss opportunities afforded by single-cell technologies for energy and environmental science and grand challenges that must be tackled to apply these approaches to plants, fungi and algae. We highlight the need to develop better and more comprehensive single-cell technologies, analysis and visualization tools, and tissue preparation methods. We advocate for the creation of a centralized, open-access database to house plant single-cell data. Finally, we consider how such efforts should balance the need for deep characterization of select model species while still capturing the diversity in the plant kingdom. Investments into the development of methods, their application to relevant species, and the creation of resources to support data dissemination will enable groundbreaking insights to propel energy and environmental science forward.

59 BASIC BIOLOGICAL SCIENCES↗