SEARCH · Engineering Papers

Results for “Data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

All sources NASA NTRS DOE OSTI

NASA NTRS2006

Cooperative Collision Avoidance Step 1 - Technology Demonstration Flight Test Report. Revision 1

The National Aeronautics and Space Administration (NASA) Access 5 Project Office sponsored a cooperative collision avoidance flight demonstration program for unmanned aircraft systems (UAS). This flight test was accomplished between September 21st and September 27th 2005 from the Mojave Airport, Mojave, California. The objective of these flights was to collect data for the Access 5 Cooperative Collision Avoidance (CCA) Work Package simulation effort, i.e., to gather data under select conditions to allow validation of the CCA simulation. Subsequent simulation to be verified were: Demonstrate the ability to detect cooperative traffic and provide situational awareness to the ROA pilot; Demonstrate the ability to track the detected cooperative traffic and provide position information to the ROA pilot; Demonstrate the ability to determine collision potential with detected cooperative traffic and provide notification to the ROA pilot; Demonstrate that the CCA subsystem provides information in sufficient time for the ROA pilot to initiate an evasive maneuver to avoid collision; Demonstrate an evasive maneuver that avoids collision with the threat aircraft; and lastly, Demonstrate the ability to assess the adequacy of the maneuver and determine that the collision potential has been avoided. The Scaled Composites, LLC Proteus Optionally Piloted Vehicle (OPV) was chosen as the test platform. Proteus was manned by two on-board pilots but was also capable of being controlled from an Air Vehicle Control Station (AVCS) located on the ground. For this demonstration, Proteus was equipped with cooperative collision sensors and the required hardware and software to place the data on the downlink. Prior to the flight phase, a detailed set of flight test scenarios were developed to address the flight test objectives. Two cooperative collision avoidance sensors were utilized for detecting aircraft in the evaluation: Traffic Alert and Collision Avoidance System-II (TCAS-II) and Automatic Dependent Surveillance Broadcast (ADS-B). A single intruder aircraft was used during all the flight testing, a NASA Gulfstream III (G-III). During the course of the testing, six geometrically different near-collision scenarios were evaluated. These six scenarios were each tested using various combinations of sensors and collision avoidance software. Of the 54 planned test points 49 were accomplished successfully. Proteus flew a total of 21.5 hours during the testing and the G-III flew 19.8 hours. The testing fully achieved all flight test objectives. The Flight IPT performed an analysis to determine the accuracy of the simulation model used to predict the location of the host aircraft downstream during an avoidance maneuver. The data collected by this flight program was delivered to the Access 5 Cooperative Collision Avoidance (CCA) Work Package Team who was responsible for reporting on their analysis of this flight data.

Trongale, Nicholas A.↗

NASA NTRS2014

Data Albums: An Event Driven Search, Aggregation and Curation Tool for Earth Science

One of the largest continuing challenges in any Earth science investigation is the discovery and access of useful science content from the increasingly large volumes of Earth science data and related information available. Approaches used in Earth science research such as case study analysis and climatology studies involve gathering discovering and gathering diverse data sets and information to support the research goals. Research based on case studies involves a detailed description of specific weather events using data from different sources, to characterize physical processes in play for a specific event. Climatology-based research tends to focus on the representativeness of a given event, by studying the characteristics and distribution of a large number of events. This allows researchers to generalize characteristics such as spatio-temporal distribution, intensity, annual cycle, duration, etc. To gather relevant data and information for case studies and climatology analysis is both tedious and time consuming. Current Earth science data systems are designed with the assumption that researchers access data primarily by instrument or geophysical parameter. Those who know exactly the datasets of interest can obtain the specific files they need using these systems. However, in cases where researchers are interested in studying a significant event, they have to manually assemble a variety of datasets relevant to it by searching the different distributed data systems. In these cases, a search process needs to be organized around the event rather than observing instruments. In addition, the existing data systems assume users have sufficient knowledge regarding the domain vocabulary to be able to effectively utilize their catalogs. These systems do not support new or interdisciplinary researchers who may be unfamiliar with the domain terminology. This paper presents a specialized search, aggregation and curation tool for Earth science to address these existing challenges. The search tool automatically creates curated "Data Albums", aggregated collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; and information about the event contained in news reports, images or videos to supplement research analysis. Curation in the tool is driven via an ontology based relevancy ranking algorithm to filter out non-relevant information and data.

Ramachandran, Rahul↗

DOE OSTI2021

Non-linear hydrologic organization

We revisit three variants of the well-known Stommel diagrams that have been used to summarize knowledge of characteristic scales in time and space of some important hydrologic phenomena and modified these diagrams focusing on spatiotemporal scaling analyses of the underlying hydrologic processes. In the present paper we focus on soil formation, vegetation growth, and drainage network organization. We use existing scaling relationships for vegetation growth and soil formation, both of which refer to the same fundamental length and timescales defining flow rates at the pore scale but different powers of the power law relating time and space. The principle of a hierarchical organization of optimal subsurface flow paths could underlie both root lateral spread (RLS) of vegetation and drainage basin organization. To assess the applicability of scaling, and to extend the Stommel diagrams, data for soil depth, vegetation root lateral spread, and drainage basin length have been accessed. The new data considered here include timescales out to 150 Myr that correspond to depths of up to 240 m and horizontal length scales up to 6400 km and probe the limits of drainage basin development in time, depth, and horizontal extent.

58 GEOSCIENCES↗

NASA NTRS1983

Pilot climate data system: A state-of-the-art capability in scientific data management

The Pilot Climate Data System (PCDS) was developed by the Information Management Branch of NASA's Goddard Space Flight Center to manage a large collection of climate-related data of interest to the research community. The PCDS now provides uniform data catalogs, inventories, access methods, graphical displays and statistical calculations for selected NASA and non-NASA data sets. Data manipulation capabilities were developed to permit researchers to easily combine or compare data. The current capabilities of the PCDS include many tools for the statistical survey of climate data. A climate researcher can examine any data set of interest via flexible utilities to create a variety of two- and three-dimensional displays, including vector plots, scatter diagrams, histograms, contour plots, surface diagrams and pseudo-color images. The graphics and statistics subsystems employ an intermediate data storage format which is data-set independent. Outside of the graphics system there exist other utilities to select, filter, list, compress, and calculate time-averages and variances for any data of interest. The PCDS now fully supports approximately twenty different data sets and is being used on a trial basis by several different in-house research grounds.

Smith, P. H.↗

NASA NTRS1991

Monte Carlo Shower Counter Studies

Activities and accomplishments related to the Monte Carlo shower counter studies are summarized. A tape of the VMS version of the GEANT software was obtained and installed on the central computer at Gallaudet University. Due to difficulties encountered in updating this VMS version, a decision was made to switch to the UNIX version of the package. This version was installed and used to generate the set of data files currently accessed by various analysis programs. The GEANT software was used to write files of data for positron and proton showers. Showers were simulated for a detector consisting of 50 alternating layers of lead and scintillator. Each file consisted of 1000 events at each of the following energies: 0.1, 0.5, 2.0, 10, 44, and 200 GeV. Data analysis activities related to clustering, chi square, and likelihood analyses are summarized. Source code for the GEANT user subprograms and data analysis programs are provided along with example data plots.

Snyder, H. David↗

NASA NTRS1992

The NSO FTS database program and archive (FTSDBM)

Data from the NSO Fourier transform spectrometer is being re-archived from half inch tape onto write-once compact disk. In the process, information about each spectrum and a low resolution copy of each spectrum is being saved into an on-line database. FTSDBM is a simple database management program in the NSO external package for IRAF. A command language allows the FTSDBM user to add entries to the database, delete entries, select subsets from the database based on keyword values including ranges of values, create new database files based on these subsets, make keyword lists, examine low resolution spectra graphically, and make disk number/file number lists. Once the archive is complete, FTSDBM will allow the database to be efficiently searched for data of interest to the user and the compact disk format will allow random access to that data.

Lytle, D. M.↗

NASA NTRS2013

A Lightweight I/O Scheme to Facilitate Spatial and Temporal Queries of Scientific Data Analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data post-processing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Temporal Queries↗

NASA NTRS2020

Constraining the Twomey effect from satellite observations: Issues and perspectives

The Twomey effect describes the radiative forcing associated with a change in cloud albedo due to an increase in anthropogenic aerosol emissions. It is driven by the perturbation in cloud droplet number concentration (ΔNd,ant) in liquid-water clouds and is currently understood to exert a cooling effect on climate. The Twomey effect is the key driver in the effective radiative forcing due to aerosol–cloud interactions which also comprises rapid adjustments. These adjustments are essentially the responses of cloud fraction and liquid water path to ΔNd,ant and thus scale approximately with it. While the fundamental physics of the influence of added aerosol particles on the droplet concentration (Nd) is well described by established theory at the particle scale (micrometres), how this relationship is expressed at the large scale (hundreds of kilometres) ΔNd,ant remains uncertain. The discrepancy between process understanding at particle scale and insufficient quantification at the climate-relevant large scale is caused by co-variability of aerosol particles and vertical wind and by droplet sink processes. These operate at scales on the order of 10s of metres at which only localized observations are available and at which no approach exists yet to quantify the anthropogenic perturbation. Different atmospheric models suggest diverse magnitudes of the Twomey effect even when applying the same anthropogenic aerosol emission perturbation. Thus, observational data are needed to quantify and constrain the Twomey effect. At the global scale, this means satellite data. There are three key uncertainties in determining ΔNd,ant, namely the quantification (i) of the cloud-active aerosol – the cloud condensation nuclei concentrations (CCN) at or above cloud base –, (ii) of Nd, as well as (iii) the statistical approach for inferring the sensitivity of Nd to aerosol particles from the satellite data. A fourth uncertainty, the anthropogenic perturbation to CCN concentrations, is also not easily accessible from observational data. This review discusses deficiencies of current approaches for the different aspects of the problem and proposes several ways forward: In terms of CCN, retrievals of optical quantities such as aerosol optical depth suffer from a lack of vertical resolution, size and hygroscopicity information, the non-direct relation to the concentration of aerosols, the impossibility to quantify it within or below clouds, and the problem of insufficient sensitivity at low concentrations, in addition to retrieval errors. A future path forward can include utilizing colocated polarimeter and lidar instruments, ideally including high spectral resolution lidar capability at two wavelengths to maximize vertically resolved size distribution information content. In terms of Nd, a key problem is the lack of operational retrievals of this quantity, and the inaccuracy of the retrieval especially in broken-cloud regimes. As for the Nd – to – CCN sensitivity, key issues are the updraught distributions and the role of Nd sink processes, for which empirical assessments for specific cloud regimes are currently the best solutions. These considerations point to the conclusion that past studies using existing approaches have likely underestimated the true sensitivity and, thus, the radiative forcing due to the Twomey effect.

Twomey effect↗

NASA NTRSDate not supplied

Simplifying Analysis of Hierarchical HDF5 and NetCDF4 Files with Xarray-Datatree

NASA’s Earth Observing System Data and Information System (EOSDIS) contains thousands of Earth science datasets from satellites, models, and field campaigns. EOSDIS data are stored in formats that are well supported by the Earth Science community. These formats include the Hierarchical Data Format (HDF), with derivative flavors such as HDF-5 and the Network Common Data Format (NetCDF-4). The HDF specification allows for a directory-like hierarchy within a single file, known as "groups". Observational data and associated metadata within a single file can be distributed amongst multiple internal groups, which can also be nested to multiple levels. Working with datasets that have a group hierarchical structure can be difficult because of the nested structure of groups. Widely used packages, such as xarray, have data models that do not accommodate the hierarchical structure within HDF files, requiring users to traverse the file and open different HDF groups as separate, unrelated objects. Xarray-datatree is a Python package developed to solve the difficulty of traversing HDFs with a hierarchical group structure by creating a tree-like hierarchical data structure in xarray. The tree-like structure allows each group to be accessed once a DataTree object is instantiated. The migration of xarray-datatree into the xarray core library will reduce barriers to accessing Earth science data by eliminating the need to understand and traverse the specific hierarchy of a grouped HDF file.

Eni Awowale↗

NASA NTRS2009

Effective Materials Property Information Management for the 21st Century

This paper discusses key principles for the development of materials property information management software systems. There are growing needs for automated materials information management in various organizations. In part these are fueled by the demands for higher efficiency in material testing, product design and engineering analysis. But equally important, organizations are being driven by the need for consistency, quality and traceability of data, as well as control of access to sensitive information such as proprietary data. Further, the use of increasingly sophisticated nonlinear, anisotropic and multi-scale engineering analyses requires both processing of large volumes of test data for development of constitutive models and complex materials data input for Computer-Aided Engineering (CAE) software. And finally, the globalization of economy often generates great needs for sharing a single "gold source" of materials information between members of global engineering teams in extended supply chains. Fortunately, material property management systems have kept pace with the growing user demands and evolved to versatile data management systems that can be customized to specific user needs. The more sophisticated of these provide facilities for: (i) data management functions such as access, version, and quality controls; (ii) a wide range of data import, export and analysis capabilities; (iii) data "pedigree" traceability mechanisms; (iv) data searching, reporting and viewing tools; and (v) access to the information via a wide range of interfaces. In this paper the important requirements for advanced material data management systems, future challenges and opportunities such as automated error checking, data quality characterization, identification of gaps in datasets, as well as functionalities and business models to fuel database growth and maintenance are discussed.

Ren, Weiju↗

NASA NTRS1994

Design document for the MOODS Data Management System (MDMS), version 1.0

The MOODS Data Management System (MDMS) provides access to the Master Oceanographic Observation Data Set (MOODS) which is maintained by the Naval Oceanographic Office (NAVOCEANO). The MDMS incorporates database technology in providing seamless access to parameter (temperature, salinity, soundspeed) vs. depth observational profile data. The MDMS is an interactive software application with a graphical user interface (GUI) that supports user control of MDMS functional capabilities. The purpose of this document is to define and describe the structural framework and logical design of the software components/units which are integrated into the major computer software configuration item (CSCI) identified as MDMS, Version 1.0. The preliminary design is based on functional specifications and requirements identified in the governing Statement of Work prepared by the Naval Oceanographic Office (NAVOCEANO) and distributed as a request for proposal by the National Aeronautics and Space Administration (NASA).

Source record↗

DOE OSTI2023

Last-level collective hardware prefetching

A last-level collective hardware prefetcher (LLCHP) is described. The LLCHP is to detect a first off-chip memory access request by a first processor core of a plurality of processor cores. The LLCHP is further to determine, based on the first off-chip memory access request, that first data associated with the first off-chip memory access request is associated with second data of a second processor core of the plurality of processor cores. The LLCHP is further to prefetch the first data and the second data based on the determination.

Michelogiannakis, Georgios↗

DOE OSTI2021

HAM: Hotspot-Aware Manager for Improving Communications with 3D-Stacked Memory

merging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this paper, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. %and an associated hotspot-aware page policy. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51\% and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81\% in average (up to 34.28\%) and power savings of 35.07\% over a standard 3D-stacked memory.

Wang, Xi↗

NASA NTRS2017

Satellite Level 3 & 4 Data Subsetting at NASA GES DISC

Earth Science data are available in many file formats (NetCDF, HDF, GRB, etc.) and in a wide range of sizes, from kilobytes to gigabytes. These properties have become a challenge to users if they are not familiar with these formats or only want a small region of interest (ROI) from a specific dataset. At NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we have developed and implemented a multipurpose subset service to ease user access to Earth Science data. Our Level 3 & 4 Regridder is capable of subsetting across multiple parameters (spatially, temporally, by level, and by variable) as well as having additional beneficial features (temporal means, regridding to target grids, and file conversion to other data formats). In this presentation, we will demonstrate how users can use this service to better access only the data they need in the form they require.

subsetting↗

DOE OSTI2023

A general spatial-temporal framework for short-term building temperature forecasting at arbitrary locations with crowdsourcing weather data

Weather forecasting has been a critical component to predict and control building energy consumption for better building energy management. Without accessibility to other data sources, the onsite observed temperatures or the airport temperatures are used in forecast models. In this paper, we present a novel approach by utilizing the crowdsourcing weather data from neighboring personal weather stations (PWS) to improve the weather forecast accuracy around buildings using a general spatial-temporal modeling framework. The final forecast is based on the ensemble of local forecasts for the target location using neighboring PWSs. Our approach is distinguished from existing literature in various aspects. First, we leverage the crowdsourcing weather data from PWS in addition to public data sources. In this way, the data is at much finer time resolution (e.g., at 5-minute frequency) and spatial resolution (e.g., arbitrary location vs grid). Second, our proposed model incorporates spatial-temporal correlation information of weather variables between the target building and a set of neighboring PWSs so that underlying correlations can be effectively captured to improve forecasting performance. Here, we demonstrate the performance of the proposed framework by comparing to the benchmark models on temperature forecasting for a building located at an arbitrary location at San Antonio, Texas, USA. In general, the proposed model framework equipped with machine learning technique such as Random Forest can improve forecasting by 50% compares with persistent model and has 90% chance to outperform airport forecast in short-term forecasting. In a real-time setting, the proposed model framework can provide more accurate temperature forecasting results compared with using airport temperature forecast for most forecast horizon. Moreover, we analyze the sensitivity of model parameters to gain insights on how crowdsourcing data from the neighboring personal weather stations impacts forecasting performance. Finally, we implement our model in other cities such as Syracuse and Chicago to test the model's performance in different landforms and climate types.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Open Data and Deep Semantic Segmentation for Automated Extraction of Building Footprints

Advances in machine learning and computer vision, combined with increased access to unstructured data (e.g., images and text), have created an opportunity for automated extraction of building characteristics, cost-effectively, and at scale. These characteristics are relevant to a variety of urban and energy applications, yet are time consuming and costly to acquire with today’s manual methods. Several recent research studies have shown that in comparison to more traditional methods that are based on features engineering approach, an end-to-end learning approach based on deep learning algorithms significantly improved the accuracy of automatic building footprint extraction from remote sensing images. However, these studies used limited benchmark datasets that have been carefully curated and labeled. How the accuracy of these deep learning-based approach holds when using less curated training data has not received enough attention. The aim of this work is to leverage the openly available data to automatically generate a larger training dataset with more variability in term of regions and type of cities, which can be used to build more accurate deep learning models. In contrast to most benchmark datasets, the gathered data have not been manually curated. Thus, the training dataset is not perfectly clean in terms of remote sensing images exactly matching the ground truth building’s foot-print. A workflow that includes data pre-processing, deep learning semantic segmentation modeling, and results post-processing is introduced and applied to a dataset that include remote sensing images from 15 cities and five counties from various region of the USA, which include 8,607,677 buildings. The accuracy of the proposed approach was measured on an out of sample testing dataset corresponding to 364,000 buildings from three USA cities. The results favorably compared to those obtained from Microsoft’s recently released US building footprint dataset.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1974

Application of ERTS-1 data to the protection and management of New Jersey's coastal environment

The author has identified the following significant results. Rapid access to ERTS data was provided by NASA GSFC for the February 26, 1974 overpass of the New Jersey test site. Forty-seven hours following the overpass computer-compatible tapes were ready for processing at EarthSat. The finished product was ready just 60 hours following the overpass and delivered to the New Jersey Department of Environmental Protection. This operational demonstration has been successful in convincing NJDEP as to the worth of ERTS as an operational monitoring and enforcement tool of significant value to the State. An erosion/ accretion severity index has been developed for the New Jersey shore case study area. Computerized analysis techniques have been used for monitoring offshore waste disposal dumping locations, drift vectors, and dispersion rates in the New York Bight area. A computer shade print of the area was used to identify intensity levels of acid waste. A Litton intensity slice print was made to provide graphic presentation of dispersion characteristics and the dump extent. Continued monitoring will lead to the recommendation and justification of permanent dumping sites which pose no threat to water quality in nearshore environments.

Yunghans, R. S.↗

DOE OSTI2020

Building fault detection data to aid diagnostic algorithm creation and performance testing

It is estimated that approximately 4-5% of national energy consumption can be saved through corrections to existing commercial building controls infrastructure and resulting improvements to efficiency. Correspondingly, automated fault detection and diagnostics (FDD) algorithms are designed to identify the presence of operational faults and their root causes. A diversity of techniques is used for FDD spanning physical models, black box, and rule-based approaches. A persistent challenge has been the lack of common datasets and test methods to benchmark their performance accuracy. This article presents a first of its kind public dataset with ground-truth data on the presence and absence of building faults. This dataset spans a range of seasons and operational conditions and encompasses multiple building system types. It contains information on fault severity, as well as data points reflective of the measurements in building control systems that FDD algorithms typically have access to. The data were created using simulation models as well as experimental test facilities, and will be expanded over time.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗