Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data requests”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

The NPOESS Preparatory Project Science Data Segment (SDS) Data Depository and Distribution Element (SD3E) System Architecture

The National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (NPP) Science Data Segment (SDS) will make daily data requests for approximately six terabytes of NPP science products for each of its six environmental assessment elements from the operational data providers. As a result, issues associated with duplicate data requests, data transfers of large volumes of diverse products, and data transfer failures raised concerns with respect to the network traffic and bandwidth consumption. The NPP SDS Data Depository and Distribution Element (SD3E) was developed to provide a mechanism for efficient data exchange, alleviate duplicate network traffic, and reduce operational costs.

Ho, Evelyn L.↗
NASA NTRS1974

Status of availability of Mariner 9 (1971-051A) TV picture data

Request procedures are described for Mariner 9 TV picture data. Image processing laboratory, user's guides, mosaics, catalogues, and microfiches are briefly discussed. Samples of the limb picture microfiche index and catalogue are presented.

Source record↗
NASA NTRS2013

Dynamic Server-Based KML Code Generator Method for Level-of-Detail Traversal of Geospatial Data

Web-based geospatial client applications such as Google Earth and NASA World Wind must listen to data requests, access appropriate stored data, and compile a data response to the requesting client application. This process occurs repeatedly to support multiple client requests and application instances. Newer Web-based geospatial clients also provide user-interactive functionality that is dependent on fast and efficient server responses. With massively large datasets, server-client interaction can become severely impeded because the server must determine the best way to assemble data to meet the client applications request. In client applications such as Google Earth, the user interactively wanders through the data using visually guided panning and zooming actions. With these actions, the client application is continually issuing data requests to the server without knowledge of the server s data structure or extraction/assembly paradigm. A method for efficiently controlling the networked access of a Web-based geospatial browser to server-based datasets in particular, massively sized datasets has been developed. The method specifically uses the Keyhole Markup Language (KML), an Open Geospatial Consortium (OGS) standard used by Google Earth and other KML-compliant geospatial client applications. The innovation is based on establishing a dynamic cascading KML strategy that is initiated by a KML launch file provided by a data server host to a Google Earth or similar KMLcompliant geospatial client application user. Upon execution, the launch KML code issues a request for image data covering an initial geographic region. The server responds with the requested data along with subsequent dynamically generated KML code that directs the client application to make follow-on requests for higher level of detail (LOD) imagery to replace the initial imagery as the user navigates into the dataset. The approach provides an efficient data traversal path and mechanism that can be flexibly established for any dataset regardless of size or other characteristics. The method yields significant improvements in userinteractive geospatial client and data server interaction and associated network bandwidth requirements. The innovation uses a C- or PHP-code-like grammar that provides a high degree of processing flexibility. A set of language lexer and parser elements is provided that offers a complete language grammar for writing and executing language directives. A script is wrapped and passed to the geospatial data server by a client application as a component of a standard KML-compliant statement. The approach provides an efficient means for a geospatial client application to request server preprocessing of data prior to client delivery. Data is structured in a quadtree format. As the user zooms into the dataset, geographic regions are subdivided into four child regions. Conversely, as the user zooms out, four child regions collapse into a single, lower-LOD region. The approach provides an efficient data traversal path and mechanism that can be flexibly established for any dataset regardless of size or other characteristics.

Baxes, Gregory↗
DOE OSTI2018

System and method of energy management control

A method of controlling energy consumption in a building includes receiving occupancy data including at least one of occupant request data and occupant schedule data, receiving weather data including at least one of current weather measurement data and weather forecast data, generating an output control signal based on the occupancy data and the weather data, and transmitting the output control signal to a building automation system (BAS) of the building. The occupant request data includes a current request, and the occupant schedule data includes a predicted occupant schedule. The output control signal adjusts a building control device in a zone in the building.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2016

Guide to Data Access and Utility Customer Confidentiality

This report explains the difference between whole-building energy usage data requests and other types of data requests utilities may receive. It outlines utility best practices for providing energy consumption information to building owners while respecting the confidentiality of utility customers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2019

ISO Data Automated Request Tool

Very early stages of a simple tool able to create multiple automated requests of different Independent System Operators (ISOs) data, and download it in comma separated value (.csv) files.

Sotorrio, Pedro↗
NASA NTRS1989

Space data management at the NSSDC (National Space Sciences Data Center): Applications for data compression

The National Space Science Data Center (NSSDC), established in 1966, is the largest archive for processed data from NASA's space and Earth science missions. The NSSDC manages over 120,000 data tapes with over 4,000 data sets. The size of the digital archive is approximately 6,000 gigabytes with all of this data in its original uncompressed form. By 1995 the NSSDC digital archive is expected to more than quadruple in size reaching over 28,000 gigabytes. The NSSDC digital archive is expected to more than quadruple in size reaching over 28,000 gigabytes. The NSSDC is beginning several thrusts allowing it to better serve the scientific community and keep up with managing the ever increasing volumes of data. These thrusts involve managing larger and larger amounts of information and data online, employing mass storage techniques, and the use of low rate communications networks to move requested data to remote sites in the United States, Europe and Canada. The success of these thrusts, combined with the tremendous volume of data expected to be archived at the NSSDC, clearly indicates that innovative storage and data management solutions must be sought and implemented. Although not presently used, data compression techniques may be a very important tool for managing a large fraction or all of the NSSDC archive in the future. Some future applications would consist of compressing online data in order to have more data readily available, compress requested data that must be moved over low rate ground networks, and compress all the digital data in the NSSDC archive for a cost effective backup that would be used only in the event of a disaster.

Green, James L.↗
DOE OSTI2026

The LCLStream Ecosystem for Multi-Institutional Dataset Exploration

We describe a new end-to-end experimental data streaming framework designed from the ground up to support new types of applications – AI training, extremely high-rate X-ray time-of-flight analysis, crystal structure determination with distributed processing, and custom data science applications and visualizers yet to be created. Throughout, we use design choices merging cloud microservices with traditional HPC batch execution models for security and flexibility. This project makes a unique contribution to the DOE Integrated Research Infrastructure (IRI) landscape. By creating a flexible, API-driven data request service, we address a significant need for high-speed data streaming sources for the X-ray science data analysis community. With the combination of data request API, mutual authentication web security framework, job queue system, high-rate data buffer, and complementary nature to facility infrastructure, the LCLStreamer framework has prototyped and implemented several new paradigms critical for future generation experiments.

Rogers, David [ORNL] (ORCID:0000000251871768)↗
NASA NTRS1997

CINTEX: International Interoperability Extensions to EOSDIS

A large part of the research under this cooperative agreement involved working with representatives of the DLR, NASDA, EDC, and NOAA-SAA data centers to propose a set of enhancements and additions to the EOSDIS Version 0 Information Management System (V0 IMS) Client/Server Message Protocol. Helen Conover of ITSL led this effort to provide for an additional geographic search specification (WRS Path/Row), data set- and data center-specific search criteria, search by granule ID, specification of data granule subsetting requests, data set-based ordering, and the addition of URLs to result messages. The V0 IMS Server Cookbook is an evolving document, providing resources and information to data centers setting up a VO IMS Server. Under this Cooperative Agreement, Helen Conover revised, reorganized, and expanded this document, and converted it to HTML. Ms. Conover has also worked extensively with the IRE RAS data center, CPSSI, in Russia. She served as the primary IMS contact for IRE-CPSSI and as IRE-CPSSI's liaison to other members of IMS and Web Gateway (WG) development teams. Her documentation of IMS problems in the IRE environment (Sun servers and low network bandwidth) led to a general restructuring of the V0 IMS Client message polling system. to the benefit of all IMS participants. In addition to the IMS server software and documentation. which are generally available to CINTEX sites, Ms. Conover also provided database design documentation and consulting, order tracking software, and hands-on testing and debug assistance to IRE. In the final pre-operational phase of IRE-CPSSI development, she also supplied information on configuration management, including ideas and processes in place at the Global Hydrology Resource Center (GHRC), an EOSDIS data center operated by ITSL.

Graves, Sara J.↗
NASA NTRS2018

Predicting the Operational Acceptance of Airborne Flight Reroute Requests Using Data Mining

For tools that generate more efficient flight routes or reroute advisories, it is important to ensure compatibility of automation and autonomy decisions with human objectives so as to ensure acceptability by the human operators. In this paper, the authors developed a proof of concept predictor of operational acceptability for route changes during a flight. Such a capability could have applications in automation tools that identify more efficient routes around airspace impacted by weather or congestion and that better meet airline preferences. The predictor is based on applying data mining techniques, including logistic regression, a decision tree, a support vector machine, a random forest and Adaptive Boost, to historical flight plan amendment data reported during operations and field experiments. Cross validation was used for model development, while nested cross validation was used to validate the models. The model found to have the best performance in predicting air traffic controller acceptance or rejection of a route change, using the available data from Fort Worth Air Traffic Control Center and its adjacent Centers, was the random forest, with an F-score of 0.77. This result indicates that the operational acceptance of reroute requests does indeed have some level of predictability, and that, with suitable data, models can be trained to predict the operational acceptability of reroute requests. Such models may ultimately be used to inform route selection by decision support tools, contributing to the development of increasingly autonomous systems that are capable of routing aircraft with less human input than is currently the case.

Operational Acceptability↗
DOE OSTI2025

Baseline Climate Variables for Earth System Modelling

The Baseline Climate Variables for Earth System Modelling (ESM-BCVs) are defined as a list of 135 variables which have high utility for the evaluation and exploitation of climate simulations. The list reflects the most frequently used elements of the Coupled Model Intercomparison Project Phase 6 (CMIP6) archive. Successive phases of CMIP have supported strong results in science and substantially influence international climate policy formulation. This paper responds to both interest in exploiting CMIP data standards in a broader range of climate modelling activities and a need to achieve greater clarity about the significance and intention of variables in the CMIP Data Request. As Earth system modelling archives grow in scale and complexity, there are emerging problems associated with weak standardisation at the variable collection level. That is, there are good standards covering how specific variables should be archived, but this paper fills a gap in the standardisation of which variables should be archived. The ESM-BCV list is intended as a resource for ESM intercomparison projects (MIPs) developing requests to enable greater consistency among MIPs and as a reference for modelling centres to enhance consistency within MIPs. Provisional planning for the CMIP7 Data Request exploits the ESM-BCVs as a core element. The baseline variable list includes 98 variables which have modest or minor data volume footprints and could be generated systematically when simulations are produced and archived for exploitation by the World Climate Research Programme (WCRP) community. A further 35 variables are classed as “high volume” and are only suitable for production when the resource implications are justified.

Juckes, Martin [University of Oxford (United Kingd↗
NASA NTRSDate not supplied

Increasing Accessibility of the Runs-on-Request Metadata, Data, and Services at the Community Coordinated Modeling Center

Space weather models are essential to our ability to understand and predict space weather events. For over 20 years, the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov) has been providing transformative tools and platforms for hosting space weather models and associated services, free and open to anyone interested in studying space weather. Runs-on-Request system (ROR) is one of the popular services at CCMC that permits researchers and other end-users to exercise cutting-edge hosted heliophysics and space weather models using a simple web interface, as well as collaborate on an extensive and continuously growing archive of over 28,000 model run results. Similar to other projects at CCMC, ROR has grown as a community project that strives to be open and transparent to its users. In this poster, we discuss some of our recent efforts to further expose ROR data, metadata, and services to the end users through both custom and community-developed access protocols. We also discuss how in-house science support provided by the CCMC team plays a paramount role in making ROR data and services truly accessible by the community.

Maksym Petrenko↗
DOE OSTI2019

CogSim: Integrate ML-based Library (LAGER) into Multi-Physics Simulation Code

In LAGER, the Mesh and Simulation History Extract (MASHExtract) package has been used for extracting various types of data from Kull runs. The package itself was written in python and completely enmeshed in the Kull codebase, making it impossible to use with other simulations. Having a version that extends to the Ale3d codebase would necessitate a complete re-write of the package, including a conversion to the C++ language. Extending the MASHExtract package to Ale3d requires a level of simulation independence, much of which would come from handling user requests, data management, and exporting. However, there is still a section of code that would need to be able to handle extracting data from the target codebase. Because of the often insurmountable differences between simulation codebases, the MASHExtract package would need a specific layer to be implemented for each codebase that handles this process of collecting and organizing the requested data. Furthermore, different simulation codebases also have different requirements for various enhancements they rely on—user interface, multi processor distribution, gpu acceleration, etc—all of which would also need to be handled by this layer. In regards to Kull and Ale3d specifically, the Kull implementation (KullMASHExtract) will need to be wrapped in Python, and Ale3d (Ale3dMASHExtract) will need to be wrapped in RAJA. Since Kull is primarily wrapped in SWIG, we should also use this library for our own wrapping to maintain consistency. There are also differences between what metrics are available for extraction. In particular, Kull has a wide array of physics metrics that may not be available in Ale3d, and these differences will need to be accounted for when re-writing MASHExtract.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Scoreboard

Emerging HPC machines have given rise to enhanced compute power that far outstrips the machine's ability to save large scale results for post-processing. To combat this, in situ data analysis techniques are slowly being adopted. With in situ data management favoring workflows composed of multiple simulations and analyses connected in transit on heterogeneous machines, scientists and engineers need a tool that enables them to create data extracts, visualizations, and interactively monitor and steer their simulations. Scoreboard Phase II is a next generation analysis software that supports composite in transit workflows on heterogeneous architectures and restores interactivity to in situ data analysis through simulation monitoring and computational steering. Scoreboard provides a simulation dashboard with graphs of metrics over time, controls for setting custom simulation steering parameters, controls for managing the set of data extracts being produced in the simulation, as well as the ability to explore data extracts, all from a web browser. Realizing the vision outlined in this project required research into making a system that integrates end to end from simulations all the way to the user. In situ tools generally suffer from complexity and excessive software dependencies. Scoreboard, by contrast, is easy to build and integrate into simulation codes and it provides first class FORTRAN support. The Scoreboard library is capable of in situ and in transit data analysis that can produce data extracts commonly needed for Computational Fluid Dynamics (CFD) analysis. Simulations can transparently stage data in transit to a Scoreboard Endpoint program, which can accept their data and produce the requested data extracts. This lets simulations return to their work while the Endpoint works on the analysis. Efficiently staging the data at scale was a topic of this research. Scoreboard provides the means to let the user manage data extracts and monitor/steer many simulations from a web browser. This area of the research focused on discovery of in transit network components to expose and control their steering parameters within an interactive browser-based user interface that includes: system topology, gathered metrics, notifications, dynamically-generated steering controls, and exploration of visualization data products.

Whitlock, BradJoseph [Intelligent Light] (00000001↗
DOE OSTI2017

Epi Archive: automated data collection of notifiable disease data

LANL has built a software program that automatically collects global notifiable disease data—particularly data stored in files—and makes it available and shareable within the Biosurveillance Ecosystem(BSVE) as a new data source. This will improve the prediction and early warning of disease events and other applications. Most countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health as exemplified by the Biosurveillance Ecosystem(BSVE).While most nations do likely store their data in a machine-readable format, the governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues. For example, an attempt by LANL to obtain a weekly version of openly available monthly data, reported by the Australian government, resulted in an onerous bureaucratic reply. The obstacles to obtaining data included: paperwork to request data from each of the Australian states and territories, a long delay to obtain data (up to 3 months) and extensive limitations on the data’s use that prohibit collaboration and sharing. This type of experience when attempting to contact public health departments or ministries of health for data is not uncommon. A survey conducted by LANL of notifiable disease data reporting in 52 countries identified only 10 as being machine-readable and 42 being reported in pdf files on a regular basis. Within the 42 nations that report in pdf files, 32 report in a structured, tabular format and10 in a non-structured way. As a result, LANL has developed a tool-Epi Archive (formerly known as EPIC)-to automatically and continuously collect global notifiable disease data and make it readily accessible. We conducted a survey of the national notifiable disease reporting systems notating how the data is reported in two important dimensions: date standards and case definitions. The development of software to regularly ingests notifiable disease data frand makes this data available involved four main steps scraping, extracting, parsing and persisting. For scraping: we would examine website designs and determine reporting mechanisms for each country/website as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of report pdf files, for each country. We stored report pdfs along with appropriate metadata for extracting and parsing. For extracting: we developed software that can extract notifiable disease data presented in tabular form from a pdf file. We combined the methodology of figure placement detection with the in-house developed table extraction and annotation heuristics. For parsing: we determined what to extract from each pdf dataset from the survey conducted. We then parsed the extracted data into uniform data structures correctly accommodating the dimensions surveyed and the various human languages. This task involved ingesting notifiable disease data in many disparate formats extracted from pdf files and coalescing the data into a standardized format. For persisting: We then store the data in the Epi Archive PostgreSQL database and make it available through the BSVE. The EpiArchive tool currently contains subnational notifiable disease data from 10 nations. When a user accesses the EpiArchive site, they are prompted with four fields: country, region, disease, and date duration. These fields allow the user to specify the location(down to the state level), the disease of interest, and the duration of interest. Upon form submission, a time series is generated from the users’ specifications. The generated time series can then be downloaded into a csv file if a user is interested in performing personal analysis. Additionally, the data from EpiArchive can be reached through an API. LANL as part of a currently funded DTRA effort so that it will automatically and continuously collect global notifiable disease data—particularly data stored in pdf files—and make it available and shareable within the Biosurveillance Ecosystem (BSVE) as a new data source. This will provide data to analytics and users that will improve the prediction and early warning of disease events and other applications.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1997

Requesting AVIRIS Data: A Guide for Principal Investigators

This guide serves as a brief overview of the AVIRIS instrument and its role in the field of imaging spectrometry. Mission planning and flight operations are discussed, and recommendations are given regarding the deployment of ground truth experiments.

VISIBLE SPECTRA↗
NASA NTRSDate not supplied

Standard Measures During Spaceflight

The key goal of the Spaceflight Standard Measures project is to ensure that a set of measures, representing the Human Research Program’s key risks and acquired with minimal impact on time and resources, is consistently captured from crewmembers through the end of the International Space Station (ISS) Program. Data collected under the Spaceflight Standard Measures project include assessments of sleep/wake cycles, cognition, immune status and function, general blood and urine chemistry (urine is collected only before flight and after landing), microbiome composition (gastrointestinal tract, saliva, and body surface), cardiovascular structure and function (carotid intima-media thickness, orthostatic responses), sensorimotor function, and team processes. Data is collected once or twice before the flight (180 and 90 days before launch), twice during the 6-month missions (fight day 30 and 30 days before return to Earth) with the exception of actigraphy, which is recorded continuously during the mission, and during two-week periods before and after the mission. In this presentation, we will review the data collected to date on twelve ISS crew members. These data are placed in the NASA Life Sciences Data Archive and are available for occupational surveillance (using non-identifiable data) Institutional Review Board-approved data sharing requests, and retrospective data requests. This data repository enables high-level monitoring of the effectiveness of countermeasures and meaningful interpretation of health and performance outcomes for various mission durations. The knowledge gained from this project informs and supports future hypothesis-driven research that will enable the success of planetary missions.

G R Clement↗