SEARCH · Engineering Papers

Results for “Data access”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Daily grass reference evapotranspiration with Meteosat Second Generation shortwave radiation and reference ET products

Here we assess the accuracy of estimating daily grass reference evapotranspiration (PM-ET o ) using daily shortwave radiation (R s ) and reference evapotranspiration (ET REF ) products provided by the Meteosat Second Generation (MSG) geostationary satellite delivered by the Satellite Applications Facility on Land Surface Analysis (LSA-SAF) framework. The accuracy of using reanalysis ERA5 shortwave radiation data (R s ERA5 ) provided by the European Center for Medium-Range Weather Forecasts (ECMWF) is also evaluated. The assessments were performed using observed weather variables at 37 weather stations distributed across continental Portugal, where climate conditions range from semi-arid to humid, and 12 weather stations located in Azores islands, characterized by humid, windy and often cloudy conditions. This study's use of data from a variety of climate conditions contributed to a unique and innovative assessment of the usability of LSA-SAF and ERA5 products for ET o estimation. The first assessment focused on comparing LSA-SAF estimates of R s (R s LSA-SAF ) against ground stations (R s ground ). The results showed a good matching between the two R s data sets for continental Portugal but a tendency for R s LSA-SAF to under-estimate R s ground in the cloudy islands of Azores. ET o values computed using R s LSA-SAF data and observed temperature, humidity and wind speed (ET o LSA-SAF ) were then compared with PM-ET o estimates with ground-based data, which were used as benchmark; input data of temperature and humidity needed for PM-ET o were quality checked for surface aridity effects. It was observed that ET o LSA-SAF is strongly correlated with PM-ET o (R 2 > 0.97) for most locations in continental Portugal, with regression coefficient of a linear regression forced to the origin ranging between 0.95 and 1.05, mean root mean square error (RMSE) of 0.13 mm d –1 , and Nash and Sutcliff efficiency of modeling (EF) above 0.95. For most Azores locations, ET o LSA-SAF over-estimated PM-ET o . This is likely a consequence of the high spatio-temporal heterogeneity of weather conditions that occur in these oceanic islands together with the different footprints of satellite (averaged over the pixel) and station observations. Reanalysis ERA5 shortwave radiation data presented similar behavior to the LSA-SAF products, however with slightly lower accuracy. The daily LSA-SAF ET REF product (ET REF LSA-SAF ) was assessed and results have shown a good accuracy of this product, with acceptable RMSE and high EF values, for continental Portugal but a low accuracy for the Azores islands. A simplified bias correction approach was shown to improve both ET o derived from the LSA-SAF products, namely for Azores stations, which seem to be representative of smaller areas. The use of the FAO-PM temperature approach (PMT) was also assessed using the R s LSA-SAF and R s ERA5 data, which showed a superiority of the LSA-SAF product for ET o estimations (ET o PMT LSA-SAF ). No significant differences (p < 0.05) were observed in terms of the median value of the RMSE when adopting ET o PMT and ET REF LSA-SAF . Differently, results showed that using the R s LSA-SAF in the PMT approach (ET o PMT LSA-SAF ) produces significantly better RMSE results than ET o PMT and ET REF LSA-SAF . Overall, the performed assessment allows concluding that the use of R s LSA-SAF, and to a lesser extent the use of the R s ERA5 , highly improves the accuracy of computation of ET o when R s observations are not available, including when only temperature data are accessible. The use of the ET REF LSA-SAF product is a good alternative when observed weather data are not available.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Cross-correlation analysis of X-ray photon correlation spectroscopy to extract rotational diffusion coefficients

Coefficients for translational and rotational diffusion characterize the Brownian motion of particles. Emerging X-ray photon correlation spectroscopy (XPCS) experiments probe a broad range of length scales and time scales and are well-suited for investigation of Brownian motion. While methods for estimating the translational diffusion coefficients from XPCS are well-developed, there are no algorithms for measuring the rotational diffusion coefficients based on XPCS, even though the required raw data are accessible from such experiments. In this paper, we propose angular-temporal cross-correlation analysis of XPCS data and show that this information can be used to design a numerical algorithm (Multi-Tiered Estimation for Correlation Spectroscopy [MTECS]) for predicting the rotational diffusion coefficient utilizing the cross-correlation: This approach is applicable to other wavelengths beyond this regime. We verify the accuracy of this algorithmic approach across a range of simulated data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗

NASA NTRS2001

CDDIS Support of IGS LEO Activities

The Crustal Dynamics Data Information System (CDDIS) has served as a global data center for the International GPS Service (IGS) since its start in June 1992, providing on-line access to data from over 175 sites on a daily and hourly basis. This paper will present an overview about the current status of the CDDIS GPS data and products archive with a look to the future support of LEO (Low Earth Orbiting missions), including the archive of high-rate data and on-board GPS receiver data.

Noll, Carey E.↗

NASA NTRS2012

Restoration of APOLLO Data by the NSSDC and PDS Lunar Data Node

The Apollo Lunar Surface Experiment Packages (ALSEPs), suites of instruments deployed by the Apollo 12. 14, 15, 16 and 17 astronauts on the lunar surface, still represent the only in-situ measurements of the Moon's environment taken over long time periods, Much of these data are housed at the National Space Science Data Center (NSSDC) at Goddard Space Flight Center but are in forms that are not readily usable, such as microfilm, hardcopy, and magnetic tapes with older, obsolete formats. The Lunar Data Node (LDN) has been formed under the auspices of the Planetary Data System (PDS) Geosciences Node to put relevant, scientifically important Apollo data into accessible digital form for use by researchers and mission planners. The LDN has prioritized the restoration of these data based on their scientific and engineering value and the level of effort required. We will report on progress made and plans for future data restorations.

Williams, David R.↗

NASA NTRS2015

The Open Data Repositorys Data Publisher

Data management and data publication are becoming increasingly important components of researcher's workflows. The complexity of managing data, publishing data online, and archiving data has not decreased significantly even as computing access and power has greatly increased. The Open Data Repository's Data Publisher software strives to make data archiving, management, and publication a standard part of a researcher's workflow using simple, web-based tools and commodity server hardware. The publication engine allows for uploading, searching, and display of data with graphing capabilities and downloadable files. Access is controlled through a robust permissions system that can control publication at the field level and can be granted to the general public or protected so that only registered users at various permission levels receive access. Data Publisher also allows researchers to subscribe to meta-data standards through a plugin system, embargo data publication at their discretion, and collaborate with other researchers through various levels of data sharing. As the software matures, semantic data standards will be implemented to facilitate machine reading of data and each database will provide a REST application programming interface for programmatic access. Additionally, a citation system will allow snapshots of any data set to be archived and cited for publication while the data itself can remain living and continuously evolve beyond the snapshot date. The software runs on a traditional LAMP (Linux, Apache, MySQL, PHP) server and is available on GitHub (http://github.com/opendatarepository) under a GPLv2 open source license. The goal of the Open Data Repository is to lower the cost and training barrier to entry so that any researcher can easily publish their data and ensure it is archived for posterity.

Astrobiology data↗

NASA NTRS2010

Issues and Solutions for Bringing Heterogeneous Water Cycle Data Sets Together

The water cycle research community has generated many regional to global scale products using data from individual NASA missions or sensors (e.g., TRMM, AMSR-E); multiple ground- and space-based data sources (e.g., Global Precipitation Climatology Project [GPCP] products); and sophisticated data assimilation systems (e.g., Land Data Assimilation Systems [LDAS]). However, it is often difficult to access, explore, merge, analyze, and inter-compare these data in a coherent manner due to issues of data resolution, format, and structure. These difficulties were substantiated at the recent Collaborative Energy and Water Cycle Information Services (CEWIS) Workshop, where members of the NASA Energy and Water cycle Study (NEWS) community gave presentations, provided feedback, and developed scenarios which illustrated the difficulties and techniques for bringing together heterogeneous datasets. This presentation reports on the findings of the workshop, thus defining the problems and challenges of multi-dataset research. In addition, the CEWIS prototype shown at the workshop will be presented to illustrate new technologies that can mitigate data access roadblocks encountered in multi-dataset research, including: (1) Quick and easy search and access of selected NEWS data sets. (2) Multi-parameter data subsetting, manipulation, analysis, and display tools. (3) Access to input and derived water cycle data (data lineage). It is hoped that this presentation will encourage community discussion and feedback on heterogeneous data analysis scenarios, issues, and remedies.

Acker, James↗

DOE OSTI2022

Electron microscopy holdings of the Protein Data Bank: the impact of the resolution revolution, new validation tools, and implications for the future

Abstract As a discipline, structural biology has been transformed by the three-dimensional electron microscopy (3DEM) “Resolution Revolution” made possible by convergence of robust cryo-preservation of vitrified biological materials, sample handling systems, and measurement stages operating a liquid nitrogen temperature, improvements in electron optics that preserve phase information at the atomic level, direct electron detectors (DEDs), high-speed computing with graphics processing units, and rapid advances in data acquisition and processing software. 3DEM structure information (atomic coordinates and related metadata) are archived in the open-access Protein Data Bank (PDB), which currently holds more than 11,000 3DEM structures of proteins and nucleic acids, and their complexes with one another and small-molecule ligands (~ 6% of the archive). Underlying experimental data (3DEM density maps and related metadata) are stored in the Electron Microscopy Data Bank (EMDB), which currently holds more than 21,000 3DEM density maps. After describing the history of the PDB and the Worldwide Protein Data Bank (wwPDB) partnership, which jointly manages both the PDB and EMDB archives, this review examines the origins of the resolution revolution and analyzes its impact on structural biology viewed through the lens of PDB holdings. Six areas of focus exemplifying the impact of 3DEM across the biosciences are discussed in detail (icosahedral viruses, ribosomes, integral membrane proteins, SARS-CoV-2 spike proteins, cryogenic electron tomography, and integrative structure determination combining 3DEM with complementary biophysical measurement techniques), followed by a review of 3DEM structure validation by the wwPDB that underscores the importance of community engagement.

Burley, Stephen K. (ORCID:0000000224879713)↗

DOE OSTI2023

Department of Energy’s Atmospheric System Research (ASR) Program’s Workshop on the Future of Atmospheric Large Eddy Simulation (LES): Workshop Report

Large-eddy simulation (LES) is used as a tool to understand physical processes such as turbulence, aerosols, clouds, precipitation, radiation, the interactions among all these, and their interactions with the underlying surface. Over the next 10 years, LES will drive fundamental progress in open scientific questions in these areas as LES is increasingly used to gain understanding of complex interacting physical processes involving atmospheric turbulence. This growth will be driven both by scientific demand and the expansion of computational resources needed to conduct LES, and the form that the growth takes will largely be determined by how computational resources are leveraged for scientific gain. In particular, we suggest that computational resources are likely to be leveraged in two separate but not necessarily distinct ways. On one hand, growth in computational resources will allow LES to be made more routine, that is, performed more frequently, while on the other hand, the computational expense (measured in total floating point operations) afforded to individual LES will expand dramatically, allowing simulations to increase in both domain size and resolution as well as physical detail. Current U.S. Department of Energy (DOE) projects such as LES ARM Symbiotic Simulation and Observation Activity (LASSO) are leading the way in conducting routine LES, building large, public databases that are accessible for data science, sensitivity studies, and training for machine learning. LES will also become more routine as it becomes more accessible for individual researchers to address their scientific questions of interest. Scientific questions addressed by LES over the next 10 years are likely to include cloud organization and aggregation; aerosol cloud interactions and atmospheric chemistry (including geo-engineering); urban-scale LES; atmospheric extreme events, ranging from small-scale severe weather to wildfires; and ocean-wave-atmosphere interactions. Further LES-related research will likely grow significantly in areas related to societal impact studies of air quality and extreme weather events, applications to renewable energy forecasting and resource assessment, and aid in decision-making processes. The growth in the use of LES in atmospheric science research will drive the need for better physical process representations (e.g., cloud aerosol microphysics, radiation, and atmospheric chemistry) at the scales resolved by LES. To date, many of the process representations used by LES have been taken directly from coarser-resolution models. Promising methods for LES process representations include superdroplet and quadrature methods for microphysics, 3D approaches for radiation, and better representation of chemistry and aerosol processes. At LES resolution, land-atmosphere interactions for complex terrains, land cover/types, biogeochemistry, and plant canopy models are needed as an improvement beyond traditional and widely used Monin-Obuhkov similarity theory.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Department of Energy’s Atmospheric System Research (ASR) Program’s Workshop on the Future of Atmospheric Large Eddy Simulation (LES) (Workshop Report)

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1986

Ada and knowledge-based systems: A prototype combining the best of both worlds

A software architecture is described which facilitates the construction of distributed expert systems using Ada and selected knowledge based systems. This architecture was utilized in the development of a Knowledge-based Maintenance Expert System (KNOMES) prototype for the Space Station Mobile Service Center (MSC). The KNOMES prototype monitors a simulated data stream from MSC sensors and built-in test equipment. It detects anomalies in the data and performs diagnosis to determine the cause. The software architecture which supports the KNOMES prototype allows for the monitoring and diagnosis tasks to be performed concurrently. The basic concept of this software architecture is named ACTOR (Ada Cognitive Task ORganization Scheme). An individual ACTOR is a modular software unit which contains both standard data processing and artificial intelligence components. A generic ACTOR module contains Ada packages for communicating with other ACTORs and accessing various data sources. The knowledge based component of an ACTOR determines the role it will play in a system. In this prototype, an ACTOR will monitor the MSC data stream.

Brauer, David C.↗

NASA NTRS1990

Prototyping a large-scale distributed system for the Great Observatories era - NASA Astrophysics Data System (ADS)

The NASA Astrophysics Data System (ADS) is a distributed information system intended to support research in the Great Observatories era, to simplify access to data, and to enable simultaneous analyses of multispectral data sets. Here, the user agent and interface, its functions, and system components are examined, and the system architecture and infrastructure is addressed. The present status of the system and related future activities are examined.

Shames, Peter↗

DOE OSTI2026

Performance and Reliability Assessment of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) Data Advisor (ADA)

The Atmospheric Radiation Measurement (ARM) User Facility provides one of the world's largest openly accessible repositories of atmospheric observations through the ARM Data Discovery platform. Although the repository contains more than three decades of measurements collected from permanent observatories, mobile facilities, aircraft campaigns, and field experiments, identifying appropriate datasets can be challenging, particularly for new users unfamiliar with ARM instrumentation and datastream organization. To improve data accessibility, the ARM Data Center developed the ARM Data Advisor (ADA), an artificial intelligence-powered assistant designed to facilitate scientific data discovery, dataset interpretation, and user guidance. This report evaluates ADA's performance as a domain-specific scientific assistant using realistic atmospheric science workflows. The evaluation examines five key capabilities: data retrieval and curation efficiency, hallucination resistance, scientific reasoning, response to ambiguous queries, and content retention and session continuity. Representative prompts were developed to simulate typical interactions between researchers and the ARM Data Discovery platform, and ADA's responses were assessed for retrieval completeness, scientific accuracy, consistency, and practical usefulness. In these representative tests, ADA reduced the complexity of discovering and accessing ARM datasets by recommending appropriate datastreams, explaining instrumentation, interpreting metadata, and assisting with data processing workflows. ADA also exhibits strong domain knowledge of atmospheric science terminology and generally resists hallucination by acknowledging unavailable datasets and requesting clarification when appropriate. Overall, the results indicate that ADA represents a promising advancement in scientific data discovery within the ARM User Facility and has considerable potential to improve researcher productivity, particularly for new users and interdisciplinary scientists seeking efficient access to ARM observations.

Salvador, Christian [ORNL] (ORCID:0000000283287777↗

DOE OSTI2025

Innovative Interactive 3D Slice Visualization Tool: Bridging Accessibility, Insight, and Immersive Analysis for Complex Data Exploration

36 - MATERIALS SCIENCE↗

DOE OSTI2025

A Performant, Scalable Processing Pipeline for High‐Quality and FAIR Environmental Sensor Data

High-resolution environmental monitoring is necessary to record, understand, and predict biogeochemical and ecological changes particularly in coastal systems but brings significant challenges in processing and making rapidly available the resulting data. The COMPASS-FME project established a network of coastal observational sites across the Chesapeake Bay and western Lake Erie regions extensively instrumented with soil, vegetation, and weather sensors logging data every 15 min. Our data processing framework, written in R and completely open source, prioritizes rapid model-experiment iteration and makes biogeochemical data rapidly available for quality assurance/quality control, analysis, and model ingestion. This pipeline is distinguished by a standardized and modular approach to data curation, extensive metadata and documentation, and its high performance. These attributes combine to make biogeochemical data rapidly accessible across COMPASS-FME and the broader community. Flexible, powerful, and reproducible approaches to handling high-volume environmental data are crucial for accelerating biogeosciences research.

Pennington, Stephanie C. [Pacific Northwest Nation↗

DOE OSTI2020

Attack on Grid Event Cause Analysis: An Adversarial Machine Learning Approach

With the ever-increasing reliance on data for data-driven applications in power grids, such as event cause analysis, the authenticity of data streams has become crucially important. The data can be prone to adversarial stealthy attacks aiming to manipulate the data such that residual-based bad data detectors cannot detect them, and the perception of system operators or event classifiers changes about the actual event. This paper investigates the impact of adversarial attacks on convolutional neural network-based event cause analysis frameworks. We have successfully verified the ability of adversaries to maliciously misclassify events through stealthy data manipulations. The vulnerability assessment is studied with respect to the number of compromised measurements. Furthermore, a defense mechanism to robustify the performance of the event cause analysis is proposed. The effectiveness of adversarial attacks on changing the output of the framework is studied using the data generated by real-time digital simulator (RTDS) under different scenarios such as type of attacks and level of access to data.

Niazazari, Iman↗

NASA NTRS2019

Fuser and Fuser in the Cloud

The ˜Fuser' is the name of the SWIM data integration service that has been built, utilized and evolved on ATD-2 to synchronize trajectory predictions from multiple decision support tools. NASA is researching ways to make this data stream available for broader consumption as an example implementation that would help reduce the burden for consumers while enabling access to data that would help industry prepare and innovate for TFDM (e.g. integrates TTP feed).

Fuser↗

NASA NTRS2018

Advanced Utilization of the Payload Executive Processor (PEP) Ethernet Port to Support JSL Based Payloads

Increased interest by Payload Developers in utilization of the Joint Station LAN (Local Area Network) (JSL) for command and data transactions has driven the investigation of providing bi-directional Ethernet communication with PEP (Payload Executive Processor). Ethernet-only payload developers are interested in taking advantage of the services provided by PEP. Enabling Ethernet communications within PEP requires more than just turning on the hardware. PEP contains no Operating System (OS) or Ethernet stack so it does not inherit any built in functionality for Ethernet support. Both hardware drivers and application software must be developed from scratch. In the absence of an OS (Operating System) the design must address security issues in terms of access and data transferred, i.e. blocking unauthorized users and preventing denial of service. Developing a design to process Ethernet data within the existing real-time constraints of PEP requires a few compromises: data rates, supported protocols, custom packet format and limited client connections to name a few. Integration of the light weight internet protocol (LwIP). Ethernet stack provides a customizable solution for the embedded real-time environment. As utilization of the JSL increases, the successful implementation of an Ethernet interface enhances the utilization of PEP, and provides a new path for future Ethernet-only payloads to gain access to around the clock command and data services.

Guyette, Greg↗