Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data sharing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79

Explicit Discontinuous Galerkin Methods for Conservation Laws

The two explicit DG methods in this study are based on a ‘predictor-corrector’ formulation, the first introduced by Lörcher, Gassner, and Munz (2007, 2008) called space–time expansion discontinuous Galerkin or STE-DG scheme, and the second, introduced independently by the author (Huynh 2006, 2013) called the upwind moment scheme. The predictor step of the two methods is essentially identical using a Cauchy-Kovalevsky (CK) procedure, which involves no interaction of the data among neighboring cells. The corrector step also shares the same space-time integration formulation and is where interaction of the data among neighboring cells takes place; the difference, however, is in how the resulting space-time volume integral is estimated. As a consequence of the different estimates, for the case of advection in one spatial dimension (1D), the moment scheme has a CFL (Courant-Friedrichs-Lewy) condition of 1 for all p and is accurate to order 2p+1, i.e., it possesses the super accuracy property, whereas the STE-DG method has a more restrictive CFL condition and is accurate to the expected order of p+1. For 1D advection, compared with the CFL conditions of 1/(2p+1) of standard RK-DG (Runge-Kutta) scheme where space and time discretization are of the same order, the moment scheme allows a significantly larger time step size. It also turns out that the scheme yields a result identical to Van Leer’s scheme III (1977), which amounts to shifting the data a distance of advection corresponding to the time step and projecting the result onto the space of polynomial solutions. Contrary to Van Leer’s approach, however, the space-time ‘predictor-corrector’ formulation facilitates extensions to the case of systems of equations. Concerning 2D extensions, in the case of advection, when the flow is along the diagonal direction, the CFL conditions for the moment schemes become restrictive as will be shown by Fourier (Von Neumann) stability and accuracy analyses. Since the moment scheme employs the right Radau points as collocation points in time, the method is closely related to the implicit Radau IIA scheme, which is stable for any time step size. The role of Radau IIA in relieving stability restriction for these explicit DG schemes remains to be explored

Discontinuous Galerkin↗

Resource Discovery for Extreme Scale Collaboration (RDESC) Final Report - RPI/TWC - Year 3

The amount of data produced in the practice of science is growing rapidly. Despite the accumulation and demand for scientific data, relatively little is actually made available for the broader scientific community. We surmise that the root of the problem is the perceived difficulty to electronically publish scientific data and associated metadata in a way that makes it discoverable. We propose to exploit Semantic Web technologies and practices to make (meta)data discoverable and easy to publish. We share our experiences in curating metadata to illustrate both the flexibility of our approach and the pain of discovering data in the current research environment. We also make recommendations by concrete example of how data publishers can provide their (meta)data by adding some limited, additional markup to HTML pages on the Web. With little additional effort from data publishers, the difficulty of data discovery/access/sharing can be greatly reduced and the impact of research data greatly enhanced.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Open Science for Plants in Space: Improvements in NASA's Open Science Data Repository

Upcoming deep space missions will rely on plants for crew and ecosystem health. Open access space biology data enables scientists to examine the biological responses of plants to ionizing radiation, altered gravity, elevated CO2, and many other abiotic stressors. NASA has declared 2023 as the ‘Year of Open Science’ and created a 5-year Transform to Open Science (TOPS) initiative designed to rapidly transform the agency toward an inclusive culture of open science. NASA’s Open Science Data Repository (OSDR) combines two databases, GeneLab and Ames Life Sciences Data Archive (ALSDA) to maximize access to standardized ‘omics (e.g., transcriptomics, proteomics) and phenotypic data (e.g., microscopy, biomass), respectively. Current OSDR standards include the ISA (Investigation-Study-Assay) experiment model, assay metadata configurations, and standardized terminology and ontologies. In 2024 OSDR will include a new suite of features for improved FAIR compliance including downloadable plant metadata templates, data submission tools and overall improved AI-readiness of plant datasets. AI/ML methods can be helpful tools to overcome the inherent challenges of space biology research (small sample size, sparse and heterogeneous data etc.). However these methods are built on an assumption of normalized and well-curated data. OSDR’s new curation tools will improve users ability to leverage ML and AI methods to model space biology data and better understand the complex effects of spaceflight on living systems across hierarchical biological levels. We look forward to sharing our advances with the spaceflight community.

FAIR↗

The Alliance of Genome Resources: Building a Modern Data Ecosystem for Model Organism Databases

Model organisms are essential experimental platforms for discovering gene functions, defining protein and genetic networks, uncovering functional consequences of human genome variation, and for modeling human disease. For decades, researchers who use model organisms have relied on Model Organism Databases (MODs) and the Gene Ontology Consortium (GOC) for expertly curated annotations, and for access to integrated genomic and biological information obtained from the scientific literature and public data archives. Through the development and enforcement of data and semantic standards, these genome resources provide rapid access to the collected knowledge of model organisms in human readable and computation-ready formats that would otherwise require countless hours for individual researchers to assemble on their own. Since their inception, the MODs for the predominant biomedical model organisms [Mus sp (laboratory mouse), Saccharomyces cerevisiae, Drosophila melanogaster, Caenorhabditis elegans, Danio rerio, and Rattus norvegicus] along with the GOC have operated as a network of independent, highly collaborative genome resources. In 2016, these six MODs and the GOC joined forces as the Alliance of Genome Resources (the Alliance). By implementing shared programmatic access methods and data-specific web pages with a unified "look and feel," the Alliance is tackling barriers that have limited the ability of researchers to easily compare common data types and annotations across model organisms. To adapt to the rapidly changing landscape for evaluating and funding core data resources, the Alliance is building a modern, extensible, and operationally efficient "knowledge commons" for model organisms using shared, modular infrastructure.

59 BASIC BIOLOGICAL SCIENCES↗

Parallel file system with metadata distributed across partitioned key-value store c

Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).

97 MATHEMATICS AND COMPUTING↗

Super-Resolution for Renewable Energy Resource Data with Wind from Reanalysis Data and Application to Ukraine

With a potentially increasing share of the electricity grid relying on wind to provide generating capacity and energy, there is an expanding global need for historically accurate, spatiotemporally continuous, high-resolution wind data. Conventional downscaling methods for generating these data based on numerical weather prediction have a high computational burden and require extensive tuning for historical accuracy. In this work, we present a novel deep learning-based spatiotemporal downscaling method using generative adversarial networks (GANs) for generating historically accurate high-resolution wind resource data from the European Centre for Medium-Range Weather Forecasting Reanalysis version 5 data (ERA5). In contrast to previous approaches, which used coarsened high-resolution data as low-resolution training data, we use true low-resolution simulation outputs. We show that by training a GAN model with ERA5 as the low-resolution input and Wind Integration National Dataset Toolkit (WTK) data as the high-resolution target, we achieved results comparable in historical accuracy and spatiotemporal variability to conventional dynamical downscaling. This GAN-based downscaling method additionally reduces computational costs over dynamical downscaling by two orders of magnitude. We applied this approach to downscale 30 km, hourly ERA5 data to 2 km, 5 min wind data for January 2000 through December 2023 at multiple hub heights over Ukraine, Moldova, and part of Romania. With WTK coverage limited to North America from 2007–2013, this is a significant spatiotemporal generalization. The geographic extent centered on Ukraine was motivated by stakeholders and energy-planning needs to rebuild the Ukrainian power grid in a decentralized manner. This 24-year data record is the first member of the super-resolution for renewable energy resource data with wind from the reanalysis data dataset (Sup3rWind).

17 WIND ENERGY↗

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗

Communication systems and methods for authenticating data packets within network flow

A communication system includes a first quantum key distribution device and an intermediary device. The first quantum key distribution device is configured to be coupled to a second quantum key distribution device over a quantum channel and to generate a shared key with the second quantum key distribution device based on a quantum state transmitted along the quantum channel. The intermediary device is disposed along a communication pathway within a network between a sender device and a receiver device. The intermediary device is communicatively connected to the first quantum key distribution device and configured to utilize the shared key to authenticate one or more data packets communicated from the sender device along the communication pathway by examining the one or more data packets for a presence of an information pattern that is associated with the shared key.

Bush, Stephen Francis↗

Visualization Component of Vehicle Health Decision Support System

The visualization front-end of a Decision Support System (DSS) also includes an analysis engine linked to vehicle telemetry, and a database of learned models for known behaviors. Because the display is graphical rather than text-based, the summarization it provides has a greater information density on one screen for evaluation by a flight controller.This tool provides a system-level visualization of the state of a vehicle, and drill-down capability for more details and interfaces to separate analysis algorithms and sensor data streams. The system-level view is a 3D rendering of the vehicle, with sensors represented as icons, tied to appropriate positions within the vehicle body and colored to indicate sensor state (e.g., normal, warning, anomalous state, etc.). The sensor data is received via an Information Sharing Protocol (ISP) client that connects to an external server for real-time telemetry. Users can interactively pan, zoom, and rotate this 3D view, as well as select sensors for a detail plot of the associated time series data. Subsets of the plotted data can be selected and sent to an external analysis engine to either search for a similar time series in an historical database, or to detect anomalous events. The system overview and plotting capabilities are completely general in that they can be applied to any vehicle instrumented with a collection of sensors. This visualization component can interface with the ISP for data streams used by NASA s Mission Control Center at Johnson Space Center. In addition, it can connect to, and display results from, separate analysis engine components that identify anomalies or that search for past instances of similar behavior. This software supports NASA's Software, Intelligent Systems, and Modeling element in the Exploration Systems Research and Technology Program by augmenting the capability of human flight controllers to make correct decisions, thus increasing safety and reliability. It was designed specifically as a tool for NASA's flight controllers to monitor the International Space Station and a future Crew Exploration Vehicle.

Jacob, Joseph↗

Background Error Covariance Estimation Using Information from a Single Model Trajectory with Application to Ocean Data Assimilation

An attractive property of ensemble data assimilation methods is that they provide flow dependent background error covariance estimates which can be used to update fields of observed variables as well as fields of unobserved model variables. Two methods to estimate background error covariances are introduced which share the above property with ensemble data assimilation methods but do not involve the integration of multiple model trajectories. Instead, all the necessary covariance information is obtained from a single model integration. The Space Adaptive Forecast error Estimation (SAFE) algorithm estimates error covariances from the spatial distribution of model variables within a single state vector. The Flow Adaptive error Statistics from a Time series (FAST) method constructs an ensemble sampled from a moving window along a model trajectory.SAFE and FAST are applied to the assimilation of Argo temperature profiles into version 4.1 of the Modular Ocean Model (MOM4.1) coupled to the GEOS-5 atmospheric model and to the CICE sea ice model. The results are validated against unassimilated Argo salinity data. They show that SAFE and FAST are competitive with the ensemble optimal interpolation (EnOI) used by the Global Modeling and Assimilation Office (GMAO) to produce its ocean analysis. Because of their reduced cost, SAFE and FAST hold promise for high-resolution data assimilation applications.

Error Covariance↗

Background Error Covariance Estimation using Information from a Single Model Trajectory with Application to Ocean Data Assimilation into the GEOS-5 Coupled Model

An attractive property of ensemble data assimilation methods is that they provide flow dependent background error covariance estimates which can be used to update fields of observed variables as well as fields of unobserved model variables. Two methods to estimate background error covariances are introduced which share the above property with ensemble data assimilation methods but do not involve the integration of multiple model trajectories. Instead, all the necessary covariance information is obtained from a single model integration. The Space Adaptive Forecast error Estimation (SAFE) algorithm estimates error covariances from the spatial distribution of model variables within a single state vector. The Flow Adaptive error Statistics from a Time series (FAST) method constructs an ensemble sampled from a moving window along a model trajectory. SAFE and FAST are applied to the assimilation of Argo temperature profiles into version 4.1 of the Modular Ocean Model (MOM4.1) coupled to the GEOS-5 atmospheric model and to the CICE sea ice model. The results are validated against unassimilated Argo salinity data. They show that SAFE and FAST are competitive with the ensemble optimal interpolation (EnOI) used by the Global Modeling and Assimilation Office (GMAO) to produce its ocean analysis. Because of their reduced cost, SAFE and FAST hold promise for high-resolution data assimilation applications.

Data Assimilation↗

High-Mileage Courier Fleet Vehicle On-road Logger Data

This dataset describes the performance and fuel efficiency of AVTA test vehicles operating in commercial courier fleets the Phoenix, AZ metro area between 2010 and 2016. Aftermarket data loggers were installed in two to four vehicles of each of 30+ distinct year/make/models (see reference ["INL Advanced Vehicle Testing Activity: On-road Logger and Laboratory Battery Pack Testing Vehicle List"](https://avt.inl.gov/sites/default/files/pdf/reports/DatasetVehicleList.pdf) for full list of vehicles). Loggers recorded vehicle operation as they were driven up to 160,000 miles in up to three years of fleet testing. Parameters were logged at 1-second intervals, including - vehicle speed, - engine and/or electric motor speed, - fuel and/or electricity consumption, and - ambient temperature. This dataset includes both raw second-by-second data and trip-level metrics. (This dataset will be shared by API; a small sample of the vehicle and logger data has been extracted and is available for download while the API is being developed.)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA-ISRO Synthetic Aperture Radar (NISAR) Mission

NISAR is a multi-disciplinary Earth-observing radar mission that makes global measurements of land surface changes that will greatly improve Earth system models. NISAR data will clarify spatially and temporally complex phenomena, including ecosystem disturbances, ice sheet collapse, and natural hazards including earthquakes, tsunamis, volcanoes, and landslides. It provides societally relevant data that will enable better protection of life and property. The mission, a NASA-ISRO partnership, uses two fully polarimetric SARs, one at L-band (L-SAR) and one at S-band (S-SAR), in exact repeating orbits every 12 days that allows interferometric combination of data on repeated passes. NASA provides the L SAR; a shared deployable reflector; an engineering payload that supports mission-specific data handling, navigation and communication functions; science observation planning and L SAR data processing. ISRO provides the S-SAR, spacecraft, launch vehicle, satellite operations, and S-SAR data processing. The mission will be launched from the Satish Dhawan Space Centre, Sriharikota, India. Mission development has addressed many unique challenges and incorporates many “firsts” for a jointly-developed free-flyer radar science mission.

Sarma, C V H S↗

DeepFreeze: Towards Scalable Asynchronous Checkpointing of Deep Learning Models

In the age of big data, deep learning has emerged as a powerful tool to extract insight and exploit its value, both in industry and scientific applications. One common pattern emerging in such applications is frequent checkpointing of the state of the learning model during training, needed in a variety of scenarios: analysis of intermediate states to explain features and correlations with training data, exploration strategies involving alternative models that share a common ancestor, knowledge transfer, resilience, etc. However, with increasing size of the learning models and popularity of distributed data-parallel training approaches, simple checkpointing techniques used so far face several limitations: low serialization performance, blocking I/O, stragglers due to the fact that only a single process is involved in checkpointing. This paper proposes a checkpointing technique specifically designed to address the aforementioned limitations, introducing efficient asynchronous techniques to hide the overhead of serialization and I/O, and distribute the load over all participating processes. Experiments with two deep learning applications (CANDLE and ResNet) on a pre-Exascale HPC platform (Theta) shows significant improvement over state-of-art, both in terms of checkpointing duration and runtime overhead.

checkpointing↗

GeneLab: A Systems Biology Platform for Spaceflight Omics Data

NASA's mission includes expanding our understanding of biological systems to improve life on Earth and to enable long-duration human exploration of space. Resources to support large numbers of spaceflight investigations are limited. NASA's GeneLab project is maximizing the science output from these experiments by: (1) developing a unique public bioinformatics database that includes space bioscience relevant "omics" data (genomics, transcriptomics, proteomics, and metabolomics) and experimental metadata; (2) partnering with NASA-funded flight experiments through bio-sample sharing or sample augmentation to expedite omics data input to the GeneLab database; and (3) developing community-driven reference flight experiments. The first database, GeneLab Data System Version 1.0, went online in April 2015. V1.0 contains numerous flight datasets and has search and download capabilities. Version 2.0 will be released in 2016 and will link to analytic tools. In 2015 Genelab partnered with two Biological Research in Canisters experiments (BBRIC-19 and BRIC-20) which examine responses of Arabidopsis thaliana to spaceflight. GeneLab also partnered with Rodent Research-1 (RR1), the maiden flight to test the newly developed rodent habitat. GeneLab developed protocols for maxiumum yield of RNA, DNA and protein from precious RR-1 tissues harvested and preserved during the SpaceX-4 mission, as well as from tissues from mice that were frozen intact during spaceflight and later dissected. GeneLab is establishing partnerships with at least three planned flights for 2016. Organism-specific nationwide Science Definition Teams (SDTs) will define future GeneLab dedicated missions and ensure the broader scientific impact of the GeneLab missions. GeneLab ensures prompt release and open access to all high-throughput omics data from spaceflight and ground-based simulations of microgravity and radiation. Overall, GeneLab will facilitate the generation and query of parallel multi-omics data, and deep curation of metadata for integrative analysis, allowing researchers to uncover cellular networks as observed in systems biology platforms. Consequently, the scientific community will have access to a more complete picture of functional and regulatory networks responsive to the spaceflight environment.. Analysis of GeneLab data will contribute fundamental knowledge of how the space environment affects biological systems, and enable emerging terrestrial benefits resulting from mitigation strategies to prevent effects observed during exposure to space. As a result, open access to the data will foster new hypothesis-driven research for future spaceflight studies spanning basic science to translational science.

proteomics↗

Computer assisted thermal-vacuum testing

In testing complex systems and components under dynamic thermal-vacuum environments, it is desirable to optimize the environment control sequence in order to reduce test duration and cost. This paper describes an approach where a computer is utilized as part of the test control operation. Real time test data is made available to the computer through time-sharing terminals at appropriate time intervals. A mathematical model of the test article and environmental control equipment is then operated on using the real time data to yield current thermal status, temperature analysis, trend prediction and recommended thermal control setting changes to arrive at the required thermal condition. The data acquisition interface and the time-sharing hook-up to an IBM-370 computer is described along with a typical control program and data demonstrating its use.

Petrie, W.↗

Integrating a Microwave Radiometer into Radar Hardware for Simultaneous Data Collection Between the Instruments

The conventional method for integrating a radiometer into radar hardware is to share the RF front end between the instruments, and to have separate IF receivers that take data at separate times. Alternatively, the radar and radiometer could share the antenna through the use of a diplexer, but have completely independent receivers. This novel method shares the radar's RF electronics and digital receiver with the radiometer, while allowing for simultaneous operation of the radar and radiometer. Radars and radiometers, while often having near-identical RF receivers, generally have substantially different IF and baseband receivers. Operation of the two instruments simultaneously is difficult, since airborne radars will pulse at a rate of hundreds of microseconds. Radiometer integration time is typically 10s or 100s of milliseconds. The bandwidth of radar may be 1 to 25 MHz, while a radiometer will have an RF bandwidth of up to a GHz. As such, the conventional method of integrating radar and radiometer hardware is to share the highfrequency RF receiver, but to have separate IF subsystems and digitizers. To avoid corruption of the radiometer data, the radar is turned off during the radiometer dwell time. This method utilizes a modern radar digital receiver to allow simultaneous operation of a radiometer and radar with a shared RF front end and digital receiver. The radiometer signal is coupled out after the first down-conversion stage. From there, the radar transmit frequencies are heavily filtered, and the bands outside the transmit filter are amplified and passed to a detector diode. This diode produces a DC output proportional to the input power. For a conventional radiometer, this level would be digitized. By taking this DC output and mixing it with a system oscillator at 10 MHz, the signal can instead be digitized by a second channel on the radar digital receiver (which typically do not accept DC inputs), and can be down-converted to a DC level again digitally. This unintuitive step allows the digital receiver to sample both the radiometer and radar data at a rapid, synchronized data rate (greater than 1 MHz bandwidth). Once both signals are sampled by the same digital receiver, high-speed quality control can be performed on the radiometer data to allow it to take data simultaneously with the radar. The radiometer data can be blanked during radar transmit, or when the radar return is of a power level high enough to corrupt the radiometer data. Additionally, the receiver protection switches in the RF front end can double as radiometer calibration sources, the short (four-microsecond level) switching periods integrated over many seconds to estimate the radiometer offset. The major benefit of this innovation is that there is minimal impact on the radar performance due to the integration of the radiometer, and the radiometer performance is similarly minimally affected by the radar. As the radar and radiometer are able to operate simultaneously, there is no extended period of integration time loss for the radiometer (maximizing sensitivity), and the radar is able to maintain its full number of pulses (increasing sensitivity and decreasing measurement uncertainty).

McLinden, Matthew↗