Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data processing and standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Tanana River Test Site Model Verification Using the Marine and Hydrokinetic Toolkit (MHKiT)

The marine energy (ME) industry historically lacked a standardized data processing toolkit for common tasks such as data ingestion, quality control, and visualization. The marine and hydrokinetic toolkit (MHKiT) solved this issue by providing a public software deployment (open-source and free) toolkit for the ME industry to store and maintain commonly used functionality for wave, tidal, and river energy. This paper demonstrates an initial model verification study in MHKiT. Using Delft3D, a numerical model of the Tanana River Test Site (TRTS) at Nenana, Alaska was created. Field data from the site was collected using an Acoustic Doppler Current Profiler (ADCP) at the proposed Current Energy Converter (CEC) locations. MHKiT is used to process model simulations from Delft3D and compare them to the transect data from the ADCP measurements at TRTS. The ability to use a single tool to process simulation and field data demonstrates the ease at which the ME industry can obtain results and collaborate across specialties, reducing errors and increasing efficiency.

13 HYDRO ENERGY↗
NASA NTRS1982

Sea-state-related altitude errors in the Seasat radar altimeter

A significant waveheight-dependent bias is identified in Seasat-measured surface heights which cannot be neglected when using Seasat data at accuracies within 10 cm. Ten-second averages of the data from 63 waveform sampling rates in the Seasat radar altimeter were least squares fitted with a six-parameter model, the time-origin parameter of which provides a measure of the position of the actual mean radar waveform relative to the sampling rate and therefore provides altitude measurement corrections. Attention is given to actual altimeter point target response function, waveform sampler noise baseline, waveform sampler gain calibration, attitude angle and sea surface skewness, as effects not accounted for in Seasat altimeter measurements and standard data processing.

Hayne, G. S.↗
NASA NTRS1987

Methodology requirements for intelligent systems architecture

The methodology required for the development of the 'intelligent system architecture' of distributed computer systems which integrate standard data processing capabilities with symbolic processing to provide powerful and highly autonomous adaptive processing capabilities must encompass three elements: (1) a design knowledge capture system, (2) computer-aided engineering, and (3) verification and validation metrics and tests. Emphasis must be put on the earliest possible definition of system requirements and the realistic definition of allowable system uncertainties. Methodologies must also address human factor issues.

Grant, Terry↗
NASA NTRS1986

Ada and knowledge-based systems: A prototype combining the best of both worlds

A software architecture is described which facilitates the construction of distributed expert systems using Ada and selected knowledge based systems. This architecture was utilized in the development of a Knowledge-based Maintenance Expert System (KNOMES) prototype for the Space Station Mobile Service Center (MSC). The KNOMES prototype monitors a simulated data stream from MSC sensors and built-in test equipment. It detects anomalies in the data and performs diagnosis to determine the cause. The software architecture which supports the KNOMES prototype allows for the monitoring and diagnosis tasks to be performed concurrently. The basic concept of this software architecture is named ACTOR (Ada Cognitive Task ORganization Scheme). An individual ACTOR is a modular software unit which contains both standard data processing and artificial intelligence components. A generic ACTOR module contains Ada packages for communicating with other ACTORs and accessing various data sources. The knowledge based component of an ACTOR determines the role it will play in a system. In this prototype, an ACTOR will monitor the MSC data stream.

Brauer, David C.↗
NASA NTRS2003

HEASARC Software Archive

(1) Chandra Archive: SAO has maintained the interfaces through which HEASARC gains access to the Chandra Data Archive. At HEASARC's request, we have implemented an anonymous ftp copy of a major part of the public archive and we keep that archive up-to- date. SAO has participated in the ADEC interoperability working group, establishing guidelines or interoperability standards and prototyping such interfaces. We have provided an NVO-based prototype interface, intending to serve the HEASARC-led NVO demo project. HEASARC's Astrobrowse interface was maintained and updated. In addition, we have participated in design discussions surrounding HEASARC's Caldb project. We have attended the HEASARC Users Group meeting and presented CDA status and developments. (2) Chandra CALDB: SA0 has maintained and expanded the Chandra CALDB by including four new data file types, defining the corresponding CALDB keyword/identification structures. We have provided CALDB upgrades for the public (CIAO) and for Standard Data Processing. Approximately 40 new files have been added to the CALDB in these version releases. There have been in the past year ten of these CALDB upgrades, each with unique index configurations. In addition, with the inputs from software, archive, and calibration scientists, as well as CIAO/SDP software developers, we have defined a generalized expansion of the existing CALDB interface and indexing structure. The purpose of this is to make the CALDB more generally applicable and useful in new and future missions that will be supported archivally by HEASARC. The generalized interface will identify additional configurational keywords and permit more extensive calibration parameter and boundary condition specifications for unique file selection. HEASARC scientists and developers from SAO and GSFC have become involved in this work, which is expected to produce a new interface for general use within the current year. (3) DS9: One of the decisions that came from last year's HEADCC meeting was to make the ds9 image display program the primary vehicle for displaying line graphics (as well as images). The first step required to make this possible was to enhance the line graphics capabilities of ds9. SAO therefore spent considerable effort upgrading ds9 to use Tcl 8.4 so that the BLT line graphics package could be built and imported into ds9 from source code, rather than from a pre-built (and generally outdated) shared library. This task, which is nearly complete, allows us to extend BLT as needed for the HEAD community. Following HEADCC discussion concerning archiving and the display of archived data, we extended ds9 to support full access to many astronomical Web-based archives sites, including HEASARC, MAST, CHANDRA, SKYVIEW, ADS, NED, SIMBAD, IRAS, NVRO, SAO TDC, and FIRST. Using ds9's new internal Web access capabilities, these archives can be accessed via their Web page. FITS images, plots, spectra, and journal abstracts can be referenced, down-loaded, and displayed directly and easily in ds9. For more information, see: http://hea-www.harvard.edu/saord/ds9. Also after the HEADCC discussion concerning region filtering, we extended the Funtools sample implementation of region filtering as described in: http://hea-www.harvard.edu/saord/funtools/regions.html. In particular, we added several new composite regions for event and image filtering, including elliptical and box annuli. We also extended the panda (Pie AND Annulus) region support to include box pandas and elliptical pandas. These new composite regions are especially useful in programs that need to count photons in each separate region using only a single pass through the data. Support for these new regions was added to ds9. In the same vein, we developed new region support for filtering images using simple FITS image masks, i.e. 8-bit or 16-bit FITS images where the value of a pixel is the region id number for that pixel. Other important enhancements to DS9 this year, include supporor multiple world coordinate systems, three dimensional event file binning, image smoothing, region groups and tags, the ability to save images in a number of image formats (such as JPEG, TIFF, PNG, FITS), improvements in support for integrating external analysis tools, and support for the virtual observatory. In particular, a full-featured web browser has been implemented within D S 9 . This provides support for full access to HEASARC archive sites such as SKYVIEW and W3BROWSE, in addition to other astronomical archives sites such as MAST, CHANDRA, ADS, NED, SIMBAD, IRAS, NVRO, SA0 TDC, and FIRST. From within DS9, the archives can be searched, and FITS images, plots, spectra, and journal abstracts can be referenced, downloaded and displayed The web browser provides the basis for the built-in help facility. All DS9 documentation, including the reference manual, FAQ, Know Features, and contact information is now available to the user without the need for external display applications. New versions of DS9 maybe downloaded and installed using this facility. Two important features used in the analysis of high energy astronomical data have been implemented in the past year. The first is support for binning photon event data in three dimensions. By binning the third dimension in time or energy, users are easily able to detect variable x-ray sources and identify other physical properties of their data. Second, a number of fast smoothing algorithms have been implemented in DS9, which allow users to smooth their data in real time. Algorithms for boxcar, tophat, and gaussian smoothing are supported.

White, Nicholas↗
NASA NTRS2019

2018 NISAR Applications Workshop: Forest and Disturbance; Workshop Report

Forest lands cover the globe and are important sources for providing ecosystem services including: carbon sequestration, biodiversity, timber, air and water quality. As such, counties around the world have dedicated programs for managing them. Accurate and timely information concerning the status of these forests (moisture, biomass, disturbance type, etc.) is essential to those Nations’ human and ecological health as well as economy. The joint NASA/US Forest Service workshop focused on arming forest land managers with observations and remote sensing information from the upcoming NASA-ISRO (Indian Space Research Organization) SAR (Synthetic Aperture Radar) (NISAR) satellite mission (expected to launch early 2022). Participants included representatives from different US Federal Agencies, private sector, and non-governmental organizations (NGO) that are key players in facilitating integration of Earth Observations (EO) into forest management and decision support workflows. They included scientists, technicians, and program managers with a responsibility for data acquisition and exploitation such as product development, delivery, and use, and capacity building. Discussions were held over two days to convey the broader forest and disturbance community information needs for various representative participants and programs and to facilitate the delivery of NISAR mission geospatial products and observational capabilities. Case studies were presented to demonstrate the current state of practice in the use of SAR remote sensing for applications of direct importance to forest and disturbance land management community. Eleven organizations presented their information requirements in response to a set of questions provided by the NASA team, then the NASA team responded by describing the degree to which NISAR could meet these requirements. Discussion ensued about needed data product specifications to increase utility (e.g., projection, latency, etc.), tools and capacity building. The general findings of this workshop were that (a) NISAR observations will be particularly useful to the global forest carbon and disturbance monitoring applications, but that certain data product design decisions (projections and radiometric and terrain corrections) need to be considered to increase utility; b) the biomass and disturbance detection algorithms meet many of the community needs, however there are other information products of value (e.g., soil moisture or disturbance classification, not just detection) and all products should be compliant with existing community standards for reporting uncertainty; c) providing SAR education to the community will be key specifically thinking about putting the information first and the SAR theory second, providing a simple guide of standard data processing steps (e.g., dB (decibel) to power conversion and speckle filtering); d) the community needs a user-friendly interface for finding free, archived data over their geographic regions of interest; e) user-friendly tools that connect to open-sources GIS (Global Information System) software (e.g., QGIS (Quantum GIS)) that include a graphical user interface (GUI) for SAR processing that enables both download and cloud processing. To integrate these findings and prepare the community before NISAR launches, it was suggested that there be a dedicated NISAR Forest and Disturbance Applications Working Group (as per the specifications in the NISAR Utilization Plan). After launch, it was decided that the community continue capacity building activities.

Stavros, Natasha↗
NASA NTRS2018

Model-Based Data Integration and Process Standardization Techniques for Fault Management: A Feasibility Study

This paper describes the theory and considerations in the application of model-based techniques to assimilate information from disjoint knowledge sources for performing NASA's Fault Management (FM)-related activities using the TEAMS® toolset. FM consists of the operational mitigation of existing and impending spacecraft failures. NASA's FM directives have both design-phase and operational-phase goals. This paper highlights recent studies by QSI and DST of the capabilities required in the TEAMS® toolset for conducting FM activities with the aim of reducing operating costs, increasing autonomy, and conforming to time schedules. These studies use and extend the analytic capabilities of QSI's TEAMS® toolset to conduct a range of FM activities within a centralized platform.

Haste, Deepak↗
NASA NTRSDate not supplied

The Nasa SRA Process as It Relates to Open-Source Workflows Developed for GeneLab Data Processing

To release open, standards-compliant processed data sets in the Open Science Data Repository (OSDR), the GeneLab Data Processing team works with the scientific community through the OSDR Analysis Working Groups to design and build open-source data processing pipelines. Once baselined internally, these pipelines are wrapped into workflows and published on the NASA GeneLab Data Processing public GitHub repository along with detailed instructions for installation and use. Each workflow must be approved through NASA's Software Release Authorization (SRA) process prior to publishing. However, the SRA process lacks sufficient documentation and clarity regarding which forms are applicable for new open-source software that utilizes publicly available 3rd party tools, and the SRA process can take several months to complete, making sharing software outside of NASA cumbersome and in contradiction with the concept of Open Science. Furthermore, the SRA process was designed as a one-size fits all approach and thus many of the questions asked are not applicable to our open-source workflows. Here we describe the software provided on the NASA GeneLab Data Processing GitHub repository, summarize our experiences with the SRA process to release these software, and propose a more stream-lined approach for review of open-source projects.

Software Release Authorization↗
NASA NTRSDate not supplied

The NASA SRA Process as it Relates to Open-Source Workflows Developed for GeneLab Data Processing

To release open, standards-compliant processed data sets in the Open Science Data Repository (OSDR), the GeneLab Data Processing team works with the scientific community through the OSDR Analysis Working Groups to design and build open-source data processing pipelines. Once baselined internally, these pipelines are wrapped into workflows and published on the NASA GeneLab Data Processing public GitHub repository along with detailed instructions for installation and use. Each workflow must be approved through NASA's Software Release Authorization (SRA) process prior to publishing. However, the SRA process lacks sufficient documentation and clarity regarding which forms are applicable for new open-source software that utilizes publicly available 3rd party tools, and the SRA process can take several months to complete, making sharing software outside of NASA cumbersome and in contradiction with the concept of Open Science. Furthermore, the SRA process was designed as a one-size fits all approach and thus many of the questions asked are not applicable to our open-source workflows. Here we describe the software provided on the NASA GeneLab Data Processing GitHub repository, summarize our experiences with the SRA process to release these software, and propose a more stream-lined approach for review of open-source projects.

Software Release Authorization↗
DOE OSTI2022

PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology

PDBx/mmCIF, Protein Data Bank Exchange (PDBx) macromolecular Crystallographic Information Framework (mmCIF), has become the data standard for structural biology. With its early roots in the domain of small-molecule crystallography, PDBx/mmCIF provides an extensible data representation that is used for deposition, archiving, remediation, and public dissemination of experimentally determined three-dimensional (3D) structures of biological macromolecules by the Worldwide Protein Data Bank (wwPDB, wwpdb.org). Extensions of PDBx/mmCIF are similarly used for computed structure models by ModelArchive (modelarchive.org), integrative/hybrid structures by PDB-Dev (pdb-dev.wwpdb.org), small angle scattering data by Small Angle Scattering Biological Data Bank SASBDB (sasbdb.org), and for models computed generated with the AlphaFold 2.0 deep learning software suite (alphafold.ebi.ac.uk). Community-driven development of PDBx/mmCIF spans three decades, involving contributions from researchers, software and methods developers in structural sciences, data repository providers, scientific publishers, and professional societies. Having a semantically rich and extensible data framework for representing a wide range of structural biology experimental and computational results, combined with expertly curated 3D biostructure data sets in public repositories, accelerates the pace of scientific discovery. Herein, we describe the architecture of the PDBx/mmCIF data standard, tools used to maintain representations of the data standard, governance, and processes by which data content standards are extended, plus community tools/software libraries available for processing and checking the integrity of PDBx/mmCIF data. Use cases exemplify how the members of the Worldwide Protein Data Bank have used PDBx/mmCIF as the foundation for its pipeline for delivering Findable, Accessible, Interoperable, and Reusable (FAIR) data to many millions of users worldwide.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1981

Frequency standard test facility data acquisition and processing system

The interim frequency standards test facility tests all frequency standards used in the DSN. It is also the central testing facility for testing all ultrastable hydrogen maser frequency standards. This proposed data acquisition/processing system was designed to support the IFSTF by acquiring and processing the large amounts of data generated there.

Knapp, L. J.↗
NASA NTRS1989

Standard services for the capture, processing, and distribution of packetized telemetry data

Standard functional services for the capture, processing, and distribution of packetized data are discussed with particular reference to the future implementation of packet processing systems, such as those for the Space Station Freedom. The major functions are listed under the following major categories: input processing, packet processing, and output processing. A functional block diagram of a packet data processing facility is presented, showing the distribution of the various processing functions as well as the primary data flow through the facility.

Stallings, William H.↗
NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan Dejesus Oribello↗
NASA NTRSDate not supplied

NASA GeneLab RNASeq Consensus Pipeline: A Nextflow Implementation

The NASA GeneLab project (genelab.nasa.gov) seeks to accelerate space biology research through cataloging and democratizing omics data. Since raw omics data is largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data has greater immediate value to a wide range of users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. Previously, GeneLab developed a standardized pipeline for processing RNAseq data, referred to as the ‘GeneLab RNAseq Consensus Pipeline (RCP)’, in collaboration with GeneLab’s Analysis Working Groups. The work presented here is a Nextflow implementation of GeneLab’s RCP that automates and accelerates data processing of RNASeq datasets hosted on GeneLab. In addition to the core data processing, the workflow also includes staging of GeneLab raw data and a robust verification and validation (V&V) program that runs after each processing step to identify errors in real-time, stop additional downstream computation, and preserve computational resources. The workflow, including the staging and V&V functionality, is open source for others to reuse and modify at https://github.com/nasa/GeneLab_Data_Processing/tree/master/RNAseq.

Jonathan D Oribello↗
NASA NTRSDate not supplied

Transcriptomics Processing Pipelines for Space Biology: An Open Source and Consensus-Driven Approach

Transcriptomics holds significant value in elucidating the relationship between gene expression, experimental factors, biological factors, and various types of omics data. Enhancing our understanding of these connections is paramount for foundational biology, which plays a pivotal role in devising solutions for challenges pertinent to both space travel and terrestrial life. The NASA GeneLab project, part of the Open Science Data Repository (OSDR.nasa.gov), seeks to accelerate space biology research through cataloging and democratizing ‘omics data, including transcriptomics. Since raw omics data are largely inaccessible to non-bioinformaticians, GeneLab works with the scientific community via the Open Science Analysis Working Groups (AWGs) to develop standard processing pipelines to generate and publish processed data. Unlike raw data, processed data have greater immediate value to diverse users with varying technical backgrounds and computational capabilities. Standardizing processing workflows is essential to match the pace of raw data generation, ensure reproducibility, and enable standardized processed data for comparison across datasets. As of June 2023, transcriptomics studies comprise over half of GeneLab datasets hosted on the OSDR, including data from bulk RNA-seq and Affymetrix or Agilent 1-Channel DNA microarray assays. In collaboration with the AWGs, GeneLab developed consensus processing pipelines for these transcriptomics data types that includes quality control, background correction (microarray only), data normalization and quantification, culminating in the detection and annotation of differentially expressed genes. The work presented here describes Nextflow implementations of GeneLab’s consensus transcriptomics pipelines that automates and accelerates processing of these datasets. In addition to the core data processing, these workflows also include raw data staging and a robust verification and validation program to identify errors in real-time, stop additional downstream computation, and preserve computational resources. These workflows are used to generate GeneLab processed data hosted on the OSDR, and are publicly available as open source software for others to use at: https://github.com/nasa/GeneLab_Data_Processing.

Jonathan Oribello↗
DOE OSTI2023

Empowering Scientific Discovery Through Computing at the Advanced Photon Source

This paper explores the challenges and solutions for managing and processing the vast amount of data generated by the Advanced Photon Source (APS), a synchrotron light source facility producing ultra-bright x-rays for diverse scientific domains. With 68 experimental beamlines covering materials research, biology, and more, the APS serves a wide user base across academia, government, and industry. The ongoing upgrade of the APS storage ring and installation of new instruments will amplify data generation and processing demands. This paper discusses the approach to address these demands through automated data processing using standardized workflows that produce faster scientific insights. The APS Data Management System coordinates various data related tasks to manage storage, data transfer, metadata cataloging, data processing, and interfaces with tools provided by Globus. Through integration with the Argonne Leadership Computing Facility (ALCF), APS users can efficiently access high-performance computing resources. Standardized workflows have led to reduced computational burdens on scientists and greater accessibility of high performance computing resources. We demonstrate how standardization and collaboration enable scientists to rapidly convert raw data into meaningful scientific results, establishing a streamlined path from data collection to analysis and ultimately to publication.

Parraga, Hannah↗