SEARCH · Engineering Papers

Results for “pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

NASA NTRS2012

Enabling Earth Science Through Cloud Computing

Cloud Computing holds tremendous potential for missions across the National Aeronautics and Space Administration. Several flight missions are already benefiting from an investment in cloud computing for mission critical pipelines and services through faster processing time, higher availability, and drastically lower costs available on cloud systems. However, these processes do not currently extend to general scientific algorithms relevant to earth science missions. The members of the Airborne Cloud Computing Environment task at the Jet Propulsion Laboratory have worked closely with the Carbon in Arctic Reservoirs Vulnerability Experiment (CARVE) mission to integrate cloud computing into their science data processing pipeline. This paper details the efforts involved in deploying a science data system for the CARVE mission, evaluating and integrating cloud computing solutions with the system and porting their science algorithms for execution in a cloud environment.

science data system↗

NASA NTRS2015

High-Energy Electromagnetic Offline Follow-Up of Ligo-Virgo Gravitational-Wave Binary Coalescence Candidate Events

We present two different search methods for electromagnetic counterparts to gravitational-wave (GW) events from ground-based detectors using archival NASA high-energy data from the Fermi Gamma-ray Burst Monitor (GBM) and RXTE All-sky Monitor (ASM) instruments. To demonstrate the methods, we use a limited number of representative GW background noise events produced by a search for binary neutron star coalescence over the last two months of the LIGO-Virgo S6/VSR3 joint science run. Time and sky location provided by the GW data trigger a targeted search in the high-energy photon data. We use two custom pipelines: one to search for prompt gamma-ray counterparts in GBM, and the other to search for a variety of X-ray afterglow model signals in ASM. We measure the efficiency of the joint pipelines to weak gamma-ray burst counterparts, and a family of model X-ray afterglows. By requiring a detectable signal in either electromagnetic instrument coincident with a GW event, we are able to reject a large majority of GW candidates. This reduces the signal-to-noise ratio of the loudest surviving GW background event by around 15-20 percent.

gamma-ray burst: general↗

NASA NTRS2015

Technology Cost and Schedule Estimation (TCASE) Final Report

During the 2014-2015 project year, the focus of the TCASE project has shifted from collection of historical data from many sources to securing a data pipeline between TCASE and NASA's widely used TechPort system. TCASE v1.0 implements a data import solution that was achievable within the project scope, while still providing the basis for a long-term ability to keep TCASE in sync with TechPort. Conclusion: TCASE data quantity is adequate and the established data pipeline will enable future growth. Data quality is now highly dependent the quality of data in TechPort. Recommendation: Technology development organizations within NASA should continue to work closely with project/program data tracking and archiving efforts (e.g. TechPort) to ensure that the right data is being captured at the appropriate quality level. TCASE would greatly benefit, for example, if project cost/budget information was included in TechPort in the future.

Wallace, Jon↗

NASA NTRS2016

Laboratory Testing and Performance Verification of the CHARIS Integral Field Spectrograph

The Coronagraphic High Angular Resolution Imaging Spectrograph (CHARIS) is an integral field spectrograph (IFS) that has been built for the Subaru telescope. CHARIS has two imaging modes; the high-resolution mode is R82, R69, and R82 in J, H, and K bands respectively while the low-resolution discovery mode uses a second low-resolution prism with R19 spanning 1.15-2.37 microns (J+H+K bands). The discovery mode is meant to augment the low inner working angle of the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) adaptive optics system, which feeds CHARIS a coronagraphic image. The goal is to detect and characterize brown dwarfs and hot Jovian planets down to contrasts five orders of magnitude dimmer than their parent star at an inner working angle as low as 80 milliarcseconds. CHARIS constrains spectral crosstalk through several key aspects of the optical design. Additionally, the repeatability of alignment of certain optical components is critical to the calibrations required for the data pipeline. Specifically the relative alignment of the lens let array, prism, and detector must be highly stable and repeatable between imaging modes. We report on the measured repeatability and stability of these mechanisms, measurements of spectral crosstalk in the instrument, and the propagation of these errors through the data pipeline. Another key design feature of CHARIS is the prism, which pairs Barium Fluoride with Ohara L-BBH2 high index glass. The dispersion of the prism is significantly more uniform than other glass choices, and the CHARIS prisms represent the first NIR astronomical instrument that uses L-BBH2as the high index material. This material choice was key to the utility of the discovery mode, so significant efforts were put into cryogenic characterization of the material. The final performance of the prism assemblies in their operating environment is described in detail. The spectrograph is going through final alignment, cryogenic cycling, and is being delivered to the Subaru telescope in April 2016. This paper is a report on the laboratory performance of the spectrograph, and its current status in the commissioning process so that observers will better understand the instrument capabilities. We will also discuss the lessons learned during the testing process and their impact on future high-contrast imaging spectrographs for wavefront control.

Coronagraphic High Angular Resolution Imaging Spec↗

NASA NTRS2016

Erasing the Milky Way: New Cleaning Technique Applied to GBT Intensity Mapping Data

We present the first application of a new foreground removal pipeline to the current leading HI intensity mapping dataset, obtained by the Green Bank Telescope (GBT). We study the 15- and 1-h field data of the GBT observations previously presented in Masui et al. (2013) and Switzer et al. (2013), covering about 41 square degrees at 0.6 less than z is less than 1.0, for which cross-correlations may be measured with the galaxy distribution of the WiggleZ Dark Energy Survey. In the presented pipeline, we subtract the Galactic foreground continuum and the point source contamination using an independent component analysis technique (fastica), and develop a Fourier-based optimal estimator to compute the temperature power spectrum of the intensity maps and cross-correlation with the galaxy survey data. We show that fastica is a reliable tool to subtract diffuse and point-source emission through the non-Gaussian nature of their probability distributions. The temperature power spectra of the intensity maps is dominated by instrumental noise on small scales which fastica, as a conservative sub-traction technique of non-Gaussian signals, can not mitigate. However, we determine similar GBT-WiggleZ cross-correlation measurements to those obtained by the Singular Value Decomposition (SVD) method, and confirm that foreground subtraction with fastica is robust against 21cm signal loss, as seen by the converged amplitude of these cross-correlation measurements. We conclude that SVD and fastica are complementary methods to investigate the foregrounds and noise systematics present in intensity mapping datasets.

Wolz, L.↗

NASA NTRS2017

Kepler Planet Detection Metrics: Window and One-Sigma Depth Functions for Data Release 25

This document describes the window and one-sigma depth functions relevant to the Transiting Planet Search (TPS) algorithm in the Kepler pipeline (Jenkins 2002; Jenkins et al. 2017). The window function specifies the fraction of unique orbital ephemeris epochs over which three transits are observable as a function of orbital period. In this context, the epoch and orbital period, together, comprise the ephemeris of an orbiting companion, and ephemerides with the same period are considered equivalent if their epochs differ by an integer multiple of the period. The one-sigma depth function specifies the depth of a signal (in ppm) for a given light curve that results in a one-sigma detection of a transit signature as a function of orbital period when averaged over all unique orbital ephemerides. These planet detection metrics quantify the ability of TPS to detect a transiting planet signature on a star-by-star basis. They are uniquely applicable to a specific Kepler data release, since they are dependent on the details of the light curves searched and the functionality of the TPS algorithm used to perform the search. This document describes the window and one-sigma depth functions relevant to Kepler Data Release 25 (DR25), where the data were processed (Thompson et al. 2016) and searched (Twicken et al. 2016) with the SOC 9.3 pipeline. In Section 4, we describe significant differences from those reported in Kepler Data Release 24 (Burke Seader 2016) and document our verification method.

Planet Detection Metrics↗

NASA NTRS2016

Processing and Managing the Kepler Mission's Treasure Trove of Stellar and Exoplanet Data

The Kepler telescope launched into orbit in March 2009, initiating NASAs first mission to discover Earth-size planets orbiting Sun-like stars. Kepler simultaneously collected data for 160,000 target stars at a time over its four-year mission, identifying over 4700 planet candidates, 2300 confirmed or validated planets, and over 2100 eclipsing binaries. While Kepler was designed to discover exoplanets, the long term, ultra- high photometric precision measurements it achieved made it a premier observational facility for stellar astrophysics, especially in the field of asteroseismology, and for variable stars, such as RR Lyraes. The Kepler Science Operations Center (SOC) was developed at NASA Ames Research Center to process the data acquired by Kepler from pixel-level calibrations all the way to identifying transiting planet signatures and subjecting them to a suite of diagnostic tests to establish or break confidence in their planetary nature. Detecting small, rocky planets transiting Sun-like stars presents a variety of daunting challenges, from achieving an unprecedented photometric precision of 20 parts per million (ppm) on 6.5-hour timescales, supporting the science operations, management, processing, and repeated reprocessing of the accumulating data stream. This paper describes how the design of the SOC meets these varied challenges, discusses the architecture of the SOC and how the SOC pipeline is operated and is run on the NAS Pleiades supercomputer, and summarizes the most important pipeline features addressing the multiple computational, image and signal processing challenges posed by Kepler.

high performance computing↗

NASA NTRS2017

GEONEX: Land Monitoring From a New Generation of Geostationary Satellite Sensors

The latest generation of geostationary satellites carry sensors such as ABI (Advanced Baseline Imager on GOES-16) and the AHI (Advanced Himawari Imager on Himawari) that closely mimic the spatial and spectral characteristics of Earth Observing System flagship MODIS for monitoring land surface conditions. More importantly they provide observations at 5-15 minute intervals. Such high frequency data offer exciting possibilities for producing robust estimates of land surface conditions by overcoming cloud cover, enabling studies of diurnally varying local-to-regional biosphere-atmosphere interactions, and operational decision-making in agriculture, forestry and disaster management. But the data come with challenges that need special attention. For instance, geostationary data feature changing sun angle at constant view for each pixel, which is reciprocal to sun-synchronous observations, and thus require careful adaptation of EOS algorithms. Our goal is to produce a set of land surface products from geostationary sensors by leveraging NASA's investments in EOS algorithms and in the data/compute facility NEX. The land surface variables of interest include atmospherically corrected surface reflectances, snow cover, vegetation indices and leaf area index (LAI)/fraction of photosynthetically absorbed radiation (FPAR), as well as land surface temperature and fires. In order to get ready to produce operational products over the US from GOES-16 starting 2018, we have utilized 18 months of data from Himawari AHI over Australia to test the production pipeline and the performance of various algorithms for our initial tests. The end-to-end processing pipeline consists of a suite of modules to (a) perform calibration and automatic georeference correction of the AHI L1b data, (b) adopt the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to produce surface spectral reflectances along with compositing schemes and QA, and (c) modify relevant EOS retrieval algorithms (e.g., LAI and FPAR, GPP, etc.) for subsequent science product generation. Initial evaluation of Himawari AHI products against standard MODIS products indicate general agreement, suggesting that data from geostationary sensors can augment low earth orbit (LEO) satellite observations.

geostationary↗

NASA NTRS2018

GEONEX: Webpage to Display NASA-NOAA Collaboration of Producing Land Surface Products from Geostationary Sensors

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline has been deployed on Amazon Web Services cloud platform and it currently leverages near-realtime geostationary data hosted in AWS public datasets under a NOAA-AWS agreement.In order to better introduce the GEONEX products to the science community, we set up a simple webpage (www.geonex.org) to describe the background and the motivation of the project, the algorithms used in deriving the products, and user manuals to the data files. We will also update the status of the data processing, in particular the near-real-time products, on the website and provide links (in text or json files) to the latest datasets.

geostationary↗

NASA NTRS2018

GEONEX: Challenges in Producing MODIS-Like Land Products from a New Generation of Geostationary Sensors

The new generation geostationary (GEO) remote sensors (GOES-R ABI, Himawari AHI, and FY4 AGRI) provide high frequency (5-15 minute) observations spatially/spectrally similar to MODIS/VIIRS for land monitoring. These new features of GEO satellite sensors make producing MODIS like land products for terrestrial monitoring possible. The NASA Earth Exchange (NEX) team developed the GEONEX pipeline that is containerized, deployable on NASA Pleiades supercomputer as well as public cloud platforms (e.g. AWS). The processing pipeline is designed to take Himawari Standard Data (HSD) and GOES-16 L1b to generate surface reflectance (SR) and other high-level land remote sensing products. In order to produce low-Earth-orbiting (LEO) remote sensing compatible land products, inter-comparison between Himawari AHI and MODIS Terra/Aqua has been conducted in this research work. Comparisons of TOA reflectance and surface reflectance between AHI and Terra/Aqua are presented. Ray-Matching method was used to locate the co-located pixels, where GEO and LEO sensors look at the land target with similar Viewing Zenith Angle (VZA) and Viewing Azimuth Angle (VAA) simultaneously. Here, we address challenges associated with the selection of qualified pixels of similar solar illumination condition and atmosphere path. We used strict criterion to constrain the pixel selection: the time difference between GEO and LEO observations is less than +-2.5 mins, the cosine of VZA difference is less than 1%, and the VAA difference is less than 10 deg. We also discuss the strong radiometric consistency that the new generation GEO sensors along with the popular LEO sensors would benefit the environmental remote sensing community.

Li, Shuang↗

NASA NTRS2015

Supercomputing in the Age of Discovering Superearths, Earths and Exoplanet Systems

With the Kepler Mission, exoplanet science has entered the era of Big Data and has benefited tremendously from the availability of the NAS (NASA Advanced Supercomputing Division) Pleiades supercomputer. This presentation provides an overview of the history of exoplanet discoveries and highlights the key enabling technologies for Kepler. I provide an overview of the Kepler science pipeline and the algorithms used to analyze the data and detect transiting planets, all of which run routinely on the NAS Pleiades system. The presentation also provides a description of the Blender analysis program used to validate Kepler planet discoveries statistically, and the effort to discover exomoons lurking in Kepler data, both of which also run on Pleiades. Finally, I describe NASA's TESS (Transiting Exoplanet Survey Satellite) Mission, which leverages Kepler heritage to provide a science pipeline similar to that for Kepler and that will also run routinely on the NAS Pleiades supercomputer.

big data↗

NASA NTRS2015

A Data Exploration Tool for Large Sets of Spectra

We present an exploration tool for very large spectrum data sets such as the SDSS (Sloan Digital Sky Survey), LAMOST (Large Sky Area Multi-Object Fiber Spectroscopic Telescope), and 4MOST (4-meter Multi-Object Spectroscopic Telescope) data sets. The tool works in two stages: the first uses batch processing and the second runs interactively. The latter employs the NASA hyperwall, a configuration of 128 workstation displays (8 by 16 array) controlled by a parallelized software suite running on NASA's Pleiades supercomputer. The stellar subset of the Sloan Digital Sky Survey, DR10, was chosen to show how the our tool may be used. In stage one, SDSS files for 569,740 stars are processed through our data pipeline. The pipeline fits each spectrum using an iterative continuum algorithm, distinguishing emission from absorption and handling molecular absorption bands correctly. It then measures 1659 discrete atomic and molecular spectral features that were carefully preselected based on their likelihood of being visible at some spectral type. The depths relative to the local continuum at each feature wavelength are determined for each spectrum: these depths, the local S/N (signal to noise ratio) level, and DR10-supplied variables such as magnitudes, colors, positions, and radial velocities are the basic measured quantities used on the hyperwall. In stage two, each hyperwall panel is used to display a 2-D scatter plot showing the depth of feature A vs the depth of feature B for all of the stars. A and B change from panel to panel. The relationships between the various (A,B) strengths and any distinctive clustering are immediately apparent when examining and inter-comparing the different panels on the hyperwall. The interactive software allows the user to select the stars in any interesting region of any 2-D plot on the hyperwall, immediately rendering the same stars on all the other 2-D plots in a unique color. The process may be repeated multiple times, each selection displaying a distinctive color on all the plots. At any time, the spectra of the selected stars may be examined in detail on a connected workstation display. We illustrate how our approach allows us to quickly isolate and examine such interesting stellar subsets as EMP (Extremely Metal‐Poor) stars, CV (Cataclymic Variable) stars and C (Carbon)-rich stars.

Data Exploration↗

NASA NTRS2019

Understanding Heating in Active Region Cores through Machine Learning. I. Numerical Modeling and Predicted Observables

To adequately constrain the frequency of energy deposition in active region cores in the solar corona, systematic comparisons between detailed models and observational data are needed. In this paper, we describe a pipeline for forward modeling active region emission using magnetic field extrapolations and field-aligned hydrodynamic models. We use this pipeline to predict time-dependent emission from active region NOAA 1158 for low-, intermediate-, and high-frequency nanoflares. In each pixel of our predicted multi-wavelength, time-dependent images, we compute two commonly used diagnostics: the emission measure slope and the time lag. We find that signatures of the heating frequency persist in both of these diagnostics. In particular, our results show that the distribution of emission measure slopes narrows and the mean decreases with decreasing heating frequency and that the range of emission measure slopes is consistent with past observational and modeling work. Furthermore, we find that the time lag becomes increasingly spatially coherent with decreasing heating frequency while the distribution of time lags across the whole active region becomes more broad with increasing heating frequency. In a follow-up paper, we train a random forest classifier on these predicted diagnostics and use this model to classify real observations of NOAA 1158 in terms of the underlying heating frequency.

UV radiation↗

NASA NTRS2019

GEONEX Data Products: Geostationary Satellite Derived Land Surface and Atmospheric Public Data Products

The latest generation of geostationary satellites carry sensors such as the Advanced Baseline Imager (GOES-16/17) and the Advanced Himawari Imager (Himawari-8/9) that closely mimic the spatial and spectral characteristics of MODIS and VIIRS, useful for monitoring land surface conditions. The NASA Earth Exchange (NEX) team at Ames Research Center has embarked on a collaborative effort among scientists from NASA and NOAA exploring the feasibility of producing operational land surface products similar to those from MODIS/VIIRS. The team built a processing pipeline called GEONEX that is capable of converting raw geostationary data into routine products of Fires, surface reflectances, vegetation indices, LAI/FPAR, ET and GPP/NPP using algorithms adapted from both NASA/EOS and NOAA/GOES-R programs. The GEONEX pipeline will begin to produce provisional data products to be consumed by external collaborators and the academic community. In order to better inform and introduce the GEONEX products to the science community, the provisional products shall be distributed from the NAS data portal, located at data.nas.nasa.gov, and simple webpage at www.nasa.gov/geonex has been deployed, which describes any algorithms used in deriving the products, user manuals and data file information. We will also update the status of the data processing, on the website and provide links to the latest datasets, and use a geonex mailing list, using lists.nasa.gov.

Wang, Weile↗

NASA NTRS2018

Automated Semantic Segmentation for Volumetric Cardiovascular Feature Quantification and Pathology Assessment

We present a pipeline method that curtails the expense and observer bias of manual cardiac evaluation by combining semantic segmentation and disease classification as a fully automatic processing pipeline. The initial element consists of a 2D U-Net convolutional neural network architecture for voxel-wise segmentation of the myocardium and ventricular cavities. The results of the segmentation were used to compute a comprehensive volumetric feature matrix that captured diagnostic clinical procedure data and that was used to model a cardiac pathology classifier.Our approach evaluated anonymized parasternal MRI cardiac images from a database of 100 patients (4 pathology groups, 1 healthy group, 20 patients per group) examined at the University Hospital of Dijon. We achieved top average Dice index scores of 0.939, 0.849, 0.886 for structure segmentation of the left ventricle (LV), right ventricle (RV) and myocardium respectively. A 5-ary pathology classification accuracy of 90% was recorded on an independent test set using our trained model.

Lindsey, Tony↗

NASA NTRS2019

GeneLab: Overview of Challenges and Opportunities

The NASA GeneLab project capitalizes on multi-omic technologies to maximize the return on spaceflight experiments. To do this, GeneLab maintains a publicly accessible database (GLDS) that houses spaceflight and spaceflight relevant multi-omics data, and collaborates with NASA principal investigators and projects to generate additional omics data. GeneLab houses more than 200 transcriptomic, proteomic, metabolomic and epigenomic datasets from plant, animal and microbial experiments, with a growing number of these having been produced by the GeneLab sample processing lab. The GLDS contains rich metadata about each experiment and has recently integrated radiation dosimetery data from experiments flown on the Space Shuttle. GeneLab has also recently implemented an effort to present processed data in the GLDS in addition to the raw omics data. The processed data will enable interpretation of the data by a larger group of students, scientists and the general public. Standard pipelines for the transformation of raw data into visualizations were developed by four GeneLab Analysis Working Groups (animals, plants, microbes, multi-omics) comprised of over 100 scientists from NASA and academia. These pipelines are now being used by a group of bioinformatics interns to provide standard basic analysis of the data for incorporation into GLDS.

Galazka, Jonathan M.↗

NASA NTRS2019

GeneLab: Open Science for Life in Space

Galazka, Jonathan M.↗

NASA NTRSDate not supplied

GeoNEX: A geostationary earth observatory at NASA Earth eXchange: Earth monitoring from operational geostationary satellite systems

The latest generation of geostationary satellites (Himawari 8/9, GOES-16/17, FY-4, GK-2A) carries sensors that closely mimic the spatial and spectral characteristics of widely used polar-orbiting, global monitoring sensors such as MODIS and VIIRS. When combined, data from various currently operating/planned geostationary platforms provide a geo-ring of hyper-temporal (5-10 minutes), multispectral observations at spatial resolutions as high as 500 m. These high frequency observations offer exciting new possibilities for monitoring our planet, including better retrievals of geophysical variables by overcoming cloud cover, enabling studies of diurnally varying phenomena in the atmosphere, land, and the oceans, and support operational decision-making in agriculture, hydrology and disaster management. The NASA Earth Exchange (NEX) team, in collaboration with scientists from JAXA, KARI, NOAA and other international institutions, created the GeoNEX (www.nasa.gov/geonex) pipeline to integrate data from all available geostationary platforms and produce and distribute spatially, temporally, and radiometrically consistent data for the earth science community. We envision various institutions adapting the Geo component (e.g., GeoNOAA, GeoKARI, GeoChiba, GeoJAXA, GeoCMA) and customizing the pipeline and downstream products to serve the local/regional research and applied science communities.

Ramakrishna R Nemani↗