Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data pipeline”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

A statistical-based scheduling algorithm in automated data path synthesis

In this paper, we propose a new heuristic scheduling algorithm based on the statistical analysis of the cumulative frequency distribution of operations among control steps. It has a tendency of escaping from local minima and therefore reaching a globally optimal solution. The presented algorithm considers the real world constraints such as chained operations, multicycle operations, and pipelined data paths. The result of the experiment shows that it gives optimal solutions, even though it is greedy in nature.

Jeon, Byung Wook↗
NASA NTRS2001

Processors, Pipelines, and Protocols for Advanced Modeling Networks

Predictive capabilities arise from our understanding of natural processes and our ability to construct models that accurately reproduce these processes. Although our modeling state-of-the-art is primarily limited by existing computational capabilities, other technical areas will soon present obstacles to the development and deployment of future predictive capabilities. Advancement of our modeling capabilities will require not only faster processors, but new processing algorithms, high-speed data pipelines, and a common software engineering framework that allows networking of diverse models that represent the many components of Earth's climate and weather system. Development and integration of these new capabilities will pose serious challenges to the Information Systems (IS) technology community. Designers of future IS infrastructures must deal with issues that include performance, reliability, interoperability, portability of data and software, and ultimately, the full integration of various ES model systems into a unified ES modeling network.

Coughlan, Joseph↗
NASA NTRS2012

Using Quality Attributes to Bridge Systems Engineering Gaps : A Juno Ground Data Systems Case Study

The Juno Mission to Jupiter is the second mission selected by the NASA New Frontiers Program. Juno launched August 2011 and will reach Jupiter July 2016. Juno's payload system is composed of nine instruments plus a gravity science experiment. One of the primary functions of the Juno Ground Data System (GDS) is the assembly and distribution of the CFDP (CCSDS File Delivery Protocol) product telemetry, also referred to as raw science data, for eight out of the nine instruments. The GDS accomplishes this with the Instrument Data Pipeline (IDP). During payload integration, the first attempt to exercise the IDP in a flight like manner revealed that although the functional requirements were well understood, the system was unable to meet latency requirements with the as-is heritage design. A systems engineering gap emerged between Juno instrument data delivery requirements and the assumptions behind the heritage flight-ground interactions. This paper describes the use of quality attributes to measure and overcome this gap by introducing a new systems engineering activity, and a new monitoring service architecture that successfully delivered the performance metrics needed to validate Juno IDP.

Ground Data Systems (GDS)↗
NASA NTRS1999

SIRTF Science Operations System Design

SIRTF Science Operations System Design William B. Green Manager, SIRTF Science Center California Institute of Technology M/S 310-6 1200 E. California Blvd., Pasadena CA 91125 (626) 395 8572 Fax (626) 568 0673 bgreen@ipac.caltech.edu. The Space Infrared Telescope Facility (SIRTF) will be launched in December 2001, and perform an extended series of science observations at wavelengths ranging from 20 to 160 microns for five years or more. The California Institute of Technology has been selected as the home for the SIRTF Science Center (SSC). The SSC will be responsible for evaluating and selecting observation proposals, providing technical support to the science community, performing mission planning and science observation scheduling activities, instrument calibration during operations and instrument health monitoring, production of archival quality data products, and management of science research grants. The science payload consists of three instruments delivered by instrument Principal Investigators located at University of Arizona, Cornell, and Harvard Smithsonian Astrophysical Observatory. The SSC is responsible for design, development, and operation of the Science Operations System (SOS) which will support the functions assigned to the SSC by NASA. The SIRTF spacecraft, mission profile, and science instrument design have undergone almost ten years of refinement. SIRTF development and operations activities are highly cost constrained. The cost constraints have impacted the design of the SOS in several ways. The Science Operations System has been designed to incorporate a set of efficient, easy to use tools which will make it possible for scientists to propose observation sequences in a rapid and automated manner. The use of highly automated tools for requesting observations will simplify the long range observatory scheduling process, and the short term scheduling of science observations. Pipeline data processing will be highly automated and data-driven, utilizing a variety of tools developed at JPL, the instrument development teams, and Space Telescope Science Institute to automate processing. An incremental ground data system development approach has been adopted, featuring periodic deliveries that are validated with the flight hardware throughout the various phases of system level development and testing. This approach minimizes development time and decreases operations risk. This paper will describe the top level architecture of the SOS and the basic design concepts. A summary of the incremental development approach will be presented. Examples of the unique science user tools now under final development prior to the first proposal call scheduled for mid-2000 will be shown.

Green, William↗
NASA NTRS2015

A Data Exploration Tool for Large Sets of Spectra

We present an exploration tool for very large spectrum data sets such as the SDSS (Sloan Digital Sky Survey), LAMOST (Large Sky Area Multi-Object Fiber Spectroscopic Telescope), and 4MOST (4-meter Multi-Object Spectroscopic Telescope) data sets. The tool works in two stages: the first uses batch processing and the second runs interactively. The latter employs the NASA hyperwall, a configuration of 128 workstation displays (8 by 16 array) controlled by a parallelized software suite running on NASA's Pleiades supercomputer. The stellar subset of the Sloan Digital Sky Survey, DR10, was chosen to show how the our tool may be used. In stage one, SDSS files for 569,740 stars are processed through our data pipeline. The pipeline fits each spectrum using an iterative continuum algorithm, distinguishing emission from absorption and handling molecular absorption bands correctly. It then measures 1659 discrete atomic and molecular spectral features that were carefully preselected based on their likelihood of being visible at some spectral type. The depths relative to the local continuum at each feature wavelength are determined for each spectrum: these depths, the local S/N (signal to noise ratio) level, and DR10-supplied variables such as magnitudes, colors, positions, and radial velocities are the basic measured quantities used on the hyperwall. In stage two, each hyperwall panel is used to display a 2-D scatter plot showing the depth of feature A vs the depth of feature B for all of the stars. A and B change from panel to panel. The relationships between the various (A,B) strengths and any distinctive clustering are immediately apparent when examining and inter-comparing the different panels on the hyperwall. The interactive software allows the user to select the stars in any interesting region of any 2-D plot on the hyperwall, immediately rendering the same stars on all the other 2-D plots in a unique color. The process may be repeated multiple times, each selection displaying a distinctive color on all the plots. At any time, the spectra of the selected stars may be examined in detail on a connected workstation display. We illustrate how our approach allows us to quickly isolate and examine such interesting stellar subsets as EMP (Extremely Metal‐Poor) stars, CV (Cataclymic Variable) stars and C (Carbon)-rich stars.

Data Exploration↗
NASA NTRS1983

Parallel processing in a host plus multiple array processor system for radar

Host plus multiple array processor architecture is demonstrated to yield a modular, fast, and cost-effective system for radar processing. Software methodology for programming such a system is developed. Parallel processing with pipelined data flow among the host, array processors, and discs is implemented. Theoretical analysis of performance is made and experimentally verified. The broad class of problems to which the architecture and methodology can be applied is indicated.

Barkan, B. Z.↗
NASA NTRS1988

VLSI high speed packet processor

The Goddard Space Flight Center Mission Operations and Data Systems Directorate has developed a packet processor card utilizing semicustom very large scale integration (VLSI) devices, microprocessors, and programmable gate arrays to support the implementation of multichannel telemetry data capture systems. This card will receive synchronized error corrected telemetry transfer frames and output annotated application packets derived from this data. An adaptable format capability is provided by the programmability of three microprocessors while the throughput capability of the packet processor is achieved by a data pipeline consisting of two separate RAM systems controlled by specially designed semicustom VLSI logic.

Grebowsky, Gerald J.↗
NASA NTRS2003

Intelligent Systems Technologies to Assist in Utilization of Earth Observation Data

With the launch of several Earth observing satellites over the last decade, we are now in a data rich environment. From NASA's Earth Observing System (EOS) satellites alone, we are accumulating more than 3 TB per day of raw data and derived geophysical parameters. The data products are being distributed to a large user community comprising scientific researchers, educators and operational government agencies. Notable progress has been made in the last decade in facilitating access to data. However, to realize the full potential of the growing archives of valuable scientific data, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications. Sponsored by NASA s Intelligent Systems Project within the Computing, Information and Communication Technology (CICT) Program, a conceptual architecture study has been conducted to examine ideas to improve data utilization through the addition of intelligence into the archives in the context of an overall knowledge building system. Potential Intelligent Archive concepts include: 1) Mining archived data holdings using Intelligent Data Understanding algorithms to improve metadata to facilitate data access and usability; 2) Building intelligence about transformations on data, information, knowledge, and accompanying services involved in a scientific enterprise; 3) Recognizing the value of results, indexing and formatting them for easy access, and delivering them to concerned individuals; 4) Interacting as a cooperative node in a web of distributed systems to perform knowledge building (i.e., the transformations from data to information to knowledge) instead of just data pipelining; and 5) Being aware of other nodes in the knowledge building system, participating in open systems interfaces and protocols for virtualization, and collaborative interoperability. This paper presents some of these concepts and identifies issues to be addressed by research in future intelligent systems technology.

Ramapriyan, Hampapuram K.↗
NASA NTRS2003

A Comparison of Solar p-Mode Parameters from MDI and Gong: Mode Frequencies and Structure Inversions

Helioseismic analysis of solar global oscillations allows investigation of the internal structure of the Sun. One important test of the reliability of the inferences from helioseismology is that the results from independent sets of contemporaneous data are consistent with one another. Here we compare mode frequencies from the Global Oscillation Network Group and Michelson Doppler Imager on board SOHO and resulting inversion results on the Sun's internal structure. The average relative differences between the data sets are typically less than 1 x 10(exp -5) substantially smaller than the formal errors in the differences; however, in some cases the frequency differences show a systematic behavior that might nonetheless influence the inversion results. We find that the differences in frequencies are not a result of instrumental effects but are almost entirely related to the data pipeline software. Inversion of the frequencies shows that their differences do not result in any significant effects on the resulting inferences on solar structure. We have also experimented with fitting asymmetric profiles to the oscillation power spectra and find that, compared with the symmetric fits, this causes no significant change in the inversion results.

Basu, S.↗
NASA NTRS2012

Interactive 3D Mars Visualization

The Interactive 3D Mars Visualization system provides high-performance, immersive visualization of satellite and surface vehicle imagery of Mars. The software can be used in mission operations to provide the most accurate position information for the Mars rovers to date. When integrated into the mission data pipeline, this system allows mission planners to view the location of the rover on Mars to 0.01-meter accuracy with respect to satellite imagery, with dynamic updates to incorporate the latest position information. Given this information so early in the planning process, rover drivers are able to plan more accurate drive activities for the rover than ever before, increasing the execution of science activities significantly. Scientifically, this 3D mapping information puts all of the science analyses to date into geologic context on a daily basis instead of weeks or months, as was the norm prior to this contribution. This allows the science planners to judge the efficacy of their previously executed science observations much more efficiently, and achieve greater science return as a result. The Interactive 3D Mars surface view is a Mars terrain browsing software interface that encompasses the entire region of exploration for a Mars surface exploration mission. The view is interactive, allowing the user to pan in any direction by clicking and dragging, or to zoom in or out by scrolling the mouse or touchpad. This set currently includes tools for selecting a point of interest, and a ruler tool for displaying the distance between and positions of two points of interest. The mapping information can be harvested and shared through ubiquitous online mapping tools like Google Mars, NASA WorldWind, and Worldwide Telescope.

Powell, Mark W.↗
NASA NTRS2015

The CHARIS IFS for high contrast imaging at Subaru

The Coronagraphic High Angular Resolution Imaging Spectrograph (CHARIS) is an integral field spectrograph (IFS) being built for the Subaru telescope. CHARIS will take spectra of brown dwarfs and hot Jovian planets in the coronagraphic image provided by the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) and AO188 adaptive optics systems. The system is designed to detect objects five orders of magnitude dimmer than their parent star down to an 80 milliarcsecond inner working angle. For characterization, CHARIS has a high-resolution prism providing an average spectral resolution of R82, R69, and R82 in J, H, and K bands respectively. The so-called discovery mode uses a second low-resolution prism with an average spectral resolution of R19 spanning 1.15-2.37 microns (J+H+K bands). This is unique compared to other high contrast IFS designs. It augments low inner working angle performance by reducing the separation at which we can rely on spectral differential imaging. The principal challenge for a high-contrast IFS is quasi-static speckles, which cause undue levels of spectral crosstalk. CHARIS has addressed this through several key design aspects that should constrain crosstalk between adjacent spectral features to be below 1%. Sitting on the Nasmyth platform, the alignment between the lenslet array, prism, and detector will be highly stable, key for the performance of the data pipeline. Nearly every component has arrived and the project is entering its final build phase. Here we review the science case, the resulting design, status of final construction, and lessons learned that are directly applicable to future exoplanet instruments.

Subaru Coronagraphic Extreme Adaptive Optics (SCEx↗
NASA NTRS2016

Enabling the Direct Detection of Earth-Sized Exoplanets with the LBTI HOSTS Project: A Progress Report

NASA has funded a project called the Hunt for Observable Signatures of Terrestrial Systems (HOSTS) to survey nearby solar type stars to determine the amount of warm zodiacal dust in their habitable zones. The goal is not only to determine the luminosity distribution function but also to know which individual stars have the least amount of zodiacal dust. It is important to have this information for future missions that directly image exoplanets as this dust is the main source of astrophysical noise for them. The HOSTS project utilizes the Large Binocular Telescope Interferometer (LBTI), which consists of two 8.4-m apertures separated by a 14.4-m baseline on Mt. Graham, Arizona. The LBTI operates in a nulling mode in the mid-infrared spectral window (8-13 micrometers), in which light from the two telescopes is coherently combined with a 180 degree phase shift between them, producing a dark fringe at the location of the target star. In doing so the starlight is greatly reduced, increasing the contrast, analogous to a coronagraph operating at shorter wavelengths. The LBTI is a unique instrument, having only three warm reflections before the starlight reaches cold mirrors, giving it the best photometric sensitivity of any interferometer operating in the mid-infrared. It also has a superb Adaptive Optics (AO) system giving it Strehl ratios greater than 98% at 10 micrometers. In 2014 into early 2015 LBTI was undergoing commissioning. The HOSTS. project team passed its Operational Readiness Review (ORR) in April 2015. The team recently published papers on the target sample, modeling of the nulled disk images, and initial results such as the detection of warm dust around eta Corvi. Recently a paper was published on the data pipeline and on-sky performance. An additional paper is in preparation on Beta Leo. We will discuss the scientific and programmatic context for the LBTI project, and we will report recent progress, new results, and plans for the science verification phase that started in February 2016, and for the survey.

Hunt for Observable Signatures of Terrestrial Syst↗
NASA NTRS2018

The Gaia Catalogue Second Data Release and Its Implications to Optical Observations of Man-Made Earth Orbiting Objects

The Gaia catalogue second data release and its implications to optical observations of man-made Earth orbiting objects. Abstract and not the Final Paper is attached. The Gaia spacecraft was launched in December 2013 by the European Space Agency to produce a three-dimensional, dynamic map of objects within the Milky Way. Gaia's first year of data was released in September 2016. Common sources from the first data release have been combined with the Tycho-2 catalogue to provide a 5 parameter astrometric solution for approximately 2 million stars. The second Gaia data release is scheduled to come out in April 2018 and is expected to provide astrometry and photometry for more than 1 billion stars, a subset of which with a the full 6 parameter astrometric solution (adding radial velocity) and positional accuracy better than 0.002 arcsec (2 mas). In addition to precise astrometry, a unique opportunity exists with the Gaia catalogue in its production of accurate, broadband photometry using the Gaia G filter. In the past, clear filters have been used by various groups to maximize likelihood of detection of dim man-made objects but these data were very difficult to calibrate. With the second release of the Gaia catalogue, a ground based system utilizing the G band filter will have access to 1.5 billion all-sky calibration sources down to an accuracy of 0.02 magnitudes or better. In this talk, we will discuss the advantages and practicalities of implementing the Gaia filters and catalogue into data pipelines designed for optical observations of man-made objects.

Frith, James M.↗
NASA NTRS2001

Using XML and Java Technologies for Astronomical Instrument Control

Traditionally, instrument command and control systems have been highly specialized, consisting mostly of custom code that is difficult to develop, maintain, and extend. Such solutions are initially very costly and are inflexible to subsequent engineering change requests, increasing software maintenance costs. Instrument description is too tightly coupled with details of implementation. NASA Goddard Space Flight Center, under the Instrument Remote Control (IRC) project, is developing a general and highly extensible framework that applies to any kind of instrument that can be controlled by a computer. The software architecture combines the platform independent processing capabilities of Java with the power of the Extensible Markup Language (XML), a human readable and machine understandable way to describe structured data. A key aspect of the object-oriented architecture is that the software is driven by an instrument description, written using the Instrument Markup Language (IML), a dialect of XML. IML is used to describe the command sets and command formats of the instrument, communication mechanisms, format of the data coming from the instrument, and characteristics of the graphical user interface to control and monitor the instrument. The IRC framework allows the users to define a data analysis pipeline which converts data coming out of the instrument. The data can be used in visualizations in order for the user to assess the data in real-time, if necessary. The data analysis pipeline algorithms can be supplied by the user in a variety of forms or programming languages. Although the current integration effort is targeted for the High-resolution Airborne Wideband Camera (HAWC) and the Submillimeter and Far Infrared Experiment (SAFIRE), first-light instruments of the Stratospheric Observatory for Infrared Astronomy (SOFIA), the framework is designed to be generic and extensible so that it can be applied to any instrument. Plans are underway to test the framework with other types of instruments, such as remote sensing earth science instruments.

Ames, Troy↗
NASA NTRS2019

GeneLab Analysis Working Group Pipelines

GeneLab must establish data processing pipelines for common data types including microarray, RNA-sequencing, and metagenomic profiling. Here we give an overview of current microarray and RNA-seq pipelines and discuss future pipelines including metagenomic profiling pipelines

Galazka, Jonathan M.↗
NASA NTRS2008

Summary on the XRT Data Products and Pipeline Production

The new code to generate light curves for GRB observed with the XRT on Swift has been distributed with the latest software release. The code has been extensively tested compared with similar output from Penn State. Adjustments in both codes were made to best represent the latest calibration information and the latest software algorithms. In this talk will be highlighted the steps taken to produce the GRB light-curves, how it is been tested for non GRB sources and last the future adjustments to be made to include a better correction for the bad columns. In addition it will be presented as a new routine developed to automatically collect spectra at different phases of the GRB decay. This new routine will be included in the next software release. The XRT pipeline at GSFC will implement the production of the data files containing the level 3 products. The talk will discuss the current status and the additional data files planned.

Angelini, Lorella↗
NASA NTRS2016

Description of the TCERT Vetting Reports for Data Release 25

This document, the Kepler Instrument Handbook (KIH), is for Kepler and K2 observers, which includes the Kepler Science Team, Guest Observers (GOs), and astronomers doing archival research on Kepler and K2 data in NASAs Astrophysics Data Analysis Program (ADAP). The KIH provides information about the design, performance, and operational constraints of the Kepler flight hardware and software, and an overview of the pixel data sets available. The KIH is meant to be read with these companion documents:1. Kepler Data Processing Handbook (KSCI-19081) or KDPH (Jenkins et al., 2016). The KDPH describes how pixels downlinked from the spacecraft are converted by the Kepler Data Processing Pipeline (henceforth just the pipeline) into the data products delivered to the MAST archive. 2. Kepler Archive Manual (KDMC-10008) or KAM (Thompson et al., 2016). The KAM describes the format and content of the data products, and how to search for them.3. Kepler Data Characteristics Handbook (KSCI-19040) or KDCH (Christiansen et al., 2016). The KDCH describes recurring non-astrophysical features of the Kepler data due to instrument signatures, spacecraft events, or solar activity, and explains how these characteristics are handled by the pipeline.4. Kepler Data Release Notes 25 (KSCI-19065) or DRN 25 (Thompson et al., 2015). DRN 25 describes signatures and events peculiar to individual quarters, and the pipeline software changes between a data release and the one preceding it.Together, these documents supply the information necessary for obtaining and understanding Kepler results, given the real properties of the hardware and the data analysis methods used, and for an independent evaluation of the methods used if so desired.

Instrument↗