Hyperspectral data compression
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Hyperspectral imaging systems onboard aircraft or spacecraft can acquire large amounts of data, putting a strain on limited downlink and storage resources. Onboard data compression can mitigate this problem but may require a system capable of a high throughput. In order to achieve a high throughput with a software compressor, a graphics processing unit (GPU) implementation of a compressor was developed targeting the current state-of-the-art GPUs from NVIDIA(R). The implementation is based on the fast lossless (FL) compression algorithm reported in "Fast Lossless Compression of Multispectral-Image Data" (NPO- 42517), NASA Tech Briefs, Vol. 30, No. 8 (August 2006), page 26, which operates on hyperspectral data and achieves excellent compression performance while having low complexity. The FL compressor uses an adaptive filtering method and achieves state-of-the-art performance in both compression effectiveness and low complexity. The new Consultative Committee for Space Data Systems (CCSDS) Standard for Lossless Multispectral & Hyperspectral image compression (CCSDS 123) is based on the FL compressor. The software makes use of the highly-parallel processing capability of GPUs to achieve a throughput at least six times higher than that of a software implementation running on a single-core CPU. This implementation provides a practical real-time solution for compression of data from airborne hyperspectral instruments.
Efficient onboard data compression can reduce the data volume from hyperspectral imagers on NASA and DoD spacecraft in order to return as much imagery as possible through constrained downlink channels. Lossless compression is important for signature extraction, object recognition, and feature classification capabilities. To provide onboard data compression, a hardware implementation of a lossless hyperspectral compression algorithm was developed using a field programmable gate array (FPGA). The underlying algorithm is the Fast Lossless (FL) compression algorithm reported in Fast Lossless Compression of Multispectral- Image Data (NPO-42517), NASA Tech Briefs, Vol. 30, No. 8 (August 2006), p. 26 with the modification reported in Lossless, Multi-Spectral Data Comressor for Improved Compression for Pushbroom-Type Instruments (NPO-45473), NASA Tech Briefs, Vol. 32, No. 7 (July 2008) p. 63, which provides improved compression performance for data from pushbroom-type imagers. An FPGA implementation of the unmodified FL algorithm was previously developed and reported in Fast and Adaptive Lossless Onboard Hyperspectral Data Compression System (NPO-46867), NASA Tech Briefs, Vol. 36, No. 5 (May 2012) p. 42. The essence of the FL algorithm is adaptive linear predictive compression using the sign algorithm for filter adaption. The FL compressor achieves a combination of low complexity and compression effectiveness that exceeds that of stateof- the-art techniques currently in use. The modification changes the predictor structure to tolerate differences in sensitivity of different detector elements, as occurs in pushbroom-type imagers, which are suitable for spacecraft use. The FPGA implementation offers a low-cost, flexible solution compared to traditional ASIC (application specific integrated circuit) and can be integrated as an intellectual property (IP) for part of, e.g., a design that manages the instrument interface. The FPGA implementation was benchmarked on the Xilinx Virtex IV LX25 device, and ported to a Xilinx prototype board. The current implementation has a critical path of 29.5 ns, which dictated a clock speed of 33 MHz. The critical path delay is end-to-end measurement between the uncompressed input data and the output compression data stream. The implementation compresses one sample every clock cycle, which results in a speed of 33 Msample/s. The implementation has a rather low device use of the Xilinx Virtex IV LX25, making the total power consumption of the implementation about 1.27 W.
Efficient on-board lossless hyperspectral data compression reduces the data volume necessary to meet NASA and DoD limited downlink capabilities. The techniques also improves signature extraction, object recognition and feature classification capabilities by providing exact reconstructed data on constrained downlink resources. At JPL a novel, adaptive and predictive technique for lossless compression of hyperspectral data was recently developed. This technique uses an adaptive filtering method and achieves a combination of low complexity and compression effectiveness that far exceeds state-of-the-art techniques currently in use. The JPL-developed 'Fast Lossless' algorithm requires no training data or other specific information about the nature of the spectral bands for a fixed instrument dynamic range. It is of low computational complexity and thus well-suited for implementation in hardware, which makes it practical for flight implementations of pushbroom instruments. A prototype of the compressor (and decompressor) of the algorithm is available in software, but this implementation may not meet speed and real-time requirements of some space applications. Hardware acceleration provides performance improvements of 10x-100x vs. the software implementation (about 1M samples/sec on a Pentium IV machine). This paper describes a hardware implementation of the JPL-developed 'Fast Lossless' compression algorithm on a Field Programmable Gate Array (FPGA). The FPGA implementation targets the current state of the art FPGAs (Xilinx Virtex IV and V families) and compresses one sample every clock cycle to provide a fast and practical real-time solution for Space applications.
On-board lossless hyperspectral data compression reduces data volume in order to meet NASA and DoD limited downlink capabilities. The technique also improves signature extraction, object recognition and feature classification capabilities by providing exact reconstructed data on constrained downlink resources. At JPL a novel, adaptive and predictive technique for lossless compression of hyperspectral data was recently developed. This technique uses an adaptive filtering method and achieves a combination of low complexity and compression effectiveness that far exceeds state-of-the-art techniques currently in use. The JPL-developed 'Fast Lossless' algorithm requires no training data or other specific information about the nature of the spectral bands for a fixed instrument dynamic range. It is of low computational complexity and thus well-suited for implementation in hardware. A modified form of the algorithm that is better suited for data from pushbroom instruments is generally appropriate for flight implementation. A scalable field programmable gate array (FPGA) hardware implementation was developed. The FPGA implementation achieves a throughput performance of 58 Msamples/sec, which can be increased to over 100 Msamples/sec in a parallel implementation that uses twice the hardware resources This paper describes the hardware implementation of the 'Modified Fast Lossless' compression algorithm on an FPGA. The FPGA implementation targets the current state-of-the-art FPGAs (Xilinx Virtex IV and V families) and compresses one sample every clock cycle to provide a fast and practical real-time solution for space applications.
Efficient on-board lossless hyperspectral data compression reduces data volume in order to meet NASA and DoD limited downlink capabilities. The technique also improves signature extraction, object recognition and feature classification capabilities by providing exact reconstructed data on constrained downlink resources. At JPL a novel, adaptive and predictive technique for lossless compression of hyperspectral data was recently developed. This technique uses an adaptive filtering method and achieves a combination of low complexity and compression effectiveness that far exceeds state-of-the-art techniques currently in use. The JPL-developed 'Fast Lossless' algorithm requires no training data or other specific information about the nature of the spectral bands for a fixed instrument dynamic range. It is of low computational complexity and thus well-suited for implementation in hardware.
Modern hyperspectral imaging systems are able to acquire far more data than can be downlinked from a spacecraft. Onboard data compression helps to alleviate this problem, but requires a system capable of power efficiency and high throughput. Software solutions have limited throughput performance and are power-hungry. Dedicated hardware solutions can provide both high throughput and power efficiency, while taking the load off of the main processor. Thus a hardware compression system was developed. The implementation uses a field-programmable gate array (FPGA). The implementation is based on the fast lossless (FL) compression algorithm reported in Fast Lossless Compression of Multispectral-Image Data (NPO-42517), NASA Tech Briefs, Vol. 30, No. 8 (August 2006), page 26, which achieves excellent compression performance and has low complexity. This algorithm performs predictive compression using an adaptive filtering method, and uses adaptive Golomb coding. The implementation also packetizes the coded data. The FL algorithm is well suited for implementation in hardware. In the FPGA implementation, one sample is compressed every clock cycle, which makes for a fast and practical realtime solution for space applications. Benefits of this implementation are: 1) The underlying algorithm achieves a combination of low complexity and compression effectiveness that exceeds that of techniques currently in use. 2) The algorithm requires no training data or other specific information about the nature of the spectral bands for a fixed instrument dynamic range. 3) Hardware acceleration provides a throughput improvement of 10 to 100 times vs. the software implementation. A prototype of the compressor is available in software, but it runs at a speed that does not meet spacecraft requirements. The hardware implementation targets the Xilinx Virtex IV FPGAs, and makes the use of this compressor practical for Earth satellites as well as beyond-Earth missions with hyperspectral instruments.
An ultrafast high accuracy PCRTM_SOLAR model is developed based on PCA compression and principal component-based radiative transfer model (PCRTM). A fast algorithm for simulation of multi-scattering properties of cloud and/or aerosols is integrated into the fast infrared PCRTM. We completed radiance simulation and training for instruments, such as IASI, AIRS, CrIS, NASTI and SHIS, under diverse conditions. The new model is 5 orders faster than 52-stream DISORT with very high accuracy for cloudy sky radiative transfer simulation. It is suitable for hyperspectral remote data assimilation and cloudy sky retrievals.
The High Data Rate Instrument Study was a joint effort between the Jet Propulsion Laboratory (JPL) and the Goddard Space Flight Center (GSFC). The objectives were to assess the characteristics of future high data rate Earth observing science instruments and then to assess the feasibility of developing data processing systems and communications systems required to meet those data rates. Instruments and technology were assessed for technology readiness dates of 2000, 2003, and 2006. The highest data rate instruments are hyperspectral and synthetic aperture radar instruments which are capable of generating 3.2 Gigabits per second (Gbps) and 1.3 Gbps, respectively, with a technology readiness date of 2003. These instruments would require storage of 16.2 Terebits (Tb) of information (RF communications case of two orbits of data) or 40.5 Tb of information (optical communications case of five orbits of data) with a technology readiness date of 2003. Onboard storage capability in 2003 is estimated at 4 Tb; therefore, all the data created cannot be stored without processing or compression. Of the 4 Tb of stored data, RF communications can only send about one third of the data to the ground, while optical communications is estimated at 6.4 Tb across all three technology readiness dates of 2000, 2003, and 2006 which were used in the study. The study includes analysis of the onboard processing and communications technologies at these three dates and potential systems to meet the high data rate requirements. In the 2003 case, 7.8% of the data can be stored and downlinked by RF communications while 10% of the data can be stored and downlinked with optical communications. The study conclusion is that only 1 to 10% of the data generated by high data rate instruments will be sent to the ground from now through 2006 unless revolutionary changes in spacecraft design and operations such as intelligent data extraction are developed.
Topics covered include: Software Tool Integrating Data Flow Diagrams and Petri Nets; Adaptive Nulling for Interferometric Detection of Planets; Reducing the Volume of NASA Earth-Science Data; Reception of Multiple Telemetry Signals via One Dish Antenna; Space-Qualified Traveling-Wave Tube; Smart Power Supply for Battery-Powered Systems; Parallel Processing of Broad-Band PPM Signals; Inexpensive Implementation of Many Strain Gauges; Constant-Differential-Pressure Two-Fluid Accumulator; Inflatable Tubular Structures Rigidized with Foams; Power Generator with Thermo-Differential Modules; Mechanical Extraction of Power From Ocean Currents and Tides; Nitrous Oxide/Paraffin Hybrid Rocket Engines; Optimized Li-Ion Electrolytes Containing Fluorinated Ester Co-Solvents; Probabilistic Multi-Factor Interaction Model for Complex Material Behavior; Foldable Instrumented Bits for Ultrasonic/Sonic Penetrators; Compact Rare Earth Emitter Hollow Cathode; High-Precision Shape Control of In-Space Deployable Large Membrane/Thin-Shell Reflectors; Rapid Active Sampling Package; Miniature Lightweight Ion Pump; Cryogenic Transport of High-Pressure-System Recharge Gas; Water-Vapor Raman Lidar System Reaches Higher Altitude; Compact Ku-Band T/R Module for High-Resolution Radar Imaging of Cold Land Processes; Wide-Field-of-View, High-Resolution, Stereoscopic Imager; Electrical Capacitance Volume Tomography with High-Contrast Dielectrics; Wavefront Control and Image Restoration with Less Computing; Polarization Imaging Apparatus; Stereoscopic Machine-Vision System Using Projected Circles; Metal Vapor Arcing Risk Assessment Tool; Performance Bounds on Two Concatenated, Interleaved Codes; Parameterizing Coefficients of a POD-Based Dynamical System; Confidence-Based Feature Acquisition; Algorithm for Lossless Compression of Calibrated Hyperspectral Imagery; Universal Decoder for PPM of any Order; Algorithm for Stabilizing a POD-Based Dynamical System; Mission Reliability Estimation for Repairable Robot Teams; Processing AIRS Scientific Data Through Level 3; Web-Based Requesting and Scheduling Use of Facilities; AutoGen Version 5.0; Time-Tag Generation Script; PPM Receiver Implemented in Software; Tropospheric Emission Spectrometer Product File Readers; Reporting Differences Between Spacecraft Sequence Files; Coordinating "Execute" Data for ISS and Space Shuttle; Database for Safety-Oriented Tracking of Chemicals; Apparatus for Cold, Pressurized Biogeochemical Experiments; Growing B Lymphocytes in a Three-Dimensional Culture System; Tissue-like 3D Assemblies of Human Broncho-Epithelial Cells; Isolation of Resistance-Bearing Microorganisms; Oscillating Cell Culture Bioreactor; and Liquid Cooling/Warming Garment.
The emergent technology of system-on-chip (SoC) devices promises lighter, smaller, cheaper, and more capable and reliable space electronic systems that could help to unveil some of the most treasured secrets in our universe. This technology is an improvement over the technology that is currently used in space applications, which lags behind state-of-the-art commercial-off-the-shelf (COTS) equipment by several generations. SoC technology integrates all computational power required by next-generation space exploration science instruments onto a single chip. This paper describes hardware/software co-verification tools for the Xilinx Zynq-based control and data handling system that have been developed at the Jet Propulsion Laboratory (JPL) for visible-infrared imaging spectrometers. The system acquires and compresses images in real-time, in addition to programming the spectrometer (frame rate, exposure time), focus step motor, and heaters and reporting telemetry.
The TextureCam project is developing a 'smart camera' that can classify geologic surfaces in planetary images. This would allow autonomous spacecraft to collect data opportunisitcally during intervals between communications with Earth, such as during long traverses. Its surface classifications can identify new targets that were not anticipated in advance. The spacecraft might use this information to target these features with high-resolution instruments such as spectrometers nd narrow-field cameras. Classifications could also inform data 'triage' decisions, identifying high value images for prioritized downlink. Finally, the surface classification can serve as compressed maps of image content. Each of these strategies can improve the science data returned at each command cycle and speed reconnaissance during site survey and astrobiology investigation. Our first year of development has completed the image analysis algorithms and validated them in software tests. Here we survey these initial results and explore several application areas relevant to Mars and beyond.
Modern infrared satellite sensors such as Atmospheric Infrared Sounder (AIRS), Cosmic Ray Isotope Spectrometer (CrIS), Thermal Emission Spectrometer (TES), Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) and Infrared Atmospheric Sounding Interferometer (IASI) are capable of providing high spatial and spectral resolution infrared spectra. To fully exploit the vast amount of spectral information from these instruments, super fast radiative transfer models are needed. This paper presents a novel radiative transfer model based on principal component analysis. Instead of predicting channel radiance or transmittance spectra directly, the Principal Component-based Radiative Transfer Model (PCRTM) predicts the Principal Component (PC) scores of these quantities. This prediction ability leads to significant savings in computational time. The parameterization of the PCRTM model is derived from properties of PC scores and instrument line shape functions. The PCRTM is very accurate and flexible. Due to its high speed and compressed spectral information format, it has great potential for super fast one-dimensional physical retrievals and for Numerical Weather Prediction (NWP) large volume radiance data assimilation applications. The model has been successfully developed for the National Polar-orbiting Operational Environmental Satellite System Airborne Sounder Testbed - Interferometer (NAST-I) and AIRS instruments. The PCRTM model performs monochromatic radiative transfer calculations and is able to include multiple scattering calculations to account for clouds and aerosols.
In recent years, some remote-sensing applications require advanced airborne multi-sensor systems to provide high performance reflective and emissive spectral imaging measurement rapidly over large areas. The key or unique problem of characteristics is associated with a black box back-end system that operates a suite of cutting-edge imaging sensors to collect simultaneously the high throughput reflective and emissive spectral imaging data with precision georeference. This back-end system needs to be portable, easy-to-use, and reliable with advanced onboard processing. The innovation of the black box backend is a miniaturized airborne imaging central server system (MAICSS). MAICSS integrates a complex embedded system of systems with dedicated power and signal electronic circuits inside to serve a suite of configurable cutting-edge electro- optical (EO), long-wave infrared (LWIR), and medium-wave infrared (MWIR) cameras, a hyperspectral imaging scanner, and a GPS and inertial measurement unit (IMU) for atmospheric and surface remote sensing. Its compatible sensor packages include NASA s 1,024 1,024 pixel LWIR quantum well infrared photodetector (QWIP) imager; a 60.5 megapixel BuckEye EO camera; and a fast (e.g. 200+ scanlines/s) and wide swath-width (e.g., 1,920+ pixels) CCD/InGaAs imager-based visible/near infrared reflectance (VNIR) and shortwave infrared (SWIR) imaging spectrometer. MAICSS records continuous precision georeferenced and time-tagged multisensor throughputs to mass storage devices at a high aggregate rate, typically 60 MB/s for its LWIR/EO payload. MAICSS is a complete stand-alone imaging server instrument with an easy-to-use software package for either autonomous data collection or interactive airborne operation. Advanced multisensor data acquisition and onboard processing software features have been implemented for MAICSS. With the onboard processing for real time image development, correction, histogram-equalization, compression, georeference, and data organization, fast aerial imaging applications, including the real time LWIR image mosaic for Google Earth, have been realized for NASA fs LWIR QWIP instrument. MAICSS is a significant improvement and miniaturization of current multisensor technologies. Structurally, it has a complete modular and solid-state design. Without rotating hard drives and other moving parts, it is operational at high altitudes and survivable in high-vibration environments. It is assembled from a suite of miniaturized, precision-machined, standardized, and stackable interchangeable embedded instrument modules. These stackable modules can be bolted together with the interconnection wires inside for the maximal simplicity and portability. Multiple modules are electronically interconnected as stacked. Alternatively, these dedicated modules can be flexibly distributed to fit the space constraints of a flying vehicle. As a flexibly configurable system, MAICSS can be tailored to interface a variety of multisensor packages. For example, with a 1,024x1,024 pixel LWIR and a 8,984x6,732 pixel EO payload, the complete MAICSS volume is approximately 7x9x11 in. (=18x23x28 cm), with a weight of 25 lb (=11.4 kg).
Topics covered include: Improved Instrument for Detecting Water and Ice in Soil; Real-Time Detection of Dust Devils from Pressure Readings; Determining Surface Roughness in Urban Areas Using Lidar Data; DSN Data Visualization Suite; Hamming and Accumulator Codes Concatenated with MPSK or QAM; Wide-Angle-Scanning Reflectarray Antennas Actuated by MEMS; Biasable Subharmonic Membrane Mixer for 520 to 600 GHz; Hardware Implementation of Serially Concatenated PPM Decoder; Symbolic Processing Combined with Model-Based Reasoning; Presentation Extensions of the SOAP; Spreadsheets for Analyzing and Optimizing Space Missions; Processing Ocean Images to Detect Large Drift Nets; Alternative Packaging for Back-Illuminated Imagers; Diamond Machining of an Off-Axis Biconic Aspherical Mirror; Laser Ablation Increases PEM/Catalyst Interfacial Area; Damage Detection and Self-Repair in Inflatable/Deployable Structures; Polyimide/Glass Composite High-Temperature Insulation; Nanocomposite Strain Gauges Having Small TCRs; Quick-Connect Windowed Non-Stick Penetrator Tips for Rapid Sampling; Modeling Unsteady Cavitation and Dynamic Loads in Turbopumps; Continuous-Flow System Produces Medical-Grade Water; Discrimination of Spore-Forming Bacilli Using spoIVA; nBn Infrared Detector Containing Graded Absorption Layer; Atomic References for Measuring Small Accelerations; Ultra-Broad-Band Optical Parametric Amplifier or Oscillator; Particle-Image Velocimeter Having Large Depth of Field; Enhancing SERS by Means of Supramolecular Charge Transfer; Improving 3D Wavelet-Based Compression of Hyperspectral Images; Improved Signal Chains for Readout of CMOS Imagers; SOI CMOS Imager with Suppression of Cross-Talk; Error-Rate Bounds for Coded PPM on a Poisson Channel; Biomorphic Multi-Agent Architecture for Persistent Computing; and Using Covariance Analysis to Assess Pointing Performance.
Topics covered include: An "Inefficient Fin" Non-Dimensional Parameter to Measure Gas Temperatures Efficiently; On-Wafer Measurement of a Multi-Stage MMIC Amplifier with 10 dB of Gain at 475 GHz; Software to Control and Monitor Gas Streams; Miniaturized Laser Heterodyne Radiometer (LHR) for Measurements of Greenhouse Gases in the Atmospheric Column; Anomaly Detection in Test Equipment via Sliding Mode Observers; Absolute Position of Targets Measured Through a Chamber Window Using Lidar Metrology Systems; Goldstone Solar System Radar Waveform Generator; Fast and Adaptive Lossless Onboard Hyperspectral Data Compression System; Iridium Interfacial Stack - IrIS; Downsampling Photodetector Array with Windowing; Optical Phase Recovery and Locking in a PPM Laser Communication Link; High-Speed Edge-Detecting Line Scan Smart Camera; Optical Communications Channel Combiner; Development of Thermal Infrared Sensor to Supplement Operational Land Imager; Amplitude-Stabilized Oscillator for a Capacitance-Probe Electrometer; Automated Performance Characterization of DSN System Frequency Stability Using Spacecraft Tracking Data; Histogrammatic Method for Determining Relative Abundance of Input Gas Pulse; Predictive Sea State Estimation for Automated Ride Control and Handling - PSSEARCH; LEGION: Lightweight Expandable Group of Independently Operating Nodes; Real-Time Projection to Verify Plan Success During Execution; Automated Performance Characterization of DSN System Frequency Stability Using Spacecraft Tracking Data; Web-Based Customizable Viewer for Mars Network Overflight Opportunities; Fabrication of a Cryogenic Terahertz Emitter for Bolometer Focal Plane Calibrations; Fabrication of an Absorber-Coupled MKID Detector; Graphene Transparent Conductive Electrodes for Next- Generation Microshutter Arrays; Method of Bonding Optical Elements with Near-Zero Displacement; Free-Mass and Interface Configurations of Hammering Mechanisms; Wavefront Compensation Segmented Mirror Sensing and Control; Long-Life, Lightweight, Multi-Roller Traction Drives for Planetary Vehicle Surface Exploration; Reliable Optical Pump Architecture for Highly Coherent Lasers Used in Space Metrology Applications; Electrochemical Ultracapacitors Using Graphitic Nanostacks; Improved Whole-Blood-Staining Device; Monitoring Location and Angular Orientation of a Pill; Molecular Technique to Reduce PCR Bias for Deeper Understanding of Microbial Diversity; Laser Ablation Electrodynamic Ion Funnel for In Situ Mass Spectrometry on Mars; High-Altitude MMIC Sounding Radiometer for the Global Hawk Unmanned Aerial Vehicle; PRTs and Their Bonding for Long-Duration, Extreme-Temperature Environments; Mid- and Long-IR Broadband Quantum Well Photodetector; 3D Display Using Conjugated Multiband Bandpass Filters; Real-Time, Non-Intrusive Detection of Liquid Nitrogen in Liquid Oxygen at High Pressure and High Flow; Method to Enhance the Operation of an Optical Inspection Instrument Using Spatial Light Modulators; Dual-Compartment Inflatable Suitlock; Large-Strain Transparent Magnetoactive Polymer Nanocomposites; Thermodynamic Vent System for an On-Orbit Cryogenic Reaction Control Engine; Time Distribution Using SpaceWire in the SCaN Testbed on ISS; and Techniques for Solution- Assisted Optical Contacting.