Neural Network Processing for Real Time, Sub-Pixel Hyperspectral Data Extraction
The goal of many applications that using hyperspectral images is to describe the constituent components of each pixel in a scene.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The goal of many applications that using hyperspectral images is to describe the constituent components of each pixel in a scene.
The Visual Data Analysis Package is a collection of programs and scripts that facilitate visual analysis of data available from NASA and NOAA satellites, as well as dropsonde, buoy, and conventional in-situ observations. The package features utilities for data extraction, data quality control, statistical analysis, and data visualization. The Hierarchical Data Format (HDF) satellite data extraction routines from NASA's Jet Propulsion Laboratory were customized for specific spatial coverage and file input/output. Statistical analysis includes the calculation of the relative error, the absolute error, and the root mean square error. Other capabilities include curve fitting through the data points to fill in missing data points between satellite passes or where clouds obscure satellite data. For data visualization, the software provides customizable Generic Mapping Tool (GMT) scripts to generate difference maps, scatter plots, line plots, vector plots, histograms, timeseries, and color fill images.
The increasing use of extraction chromatography resins across fields such as hydrometallurgy, nuclear medicine, and environmental analysis has created a need for a deeper understanding of their interactions with transition metals. Despite extensive research on f-element separations, the behavior of transition metals in these systems remains relatively understudied. This review provides a comprehensive overview of the current state of knowledge on the extraction behavior of transition metals with neutral extractants, including TODGA, TEHDGA, TBP, and CMPO, and their corresponding resins, such as DGA, BDGA, UTEVA, TBP, and TRU. The review summarizes extraction data, extracted complex coordination environments, separation reaction stoichiometries, and associated thermodynamics, highlighting inconsistencies and knowledge gaps in the literature. The study emphasizes the need for further research using spectroscopy and computational methods to elucidate extraction mechanisms and to improve the efficiency and selectivity of transition metal separations. By identifying areas for future research and development, this review aims to stimulate advancements in the field and promote the development of innovative separation technologies. The implications of this research are far-reaching, with potential applications in nuclear waste management, nuclear forensics, metal recovery, and environmental remediation. Overall, this review provides a foundation for future studies on the extraction of transition metals using neutral extractants and resins.
We present the data reduction pipeline for CHARIS, a high-contrast integral-field spectrograph for the Subaru Telescope. The pipeline constructs a ramp from the raw reads using the measured nonlinear pixel response and reconstructs the data cube using one of three extraction algorithms: aperture photometry, optimal extraction, or chi-squared fitting. We measure and apply both a detector flatfield and a lenslet flatfield and reconstruct the wavelength- and position-dependent lenslet point-spread function (PSF) from images taken with a tunable laser. We use these measured PSFs to implement a chi-squared-based extraction of the data cube, with typical residuals of approximately 5 percent due to imperfect models of the under-sampled lenslet PSFs. The full two-dimensional residual of the chi-squared extraction allows us to model and remove correlated read noise, dramatically improving CHARIS's performance. The chi-squared extraction produces a data cube that has been deconvolved with the line-spread function and never performs any interpolations of either the data or the individual lenslet spectra. The extracted data cube also includes uncertainties for each spatial and spectral measurement. CHARIS's software is parallelized, written in Python and Cython, and freely available on github with a separate documentation page. Astrometric and spectrophotometric calibrations of the data cubes and PSF subtraction will be treated in a forthcoming paper.
The percentage of incident solar flux reflected by a surface is a quantity of considerable interest in remote sensing studies. To calculate reflectance from remotely sensed radiance data some estimate of incident flux is needed. Since simultaneous ground-based radiometric measurements are often not available for observations by aircraft or satellite sensors, a procedure based on modeling atmospheric transmittance and scattering was developed. The primary application is to an aircraft data set collected with the NASA C-130 over the Superior National Forest, Minnesota. Atmospherically corrected multiple angle reflectance data sets and reflectance images are generated for areas of natural forest vegetation. These data and the technique may be useful for studies of the interactions of light with forested canopies.
Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.
A computer program has been written to replace the original software of Racal Storeplex Delta tape recorders, which are used at Stennis Space Center. The original software could be activated by a command- line interface only; the present software offers the option of a command-line or graphical user interface. The present software also offers the option of batch-file operation (activation by a file that contains command lines for operations performed consecutively). The present software is also more reliable than was the original software: The original software was plagued by several deficiencies that made it difficult to execute, modify, and test. In addition, when using the original software to extract data that had been recorded within specified intervals of time, the resolution with which one could control starting and stopping times was no finer than about a second (or, in some cases, several seconds). In contrast, the present software is capable of controlling playback times to within 1/100 second of times specified by the user, assuming that the tape-recorder clock is accurate to within 1/100 second.
A computer program has been written to replace the original software of Racal Storeplex Delta tape recorders, which are still used at Stennis Space Center but have been discontinued by the manufacturer. Whereas the original software could be activated by a command-line interface only, the present software offers the option of a command-line or graphical user interface. The present software also offers the option of batch-file operation (activation by a file that contains command lines for operations performed consecutively). The present software is also more reliable than was the original software: The original software was plagued by several deficiencies that made it difficult to execute, modify, and test. In addition, when using the original software to extract data that had been recorded within specified intervals of time, the resolution with which one could control starting and stopping times was no finer than about a second (or, in some cases, several seconds). In contrast, the present software is capable of controlling playback times to within 1/100 second of times specified by the user, assuming that the tape-recorder clock is accurate to within 1/100 second.
A computer program has been written to replace the original software of Racal Storeplex Delta tape recorders, which are still used at Stennis Space Center but have been discontinued by the manufacturer. Whereas the original software could be activated by a command-line interface only, the present software offers the option of a command-line or graphical user interface. The present software also offers the option of batch-file operation (activation by a file that contains command lines for operations performed consecutively). The present software is also more reliable than was the original software: The original software was plagued by several deficiencies that made it difficult to execute, modify, and test. In addition, when using the original software to extract data that had been recorded within specified intervals of time, the resolution with which one could control starting and stopping times was no finer than about a second (or, in some cases, several seconds). In contrast, the present software is capable of controlling playback times to within 1/100 second of times specified by the user, assuming that the tape-recorder clock is accurate to within 1/100 second.
The design of an experiment to measure communication characteristics of wideband satellite-to-ground links is reported. Of special concern are the effects of rainstorms and atmospheric turbulence on path attenuation and phase fluctuation. Multi-tone and pulse probing are considered. A multi-tone technique which is a modification of ATS-5 and ATS-F hardware is recommended. Data extraction and data processing techniques and key hardware requirements for the experiment are reviewed.
The Display Audit Suite is an integrated package of software tools that partly automates the detection of Portable Computer System (PCS) Display errors. [PCS is a lap top computer used onboard the International Space Station (ISS).] The need for automation stems from the large quantity of PCS displays (6,000+, with 1,000,000+ lines of command and telemetry data). The Display Audit Suite includes data-extraction tools, automatic error detection tools, and database tools for generating analysis spread sheets. These spread sheets allow engineers to more easily identify many different kinds of possible errors. The Suite supports over 40 independent analyses, 16 NASA Tech Briefs, November 2008 and complements formal testing by being comprehensive (all displays can be checked) and by revealing errors that are difficult to detect via test. In addition, the Suite can be run early in the development cycle to find and correct errors in advance of testing.
Tropical ecosystems contain the world's largest biodiversity of vascular plants. Yet, our understanding of tropical functional diversity and its contribution to global diversity patterns is constrained by data availability. This discrepancy underscores an urgent need to bridge data gaps by incorporating comprehensive tropical root data into global datasets. Here, we provide a database of tropical root characteristics. This new database, TropiRoot 1.0, will be instrumental in evaluating an array of hypotheses pertaining to root functional ecology and plant biogeography, both within the tropics and relative to other global biomes. The data compilation was conducted by the TropiRoot Initiative, in partnership with the Fine-Root Ecology Database (FRED) and the Global Root Trait (GRooT) database, Colorado State University (CSU) and the Smithsonian Tropical Research Institute (STRI). Literature search and data extraction were conducted between 2020 and 2024. Literature was identified using Web of Science, Scopus, and complemented using the expert knowledge of members of TropiRoot. To provide broad environmental and geographical distributions, literature searches included root characteristics (traits) across global change drivers, natural gradients, and from different continents. We adopted FRED standardized data columns and streamlined the format to enhance accessibility for data extraction across various user groups. This optimized framework resulted in a smaller, yet comprehensive datasheet. To make the database compatible with other global root trait initiatives, column identification was standardized following the codes provided by FRED. These efforts culminated in data extracted from 104 new sources, resulting in more than 8000 rows of data (either species or community data). Most of the data in TropiRoot 1.0 include root characteristics such as root biomass, morphology, root dynamics, mass fraction, architecture, anatomy, physiology, and root chemistry. This initiative represents a 30% increase in the currently available data for tropical roots in FRED. TropiRoot 1.0 contains root characteristics from 25 different countries, where seven are located in Asia, six in South America, five in Central America and the Caribbean, four in Africa, two in North America, and 1 in Oceania. Due to the volume of data, when ancillary data were available, including soil data, these data were either extracted and included in the database or its availability was recorded in an additional column. Multiple contributors checked the entries for outliers during the collation process to ensure data quality. For text-based observations, we examined all cells to ensure that their content relates to their specific categories. For numerical observations, we ordered each numerical value from least to greatest and plotted the values, checking apparent outliers against the data in their respective sources and correcting or removing incorrect or impossible values. Some data (soil and aboveground) have different columns for the same variable presented in different units, including originally published units, but root characteristics data had units converted to match those reported in FRED. By filling a gap from global databases, TropiRoot 1.0 expands our knowledge of otherwise so far underrepresented regions and our ability to assess global trends. This advancement can be used to improve tropical forest representation in vegetation models. The data are freely available and should be cited when used.
Image reconstruction and data extraction techniques were considered with respect to their application to combustion diagnostics. A system was designed and constructed that possesses sufficient stability and resolution to make quantitative data extraction possible. Example data were manually processed using the system to demonstrate its feasibility for the purpose intended. The system was interfaced with the PDP-11-04 computer for maximum design capability. It was concluded that the use of specialized digital hardware controlled by a relatively small computer provides the best combination of accuracy, speed, and versatility for this particular problem area.
This volume in the AGARD Flight Instrumentation Series provides flight test instrumentation engineers with an introduction to digital measurement processes on aircraft. Flight test instrumentation systems are rapidly evolving from analog-intensive to digital-intensive systems, including the use of onboard digital computers. Topics include measurements that are digital in origin, as well as sampling, encoding, transmitting, and storing of data. Particular emphasis is placed on modern avionic data bus architectures and what to be aware of when extracting data from them. Examples of data extraction techniques are given. Tradeoffs between digital logic families, trends in digital development, and design testing techniques are discussed. An introduction to digital filtering is also covered.
This volume in the AGARD Flight Instrumentation Series provides flight test instrumentation engineers with an introduction to digital processes on aircraft. Flight test instrumentation systems are rapidly evolving from analog intensive to digital intensive systems, including the use of onboard digital computers. Topics include: measurements that are digital in origin, sampling, encoding, transmitting, and storing of data. Particular emphasis is placed on modem avionic data bus architectures and what to be aware of when extracting data from them. Some example data extraction techniques are given. Tradeoffs between digital logic families, trends in digital development, and design testing techniques are discussed. An introduction to digital filtering is also covered.
A method of classification of digitized multispectral image data is described. It is designed to exploit a particular type of dependence between adjacent states of nature that is characteristic of the data. The advantages of this, as opposed to the conventional per point approach, are greater accuracy and efficiency, and the results are in a more desirable form for most purposes. Experimental results from both aircraft and satellite data are included.
A classification method for digitized multispectral-image data is described. This method is designed to exploit a particular type of dependence between adjacent states of nature that is characteristic of the data. The advantages of this, as opposed to the conventional 'per point' approach, are greater accuracy and efficiency, and the results are in a more desirable form for most purposes. Experimental results from both aircraft and satellite data are included.
The Tropical Rainfall Measuring Mission (TRMM) Ground Validation (GV) rain products are generated from ground-based radar observations using a bulk adjusted FACE radar reflectivity (Ze) - rain rate (R) relationship. Monthly Ze-R relationships are obtained by extracting the Ze data with a horizontal resolution of 2 km from an altitude of 1.5 km (from Constant Altitude PPIS, CAPPIS) over the locations of the gauges. The gauge and radar data are merged, and a quality control procedure is applied that eliminates poorly correlated gauge-radar data. Bulk adjusted Ze-R relations are then derived from this quality-controlled merged data base. These Z-R relations are applied to the base scan reflectivity to produce rain maps. The present study examines the effect of these different planes on the total accumulations from the radar and the associated effect on the bulk adjustment coefficient. In order to evaluate the credibility of these GV products, the effect of the different planes involved in the bulk adjustment method will be discussed, The gauge data will also be used to assess the impact of missing radar data (i.e., data gaps) upon the monthly radar-derived rainfall estimates. Experiments were also performed varying both the area of the radar data extracted over the gauges and the use of different weighting coefficients. Preliminary results from these sensitivity tests indicate improved agreement and consistency between the gauge and radar data.