Results for “multispectral imaging” — Page 4

DOE OSTI2023

Gumby: Quantifying multi-modal model resiliency

With the rise of cheap data and sensors, more use cases are emerging for multi-input models. Research has shown that including multiple data modalities can improve performance, suggesting that deep learning models can successfully learn to leverage complementary information from different modalities. However, this improved predictive power comes with unanticipated costs: additional inputs change model resiliency and expand the threat space for adversarial attacks. We first provide theoretical underpinnings for how adversarial success scales with input dimension. We then characterize the performance of a suite of multispectral deep learning models with different fusion approaches, quantify their relative reliance on different input bands, and evaluate their robustness to naturalistic and adversarial image corruptions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Predicting Switchgrass Biomass Yields Using a Spectral Vegetation Index Derived from Multispectral Satellite Imagery

Successful scaling of perennial bioenergy crop production requires a landscape design that optimizes the benefits of finite lands for people, communities, and environments. Utilizing marginal areas is the key to sustainable bioenergy crop production (Ssegane et al., 2015, 2016). Marginal areas are often small-sized lands and unevenly distributed across the agricultural landscape (Ssegane et al., 2016); thus, a systematic, semi-automated remote sensing method is needed as an effective means of estimating bioenergy crop yields across landscapes. Argonne National Laboratory (Argonne) is currently developing a tool, Scaling Up Perennial Bioenergy Economics and Ecosystem Services Tool (SUPERBEEST), to identify marginal agricultural lands and quantify environmental and economic effects of perennial bioenergy crop production systems. The tool aims to provide users a path to foster the sustainable and productive integration of bioenergy crops in the Midwestern agricultural landscape. Reliable, cost-effective, and timely estimation of bioenergy crop yields using remote sensing would help calculate and track the success of integrated bioenergy crops in the landscape for those communities. Argonne previously conducted feasibility studies for estimating biomass yields for bioenergy feedstock, corn and perennial grass using spectral vegetation indices (SVIs)1 derived from optical imagery (Hamada et al., 2015, 2021). In both studies, SVIs, more specifically those sensitive to plant chlorophyll or nitrogen contents, showed potential for estimating or predicting biomass yields with a correlation of determination (R2) ranging from 0.54 to 0.96, indicating a value for further investigation as a viable means of quantifying bioenergy feedstock production across large landscapes. Thus, the goal of this study is to evaluate the feasibility of use of SVIs as a means of estimating or predicting switchgrass biomass yields at harvest using publicly available multispectral satellite imagery. The feasibility analysis was performed using four study areas of mature switchgrass located in Virginia. Objectives are to (1) examine the SVIs and establish their relationships with switchgrass biomass yields at harvest, (2) develop a parsimonious image processing model for predicting at-harvest yields by applying the relationships with the most promising spectral index and (3) map switchgrass yields predicted by the image processing model across the study sites. The calibration to field data will rely on switchgrass biomass yields determined by the baling method, representing a potential challenge to the analysis but an important practical aspect for future applications. With this research design, the study aimed to gain insights into enabling remote sensing-based estimation of bioenergy crop yields in a reliable, cost- effective, and timely manner across large, heterogeneous landscapes.

09 BIOMASS FUELS↗

DOE OSTI2026

Software for High-Resolution Flood Mapping With Sentinel-1 and Sentinel-2 via Misalignment-Robust Cross-Sensor Learning and Generative Despeckling

SF-26-078 This software provides a high-level workflow for satellite-based flood and surface-water mapping using Sentinel-1 SAR and Sentinel-2 multispectral imagery. It supports preparation of remote-sensing data, training and evaluation of deep-learning models, and generation of high-resolution water extent predictions, including methods to improve robustness to SAR speckle noise and cross-sensor image misalignment.

Feinstein, Jeremy [Argonne National Laboratory (AN↗

DOE OSTI2022

RiverPIXELS: paired Landsat images and expert-labeled sediment and water pixels for a selection of rivers v1.0

RiverPIXELS contains GeoTIFFs of hand-labeled water and sediment pixels from Landsat images containing rivers. Each of the 104 labeled patches contains 256 x 256 Landsat pixels (30 meter resolution). Our aim in releasing RiverPIXELS is to provide an "off-the-shelf" training and testing dataset for building machine-learned models to automatically identify rivers from multispectral imagery. While a number of trained models and/or surface water products already exist, RiverPIXELS aims for pixel-level accuracy in order to precisely identify river boundaries in particular. Our selection of rivers is heavily Arctic, but we include tropical and temperate rivers as well. Patches are provided for the Colville (7), Indigirka (6), Kolyma (4), Ucayali (54), Waitaki (21), and Yana (12) Rivers. For each patch, all surface water pixels are labeled (1) and all in-channel sediment pixels are labeled (2). Sediments not in-channel are considered part of the land (0) class. RiverPIXELS also includes paired surface water data from the Global Surface Water dataset that may be useful as additional features in machine learning models. Each patch therefore contains four aligned GeoTIFFs: labeled, landsat, gswmo, and gswocc.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Deep Convolutional Neural Networks Exploit High-Spatial- and -Temporal-Resolution Aerial Imagery to Phenotype Key Traits in Miscanthus

Miscanthus is one of the most promising perennial crops for bioenergy production, with high yield potential and a low environmental footprint. The increasing interest in this crop requires accelerated selection and the development of new screening techniques. New analytical methods that are more accurate and less labor-intensive are needed to better characterize the effects of genetics and the environment on key traits under field conditions. We used persistent multispectral and photogrammetric UAV time-series imagery collected 10 times over the season, together with ground-truth data for thousands of Miscanthus genotypes, to determine the flowering time, culm length, and biomass yield traits. We compared the performance of convolutional neural network (CNN) architectures that used image data from single dates (2D-spatial) versus the integration of multiple dates by 3D-spatiotemporal architectures. The ability of UAV-based remote sensing to rapidly and non-destructively assess large-scale genetic variation in flowering time, height, and biomass production was improved through the use of 3D-spatiotemporal CNN architectures versus 2D-spatial CNN architectures. The performance gains of the best 3D-spatiotemporal analyses compared to the best 2D-spatial architectures manifested in up to 23% improvements in R2, 17% reductions in RMSE, and 20% reductions in MAE. The integration of photogrammetric and spectral features with 3D architectures was crucial to the improved assessment of all traits. In conclusion, our findings demonstrate that the integration of high-spatiotemporal-resolution UAV imagery with 3D-CNNs enables more accurate monitoring of the dynamics of key phenological and yield-related crop traits. This is especially valuable in highly productive, perennial grass crops such as Miscanthus, where in-field phenotyping is especially challenging and traditionally limits the rate of crop improvement through breeding.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Topography and Functional Traits Control the Distribution of Key Shrub Plant Functional Types in Low-Arctic Tundra: Supporting Data

High-resolution classification maps derived from occupied aerial systems (UASs). The UAS data were collected in August 2021 using a Skydio 2+ drone equipped with a 4K resolution red-green-blue (RGB) camera (2024 Skydio Inc) and a 3DR SOLO Quadcopter carried a Parrot Sequoia+ Multispectral Sensor (2023 Parrot Drone SAS). This package includes vegetation classification maps at four locations around Next Generation Ecosystem Experiment Arctic (NGEE Arctic) Council watershed study site on the Seward Peninsula, Alaska. The classification maps were generated using a combination of RGB and canopy height information. The map data and metadata are provided as ENVI image (.dat) and text (.txt, *hdr) formats. Additional map quicklooks are provided as GIS *.kml files. These datasets are provided in support of Yang et al., (In review), “Topography and Functional Traits Control the Distribution of Key Shrub Plant Functional Types in Low-Arctic Tundra”.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

AICCA: AI-Driven Cloud Classification Atlas

Clouds play an important role in the Earth’s energy budget, and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help in understanding cloud responses, but decades and petabytes of multispectral cloud imagery have to date received only limited use. This study describes a new analysis approach that reduces the dimensionality of satellite cloud observations by grouping them via a novel automated, unsupervised cloud classification technique based on a convolutional autoencoder, an artificial intelligence (AI) method good at identifying patterns in spatial data. Our technique combines a rotation-invariant autoencoder and hierarchical agglomerative clustering to generate cloud clusters that capture meaningful distinctions among cloud textures, using only raw multispectral imagery as input. Cloud classes are therefore defined based on spectral properties and spatial textures without reliance on location, time/season, derived physical properties, or pre-designated class definitions. We use this approach to generate a unique new cloud dataset, the AI-driven cloud classification atlas (AICCA), which clusters 22 years of ocean images from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua and Terra instruments—198 million patches, each roughly 100 km × 100 km (128 × 128 pixels)—into 42 AI-generated cloud classes, a number determined via a newly-developed stability protocol that we use to maximize richness of information while ensuring stable groupings of patches. AICCA thereby translates 801 TB of satellite images into 54.2 GB of class labels and cloud top and optical properties, a reduction by a factor of 15,000. The 42 AICCA classes produce meaningful spatio-temporal and physical distinctions and capture a greater variety of cloud types than do the nine International Satellite Cloud Climatology Project (ISCCP) categories—for example, multiple textures in the stratocumulus decks along the West coasts of North and South America. We conclude that our methodology has explanatory power, capturing regionally unique cloud classes and providing rich but tractable information for global analysis. AICCA delivers the information from multi-spectral images in a compact form, enables data-driven diagnosis of patterns of cloud organization, provides insight into cloud evolution on timescales of hours to decades, and helps democratize climate research by facilitating access to core data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Assessing Heterogeneity of Surface Water Temperature Following Stream Restoration and a High-Intensity Fire from Thermal Imagery

Thermal heterogeneity of rivers is essential to support freshwater biodiversity. Salmon behaviorally thermoregulate by moving from patches of warm water to cold water. When implementing river restoration projects, it is essential to monitor changes in temperature and thermal heterogeneity through time to assess the impacts to a river’s thermal regime. Lightweight sensors that record both thermal infrared (TIR) and multispectral data carried via unoccupied aircraft systems (UASs) present an opportunity to monitor temperature variations at high spatial (<0.5 m) and temporal resolution, facilitating the detection of the small patches of varying temperatures salmon require. Here, we present methods to classify and filter visible wetted area, including a novel procedure to measure canopy cover, and extract and correct radiant surface water temperature to evaluate changes in the variability of stream temperature pre- and post-restoration followed by a high-intensity fire in a section of the river corridor of the South Fork McKenzie River, Oregon. We used a simple linear model to correct the TIR data by imaging a water bath where the temperature increased from 9.5 to 33.4 °C. The resulting model reduced the mean absolute error from 1.62 to 0.35 °C. We applied this correction to TIR-measured temperatures of wetted cells classified using NDWI imagery acquired in the field. We found warmer conditions (+2.6 °C) after restoration (p < 0.001) and median absolute deviation for pre-restoration (0.30) to be less than both that of post-restoration (0.85) and post-fire (0.79) orthomosaics. In addition, there was statistically significant evidence to support the hypothesis of shifts in temperature distributions pre- and post-restoration (KS test 2009 vs. 2019, p < 0.001, D = 0.99; KS test 2019 vs. 2021, p < 0.001, D = 0.10). Moreover, we used a Generalized Additive Model (GAM) that included spatial and environmental predictors (i.e., canopy cover calculated from multispectral NDVI and photogrammetrically derived digital elevation model) to model TIR temperature from a transect along the main river channel. This model explained 89% of the deviance, and the predictor variables showed statistical significance. Collectively, our study underscored the potential of a multispectral/TIR sensor to assess thermal heterogeneity in large and complex river systems.

Barker, Matthew I. (ORCID:0000000252864930)↗

DOE OSTI2023

Validating a Tethered Balloon System and Optical Technologies for Marine Wildlife Detection and Tracking

The interactions between marine wildlife and marine energy devices are not well understood, leading to regulatory delays for device deployments and testing. Technologies that enable marine wildlife observations can help to fill data gaps and reduce uncertainties about animal–device interactions. A validation test conducted in Galveston Bay near La Porte, Texas, in December 2022 used a technology package consisting of a tethered balloon system and three independent sensor systems, including three-band visible, eight-band multispectral, and single-band thermal to detect three marine-mammal-shaped surrogates. The field campaign aimed to provide an initial step to evaluating the use of the TBS and the effectiveness of the sensor suite for marine wildlife observations and detection. From 2 December to 7 December 2022, 6 flights were conducted under varying altitudes and environmental conditions resulting in the collection of 5454 images. A subset of the images was classified and analyzed with two collection criteria including Beaufort wind force scale and TBS altitude to assess a range of observations of a surrogate from near-shore to offshore based on pixel count. The results of this validation test demonstrate the potential for using TBSs and imaging sensors for marine wildlife observations and offer valuable information for further development and application of this technology for marine energy and other blue economy sectors.

47 OTHER INSTRUMENTATION↗

DOE OSTI2025

Mapping tree height in complex terrain of northern China using ultra-high-resolution images

Tree height is a key parameter for estimating forest biomass and carbon sequestration. In recent years, notable progress has been made in mapping tree height using satellite imagery. However, existing tree height products show low accuracy in mountainous and complex terrains, and few studies typically addressed tree height estimations in mountain areas. This study examines the Mentougou district of Beijing, China, characterized by complex terrain and mountainous landscapes. We analyzed two methods for estimating tree height: one using only spectral features and another combining spectral features with topographic factors (elevation, slope, aspect). We used 3-m resolution PlanetScope 8-band multispectral imagery, with 710 field-measured individual tree heights averaged to obtain 471 pixel-level tree height values as ground-truth, to develop tree height prediction models using eXtreme Gradient Boosting (XGBoost), Random Forest (RF), and Gradient Boosting Machine (GBM) models. The results show that the XGBoost model consistently presented the highest accuracy for both methods evaluated. Specifically, the XGBoost model that combined spectral data with elevation and slope variables with an R² of 0.75 and an RMSE of 2.69 m. Using the XGBoost model, we generated the tree height map for the Mentougou area at 3 m resolution, showing tree heights ranging from 0.5 to 30.4 m, and the model’s prediction error standard deviations ranged from 2.50 to 4.71 m, indicating reliable performance across varied terrain. Additionally, we compared and evaluated the global tree height products, identifying limitations in the accuracy within complex terrains. This study demonstrates the potential for accurately predicting tree heights by combining high-resolution multispectral satellites with a terrain factor modeling approach.

Complex terrain↗

DOE OSTI2021

MULTI-MODAL global surveillance methodology for predictive and on-demand characterization of localized processes using cube satellite platforms and deep learning techniques

This paper presents the work completed towards the development of a multi-modal global surveillance methodology using cube satellite (CubeSat) platforms and novel data analysis techniques. A CubeSat system equipped with adequate sensors and data analytics capabilities can autonomously characterize various phenomena of interest on the Earth’s surface. CubeSats are advantageous over conventional satellites in certain remote monitoring applications because of their reduced construction costs (due to the availability of commercially-off-the-shelf components) and are easier to launch. The CubeSat surveillance system developed in this paper focused on phenomena of interest surrounding the nuclear fuel cycle in support of nuclear non-proliferation and emergency response. To observe the phenomena, a constellation of 3U and 6U CubeSats deployed from the ISS with adequate components was chosen. Four different sensor configurations were identified for remote sensing: panchromatic/multispectral in the visible and near-infrared spectrum, multispectral in infrared spectrum, hyperspectral in infrared spectrum, and multispectral in ultraviolet spectrum. While a panchromatic/multispectral sensor configuration has CubeSat flight heritage at the required spatial resolutions, the other three sensor types need future 3 development to meet signature and system requirements. Once each sensor onboard the CubeSat system collects data on a target of interest, the onboard computers would then apply the deep learning-based characterization methodology developed in this paper to identify phenomena. Four surrogate datasets containing representative simplified “images” were created for each sensor type to train the characterization methodology. A convolutional neural network was applied to each dataset and produced recall rates for the phenomena between 89.7% - 99.3% and precision rates between 92.3% - 99.9%. Each phenomenon’s presence probability from each network is then combined into a final characterization solution for a target area. This paper covers multiple interdisciplinary areas to develop the foundation for a CubeSat surveillance system focused on phenomena surrounding the nuclear fuel cycle.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2022

Evaluation of UAV-derived multimodal remote sensing data for biomass prediction and drought tolerance assessment in bioenergy sorghum

Screening for drought tolerance is critical to ensure high biomass production of bioenergy sorghum in arid or semi-arid environments. The bottleneck in drought tolerance selection is the challenge of accurately predicting biomass for a large number of genotypes. Although biomass prediction by low-altitude remote sensing has been widely investigated on various crops, the performance of the predictions are not consistent, especially when applied in a breeding context with hundreds of genotypes. In some cases, biomass prediction of a large group of genotypes benefited from multimodal remote sensing data; while in other cases, the benefits were not obvious. In this study, we evaluated the performance of single and multimodal data (thermal, RGB, and multispectral) derived from an unmanned aerial vehicle (UAV) for biomass prediction for drought tolerance assessments within a context of bioenergy sorghum breeding. The biomass of 360 sorghum genotypes grown under well-watered and water-stressed regimes was predicted with a series of UAV-derived canopy features, including canopy structure, spectral reflectance, and thermal radiation features. Biomass predictions using canopy features derived from the multimodal data showed comparable performance with the best results obtained with the single modal data with coefficients of determination (R 2 ) ranging from 0.40 to 0.53 under water-stressed environment and 0.11 to 0.35 under well-watered environment. The significance in biomass prediction was highest with multispectral followed by RGB and lowest with the thermal sensor. Finally, two well-recognized yield-based drought tolerance indices were calculated from ground truth biomass data and UAV predicted biomass, respectively. Results showed that the geometric mean productivity index outperformed the yield stability index in terms of the potential for reliable predictions by the remotely sensed data. Collectively, this study demonstrated a promising strategy for the use of different UAV-based imaging sensors to quantify yield-based drought tolerance.

09 BIOMASS FUELS↗

DOE OSTI2020

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗

DOE OSTI2022

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗

DOE OSTI2023

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗

DOE OSTI2023

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗

DOE OSTI2024

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗

DOE OSTI2024

Window obscuration sensors for mobile gas and chemical imaging cameras

An infrared (IR) imaging system for determining a concentration of a target species in an object is disclosed. The imaging system can include an optical system including a focal plane array (FPA) unit behind an optical window. The optical system can have components defining at least two optical channels thereof, said at least two optical channels being spatially and spectrally different from one another. Each of the at least two optical channels can be positioned to transfer IR radiation incident on the optical system towards the optical FPA. The system can include a processing unit containing a processor that can be configured to acquire multispectral optical data representing said target species from the IR radiation received at the optical FPA. One or more of the optical channels may be used in detecting objects on or near the optical window, to avoid false detections of said target species.

Mallery, Ryan↗