SEARCH · Engineering Papers

Results for “Unmanned Aerial Vehicles”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Multispectral UAV imagery of experimental freshwater wetlands under 5 ppt saltwater intrusion, Louisiana, 2023 and 2024

Multispectral imagery was collected using an unmanned aerial vehicle (UAV) to evaluate how freshwater vegetation responds to short-term simulated saltwater intrusion events. The purpose of this data collection was to understand how plant health changes in response to acute salinity exposure, which is increasingly relevant in coastal wetland ecosystems facing sea level rise and storm surge events, such as in coastal Louisiana. Three experimental saltwater intrusions were conducted at a salinity of approximately 5 parts per thousand (ppt) for durations of 6-days, 10-days, and 17-days. UAV flights occurred both before and after each treatment. The resulting imagery was processed using Pix4DMapper software to georeference the images and generate orthomosaics. The multispectral sensor used in this study captures reflectance in five bands: blue, green, red, red-edge, and near-infrared. The uploaded data consist of georeferenced .tif orthomosaics for each spectral band, which are compatible with GIS software for vegetation analysis. This imagery can be utilized in investigations into vegetation stress, remote sensing of freshwater wetland ecosystems, and modeling of plant response to environmental changes.

EARTH SCIENCE > BIOSPHERE > ECOSYSTEMS↗

DOE OSTI2020

Report from the Inaugural Meeting on the Opportunities at the SEEMS Facility June 11-12, 2019

The Inaugural Meeting on the Opportunities at the Single Event Effects—Muon Spectroscopy (SEEMS) facility was held June 11–12, 2019, at Oak Ridge National Laboratory (ORNL). Capabilities of the SEEMS facility concept were reviewed, with emphasis on particle beams for single event effects (SEE) testing of electronic devices (integrated circuits) and systems—a topic of growing interest and concern in aerospace avionics, satellites, supercomputing, autonomous cars, unmanned aerial vehicles and more. Participants from these industries and their regulators attended. A clear consensus on the growing and urgent need for SEE test capabilities like those offered by SEEMS was expressed. Subsequent to the meeting, letters of support for SEEMS were sent by The Boeing Company and Honeywell to the ORNL Laboratory Director, Thomas Zacharia.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

Material Developments for 3D/4D Additive Manufacturing (AM) Technologies

Additive Manufacturing (AM), or 3D printing, is a unique technology in which structurally complex objects can be easily manufactured. While AM allows for the creation of intricate 3D objects, these objects are inactive and motionless. With recent incorporation of a “pre-programmed functionality’’ to the 3D printed objects, a new concept has emerged, 4D printing. Specifically, the 4D printing technology enables a static 3D printed object to change its shape, functionality or property over time upon exposure to a specific stimulus such as heat, stress, light, pH, and moisture, etc. We designed and produced compact 3D/4D printable materials that can be used as (a) structural components of the unmanned aerial vehicles (UAVs), and (b) as hydrogen storage materials that supply hydrogen to a fuel cell to expand the operational reach, namely flight time, and endurance of the UA Vs.

36 MATERIALS SCIENCE↗

DOE OSTI2021

A Heterogeneous System for Eagle Detection, Deterrent, and Wildlife Collision Detection for Wind Turbines (Final Technical Report)

This report summarizes the design, implementation, and test of an integrated system for automated detection and deterrence of eagles, with included wind turbine blade strike detection and imaging functionality. A machine learning approach was used in conjunction with a 360° camera system for automated detection and classification of golden eagles. This was developed using footage obtained from trained golden eagles and other raptors, in collaboration with wildlife biologists and professional bird handlers. Oregon State University developed a visual deterrent system, which uses inflatable anthropomorphic sculptures with random, kinetic motion to deter eagles, and conducted limited field testing on live eagles; the deterrent can be triggered by the visual detection of eagles using the vision system. Finally, a multi-sensor module was developed that is mounted at the turbine blade root. This module measures vibration and other motions to detect blade strikes, and an integrated on-blade camera captures an image of any impacting objects. Long-term, this blade strike detection system is intended to support an automatic monitoring and certification system for the eagle detection and deterent system. Independent field testing of each system component is described. Testing of the integrated system on an operational wind turbine was conducted across three separate field tests. This includes multi-day fields tests on a General Electric 1.5MW wind turbine at the National Renewable Energy Laboratory (NREL) National Wind Technology Center (NWTC) in Boulder, CO in October 2018 and July 2019; installation procedures, test procedures, and a summary of collected data are presented. A third multi-day on-turbine field test is also presented, which was performed using a General Electric 1.5MW wind turbine at the North American Wind Research and Training Center (NAWRTC) at Mesalands Community College, Tucumcari, NM in April 2019. Across these field tests, the vision system was demonstrated using unmanned aerial vehicles (UAV), and the eagle classification algorithm was not tested; the visual deterrent system was demonstrated, including automatic, remote deployment following surrogate visual detections; and, multi-sensor on-blade data was recorded across multiple wind turbine operational conditions and through more than 100 surrogate blade strikes using soft projectiles, including the successful demonstration of automatic image capture of striking objects. This data set was also used for offline development and validation of enhanced collision detection algorithms. As summarized in this report, the development and field validation of an integrated detection, deterrent, and blade collision detection system represents a critical proof of concept for future technology development of related detection and deterrent technologies, where both deterrent as well as collision detection recording devices are needed for future siting, monitoring, and operation of wind turbine installations, both onshore and offshore.

17 WIND ENERGY↗

DOE OSTI2022

Material Developments for 3D/4D Additive Manufacturing (AM) Technologies

Additive Manufacturing (AM), or 3D printing, is a unique technology in which structurally complex objects can be easily manufactured. While AM allows for the creation of intricate 3D objects, these objects are inactive and motionless. With recent incorporation of a “pre-programmed functionality” to the 3D printed objects, a new concept has emerged, 4D printing. Specifically, the 4D printing technology enables a static 3D printed object to change its shape, functionality or property over time upon exposure to a specific stimulus such as heat, stress, light, pH, and moisture, etc. We designed and produced compact 3D/4D printable materials that can be used as (a) structural components of the unmanned aerial vehicles (UAVs), and (b) as hydrogen storage materials that supply hydrogen to a fuel cell to expand the operational reach, namely flight time, and endurance of the UAVs.

08 HYDROGEN↗

DOE OSTI2022

Legacy surface change analyses of the 1993 Rock Valley earthquake sequence for direct comparison to planned NA-22 underground conventional high-explosive experiments (Source Physics Experiment 3 (SPE3) - RV/DC Task 1.3 FY22 Final Report)

Recent work under two previous phases of the NNSA NA-22 Source Physics Experiment (SPE) have shown that underground chemical high-explosive experiments can produce detectable surface changes that differ in spatial extent and vertical magnitude depending on the geologic media at the site (Schultz-Fellenz et al., 2018; 2020; Crawford et al., 2021). Neither of these two prior phases of SPE identified natural earthquake-related surface effects in the same region that occurred at a similar depth as the explosive experiments for direct comparison. While earthquakes can also produce surface changes that are detectable using remote sensing data analyses, it is expected that the pattern and spatial extent of surface changes would vary between earthquakes and explosions. However, no direct-observed surface-change signature comparison between earthquakes and explosions has ever been performed. The SPE Phase 3 Rock Valley/Direct Comparison (RV/DC) program presents a unique opportunity to investigate and characterize co-occurrence of both earthquakes and explosions. In this project, we worked to address five tasks in a workflow, as follows: 1. Identify and obtain existing high-resolution legacy satellite and aerial imagery as close in time before and after the 1993 Rock Valley earthquake sequence to temporally constrain the analyses. 2. Transform these pre-earthquake and post-earthquake datasets into digital elevation models (DEMs) using geospatial analysis software packages (e.g., ArcGIS, Agisoft Metashape, and Google Earth Engine). 3. Perform DEM differencing analyses to assess and quantify earthquake-related changes from the 1993 sequence, and develop map products that visualize these analyses. 4. Use the analyses from (3) to: (a) assess spatial distribution and magnitude of surface changes due to the 1993 Rock Valley earthquake sequence, and (b) determine parameters of forthcoming, planned explosion-related surface change data collection from sensors mounted on unmanned aerial vehicles (UAVs) (e.g., spatial extent of collection, design and density of survey control, sensors to deploy, forward speed and line spacing of UAV flight lines, flight altitude). 5. Develop a summary report on the analyses, including how the analyses define parameters and identify focus areas for any future surface change analytical field campaigns related to the explosive experiment. Analyzing these legacy data and identifying whether they can detect any surface changes related to the earthquake sequence facilitates opportunities for direct signature comparison of surface change from explosions at one location, which has never previously been performed. Comparing the surface change signatures from a co-located and depth-equivalent earthquake and an explosion could help to advance remote sensing event discrimination techniques. This report summarizes the work completed toward this ambitious goal.

42 ENGINEERING↗

DOE OSTI2024

Mid-IR UAV-based sensing platform with deep learning to Identify and Quantify Gaseous Emission in Gas Flares

This report details the development and evaluation of a Mid-Infrared (Mid-IR) Unmanned Aerial Vehicle (UAV)-based sensing platform integrated with deep learning algorithms for the identification and quantification of gaseous emissions in gas flares. The project, spearheaded by Omega Optics, Inc., aimed to address environmental monitoring challenges by leveraging advanced photonic technologies and autonomous UAV operations. The research focused on designing, optimizing, and fabricating photonic crystal waveguides and grating couplers to enhance the sensitivity and accuracy of gas detection. A comprehensive drone-based system was developed, featuring a miniaturized sensor, GPS module, and microcontroller communication network for real-time gas concentration monitoring. The system's adaptive sampling algorithm, implemented using the Robot Operating System (ROS), enables autonomous detection and localization of gas emission sources. Preliminary results demonstrate the platform's capability to detect and monitor gas emissions with high precision, cost-effectiveness, and scalability. Future work will expand upon this foundation by introducing 3D wind model-based learning for dynamic environmental conditions and further enhancing the user interface and data processing algorithms to support broader environmental monitoring applications. Overall, this project represents a significant step forward in UAV-based environmental sensing technologies, offering robust solutions for detecting and mitigating the impacts of gaseous emissions on public health and safety.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Evaluating the Effectiveness of a Detection and Deterrent System in Reducing Golden Eagle Fatalities at Operational Wind Facilities

The Renewable Energy Wildlife Institute (REWI) was appointed as the prime awardee of DOE award number DE-EE0007883 to lead a team of scientists, wind developers, and technology manufacturers toward the overarching goal of evaluating the effectiveness of the current DTBird system in minimizing the risk of golden eagles (Aquila chrysaetos) and other large soaring raptors from approaching the rotor-swept zone (RSZ) of operating wind turbines. As part of this goal, the team set out to 1) quantify the expected reduction in collision risk for golden eagles from operation of the detection and deterrence modules in a manner that supports the approach used by the U.S. Fish and Wildlife Service (USFWS) to assess and credit facility operators for their efforts to minimize predicted collision fatalities and 2) provide information to help improve the technology to maximize its effectiveness. DTBird is an automated detection and audio deterrent system created by the Spanish company Liquen, designed to discourage birds from entering the RSZ of spinning wind turbines. The system uses cameras to automatically detect airborne targets of interest, records each such event in an online database, and triggers a warning signal (loud sound) if the tracked object has moved close to the turbine. If the object moves even closer to the RSZ, a more aggressive dissuasion signal is broadcast. To meet our objectives, the team conducted a two-year experiment at the Goodnoe Hills wind facility in Washington state, in which 14 turbines were outfitted with DTBird units. Daily, each DTBird-equipped turbine was randomly assigned to a control or treatment group. Treatment turbines operated with DTBird running as intended—broadcasting warning or deterrent signals when DTBird detected a target within range. On control turbines, no sound signals were broadcast if a moving target triggered the DTBird system. The team also flew unmanned aerial vehicles (UAVs) designed to coarsely mimic the general size, weight, and coloration of golden eagles in programmed flight transects across DTBird detection ranges to quantify DTBird’s ability to detect intended targets and to evaluate factors that influence the probability of detection and DTBird’s response distances. Additionally, the team evaluated the behavioral responses of in situ eagles exposed to spinning turbines alone (visual and sound influences) versus spinning turbines plus broadcasted DTBird audio deterrents, to estimate the effectiveness of deterrence by the DTBird system. The data and results from these investigations were combined with those from a pilot study conducted at the Manzana Wind Power Project in California to better evaluate DTBird’s effectiveness across different landscapes.

17 WIND ENERGY↗

DOE OSTI2023

A Power Takeoff Device for a Small Marine Hydrokinetic Turbine Deployed from an Unmanned Floating Platform

The design, development, and field-testing of a power takeoff (PTO) device equipped with a ball-type continuously-variable transmission (B-CVT) for a small marine hydrokinetic (MHK) turbine deployed from a floating unmanned autonomous mobile catamaran platform is described. The turbine is a partially-submerged multi-blade undershot waterwheel (USWW). The objective is to develop a PTO for the optimal conversion of the MHK energy harnessed by the turbine to electric power, which is stored in battery banks onboard the MHK platform. Modeling, simulation, and bench testing of the USWW and PTO show the feasibility of utilizing the B-CVT's variable gear ratio to decouple the USWW and generator speeds, maintaining the waterwheel within its optimal tip speed ratio (TSR) while varying the generator speed, thereby increasing the efficiency of the PTO. Results of bench and field testing in support of characterizing the power conversion capabilities of the PTO are described. The system being developed is in support of potential self-powered autonomous mobile recharge stations for unmanned aerial vehicles (U A V s) operating in coastal zones. Field tests of the complete MHK platform with 9 blades on the waterwheel and wheel sub mergences of 10 and 12 inches (full blade submergence) were performed. The overall proof-of-concept was successfully demonstrated, with the system satisfactorily capturing and converting water flow energy into electricity. The feasibility of utilizing the B-CVT as a means of increasing the PTO power capture capabilities and efficiency is analyzed.

Pimentel, Hugo↗

DOE OSTI2022

Sustainability Trait Modeling of Field-Grown Switchgrass (Panicum virgatum) Using UAV-Based Imagery

CBI↗

DOE OSTI2022

Intraspecific Variation for Leaf Physiological and Root Morphological Adaptation to Drought Stress in Alfalfa ( Medicago sativa L.)

Drought stress reduces crop biomass yield and the profitability of rainfed agricultural systems. Evaluation of populations or accessions adapted to diverse geographical and agro-climatic environments sheds light on beneficial plant responses to enhance and optimize yield in resource-limited environments. This study used the morphological and physiological characteristics of leaves and roots from two different alfalfa subspecies during progressive drought stress imposed on controlled and field conditions. Two different soils (Experiments 1 and 2) imposed water stress at different stress intensities and crop stages in the controlled environment. Algorithm-based image analysis of leaves and root systems revealed key morphological and physiological traits associated with biomass yield under stress. The Medicago sativa subspecies (ssp.) sativa population, PI478573, had smaller leaves and maintained higher chlorophyll content (CC), leaf water potential, and osmotic potential under water stress. In contrast, M. sativa ssp. varia, PI502521, had larger leaves, a robust root system, and more biomass yield. In the field study, an unmanned aerial vehicle survey revealed PI502521 to have a higher normalized difference vegetation index (vegetation cover and plant health characteristics) throughout the cropping season, whereas PI478573 values were low during the hot summer and yielded low biomass in both irrigated and rainfed treatments. RhizoVision Explorer image analysis of excavated roots revealed a smaller diameter and a narrow root angle as target traits to increase alfalfa biomass yield irrespective of water availability. Root architectural traits such as network area, solidity, volume, surface area, and maximum radius exhibited significant variation at the genotype level only under limited water availability. Different drought-adaptive strategies identified across subspecies populations will benefit the plant under varying levels of water limitation and facilitate the development of alfalfa cultivars suitable across a broad range of growing conditions. The alleles from both subspecies will enable the development of drought-tolerant alfalfa with enhanced productivity under limited water availability.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

A novel transfer learning framework for sorghum biomass prediction using UAV-based remote sensing data and genetic markers

Yield for biofuel crops is measured in terms of biomass, so measurements throughout the growing season are crucial in breeding programs, yet traditionally time- and labor-consuming since they involve destructive sampling. Modern remote sensing platforms, such as unmanned aerial vehicles (UAVs), can carry multiple sensors and collect numerous phenotypic traits with efficient, non-invasive field surveys. However, modeling the complex relationships between the observed phenotypic traits and biomass remains a challenging task, as the ground reference data are very limited for each genotype in the breeding experiment. In this study, a Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) model is proposed for sorghum biomass prediction. The architecture is designed to exploit the time series remote sensing and weather data, as well as static genotypic information. As a large number of features have been derived from the remote sensing data, feature importance analysis is conducted to identify and remove redundant features. A strategy to extract representative information from high-dimensional genetic markers is proposed. To enhance generalization and minimize the need for ground reference data, transfer learning strategies are proposed for selecting the most informative training samples from the target domain. Consequently, a pre-trained model can be refined with limited training samples. Field experiments were conducted over a sorghum breeding trial planted in multiple years with more than 600 testcross hybrids. The results show that the proposed LSTM-based RNN model can achieve high accuracies for single year prediction. Further, with the proposed transfer learning strategies, a pre-trained model can be refined with limited training samples from the target domain and predict biomass with an accuracy comparable to that from a trained-from-scratch model for both multiple experiments within a given year and across multiple years.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Optimizing Deep Learning Models for Climate-Related Natural Disaster Detection from UAV Images and Remote Sensing Data

This research study utilized artificial intelligence (AI) to detect natural disasters from aerial images. Flooding and desertification were two natural disasters taken into consideration. The Climate Change Dataset was created by compiling various open-access data sources. This dataset contains 6334 aerial images from UAV (unmanned aerial vehicles) images and satellite images. The Climate Change Dataset was then used to train Deep Learning (DL) models to identify natural disasters. Four different Machine Learning (ML) models were used: convolutional neural network (CNN), DenseNet201, VGG16, and ResNet50. These ML models were trained on our Climate Change Dataset so that their performance could be compared. DenseNet201 was chosen for optimization. All four ML models performed well. DenseNet201 and ResNet50 achieved the highest testing accuracies of 99.37% and 99.21%, respectively. This research project demonstrates the potential of AI to address environmental challenges, such as climate change-related natural disasters. This study’s approach is novel by creating a new dataset, optimizing an ML model, cross-validating, and presenting desertification as one of our natural disasters for DL detection. Three categories were used (Flooded, Desert, Neither). Our study relates to AI for Climate Change and Environmental Sustainability. Drone emergency response would be a practical application for our research project.

AI↗

DOE OSTI2020

Flexible Magnetic Polymer Composite Substrate with Ba 1.5 Sr 1.5 Z Hexaferrite Particles of VHF/Low UHF Patch Antennas for UAVs and Medical Implant Devices

Our goal is to fabricate flexible magnetic polymer composites as antenna substrates for very high frequency (VHF)/low ultra high frequency (UHF) antennas for unmanned aerial vehicles (UAVs) and medical devices. Magnetodielectric materials, which have permeability (μ) similar to permittivity (ε), have attracted great attention, because they facilitate miniaturization of microwave devices while keeping or enhancing electromagnetic characteristics. Mechanically millled Ba 1.5 Sr 1.5 Co 2 Fe 24 O 41 (Ba 1.5 Sr 1.5 Z) hexaferrite particles were used to increase permeability in the interesting frequency band. The microwave properties of Ba 1.5 Sr 1.5 Z composites were predicted and measured. Hansen’s zero-order analysis of antenna bandwidth and electromagnetic field simulation showed that the hexaferrite-based flexible composite could enhance a bandwidth and achieve the miniaturization of antennas as well. The magnetic antenna substrates can be a good solution to integrate antennas into the UAVs whose dimensions are comparable to or larger than communication wavelength.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Sustainability Trait Modeling of Field-Grown Switchgrass ( Panicum virgatum ) Using UAV-Based Imagery

Unmanned aerial vehicles (UAVs) provide an intermediate scale of spatial and spectral data collection that yields increased accuracy and consistency in data collection for morphological and physiological traits than satellites and expanded flexibility and high-throughput compared to ground-based data collection. In this study, we used UAV-based remote sensing for automated phenotyping of field-grown switchgrass (Panicum virgatum), a leading bioenergy feedstock. Using vegetation indices calculated from a UAV-based multispectral camera, statistical models were developed for rust disease caused by Puccinia novopanici, leaf chlorophyll, nitrogen, and lignin contents. For the first time, UAV remote sensing technology was used to explore the potentials for multiple traits associated with sustainable production of switchgrass, and one statistical model was developed for each individual trait based on the statistical correlation between vegetation indices and the corresponding trait. Also, for the first time, lignin content was estimated in switchgrass shoots via UAV-based multispectral image analysis and statistical analysis. The UAV-based models were verified by ground-truthing via correlation analysis between the traits measured manually on the ground-based with UAV-based data. The normalized difference red edge (NDRE) vegetation index outperformed the normalized difference vegetation index (NDVI) for rust disease and nitrogen content, while NDVI performed better than NDRE for chlorophyll and lignin content. Overall, linear models were sufficient for rust disease and chlorophyll analysis, but for nitrogen and lignin contents, nonlinear models achieved better results. As the first comprehensive study to model switchgrass sustainability traits from UAV-based remote sensing, these results suggest that this methodology can be utilized for switchgrass high-throughput phenotyping in the field.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

Advancing Tassel Detection and Counting: Annotation and Algorithms

Tassel counts provide valuable information related to flowering and yield prediction in maize, but are expensive and time-consuming to acquire via traditional manual approaches. High-resolution RGB imagery acquired by unmanned aerial vehicles (UAVs), coupled with advanced machine learning approaches, including deep learning (DL), provides a new capability for monitoring flowering. In this article, three state-of-the-art DL techniques, CenterNet based on point annotation, task-aware spatial disentanglement (TSD), and detecting objects with recursive feature pyramids and switchable atrous convolution (DetectoRS) based on bounding box annotation, are modified to improve their performance for this application and evaluated for tassel detection relative to Tasselnetv2+. The dataset for the experiments is comprised of RGB images of maize tassels from plant breeding experiments, which vary in size, complexity, and overlap. Results show that the point annotations are more accurate and simpler to acquire than the bounding boxes, and bounding box-based approaches are more sensitive to the size of the bounding boxes and background than point-based approaches. Overall, CenterNet has high accuracy in comparison to the other techniques, but DetectoRS can better detect early-stage tassels. The results for these experiments were more robust than Tasselnetv2+, which is sensitive to the number of tassels in the image.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Detection of Outliers in LiDAR Data Acquired by Multiple Platforms over Sorghum and Maize

High-resolution point cloud data acquired with a laser scanner from any platform contain random noise and outliers. Therefore, outlier detection in LiDAR data is often necessary prior to analysis. Applications in agriculture are particularly challenging, as there is typically no prior knowledge of the statistical distribution of points, plant complexity, and local point densities, which are crop-dependent. The goals of this study were first to investigate approaches to minimize the impact of outliers on LiDAR acquired over agricultural row crops, and specifically for sorghum and maize breeding experiments, by an unmanned aerial vehicle (UAV) and a wheel-based ground platform; second, to evaluate the impact of existing outliers in the datasets on leaf area index (LAI) prediction using LiDAR data. Two methods were investigated to detect and remove the outliers from the plant datasets. The first was based on surface fitting to noisy point cloud data via normal and curvature estimation in a local neighborhood. The second utilized the PointCleanNet deep learning framework. Both methods were applied to individual plants and field-based datasets. To evaluate the method, an F-score was calculated for synthetic data in the controlled conditions, and LAI, the variable being predicted, was computed both before and after outlier removal for both scenarios. Results indicate that the deep learning method for outlier detection is more robust than the geometric approach to changes in point densities, level of noise, and shapes. The prediction of LAI was also improved for the wheel-based vehicle data based on the coefficient of determination (R2) and the root mean squared error (RMSE) of the residuals before and after the removal of outliers.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Implementing Spatio-Temporal 3D-Convolution Neural Networks and UAV Time Series Imagery to Better Predict Lodging Damage in Sorghum

Unmanned aerial vehicle (UAV)-based remote sensing is gaining momentum in a variety of agricultural and environmental applications. Very-high-resolution remote sensing image sets collected repeatedly throughout a crop growing season are becoming increasingly common. Analytical methods able to learn from both spatial and time dimensions of the data may allow for an improved estimation of crop traits, as well as the effects of genetics and the environment on these traits. Multispectral and geometric time series imagery was collected by UAV on 11 dates, along with ground-truth data, in a field trial of 866 genetically diverse biomass sorghum accessions. We compared the performance of Convolution Neural Network (CNN) architectures that used image data from single dates (two spatial dimensions, 2D) versus multiple dates (two spatial dimensions + temporal dimension, 3D) to estimate lodging detection and severity. Lodging was detected with 3D-CNN analysis of time series imagery with 0.88 accuracy, 0.92 Precision, and 0.83 Recall. This outperformed the best 2D-CNN on a single date with 0.85 accuracy, 0.84 Precision, and 0.76 Recall. The variation in lodging severity was estimated by the best 3D-CNN analysis with 9.4% mean absolute error (MAE), 11.9% root mean square error (RMSE), and goodness-of-fit (R2) of 0.76. This was a significant improvement over the best 2D-CNN analysis with 11.84% MAE, 14.91% RMSE, and 0.63 R2. The success of the improved 3D-CNN analysis approach depended on the inclusion of “before and after” data, i.e., images collected on dates before and after the lodging event. The integration of geometric and spectral features with 3D-CNN architecture was also key to the improved assessment of lodging severity, which is an important and difficult-to-assess phenomenon in bioenergy feedstocks such as biomass sorghum. This demonstrates that spatio-temporal CNN architectures based on UAV time series imagery have significant potential to enhance plant phenotyping capabilities in crop breeding and Precision agriculture applications.

3D-convolution neural networks↗