Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “cloud classification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

AICCA: AI-Driven Cloud Classification Atlas

Clouds play an important role in the Earth’s energy budget, and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help in understanding cloud responses, but decades and petabytes of multispectral cloud imagery have to date received only limited use. This study describes a new analysis approach that reduces the dimensionality of satellite cloud observations by grouping them via a novel automated, unsupervised cloud classification technique based on a convolutional autoencoder, an artificial intelligence (AI) method good at identifying patterns in spatial data. Our technique combines a rotation-invariant autoencoder and hierarchical agglomerative clustering to generate cloud clusters that capture meaningful distinctions among cloud textures, using only raw multispectral imagery as input. Cloud classes are therefore defined based on spectral properties and spatial textures without reliance on location, time/season, derived physical properties, or pre-designated class definitions. We use this approach to generate a unique new cloud dataset, the AI-driven cloud classification atlas (AICCA), which clusters 22 years of ocean images from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua and Terra instruments—198 million patches, each roughly 100 km × 100 km (128 × 128 pixels)—into 42 AI-generated cloud classes, a number determined via a newly-developed stability protocol that we use to maximize richness of information while ensuring stable groupings of patches. AICCA thereby translates 801 TB of satellite images into 54.2 GB of class labels and cloud top and optical properties, a reduction by a factor of 15,000. The 42 AICCA classes produce meaningful spatio-temporal and physical distinctions and capture a greater variety of cloud types than do the nine International Satellite Cloud Climatology Project (ISCCP) categories—for example, multiple textures in the stratocumulus decks along the West coasts of North and South America. We conclude that our methodology has explanatory power, capturing regionally unique cloud classes and providing rich but tractable information for global analysis. AICCA delivers the information from multi-spectral images in a compact form, enables data-driven diagnosis of patterns of cloud organization, provides insight into cloud evolution on timescales of hours to decades, and helps democratize climate research by facilitating access to core data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Using Downwelling Far- and Thermal-Infrared Hyperspectral Radiance for Cloud Phase Classification in the Antarctic

The cloud phase is one of the most important parameters of clouds. In this paper, we propose a method for cloud phase classification that synergistically utilizes the far- and thermal-infrared bands based on the Atmospheric Emitted Radiance Interferometer (AERI) at the Atmospheric Radiation Measurement West Antarctic Radiation Experiment (AWARE) observatory in 2016. The possible features in the far- and thermal-infrared bands are analyzed based on the differences in the simulated cloud brightness temperature (BT) spectra with different cloud phases. Using the support vector machine (SVM) algorithm, four features are determined to identify the cloud phase, which include the BT at 900 cm -1 , the slope of the fitted function of BT in the 900–1000 cm -1 interval, the BT difference (BTD) between 512 cm -1 and 726 cm -1 , and the BTD between 550 cm -1 and 726 cm -1 . Here, the performance of the proposed method is evaluated with Shupe’s and Turner’s method. The monthly average accuracy of the proposed method, the method without the two far-infrared features, and Turner’s method are about 76%, 36%, and 49%, respectively, which infer the good performance of the proposed method and also indicate that the far-infrared band features can effectively enhance cloud phase classification. It is notable that, compared to Shupe’s method, the accuracy for the proposed method is only 61% during the Antarctic summer, which results from the definitions of cloud phase and radiative effect. In addition, the accuracy is only 44% for Turner’s method in seasons with a low frequency of mixed clouds due to the significant effect of water vapor.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Thermodynamic Cloud Phase Classifications Using Machine Learning at NSA and ANX

Vertically resolved thermodynamic cloud phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) THERMOCLDPHASE Value-Added Product (VAP) uses a multi-sensor approach to classify thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave radiometer-derived liquid water path, and radiosonde temperature measurements. The measured voxels are classified as ice, snow, mixed-phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multilayer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with one year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1-score, and mean Intersection over Union (IOU). Analysis of ML confidence scores shows ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential datastreams for ML thermodynamic cloud phase predictions. The ML models’ generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. Finally, we evaluate the ML models’ response to simulated instrument outages and signal degradation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Classification of Cloud Particle Imagery and Thermodynamics (COCPIT): A New Databasing Tool for the Characterization of Cloud Particle Images Captured During DOE Field Campaigns

The Department of Energy for decades has explored the earth system and atmosphere through research and deployment of in-situ and remote sensing platforms during field campaigns. Among these datasets exists a vast supply of cloud particle images that provide visual insight into the complex microphysics in the clouds that span our globe. The millions of images collected over decades of deployments provides a unique opportunity to further our understanding of our atmosphere down to the crystal size. This work over the past 5 years has sought to organize these images into digestible datasets that can then be used by scientists to further our understanding of microphysics. A machine learning model was developed that categorizes over 1.5 million images across 11 weather events with over 90% accuracy according to particle type. The database was then extended to include dimensional characteristics of the particle as well as co-location of environmental properties, such as temperature and water content. Then, to initialize the connection between these data and our understanding of how crystals form and grow, weather research and forecasting simulations were run to generate the growth histories of the classified crystals. This research culminates with 2 databases per event: (1) a database of all classified crystals and their dimensional and environmental properties and (2) simulated growth histories of each crystal. Finally, a user interface was created to allow researchers to explore data statistics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Spatially Aware Linear Transformer (SAL-T) for Particle Jet Tagging

Transformers are very effective in capturing both global and local correlations within high-energy particle collisions, but they present deployment challenges in high-data-throughput environments, such as the CERN LHC. The quadratic complexity of transformer models demands substantial resources and increases latency during inference. In order to address these issues, we introduce the Spatially Aware Linear Transformer (SAL-T), a physics-inspired enhancement of the linformer architecture that maintains linear attention. Our method incorporates spatially aware partitioning of particles based on kinematic features, thereby computing attention between regions of physical significance. Additionally, we employ convolutional layers to capture local correlations, informed by insights from jet physics. In addition to outperforming the standard linformer in jet classification tasks, SAL-T also achieves classification results comparable to full-attention transformers, while using considerably fewer resources with lower latency during inference. Experiments on a generic point cloud classification dataset (ModelNet10) further confirm this trend. Our code is available at https://github.com/aaronw5/SAL-T4HEP.

Wang, Aaron [Illinois U., Chicago] (ORCID:00000003↗
DOE OSTI2023

Warm‐Season Afternoon Precipitation Peak in the Central Bay of Bengal: Process‐Oriented Diagnostics

Abstract Past studies have indicated that precipitation over tropical open oceans generally peaks in the early morning. However, an intriguing departure from this pattern is observed in the central Bay of Bengal (CBoB), where rainfall exhibits a distinct afternoon peak during the South Asian summer monsoon season. By using a novel satellite‐based cloud classification and tracking data set, we found that more than 75% of the afternoon rainfall (15–17 LST) over the CBoB comes from mesoscale convective systems (MCSs). Most of the MCSs contributing to the CBoB afternoon rainfall peak originate either locally over the CBoB or near the west and east coasts of the BoB, in contrast to the northern BoB as highlighted in previous studies. Analyses show that MCSs initiated near coastlines are primarily influenced by land‐sea breezes, whereas MCSs initiated over the BoB open ocean during early morning are strongly associated with diurnal radiative forcings. In addition, there are clear diurnal propagating MCS initiation signals from the west and north coastlines of the BoB to the CBoB, which are related to diurnal gravity waves emitted from the coastlines. The thermodynamic conditions conducive to MCS initiation over different sub‐regions of the BoB are also investigated. No systematic differences found in environmental convective available potential energy between days with and without MCS initiation. However, over most sub‐regions, days with MCS initiation generally have higher total column water vapor than days without MCS initiation. This difference suggests that the lower‐free‐tropospheric moisture content plays an important role in MCS initiation over the BoB.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Performance of New Near-Real-Time PERSIANN Product (PDIR-Now) for Atmospheric River Events over the Russian River Basin, California

Most heavy precipitation events and extreme flooding over the U.S. Pacific coast can be linked to prevalent atmospheric river (AR) conditions. Thus, reliable quantitative precipitation estimation with a rich spatiotemporal resolution is vital for water management and early warning systems of flooding and landslides over these regions. At the same time, high-quality near-real-time measurements of AR precipitation remain challenging due to the complex topographic features of land surface and meteorological conditions of the region: specifically, orographic features occlude radar measurements while infrared-based algorithms face challenges, differentiating between both cold brightband (BB) precipitation and the warmer nonbrightband (NBB) precipitation. It should be noted that the latter precipitation is characterized by greater orographic enhancement. In this study, we evaluate the performance of a recently developed near-real-time satellite precipitation algorithm: Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN) Dynamic Infrared–Rain Rate-Now (PDIR-Now). This model is primarily dependent on infrared information from geostationary satellites as input; consequently, PDIR-Now has the advantage of short data latency, 15–60-min delay between observation to precipitation product delivery. The performance of PDIR-Now is analyzed with a focus on AR-related events for cases dominated by NBB and BB precipitation over the Russian River basin. In our investigations, we utilize S-band (3-GHz) precipitation profilers with Joss/Parsivel disdrometer measurements at the Middletown and Santa Rosa stations to classify BB and NBB precipitation events. In general, our analysis shows that PDIR-Now is more skillful in retrieving precipitation rates over both BB and NBB events across the topologically complex study area as compared to PERSIANN-Cloud Classification System (CCS). Also, we discuss the performance of well-known operational near-real-time precipitation products from 2017 to 2019. Conventional categorical and volumetric categorical indices, as well as continuous statistical metrics, are used to show the differences between various high-resolution precipitation products such as Multi-Radar Multi-Sensor (MRMS).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

The NASA ACTIVATE Mission

The NASA Aerosol Cloud Meteorology Interactions over the Western Atlantic Experiment (ACTIVATE) conducted 162 joint flights with two aircraft over the northwest Atlantic to study aerosol–cloud interactions (ACIs), which represent the largest uncertainty in estimating total anthropogenic radiative forcing. The combination of a high-flying King Air and low-flying HU-25 Falcon, equipped with remote sensing and in situ instruments, characterized trace gases, aerosol particles, clouds, and meteorological variables with data collected nearly simultaneously below, within, and above marine boundary layer (MBL) clouds. Flights spanning warm and cold seasons across 3 years (2020–22) provided a broad range of conditions associated with aerosol particles, cloud properties (including particle size and phase), and meteorology, ideally suited for robust ACI calculations and assessing how well models simulate a wide range of MBL clouds from stratiform to cumulus. ACTIVATE data suggest that drivers of cloud droplet number concentration N d , including aerosol particles and MBL dynamics, vary between winter and summer months with a stronger potential to convert aerosol particles into cloud droplets in winter. Models of varying complexity not only highlight some skills in simulating winter and summer cloud types but also identify challenges that still need to be addressed such as treatment of turbulence, wet scavenging, and mesoscale organization. Remote sensing advances range from new retrieval methods for N d , cloud phase classification, vertically resolved aerosol and cloud condensation nuclei number concentration, and ocean surface wind speed. This work describes these scientific and technological advances along with efforts in outreach and open data science.

aerosol indirect effect↗
DOE OSTI2024

Warm boundary-layer cloud properties and cloud susceptibilities product at the ARM ENA observatory

Cloud liquid water path, albedo, and cloud fraction susceptibilities for marine warm boundary-layer clouds are quantified using the Meteosat retrieved cloud microphysical properties at the ARM Eastern North Atlantic (ENA) observatory in the Azores. The Meteosat cloud retrievals are based on the methods applied by the Clouds and the Earth’s Radiant Energy System (CERES) project, which are specifically tailored to support the ARM user facility over the ARM ground-based observation sites (enavisstpx2dm11minnisX1.c1). Cloud susceptibility is quantified as the slope between cloud properties and Nd within a 1-degree-by-1-degree grid box at each satellite time step using a least-square regression. To reduce uncertainties associated with the adiabatic assumption in the Nd retrieval, for each cloud object, we exclude cloudy pixels at the cloud edge, defined as those adjacent to cloud-free pixels. To minimize influence of cloud heterogeneity and co-variability on the retrieved cloud susceptibility, cloud properties are averaged to 0.25-degree resolution using the 3-km pixel-level retrieval, and the 1-degree cloud susceptibilities are estimated using the smoothed data. Cloud type classification is applied to distinct cloud object, labeling all contiguous cloudy pixels. Marine warm boundary-layer clouds are defined as clouds with 90% of cloud tops below 3km, and Meteosat retrieved cloud phase as liquid.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Classifying thermodynamic cloud phase using machine learning models

Vertically resolved thermodynamic cloud-phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Thermodynamic Cloud Phase (THERMOCLDPHASE) value-added product (VAP) uses a multi-sensor approach to classify the thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave-radiometer-derived liquid water path, and radiosonde temperature measurements. The measured pixels are classified as ice, snow, mixed phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multi-layer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with 1 year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1 score, and mean intersection over union (IOU). Analysis of ML confidence scores shows that ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential data streams for ML thermodynamic cloud-phase predictions. Lidar measurements exhibit lower feature importance due to rapid signal attenuation caused by the frequent presence of persistent low-level clouds at the NSA site. The ML models' generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. The models demonstrated similar performance to that observed at the NSA site. Finally, we evaluate the ML models' response to simulated instrument outages and signal degradation and show that a CNN U-Net model trained with input channel dropouts performs better when input fields are missing.

ARM Aerial Facility↗
DOE OSTI2025

ARM Thermodynamic Cloud Phase (THERMOCLDPHASE) Value-Added Product Report

The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) User Facility’s Thermodynamic Cloud Phase (THERMOCLDPHASE) Value-Added Product (VAP) provides vertically resolved thermodynamic cloud phase classifications (Zhang and Levin 2024). This VAP applies the multi-sensor methodology introduced by Shupe (2007) to identify cloud phase at the pixel level as liquid, drizzle, liquid + drizzle, rain, ice, snow, or mixed-phase. In addition, the VAP determines the overall cloud layer phase—classified as liquid, mixed-phase, or ice—based on the fraction of ice-containing pixels within the entire layer.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Radar Retrieval Evaluation and Investigation of Dendritic Growth Layer Polarimetric Signatures in a Winter Storm

Abstract This study evaluates ice particle size distribution and aspect ratio φ Multi-Radar Multi-Sensor (MRMS) dual-polarization radar retrievals through a direct comparison with two legs of observational aircraft data obtained during a winter storm case from the Investigation of Microphysics and Precipitation for Atlantic Coast-Threatening Snowstorms (IMPACTS) campaign. In situ cloud probes, satellite, and MRMS observations illustrate that the often-observed K dp and Z DR enhancement regions in the dendritic growth layer can either indicate a local number concentration increase of dry ice particles or the presence of ice particles mixed with a significant number of supercooled liquid droplets. Relative to in situ measurements, MRMS retrievals on average underestimated mean volume diameters by 50% and overestimated number concentrations by over 100%. IWC retrievals using Z DR and K dp within the dendritic growth layer were minimally biased relative to in situ calculations where retrievals yielded −2% median relative error for the entire aircraft leg. Incorporating φ retrievals decreased both the magnitude and spread of polarimetric retrievals below the dendritic growth layer. While φ radar retrievals suggest that observed dendritic growth layer particles were nonspherical (0.1 ≤ φ ≤ 0.2), in situ projected aspect ratios, idealized numerical simulations, and habit classifications from cloud probe images suggest that the population mean φ was generally much higher. Coordinated aircraft radar reflectivity with in situ observations suggests that the MRMS systematically underestimated reflectivity and could not resolve local peaks in mean volume diameter sizes. These results highlight the need to consider particle assumptions and radar limitations when performing retrievals. significance statement Developing snow is often detectable using weather radars. Meteorologists combine these radar measurements with mathematical equations to study how snow forms in order to determine how much snow will fall. This study evaluates current methods for estimating the total number and mass, sizes, and shapes of snowflakes from radar using images of individual snowflakes taken during two aircraft legs. Radar estimates of snowflake properties were most consistent with aircraft data inside regions with prominent radar signatures. However, radar estimates of snowflake shapes were not consistent with observed shapes estimated from the snowflake images. Although additional research is needed, these results bolster understanding of snow-growth physics and uncertainties between radar measurements and snow production that can improve future snowfall forecasting.

Meteorology & Atmospheric Sciences↗
DOE OSTI2025

Mechanisms Behind the Long‐Distance Diurnal Offshore Precipitation Propagation in Northwestern South America

Abstract Northwestern South America (NWSA) is the rainiest region on Earth, with diurnal precipitation exhibiting extensive westward offshore propagation of up to about 1,200 km in boreal spring (March‐May). The diurnal offshore precipitation propagation begins slowly (3–10 m s −1 ) near the coast of NWSA (<200 km) but accelerates significantly (∼20 m s −1 ) and shows an afternoon enhancement far from the coast (>400 km). However, the driving mechanisms behind this long‐distance precipitation propagation remain unclear. Using a new cloud tracking and classification data set, we found that mesoscale convective systems (MCSs) are the dominant precipitation contributors in the offshore region of NWSA. Cloud tracking shows that the long‐distance propagation and the afternoon enhancement of diurnal precipitation primarily originate from MCSs initiated in the early morning, either over open oceans or from the coast of Central America. Composite tendency analysis shows that MCSs initiated near the coast of Central America have significant upward cooling and moistening signals starting from the surface before initiation. Further analysis of surface diurnal perturbation fields indicates that the land breeze is the primary driving mechanism for MCS initiation. Conversely, for MCSs initiated over open oceans, a significant downward cooling signal from 400 hPa is observed ∼7 hr before initiation, corresponding to the passage of diurnal gravity waves emitted from the Andes. Additionally, our findings highlight the critical role of lower and mid‐level moisture conditions in MCS initiation, alongside the influence of gravity waves.

Hu, Jingyi [Department of Meteorology and Atmosphe↗
DOE OSTI2025

Marine Boundary Layer Cloud Boundaries and Phase Estimation Using Airborne Radar and In Situ Measurements During the SOCRATES Campaign over Southern Ocean

The Southern Ocean Clouds, Radiation, Aerosol Transport Experimental Study (SOCRATES) was an aircraft-based campaign (15 January–26 February 2018) that deployed in situ probes and remote sensors to investigate low-level clouds over the Southern Ocean (SO). A novel methodology was developed to identify cloud boundaries and classify cloud phases in single-layer, low-level marine boundary layer (MBL) clouds below 3 km using the HIAPER Cloud Radar (HCR) and in situ measurements. The cloud base and top heights derived from HCR reflectivity, Doppler velocity, and spectrum width measurements agreed well with corresponding lidar-based and in situ estimates of cloud boundaries, with mean differences below 100 m. A liquid water content–reflectivity (LWC-Z) relationship, LWC = 0.70Z0.29, was derived to retrieve the LWC and liquid water path (LWP) from HCR profiles. The cloud phase was classified using HCR measurements, temperature, and LWP, yielding 40.6% liquid, 18.3% mixed-phase, and 5.1% ice samples, along with drizzle (29.1%), rain (3.2%), and snow (3.7%) for drizzling cloud cases. The classification algorithm demonstrates good consistency with established methods. This study provides a framework for the boundary and phase detection of MBL clouds, offering insights into SO cloud microphysics and supporting future efforts in satellite retrievals and climate model evaluation.

MBL clouds over Southern Ocean↗
DOE OSTI2023

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ~85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Additionally, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

BFSVBF (BatFinder Smart Video BioFilter) [SWR-22-87] and Multi-class BatFinder Smart Video BioFilter Keras

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The code has gone through five rounds of development with images used to train the models. This advancement allows for automated real-time data collection, identification, and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. We will discuss the two open source and publicly available machine learning models developed within this scope of this work: 1) a binary model with a 97.5% accuracy in identifying the difference between an object and an empty scene, including wind turbine and clouds; and 2) a multiple classification model with the capability of identifying the type of object detected: bats (90% accuracy), birds (83% accuracy), insects (69% accuracy) and non-biological (99% accuracy).

Yarbrough, John↗