New UrbanSARFloods Dataset Solves Flood Detection Challenges

3 main points
✔️Building UrbanSARFloods, a large dataset coveringurban and ruralflooding
✔️ Evaluating flood detection models using SAR data and identifying challenges
✔️ Evaluating state-of-the-art semantic segmentation methods using UrbanSARFloods

UrbanSARFloods: Sentinel-1 SLC-Based Benchmark Dataset for Urban and Open-Area Flood Mapping
written by Jie Zhao, Zhitong Xiong, Xiao Xiang Zhu
(Submitted on 6 Jun 2024)
Comments: Accepted by CVPR 2024 EarthVision Workshop
Subjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

code：

The images used in this article are from the paper, the introductory slides, or were created based on them.

Summary

Flooding is one of the world's most serious natural disasters, affecting billions of people. In recent years, it has been reported that the frequency and magnitude of floods have increased due to the rapid increase in rainfall caused by extreme weather events, and this has led to the need to predict, monitor, and map floods.

Against this backdrop, satellite data has become an indispensable tool for understanding flood conditions. In particular, mapping large-scale floods using optical satellite and synthetic aperture radar (SAR) data is receiving more attention than ever before.

Whileoptical satellitedata provides data over a long period of time in the past, it is subject to the problem that information is not available when covered by cloud cover or other obstructions. On the other hand, SAR data is useful for mapping floods because it can be acquired regardless of weather conditions, day or night.

However, studies on flood mapping to date have mainly focused on bare land or areas with little vegetation, and few studies have been conducted on large areas, including urban areas. In urban areas, buildings and infrastructure are more susceptible to the effects of flooding, making flood forecasting more difficult. Recent studies have proposed methods to detect flooding in urban areas using SAR data and have shown their effectiveness, but these methods do not yet have sufficient generalization performance.

In addition,deep learning, which has receivedmuch attention in recent years,has limited application in mapping large-scale urban floods using SAR data. One reason for this is the lack of benchmark datasets. While large datasets such as ImageNet have contributed significantly to research progress in the field of computer vision, such datasets do not yet exist in the field of urban flooding.

To address these challenges, this paper proposes a large dataset of floods called UrbanSARFloods. This datasetcoversfloods in and around urban areas and uses SAR data from the Sentinel-1 satellite to document pre- and post-flood conditions (SAR intensity and InSAR coherence data) in detail.It contains data from 18 floods over 807,500 km² and consists of 8,879 images of 512 x 512 pixels (20 m resolution). It provides extensive geographic coverage and can be used to evaluate the performance of deep learning models for flood detection.

This article presents the results of a study utilizing the UrbanSARFloods dataset and highlights new possibilities and challenges in urban flood detection.

Dataset Overview

The dataset includes data from 18 urban floods observed by the Sentinel-1 satellite. Detailed information is provided in the table below.

This data set contains eight bands that capture the pre- and post-flood conditions.It includestwo bands ofVVandVH polarization intensity datataken before the flood,two bands ofVVandVH polarizationintensity dataafter the flood,twomorebands ofVVandVH polarizationcoherence databefore the flood, andtwo more bands ofVVandVH polarizationintensity dataduring the flood event.

Overview of Training Data, Validation Data, and Test Data

In the UrbanSARFloods dataset, we have devised a method for partitioning the data to accommodate a variety of real-world flooding scenarios. In addition to simply increasing the amount of imagery, thetrainingand validation dataincludes floods that occur on different lands and under different conditions to accommodate a variety of scenarios. We also use data featuring floods at different locations in the test data to allow us to evaluate how flexible different methods can be.

In particular, threefloodsin Africa, Asia, and Europe, for which high-resolution optical data are available, were selected as test data, while the other 15floods were used as training and validation data.

Forthe 15floodsused in thetraining and validationdata, all images were split into 512 x 512 segments, including images that do not contain floods. This is because we believe that non-flood data can also help improve the accuracy of flood detection. Especially in mapping large floods, non-flood areas that have similar characteristics tofloodareascan be confusing. For example, scrubby areas around Beledeweyne, Somalia, can cause false positives in flood detection models.

Flood areas are often a small portion of the overall scene, and the data imbalance is particularly pronounced in urban floods.For this reason, when selectingtraining and validationdata, we use stratified sampling based on the flood case.

Specifically, all tiles (units of image segmentation) are classified into non-flooded tiles (NF), flooded open area tiles (FO), and flooded urban area tiles (FU) based on the presence or absence of flooding and its extent. Inundated tiles are further subdivided according to the percentage of flooded pixels and finally classified into five categories (NF, FO1, FO2, FU1, and FU2). Seventy percent of each category is then classified into the training data and the remainder into the validation data.

This method resulted in a total of 8,879 images covering 807,500 km², resulting in 2,408 images from three flood data cases for testing, 4,501 for training and 1,970 for validation.

The distribution of land cover classifications in thetraining, validation, and test data is also analyzed. The Copernicus Global Land Cover Map 2019 was used for this analysis.Thefigure belowshows that thetraining, validation, and testdata all contain a variety of land cover classifications and their distributions are similar.

In addition, the figure below also shows the distribution of labels in each set, indicating that the data annotations follow a "long-tail distribution".

This indicates that certain classes (e.g., flood zones) are very underrepresented compared to other classes, indicating the existence of a serious data imbalance.Such imbalanced data distribution is a particular challenge in the development of flood detection models, and methods to overcome it are also being studied by utilizing this dataset.

Experimental Results

Nine semantic segmentation models are evaluatedusing the UrbanSARFloods dataset; the performance of the models for two flooding classes is evaluated; and the performance of the models for the two flooding classes is evaluated usingthe UrbanSARFloods dataset.

• Unet.
• Unet++.
• MANet
• Linknet
• Fast Fourier Transform Nearly Normal
• PSPNet
• PAN
• DeepLabV3
• DeeplabV3+

First, the F1 scores for flooded open areas (FO) range from 0.51 to 0.77 for Weihui and Jubba. This indicates that the model may be overestimating the FO.

In the Weihui example shown below, the FO is shown in blue and the flooded urban area (FU) in red. In the RGB image of combined pre- and post-event intensity, FO is shown in cyan and FU in red. In the RGB image of coherence, FUs are shown in cyan.

Combining and analyzing this data shows that most of the FO overestimation is concentrated at the boundaries of the FO. However, NovaKakhovka has an even lower F1 score for the FO, with an accuracy of less than 0.2, likely due to the fact that the agricultural land not detected as flooding and the wind-driven rippled water surface have similar characteristics to the FO.

Next, the quantitative assessment of FUs shows very low F1 scores and accuracies of less than 0.1. This indicates that many pixels are misclassified as FUs and that the model does not distinguish well between inundated urban pixels and other similar pixels.

As indicated by the yellow boxes in the figure below (reproduced), we can see that pixels that do not clearly resemble FUs are also misclassified.

Furthermore, all flood maps were evaluated using manually annotated data, and no significant differences in model performance were found. These resultssuggest thatthe reason for the poor performance of all models is not due to the quality of the semi-automatically labeled data, but rather to the difficulties inherent in the application of mapping large floods.

Summary

A challenge in utilizing deep learning for mapping large-scale urban floods is the lack of suitable publicly available datasets. To address this challenge, this paperbuilds "UrbanSARFloods" using the pre-processed Sentinel-1 dataset, which containsboth urban and ruralflooddata.

This benchmark makes it easier for other researchers to develop new techniques based on this research.It is hoped thatthe development of generic models that accurately capture urban and rural floodingwill provide global flood maps andcontribute to solving societal problems.

• 
• 
• 

Categories related to this article

• Dataset
• Datasets
• Flood Prediction
• Flood Mapping