CLOUD-COVER ASSESSMENT: FROM SPECTRAL PROPERTIES TO SPATIAL DOMAIN NATURAL SCENE STATISTIC Shuigen Wang, Chenwei Deng, Xun Liu, Zhenzhen Li, Fan Feng, Baojun Zhao Beijing Key Laboratory of Embedded Real-Time Information Processing Technology Email:{sgwang, cwdeng, liuxun, lizhenzhen, fengfan, zbj}@bit.edu.cn ABSTRACT Cloud contamination is the most common defect leading to quality degradation in remote sensing images. Numerous cloud-cover assessment (CCA) methods have been developed in the literature. The traditional Landsat 7 CCA algorithm attempted to detect clouds by taking advantages of different spectral properties from five spectral bands. However, it suffers the weakness of omitting thin cirrus clouds and the requirement of thermal bands. In this paper, we derived an automated CCA (ACCA) model that measures statistical deviations in spatial domain between cloud and clear images. Moreover, it only conducts on panchromatic band image, which can successfully address the limitation of unavailable thermal bands for satellite missions without thermal infrared sensors on board. A database with 400 clear/cloud images is then built for performance testing. Experimental results on the database show that our approach is more consistent with ground truths than the latest Landsat 8 ACCA results. Index Terms— cloud-cover assessment, spatial domain, natural scene statistic, thin cirrus/stratus cloud. 1. INTRODUCTION Cloud-cover is the most common contaminant in remote sensing (RS) images. It impedes satellites from obtaining clear views of the earth surface, and may cause important object information loss which makes negative effects on the use of acquired data. Being constrained by limitations such as energy, weight, etc, the onboard computing and storage resources are extremely limited, as well as the space downlink bandwidths. If the onboard system is able to make an adequate decision about whether to process an acquisition or not according to a computed automated cloud-cover assessment (ACCA) score in real-time, thus, a large amount of resources could be saved up for carrying out more significant tasks. This work was partially supported by the National Natural Science Foundation of China under Grant 61301090, 111 Project of China under Grant B14010 and Major Special Project-the China High-Resolution Earth Observation System (NO.30-Y20A06-9003-15/16).
,(((
In the past decades, a number of ACCA algorithms have been proposed by different researchers. In 1987, the first recorded ACCA model by the United States Geological Survey (USGS) [1] was proposed to measure the cloud-cover percentages of Landsat 5 band images by the EROS Data Center (EDC) [2]. It uses threshold values in bands 3, 5, and 6, i.e., red, middle infrared, and thermal bands, to eliminate dark image features, snow, and warm image features, respectively. However, it has troubles with warm clouds and discriminating clouds from snow and ice. Hollingsworth et al. [2] promoted the Landsat 5 ACCA metric using a supervised classification technique where the estimation accuracy is determined by human visual assessment. After the Landsat 7 mission was launched in 1999, Richard R. Irish proposed the Landsat 7 ACCA algorithm employing two passes through ETM+ data [3, 4]. In the framework of this method, five ETM+ spectral bands (Band 2 through Band 6) of Landsat 7 data and eight different filters are used to isolate clouds resulting in clouds, non-clouds and ambiguous clouds which are re-examined in the following pass two using thermal analysis of Band 6. This Landsat 7 ACCA algorithm is still under used in the Landsat 8 ACCA system. Scaramuzza et al. [5] proposed other two approaches, i.e., AT(Artificial Thermal)-ACCA and C5-ACCA, for Landsat 8 ACCA. Both the AT-ACCA and C5-ACCA are designed only using optical data of the Operational Land Imager (OLI) for the potential absence of a thermal band, since the Thermal Infrared Sensor (TIRS) on the Landsat 8 only has a three-year design life [5]. Besides, the Landsat 8 contains Band 9 which is specific for cirrus cloud detection. All the described ACCA algorithms above share two common characteristics: (1) relying on the observation that clouds are highly reflective and cold; (2) built on using multiple spectral bands, leading to the omission of thin cirrus clouds and complex frameworks, respectively. Unlike them, in this paper, we proposed a novel ACCA method by computing the distances of statistical regularities between cloud images and clear images. The method is conducted only on the panchromatic band, which has much lower computational complexity than the existing ACCA ones. In addition, experimental results on our designed clear/cloud image database demonstrate that our approach has higher assessment accuracy in detecting thin cirrus and stratus clouds than Landsat 8 ACCA metric.
,*$566
Fig. 1. Flow diagram of the proposed NSS-based CCA model. and σ are local mean and variance
2. NATURAL SCENE STATISTIC BASED CLOUD-COVER ASSESSMENT MODEL In this section, we are to describe the proposed natural scene statistic (NSS) based CCA algorithm. Fig. 1 shows the flow diagram which contains four main steps. Given a testing image, first compute locally normalized luminances of clear and cloud image patches separately via mean subtracted contrast normalization (MSCN) [6] resulting in MSCN coefficients. The asymmetric generalized Gaussian distribution (AGGD) [7] is then used to model the MSCN distributions. The resultant AGGD feature parameters are further fitted by a multivariate Gaussian (MVG) model. Finally, we compute the statistical deviation between the clear and cloud images based on the obtained mean and covariance values of their corresponding MVG models. We found that the bigger the deviation is, the higher the cloud amount is. 2.1. Spatial Domain NSS of Cloud Images It has been widely demonstrated that the perceptual spatial domain NSS features extracted from local image patches have great ability to represent different distorted images [8, 9]. Here, based on the observation that thick/opaque, thin cirrus and stratus clouds generally produce various degrees of degradation on image contrast, their spatial domain NSS should be distinguished from each other. Fig. 2 exemplifies a group of image patches with no cloud, thick cloud and thin cirrus clouds. Given an image patch I, we do the process of local MSCN:
μ(i, j) =
L K
ωk,l I(i + k, j + l)
(2)
k=−K l=−L
K L 2 σ(i, j) = ωk,l [I(i + k, j + l) − μ(i, j)] (3) k=−K l=−L
where ω = {ωk,l |k = −K, . . . , K, l = −L, . . . , L} is a 2D circularly-symmetric Gaussian weighting function sampled out to 3 standard deviations (i.e., K = L = 3) and rescaled to unit volume. Fig. 2d presents the MSCN distributions of Fig. 2a to Fig. 2c. One can see that the coefficients of the clear patch (Fig. 2a) reliably follow a Gaussian distribution (red curve), however, clouds modify the ideal distribution when the patches are subjected to some clouds. Furthermore, patches with thick clouds have sharper distributions than thin cirrus clouds. As shown in Fig. 3, the MSCN coefficients of Fig. 3a (almost full of clouds) are almost zeros shown as Fig. 3b. 2.2. AGGD and MVG Modeling
(1)
Prior studies have shown that the AGGD effectively captures the behaviors of NSS-based MSCN coefficients [8, 9]. In this work, we follow [8] to use a zero mode AGGD [7] for modeling obtained MSCN distributions. Given a variable x, its zero mode AGGD modeling is ⎧
γ ⎪ ⎪ , ∀x ≤ 0 ⎨ (βl +βγr )Γ( 1 ) exp − −x β l γ
f (x; γ, βl , βr ) = γ ⎪ ⎪ , ∀x ≥ 0 ⎩ (β +βγ )Γ( 1 ) exp − βxr
where i, j are spatial indices with i ∈ {1, 2, . . . , M }, j ∈ {1, 2, . . . , N }, M and N are the image dimensions, and μ
(4) where the parameters (γ, βl , βr ) can be estimated using the moment-matching based approach in [7]. Besides, the mean
ˆ j) = I(i, j) − μ(i, j) I(i, σ(i, j) + 1
l
r
γ
4
104
thick cloud 3.5 3 2.5 2 1.5 1 0.5 0 -40
(a)
(a)
(b) no cloud thick cloud thin cirrus cloud
2500
-20
-10
0
10
20
30
40
(b)
Fig. 3. Image patch full of thick clouds and its corresponding MSCN distribution.
Manu National Park
3000
-30
distinct from those of clear images. Hence, the amount and thickness of cloud can be measured by the distance between the fitted MVG models of clear and cloud images:
2000
1500
1000
500
0 -0.1
D(νr , νd , Σr , Σd ) = -0.05
(c)
0
0.05
0.1
0.15
(d)
Fig. 2. Exemplified image patches with no cloud, thick cloud and thin cirrus cloud, and their corresponding MSCN distributions. (a) Clear patch; (b) Thick cloud patch; (c) Thin cirrus cloud patch; (d) MSCN distributions of (a)-(c). of the distribution is also computed:
2
Γ γ η = (βr − βl ) 1 Γ γ
(5)
Since the MSCN distributions (1) of cloud images don’t follow a fairly regular structure, we also need to capture the deviations by modeling the distributions of the products of adjacent coefficients along horizontal, vertical and two diˆ j)I(i, ˆ j + 1), I(i, ˆ j)I(i ˆ + 1, j), agonal orientations as: I(i, ˆ j)I(i ˆ + 1, j + 1) and I(i, ˆ j)I(i ˆ + 1, j − 1). Thus, there I(i, are 16 parameters over four orientations. In order to get more compact representation, an MVG model is employed to fit the resultant AGGD features: 1 1 T −1 f (x1 , . . . , xk ) = exp − Σ (x−ν) (x−ν) 1/2 2 (2π)k/2 |Σ| (6) where (x1 , . . . , xk ) are the AGGD parameters, and ν and Σ denote the mean and covariance matrix of the fitted MVG model. The MVG models of clear and cloud images are different in the two features, i.e., ν and Σ, which are adopted to compute the distance between clear and cloud images. 2.3. Statistical Deviation Computing As shown in Fig. 2 and Fig. 3, it can be known that the MSCN distributions of images with more and thicker clouds are more
Σ + Σ −1 r d (νr −νd )T (νr −νd ) 2
12
(7) where νr , νd and Σr , Σd are the mean vectors and covariance matrices of the MVG models of clear and cloud images. 3. EXPERIMENTAL RESULTS We first built a clear/cloud RS image database for testing the CCA performance of the proposed algorithm (we will public the database online later). Our proposed NSS-based spatial domain CCA method is compared with the Landsat 8 ACCA (containing Landsat 7 ACCA) results provided by the USGS. The correlation value between our predicted distances D via (7) and the ground truths is also presented in this section. 3.1. Image Database and Experiment Settings To test CCA performance, in this work, we collected 400 image patches from 10 different scenes with 20 Landsat 8 panchromatic bands for each scene. The 10 image scenes cover harbor, city, sea, forest, and desert around the world. The database contains 150 clear and 250 cloud patches and the size of each patch is 2400 × 2400. Fig. 4 demonstrates five example images and their corresponding ground truths in the database. The left two images in the first row are harbor scenes with small cirrus and stratus clouds, respectively. The middle one shows a sea image corrupted by both opaque clouds and thin cirrus clouds. The first right patch contains a large amount of cirrus clouds in both thick and thin, while the last one is a clear forest image. In our experiments, each 2400 × 2400 image is divided into 128*128 smaller patches for MSCN computing in (1). In order to compute the distance D, 150 clear images are first adopted to learn the representative MVG features of them.
Fig. 4. Five example images in our clear/cloud database and their corresponding ground truths and Landsat 8 ACCA results. 3.2. CCA Performance Comparison In Fig. 4, the ground truths and Landsat 8 ACCA results of the five example images are shown in the second and third row, separately. We can see that the Landsat 8 ACCA takes constructions with high reflectance as clouds in the left two images and omits some thin cirrus clouds in third and fourth image. The percentages of CC computed via ground truths and Landsat 8 results are 0.0220, 0.3452, 0.6065, 0.7858, 0 and 0.0291, 0.7140, 0.3984, 0.6690, 0, respectively. However, the distances computed by our method are 5.8162, 6.6505, 7.4664, 8.2837, and 5.6450, which are consistent with ground truths. Also, the correlation score between the ground truths and our predicted values reaches 0.93 for the whole database. 4. CONCLUSION In this paper, we proposed a new framework for cloud-cover assessment. Instead of using multi-spectral bands to find each cloud, a statistical distance between cloud and clear panchromatic band is computed as the measurement of cloud amount, which makes it more possible to be used on board. Moreover, it is able to cover both thick and thin clouds. Experimental results have also shown high correlation with ground truths. 5. REFERENCES [1] R. Irish, “Automatic cloud cover assessment (acca) landsat 7 acca,” in Goddard Space Flight Center. LANDSAT-7 Science Team Meeting (December 1-3), 1998.
[2] B. V. Hollingsworth, L. Chen, and R. R. Irish, “Automated cloud cover assessment for landsat tm images,” in ISOSEI, 1996, pp. 170–179. [3] R. R. Irish, “Landsat 7 automatic cloud cover assessment,” in AeroSense, 2000, pp. 348–355. [4] R. R. Irish, J. L. Barker, and T. Arvidson, “Characterization of the landsat-7 etm+ automated cloud-cover assessment (acca) algorithm,” Photogrammetric Engineering & Remote Sensing, vol. 72, no. 10, pp. 1179–1188, 2006. [5] P. L. Scaramuzza, M. A. Bouchard, and J. L. Dwyer, “Development of the landsat data continuity mission cloudcover assessment algorithms,” IEEE TGRS, vol. 50, no. 4, pp. 1140–1154, 2012. [6] D. L. Ruderman, “The statistics of natural images,” Network: computation in neural systems, vol. 5, no. 4, pp. 517–548, 1994. [7] N.-E. Lasmar, Y. Stitou, and Y. Berthoumieu, “Multiscale skewed heavy tailed model for texture analysis,” in 16th IEEE ICIP, 2009, pp. 2281–2284. [8] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a ‘completely blind’ image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. [9] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4695–4708, 2012.
