Deep-Satellite -- A novel generative deep neural network for satellite image analysis

Dr. Rahul Remanan

CEO, Moad Computer

Described in this article here, is a novel deep convolutional generative neural network called the Deep-Satellite.  It was developed to predict the changes in cropland satellite imagery data for Sri Lanka. The goal of the Deep-Satellite model was to forecast the one time-step ahead satellite radar data.

This model was trained using the data provided by the International Telecommunications Union (ITU) GeoAI challenge. Our model demonstrated reasonably good performance in generating the satellite radar data predictions, given the relatively small sample size of the training dataset.

We have demonstrated here, a technically viable approach of using a deep convolutional generative neural network model to quantitatively measure cropland changes, using satellite radar data, collected over several time-separated observations.

Data processing:

The GeoAI provided satellite radar data for the country Sri Lanka had 24 ‘.tif’ files that followed the naming convention ‘LK_yyyy-mm-dd_SR.tif’. These files contained 10 channels of satellite radar information.

The Sentinel 2 earth observation mission by the European Space Agency (ESA) was mentioned on the dataset webpage as the source of the satellite radar data. Even though the Sentinel 2 documentation from the ESA states that their satellite has radars covering 13 different frequencies, and since no additional documentation was provided by the GeoAI team, the relevance of these 10 channels in the ‘.tif’ files were unclear during the model development process.

All 10 channels of information provided in the GeoAI dataset was used to create the training dataset for the Deep-Satellite model. The data preparation steps were performed in Python3 using the NumPy and OpenCV libraries. The entire 24 files were used to create a 256 x 256 x 10 dimensional NumPy array. The resizing was performed using OpenCV. Intermediary steps were performed to ensure proper resizing of the arrays by OpenCV.

Image 1: The plot shows an example 10 channel NumPy array created using satellite radar spectra data in the GeoAI cropland imaging dataset. The image in this example was generated before the resizing step.

Algorithm:

A deep convolutional generative neural network called the Deep-Satellite was created using Tensorflow. This network had a matching 256 x 256 x 10 input and label dimensions. The model followed the general encoder-decoder architecture commonly used in generative networks.

The Deep_Satellite model had two encoders:

1. A primary shallow convolutional encoder for representing low level input features
   
2. A secondary deep convolutional encoder for representing, more abstract, high level input features
   

The shallow encoder comprised of 2 x two dimensional convolutional layers stacked on top of each other. These layers were passed through a Leaky ReLU activation layer. The deep convolutional encoder was created using the EfficientNet B5 network. The deep encoder received the inputs from a two dimensional convolutional layer, that acted as a resizing filter. The purpose of this resizing filter was to match the required input dimensions of the EfficientNet network.

The output of the shallow encoder was flattened and passed through a dense layer, before concatenating with the global average pooled output of the deep encoder. The decoder was built using 5 x two dimensional de-convolutional layer blocks stacked on top of each other.

Image 2: The summary of the Deep-Satellite architecture. The left-hand-side image shows the layers in the encoder network. The right-hand-side image shows the layers in the de-convolutional decoder block.

Metrics:

The model was trained using cosine similarity as the loss function. The training was done using the Adam optimizer with an initial learning rate of 10-5. The learning rate was decreased progressively when the training loss function did not improve for 3 consecutive epochs.

The pair-wise accuracy was also tracked during the model training. Due to the limited number of samples in the dataset, the model only achieved a reasonable result of 0.61731 peak validation pair-wise accuracy and the lowest loss metric of 0.0256 using the cosine similarity function.

Image 3: Output prediction generated by the Deep-Satellite network using the cropland satellite radar data. The image was generated by individually plotting the 10 channels in the prediction array.

Novelty and conclusion:

The generative deep neural networks are growing in popularity. Our approach towards constructing the Deep-Satellite model was to combine the power of a generative deep neural network with the satellite imaging data. Using this approach, we have demonstrated a trainable, deep convolutional generative neural network, capable of learning features that are relevant for predicting the cropland changes over time.

Although the model needs further fine-tuning, such as training using more data and experimenting with various encoder and decoder architectures, so as to improve the overall model performance; our demonstration of a trainable generative deep neural network using extremely limited number of training samples, is an important step towards applying such networks for analyzing real-world satellite data.

References:

1. GeoAI Challenge -- Cropland mapping with satellite imagery