You are on page 1of 5

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL.

21, 2024 2501405

Automatic Identification of Tree Species


From Sentinel-2A Images Using Band
Combinations and Deep Learning
Vaghela Himali P. and R. A. Alagu Raja

Abstract— Tree species identification using satellite images has land cover and crop type classification, Kussul et al. [10]
been a prominent research topic in the field of remote sensing introduced 1D and 2D-CNN that ensemble with a random
image analysis. So, this letter discussed an approach that explores forest (RF) classifier with nineteen multitemporal scenes while
various band combinations and deep learning models to identify Aboneh et al. [11] introduced the stacked ensemble classifier
tree species in the Madurai region from Sentinel-2A images.
In the subject of identifying tree species, many machine-learning
technique and proved that the proposed method outperformed
algorithms have been created. However, the ML model requires any single classifier. To classify woody and herbaceous crops,
user intervention in selecting the features to process, which is Peña et al. [8] and Ma et al. [5] suggested SVM, Decision Tree
time-consuming and based on trial and error. Owing to the (DT), RF, and logistic regression with feature extraction using
existence of such gaps, this letter discusses a hybrid deep learning the gray level co-occurrence matrix (GLCM) feature extractor.
approach where feature extraction is performed using neural But GLCM is not suitable when an image consists of a high
network blocks such as VGG, MobileNet, and ResNet, and level of noise [9]. Hu et al. [6] introduced SVM classifier to
classification is performed using a random forest (RF) classifier. identify tree species based on individual and combined spectral
Among all combinations, ResNet-RF gives 90.75% accuracy, and
it outperforms gray level co-occurrence matrix (GLCM)-RF and and textural information while Axelsson et al. [7] introduced
other state-of-the-art deep learning models. Bayesian inference using satellite-based Sentinel-2A images.
Sonobe et al. [12] introduced the performance of several
Index Terms— Deep learning, feature extraction, random forest
(RF), ResNet block, stacking bands, tree species identification.
vegetation indices and spectral bands to distinguish between
the different types of crops [13]. But vegetation indices are
sensitive to atmospheric effects such as aerosols, which may
I. I NTRODUCTION lead to misclassification in identifying different crops as well
as analyzing the features from indices are time-consuming and
T HERE are dominating tree species such as banana,
coconut, guava, mango, and papaya that are available
in Madurai, South India. These trees are both an essential
expensive. So, this research emphasis on for making process
time efficient.
component of the rural economy in Indian states and a By analyzing the relevant research publications, it is
significant source of revenue for the development of the nation. clear that the feature engineering performed by hand-crafted
As a result, accurate identification is a task that is challenging features has constraints, which may be caused by noise,
to perform, and there is no such system built in India that lighting conditions, and image distortions as well as it is also a
can monitor species using multispectral data. Physical ground time-consuming task to choose proper feature extractors. This
surveys to monitor tree species are time-consuming, expensive, problem can be overcome by using neural networks because
and prone to human error. However, remote sensing-based they perform the feature engineering on their own. However,
technology gives the best solution for species identification, as the number of layers increases in neural networks, the usage
area coverage analysis, yield estimation, disease identification, of pooling layers diminishes the features. So, to address these
and so on. So, this research aims to identify tree species using challenges, this study employs a hybrid approach where fea-
Sentinel-2A bands where different bands are used for a wide ture extractions are performed using blocks of neural network
range of applications, including monitoring land and coastal and classification is performed using an RF classifier with the
help of the four different band combinations. So, the contri-
areas, vegetation monitoring [1], forest stress monitoring,
bution of this research is to use neural network-based features
urban waterbody management [2], quantifying above-ground instead of hand-crafted features with four band combinations.
biomass, and so on.
For early diagnosis of disease in crops, Hassan et al. [3]
introduced shallow VGG networks with RF and XGBoost clas- II. DATASET C OLLECTION
sifiers and Kumar and Kannan [4] introduced the hybrid model This letter addresses the problem of identification of tree
of XGBoost and the Inception ResNet deep learning model species such as banana, coconut, guava, mango, and papaya
and proved that it outperformed all comparative methods. For in Madurai, Tamil Nadu, India (10.840390◦ N to 9.84553◦ N,
77.74750◦ E to 78.73077◦ E) using Sentinel-2A data. Sentinel-
Manuscript received 19 May 2023; revised 3 January 2024; 2A has a total of 13 bands, including blue (band 2), green
accepted 10 January 2024. Date of publication 16 January 2024; date of (band 3), red (band 4), and near-infrared (NIR-band 8) with a
current version 5 February 2024. (Corresponding author: Vaghela Himali P.)
The authors are with the Remote Sensing and GIS Laboratory,
resolution of 10 m, red-edge (bands 5-6-7), NIR-2 (band 8A),
Thiagarajar College of Engineering, Madurai 625015, India (e-mail: and short-wave infrared (SWIR-band 11-12) with a resolution
vaghela@student.tce.edu; alaguraja@tce.edu). of 20 m, and three atmospheric bands (band 1, band 9, and
Digital Object Identifier 10.1109/LGRS.2024.3354814 band 10) with a resolution of 60 m. However, not all bands are
1558-0571 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
2501405 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 21, 2024

Fig. 1. Madurai region study area and examples of labeled training data.

TABLE I
T REE S PECIES S AMPLE C OLLECTION A ND I MAGE PATCH C OUNTS

useful for identifying tree species; hence, 60 m bands are not


taken into account in this study. So, among the 13 Sentinel-
2 bands, this study used band combinations containing bands
8-8A-11-12 since combinations containing these bands helped
in identifying different types of vegetation [14].
Fig. 2. Methodological framework for tree species classification using
The reference data for all tree species is collected through Sentinel-2A imagery.
field visits, along with their latitude and longitude values. The
list of the collected samples and the number of image patches TABLE II
used for this study are shown in Table I. The required image is S ATELLITE BAND C OMBINATIONS F OR
downloaded from scihub.copernicus.eu for the date of January V EGETATION A ND T REE S PECIES A NALYSIS
18, 2021, and different patches of the study area are collected
from LandViewer. Fig. 1 shows the study area of the Madurai
region and sample patches collected for different tree species
with their labeled data.

III. M ETHODOLOGY
The identification of tree species in India is underexplored
as a nature management tool, but with the help of machine
learning and deep learning techniques, it is possible to
identify them automatically. So, this letter deals with
the identification of tree species using Sentinel-2A band
combinations and a hybrid deep learning approach. Fig. 2
shows the methodological framework, where training involves
the use of Sentinel-2A satellite images with two sets of bands:
Bands 2, 3, 4, 8 with a resolution of 10 m, and Bands 5, 6,
7, 8A, 11, 12 with a resolution of 20 m. The latter set is
resampled to 10 m using transposed convolution to match the and SWIR, feature extraction is performed on each band
resolution of the first set. combination, utilizing four different methods: GLCM, VGG,
Using these two sets, multiple band combinations are MobileNet, and ResNet. The extracted features from each band
created which are given in Table II. Using these four band combination are then stacked in layers and input into an RF
combinations, namely FCC, agriculture, vegetation analysis, classifier which trains these images with its labeled data.

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
HIMALI AND RAJA: AUTOMATIC IDENTIFICATION OF TREE SPECIES FROM SENTINEL-2A IMAGES 2501405

TABLE III
C OMPARATIVE A NALYSIS OF S TACKED BAND C OMBINATIONS
IN S ATELLITE I MAGING C ASES

TABLE IV Fig. 3. Tree species spectral reflectance patterns in Sentinel-2A bands.


D IVERSE F EATURE E XTRACTION A PPROCHES
IN T REE S PECIES I DENTIFICATION
faster to use because they do not require any training. The
pretrained weights and biases can be directly used to make
predictions on new data. It can lead to improved accuracy,
be less prone to overfitting, require less data, and be more
time-saving compared to hand-crafted features. Here, six
GLCM features are used, so single band combinations will
give six GLCM features, and for four band combinations, the
features will be 6 × 4 = 24. So, a total of 24 features are
stacked together as per Case 4. For the neural network blocks
such as VGG, MobileNet, and ResNet, a single block extracted
64 features for a single band combination. As a result of four
different band combinations yielding 64 × 4 = 256 features,
these 256 auto-generated features are fed into the RF classifier.
This classifier performs on a given set of combinations and
generates an output of the area covered by tree species.
The performance of each of the four different scenarios is
evaluated based on sensitivity or recall, specificity, precision,
f1-score, and overall accuracy. Sensitivity or recall is
calculated by (1), which shows that out of all tree species
pixels, how many are correctly predicted as tree species, and
specificity is calculated by (2) shows the ability of the model
to predict the pixels that do not belong to tree species (TN-
True Negative). It can be defined as follows:
TP
SensitivityorRecall = (1)
TP + FN
TN
Specificity = . (2)
TN + FP
The precision refers to the true positive (tree species pixels)
out of the total number of positive predictions (pixels predicted
as tree species). The harmonic means of precision and recall,
known as the f1-score, is used to determine which case among
the four produces better results. Overall accuracy (OA) is the
probability that an individual will be correctly classified by
a test. The terms precision, f1-score, and OA can be given
by (3)–(5), respectively,
TP
Precision = (3)
TP + FP
2 ∗ Precision ∗ Recall
F1-Score = (4)
Here, labeled data indicated the location of tree species in Precision + Recall
Sum of all diagonal elements
an image. Using these band combinations, the experiments OverallAccuracy = . (5)
are conducted in the four cases mentioned in Table III, which Total number of pixels
describes how various band combinations are stacked in each
case.
Table IV gives an idea of the GLCM features and blocks IV. R ESULT AND D ISCUSSION
of neural networks that are used as feature extractors. These The automatic identification of tree species from satellite
blocks are used as non-trainable parameters in Python. The imagery plays a crucial role in forestry management and
advantages of using non-trainable networks are that they are conservation efforts. So, this study examines Sentinel-2A

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
2501405 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 21, 2024

Fig. 4. (a)–(f) Overview of classification results for tree species using various convolutional neural networks and RF. (a) Original Images. (b) Labeled data.
(c) Result obtained by GLCM-RF (Case 4). (d) Result obtained by VGG-RF. (e) Result obtained by MobileNet-RF (Case 4). (f) Result obtained by ResNet-RF
(Case 4).

band combinations and hybrid deep learning methods for TABLE V


tree species identification in the Madurai region. The various RF C LASSIFIER H YPERPARAMETERS C ONFIGURATION
band combinations used in this study are highly related
to vegetation, enhance spectral information, and improve
discrimination between different tree species. Another reason
to choose these band combinations is the mean spectral
reflectance, which helps to analyze the reflectance of each
type of tree in each band. As shown in Fig. 3, our findings
suggest that bands 2-3-4-6-7-8-8A have higher reflectance for
banana; bands 6-7-11-12 have higher reflectance for coconut;
and bands 8-8A-11-12 are much more useful for finding all
types of tree species, and these bands are showing clear
discrimination among all tree species. Therefore, experiments
are performed with only four band combinations that contain
bands 8, 8A, 11, and 12. The reason to use limited bands is that neural networks like VGG, MobileNet, and ResNet is the
increasing the number of bands will increase the complexity of maxpooling layers that reduce the spatial dimensionality of
the model exponentially. So, this study shows the effectiveness the features and cause a loss of spatial information that
of stacking different band combinations. The choice of band is crucial for semantic segmentation. So, by analyzing the
combinations significantly influenced the accuracy of species different research works, this letter introduced non-trainable
identification. After choosing appropriate band combinations, networks for feature extractions that the makes process fast
complex patterns and features are extracted using hand-crafted and computationally less expensive. These deep learning
features like GLCM and different neural network blocks such blocks exhibited remarkable capabilities in extracting intricate
as VGG, MobileNet, and ResNet. features and patterns. After extracting features from different
By introducing GLCM-RF, Ma et al. [5] obtained an OA band combinations, these features are fed to the RF classifier.
of 86.49%, whereas for tree species identification, GLCM- The required hyperparameters used for the RF classifier is
RF achieved an OA of 55.97% for case 4. Because GLCM listed in Table V.
is not discriminative enough to distinguish between different The training of an RF classifier is performed with
classes in satellite images, and because objects in satellite different feature extractors and with stacking of different band
images have complex interactions between texture, shape, combinations (Case 1, Case 2, Case 3, and Case 4). As the
size, and color, it is challenging to distinguish between stacking of band combinations increased, it became more
them based on texture alone. By using Bayesian inference, effective in capturing relevant spectral information, which the
Axelsson et al. [7] achieved an OA of 87% to classify the classification performance across all models.
tree species in Sweden. Hu et al. [6] achieved an OA of 89% Table VI presents the performance analysis of the proposed
by combining different structural and textural information. method with different band combinations using GLCM-RF,
A shallow VGG structure with RF classifier gives 90.37%, VGG-RF, MobileNet-RF, and ResNet-RF. The performance
as suggested by Hassan et al. [3]. But the limitation of metrics evaluated include specificity, precision, recall, F1

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
HIMALI AND RAJA: AUTOMATIC IDENTIFICATION OF TREE SPECIES FROM SENTINEL-2A IMAGES 2501405

TABLE VI inclusion of relevant band combinations (Cases 1–4) in the


P ERFORMANCE M ETRICS OF T REE S PECIES I DENTIFICATION U SING analysis led to improved performance in all the models.
VARIOUS F EATURE E XTRACTION AND RF M ETHODS Among the models, ResNet-RF consistently outperformed
the others, achieving the highest OA in all cases. The OA
achieved by ResNet-RF is 90.75%, which is 33.65%, 21.63%,
and 15.22% higher compared to GLCM-RF, VGG-RF, and
MobileNet-RF, respectively. These findings have significant
implications for forestry management, conservation efforts,
and monitoring tree species distribution on a larger scale. But
further accuracy can be improved by providing time-series data
for a combination of different bands.

ACKNOWLEDGMENT
The authors would like to thank the All India Council
for Technical Education (AICTE), New Delhi, Government
of India for the National Doctoral Fellowship (NDF/ADF)
scheme.

R EFERENCES
[1] L. Karthikeyan, I. Chawla, and A. K. Mishra, “A review of remote
sensing applications in agriculture for food security: Crop growth and
score, and OA. The GLCM-RF achieved relatively lower yield, irrigation, and crop losses,” J. Hydrol., vol. 586, Jul. 2020,
Art. no. 124905, doi: 10.1016/j.jhydrol.2020.124905.
performance compared to the other models. In Case 1,
[2] X. Yang, S. Zhao, X. Qin, N. Zhao, and L. Liang, “Mapping of urban
it achieved an OA of 52.14% with low precision, recall, surface water bodies from Sentinel-2 MSI imagery at 10 m resolution
and F1-score values. As the stacking of band combinations via NDWI-based image sharpening,” Remote Sens., vol. 9, no. 6, p. 596,
increased (from Case 2 to Case 4), there was a gradual Jun. 2017, doi: 10.3390/rs9060596.
improvement in the performance metrics, with OA reaching [3] S. M. Hassan, M. Jasinski, Z. Leonowicz, E. Jasinska, and A. K. Maji,
“Plant disease identification using shallow convolutional neural
57.10% in Case 4. However, the OA remained relatively low network,” Agronomy, vol. 11, no. 12, p. 2388, Nov. 2021.
because GLCM is poor at capturing complex and abstract [4] K. K. Kumar and E. Kannan, “An efficient deep neural network for
features from satellite data. The VGG-RF model outperformed disease detection in rice plant using XGBOOST ensemble learning
the GLCM-RF model in all cases. It demonstrated better framework,” Int. J. Intell. Syst. Appl. Eng., vol. 10, no. 3, pp. 116–128,
performance with higher OA values ranging from 65.42% 2022.
[5] M. Ma, J. Liu, M. Liu, J. Zeng, and Y. Li, “Tree species classification
(Case 1) to 69.12% (Case 4). The precision, recall, and based on Sentinel-2 imagery and random forest classifier in the eastern
F1 score also improved consistently as the stacking of regions of the Qilian Mountains,” Forests, vol. 12, no. 12, p. 1736,
band combinations increased. This indicates that the VGG- Dec. 2021, doi: 10.3390/f12121736.
RF model effectively captured important features from [6] B. Hu, Q. Li, and G. B. Hall, “A decision-level fusion approach to tree
the input data and achieved better classification accuracy. species classification from multi-source remotely sensed data,” ISPRS
Open J. Photogramm. Remote Sens., vol. 1, Oct. 2021, Art. no. 100002,
The MobileNet-RF model showed further improvement in doi: 10.1016/j.ophoto.2021.100002.
performance compared to the VGG-RF model. It achieved [7] A. Axelsson, E. Lindberg, H. Reese, and H. Olsson, “Tree species
higher OA values ranging from 71.16% (Case 1) to 75.53% classification using Sentinel-2 imagery and Bayesian inference,” Int. J.
(Case 4), as well as precision, recall, and F1-score that Appl. Earth Observ. Geoinf., vol. 100, Aug. 2021, Art. no. 102318, doi:
10.1016/j.jag.2021.102318.
consistently increased, indicating the model’s ability to
[8] J. Peña, P. Gutiérrez, C. Hervás-Martínez, J. Six, R. Plant, and
extract more discriminative features. The ResNet-RF model F. López-Granados, “Object-based image classification of summer
demonstrated the highest performance among all the models crops with machine learning methods,” Remote Sens., vol. 6, no. 6,
evaluated. It achieved significantly higher OA values, ranging pp. 5019–5041, May 2014, doi: 10.3390/rs6065019.
from 78.81% (Case 1) to 90.75% (Case 4). The precision, [9] A. Humeau-Heurtier, “Texture feature extraction methods:
recall, and F1-scores also showed consistent improvement A survey,” IEEE Access, vol. 7, pp. 8975–9000, 2019, doi:
10.1109/ACCESS.2018.2890743.
across different band combinations. This suggests that the [10] N. Kussul, M. Lavreniuk, S. Skakun, and A. Shelestov, “Deep learning
ResNet-RF model effectively learned complex patterns and classification of land cover and crop types using remote sensing data,”
features, resulting in accurate tree species identification. It has IEEE Geosci. Remote Sens. Lett., vol. 14, no. 5, pp. 778–782, May 2017,
improved OA of 33.65%, 21.63%, and 15.22% compared to doi: 10.1109/LGRS.2017.2681128.
[11] T. Aboneh, A. Rorissa, and R. Srinivasagan, “Stacking-based ensemble
GLCM-RF, VGG-RF, and MobileNet-RF, respectively. Thus, learning method for multi-spectral image classification,” Technologies,
much improved results are obtained for all tree species by vol. 10, no. 1, p. 17, Jan. 2022, doi: 10.3390/technologies10010017.
stacking the band combinations. Fig. 4 shows the results [12] R. Sonobe, Y. Yamaya, H. Tani, X. Wang, N. Kobayashi, and
predicted by the RF classifier for all four cases. This proposed K.-I. Mochizuki, “Crop classification from Sentinel-2-derived vegetation
method was executed on a system with a Ryzen 5 AMD indices using ensemble learning,” J. Appl. Remote Sens., vol. 12, no. 2,
p. 1, May 2018, doi: 10.1117/1.jrs.12.026019.
processor and 8 GB of RAM with Python 3.6.8. [13] N. Kobayashi, H. Tani, X. Wang, and R. Sonobe, “Crop classi-
fication using spectral indices derived from Sentinel-2A imagery,”
J. Inf. Telecommun., vol. 4, no. 1, pp. 67–90, Jan. 2020, doi:
V. C ONCLUSION 10.1080/24751839.2019.1694765.
[14] T. Zhang, J. Su, C. Liu, W.-H. Chen, H. Liu, and G. Liu, “Band
The study explored the automatic identification of tree selection in Sentinel-2 satellite for agriculture applications,” in Proc.
species from Sentinel-2A images using band combinations 23rd Int. Conf. Autom. Comput. (ICAC), Sep. 2017, pp. 1–6, doi:
and deep learning techniques. The results suggested that the 10.23919/ICONAC.2017.8081990.

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.

You might also like