You are on page 1of 14

Journal of Integrative Agriculture 2021, 20(7): 1944–1957

Available online at www.sciencedirect.com

ScienceDirect

RESEARCH ARTICLE

Monthly composites from Sentinel-1 and Sentinel-2 images for


regional major crop mapping with Google Earth Engine

LUO Chong1, LIU Huan-jun1, 2, LU Lü-ping2, LIU Zheng-rong2, KONG Fan-chang2, ZHANG Xin-le2

1
Northeast Institute of Geography and Agroecology, Chinese Academy of Sciences, Changchun 130102, P.R.China
2
School of Pubilc Adminstration and Law, Northeast Agricultural University, Harbin 150030, P.R.China

Abstract
Rapid and accurate access to large-scale, high-resolution crop-type distribution maps is important for agricultural
management and sustainable agricultural development. Due to the limitations of remote sensing image quality and data
processing capabilities, large-scale crop classification is still challenging. This study aimed to map the distribution of crops
in Heilongjiang Province using Google Earth Engine (GEE) and Sentinel-1 and Sentinel-2 images. We obtained Sentinel-1
and Sentinel-2 images from all the covered study areas in the critical period for crop growth in 2018 (May to September),
combined monthly composite images of reflectance bands, vegetation indices and polarization bands as input features,
and then performed crop classification using a Random Forest (RF) classifier. The results show that the Sentinel-1 and
Sentinel-2 monthly composite images combined with the RF classifier can accurately generate the crop distribution map of
the study area, and the overall accuracy (OA) reached 89.75%. Through experiments, we also found that the classification
performance using time-series images is significantly better than that using single-period images. Compared with the use
of traditional bands only (i.e., the visible and near-infrared bands), the addition of shortwave infrared bands can improve the
accuracy of crop classification most significantly, followed by the addition of red-edge bands. Adding common vegetation
indices and Sentinel-1 data to the crop classification improved the overall classification accuracy and the OA by 0.2 and
0.6%, respectively, compared to using only the Sentinel-2 reflectance bands. The analysis of timeliness revealed that when
the July image is available, the increase in the accuracy of crop classification is the highest. When the Sentinel-1 and
Sentinel-2 images for May, June, and July are available, an OA greater than 80% can be achieved. The results of this study
are applicable to large-scale, high-resolution crop classification and provide key technologies for remote sensing-based
crop classification in small-scale agricultural areas.

Keywords: Sentinel-1, Sentinel-2, monthly composites, crop mapping, Google Earth Engine

1. Introduction

Received 8 March, 2020 Accepted 17 June, 2020 The United Nations predicts that the world population will
LUO Chong, E-mail: luochong93@yeah.net; Correspondence
increase to 11.2 billion in 2100, which will inevitably lead to
LIU Huan-jun, E-mail: huanjunliu@yeah.net
an increase in the human demand for food (Belgiu and Csillik
© 2021 CAAS. Published by Elsevier B.V. This is an open
2018). In this context, there is an urgent need to optimize
access article under the CC BY-NC-ND license (http://
creativecommons.org/licenses/by-nc-nd/4.0/). crop distributions and to strengthen the intensification of
doi: 10.1016/S2095-3119(20)63329-9 agriculture to ensure global food and nutrition security (Davis
LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1945

et al. 2017; Rockström et al. 2017). The accurate mapping management, crop varieties and other environmental
of regional crop types is the basis of crop distribution factors, a given crop type may exhibit different spectral
optimization and the intensive development of agriculture, characteristics (Griffiths et al. 2019; Zhang et al. 2019).
which are essential for food macro regulation and the Therefore, the use of multitemporal remote sensing data
development of scientific agricultural management and to obtain more detailed spectral features of crops has
practices (You et al. 2009; Conrad et al. 2010). The use become key to crop-type mapping. Many recent studies
of remote sensing techniques to map regional crop types have used multitemporal remote sensing data for crop-
has been shown to be feasible in many parts of the world type mapping. For example, Vuolo et al. (2018) found that
(Waldhoff et al. 2017; Liu et al. 2019). using multitemporal Sentinel-2 data and Random Forest
The use of remote sensing for crop growth, disaster (RF) models can significantly improve crop classification
monitoring and crop yield estimation is very common (Meng accuracy. Belgiu and Csillik (2018) used Sentinel-2 time-
et al. 2019; Luo et al. 2020). However, for most applications, series data to compare the time-weighted dynamic time
it is necessary to distinguish between crop types to improve warping classification method with the accuracy differences
the monitoring accuracy. Unfortunately, few large-scale, of RFs. Inglada et al. (2016) successfully mapped crop types
high-resolution crop-type maps are available (Griffiths et al. using optical images and synthetic aperture radar (SAR)
2019). Current crop-type products typically only cover data. However, these studies usually have narrow scopes,
some developed countries. For example, the Cropland and large-scale, high spatial-resolution crop classification
Data Layer (CDL), which has been produced by the studies are still relatively rare.
USDA National Agricultural Statistics Service since 2008, The successful launch of Sentinel-2B in 2017 reduced
covers the entire United States using multiple sources of the return visit period of Sentinel-2 to 5 days, which has
data (e.g., Landsat-8 and Sentinel-2); while supervised made it possible to acquire sufficient time series of optical
classification methods provide government agencies and images (Chen et al. 2017). The C-band of Sentinel-1 can
farmers with crop plots which have highly accurate plot penetrate clouds; thus, in Heilongjiang Province, China,
sizes (Boryan et al. 2011; Belgiu and Csillik 2018). In 2012, during the hot and rainy period, which is the critical period of
optical maps and radar images were used to produce crop crop growth, auxiliary information can be provided. Google
classification maps entirely covering Canada (Davidson Earth Engine (GEE) is a cloud-based platform designed to
et al. 2017); in addition, some countries in Europe have process satellite imagery and other terrestrial data. The
also produced national-scale crop classification maps with GEE platform stores Petabyte-level processing-ready data,
the support of the Sentinel agricultural project. enabling researchers to quickly process millions of images
Although large-scale farming is a trend in world in parallel, and thereby greatly improving the efficiency of
agricultural development, smallholder farming is still the image analyses.
main agricultural management method in most developing Equal-interval synthesis has become a feasible solution
countries, such as China and India (Altieri et al. 2012). Small for balancing the spatial and temporal resolutions of large-
planting plots are one of the main features of smallholder scale remote sensing images. Such methods, including the
farming; thus, it is difficult to classify crops in these areas. maximum normalized difference vegetation index (NDVI)
Furthermore, because these developing countries often face synthesis method, were originally designed to reduce
greater food security problems, it is important to understand cloud effects for low- and medium-resolution sensors (e.g.,
the distributions of crop types in these countries. However, advanced very high resolution radiometer (AVHRR) and
due to technical limitations, developing countries usually MODIS) (Cihlar et al. 1994; Wolfe et al. 1998). Calculating
use sample survey methods to understand the distribution the median reflectance or index median of a time-series
of crops in a region; but such methods require significant image is a commonly used pixel synthesis method (Potapov
labor resources and other resources, and the accuracy and et al. 2012). Equal-interval synthesis can be used to obtain
timeliness of the data cannot be guaranteed. Moderate cloud-free composite images of a large-scale regional time
resolution imaging spectroradiometer (MODIS) images are series, and it can provide data support for classification
commonly used in large-scale crop classification tasks, but tasks and other work. Relevant research has been used
the 500-m resolution of such images causes large errors to study monthly synthetic data for wetland classification
in areas with small plots; thus, remote sensing-based crop (Mahdianpari et al. 2019).
classification on the regional scale with a medium-high The research objectives of this paper are: 1) to evaluate
spatial resolution is necessary. the potential of crop mapping at a regional scale using
Many scholars have studied small-scale, high-resolution monthly composite images; 2) to evaluate the impact of
crop-type mapping (Kussul et al. 2016, 2017; Liu et al. 2019; including shortwave infrared (SWIR) bands, red-edge
Hao et al. 2020). Due to differences in local agricultural bands, common vegetation indices and the backscattering
1946 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

coefficient on crop classification; 3) to evaluate the thus, the study period is set to May–September 2018 to
timeliness of crop classification; and 4) to map the spatial reduce data redundancy.
distributions of crops at a regional scale.
2.2. Reference data
2. Materials and methods
In this paper, the cultivated crops in Heilongjiang Province
2.1. Study area are divided into five categories: soybeans, corn, rice, other
crops and fallow land. Each year, the agricultural insurance
The study area is located in Heilongjiang Province in company collects data on the types of crops in the insured
the northeastern part of China, from 43°26´–53°33´N plots for future management. We obtained the data from 246
and 121°11´–135°05´E (Fig.  1). The total land area of 189 parcels of insured land in Heilongjiang Province in 2018
Heilongjiang Province is 473 000 square kilometers, of from the agricultural insurance company, randomly selected
which, the cultivated land area is 119 000 square kilometers, 3 209 plots, and used ArcGIS 10.2 to convert the surface
accounting for approximately 25% of the total land area. The into points to reduce the number of calculations required.
area is characterized by a cold temperate and temperate The crop types mainly include corn, soybeans, rice and
continental monsoon climate, and the precipitation shows other crops, and there are 1 390 corn samples, 712
obvious monsoon characteristics. Summer precipitation soybean samples, 846 rice samples, and 261 other crop
is abundant, the winter is dry, and the annual average samples. In addition, due to the 2018 rice–fallow policy
precipitation is 360–830 mm (Jiang et al. 2018). The main in Heilongjiang Province, some cultivated lands were not
crops in the region are corn, soybeans and rice. The planting planted. To identify fallow land, we obtained 35 plots of
area of these three crops accounts for more than 95% of fallow land by field sampling so that a total of 3 244 samples
the total planted area (Hu et al. 2017). in this study were used to train and verify the sample.
Due to the climate, the planting patterns in the region The selected samples were confirmed on-site from July
encompass all seasons. The crop calendar for the main to August 2018 (Fig. 1), and the ratios of sample points
crop types of corn, soybeans, rice and wheat in the region of different crop types are roughly the same as the area
(Table  1) shows that the key growth periods of the main ratios between the actual crop types in the study area.
crops in Heilongjiang Province are from May to September; Subsequently, we randomly selected 70% of the samples

120°0´0´´E 125°0´0´´E 130°0´0´´E 135°0´0´´E

N
50°0´0´´N
50°0´0´´N

45°0´0´´N
45°0´0´´N

Soybeans
Corn
Rice
Other crops
0 150 300 Fallow
Heilongjiang boundary
km
Cultivated land

125°0´0´´E 130°0´0´´E 135°0´0´´E

Fig. 1 Overview of Heilongjiang Province, China.


LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1947

as training samples and 30% of the samples as verification maximum number of observations is 199.
samples. In this article, Sentinel-2 was selected for classification for
all 10- and 20-m resolution bands, including blue, green, red,
2.3. Sentinel-2 imagery and preprocessing near-infrared (NIR), red-edge 1, red-edge 2, red-edge 3, red-
edge 4, SWIR1, and SWIR2. The first four bands of 10-m
In this study, 5 532 Sentinel-2 images (Level-1C) covering resolution are the bands normally used for classification.
Heilongjiang Province from May 1, 2018, to September 30, The red-edge bands are related to the chlorophyll content of
2018, were selected for crop classification. The total number the crops. The SWIR bands are related to the water content
of observations for each pixel in different regions during the of vegetation. These bands have been proven to have the
study period is shown in Fig. 2-A. The average number of ability to distinguish crop types (Chen et al. 2005; Clevers
observations per pixel for Sentinel-2 is 65, the minimum and Gitelson 2013). In addition, we used several commonly
number of observations is 27, and the maximum number used indices to assist in the classification, including the
of observations is 343. The warmer the color in Fig. 2-A, normalized vegetation index (NDVI) (Tucker 1979), the
the higher the frequency of observation. enhanced vegetation index (EVI) (Huete et al. 2002), the
We built a function in GEE using the quality assessment land surface water index (LSWI) (Xiao et al. 2002) and the
(QA60) band detection and masking of clouds provided in green chlorophyll vegetation index (GCVI) (Gitelson et al.
the metadata to obtain a cloudless set of Sentinel-2 images 1996), which have the following formulas:
(Fig.  2-B) and divided all the band values by 10 000 to pNIR–pRed
NDVI= (1)
obtain the band reflectance values. The number of effective pNIR+pRed 
observations for each pixel in different regions of Sentinel-2 pNIR–pRed
EVI=2.5× (2)
during the study period is shown in Fig. 2-B. The average pNIR+6×pRed–pBlue+1 
effective number of observations per pixel for Sentinel-2 pNIR–pSWIR
LSWI= (3)
is 30, the minimum number of observations is 2, and the pNIR+pSWIR 

Table 1 Heilongjiang main crop calendar1)


Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sep. Oct. Nov. Dec.
Corn Sowing Growing Growing Growing Harvesting Harvesting
Rice Sowing Sowing Growing Growing Growing Harvesting Harvesting
Soybeans Sowing Growing Growing Growing Harvesting
Wheat Sowing Growing Growing Growing Harvesting
1)
The blank spaces indicate that the month is not a growing period for the indicated crop.

A B
N N

Sentinel-2
Total number of Sentinel-2
observations Good observations
27–40 2–20
41–60 21–40
61–100 41–60
101–200 0 150 300 61–100 0 150 300
201–343 km 101–199 km

Fig. 2 Number of observations from Sentinel-2 in Heilongjiang Province from May 1 to September 30, 2018. A, total number of
observations. B, good observations.
1948 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

pNIR
GCVI= –1 (4) N
pGreen 
where pBlue, pGreen, p Red, p NIR and pSWIR are the surface
reflectance values of Band 2, Band 3, Band 4, Band 8 and
Band 11, respectively, in the Sentinel-2 Level-1C products.

2.4. Sentinel-1 imagery and preprocessing

In this study, all images from Heilongjiang Province with


VV (single polarization, vertical emission/vertical reception)
and VH (double polarization, vertical emission/horizontal
reception) polarization from May 1, 2018, to September
30, 2018, were selected. In total, 475 Sentinel-1 dual-
polarized C-band SAR instrument images were used Sentinel-1
for crop classification, and these data were archived Total number of observations
9–10
in GEE in the form of Sentinel-1 SAR Ground Range 11–20
21–30 0 150 300
Detected (GRD) datasets. All images were preprocessed km
31–39
by the Sentinel-1 toolbox using the following steps:
(1) thermal noise removal; (2) radiometric calibration;
Fig. 3 Total number of observations from Sentinel-1 in
(3) terrain correction using the Shuttle Radar Terrain Mission Heilongjiang Province, China from May 1 to September 30,
(SRTM) or Advanced Spaceborne Thermal Emissions and 2018.
Reflection Radiometer (ASTER) digital elevation model
(DEM); and (4) conversion to a backscattering coefficient
usually used to distinguish different land types (Mahdianpari
(σ0) in decibels (dB).
et al. 2019; Tian et al. 2019). The median of the time-series
Fig. 3 shows the number of Sentinel-1 observations for
images is a commonly used statistical indicator that has
each pixel from May 1 to September 31, 2018. Sentinel-1
been successfully applied in land use classification (Liu
has an average of 20 observations, a minimum of nine
et al. 2019; Mahdianpari et al. 2019; Tian et al. 2019).
observations, and a maximum of 39 observations. Finally,
Considering the data availability, the complexity of the
we used the Refined Lee Filter to filter the backscattering
calculation and the classification accuracy, the monthly
time series to remove speckle noise in the Sentinel-1
(30-day) synthetic data were selected as the input image.
dataset (Lee et al. 2008; Yommy et al. 2015). For this
Therefore, in this paper, the monthly image median was
classification task, this paper selected the VV polarization
selected as a statistical indicator of the time-series images.
and VH polarization of Sentinel-1, which are commonly used
We wrote code on GEE for image recombination, which
in land use classification tasks.
produced a cloudless composite image of the Sentinel-2
2.5. Auxiliary data reflectance band and vegetation index for each month from
May to September in the Heilongjiang crop-growing season,
Many studies have shown that remote sensing images used including three visible light bands and one NIR band, four
for crop classification may be based on land that is not red-edge bands, two SWIR bands and the composite median
cultivated, such as woodlands, grasslands, and wetlands of the four vegetation indices (Fig.  4-A). We used the
(Griffiths et al. 2019). To avoid this error, a digital “cultivated same method to address the two kinds of SAR polarization
land” mask at a 30-m spatial resolution (Northeast Institute characteristics (VV polarization and VH polarization) of the
of Geography and Agroecology, Chinese Academy of Sentinel-1 images, and obtained the VV polarization and
Sciences, unpublished data) was superimposed onto the VH of the Sentinel-1 images for every month from May to
crop map to help select crop sampling locations. September in the Heilongjiang crop-growing season and the
composite median of polarization (Fig. 4-B).
2.6. Image compositing
2.7. Random Forest classifier
In the time-series analysis, to reduce the uncertainty in the
number of effective pixels caused by the rainy season and RF classifier is a classification technique that uses
the satellite revisit period, the time series of the image is fully multiple trees to train and predict samples. Because it
utilized, and the statistical indicators of the time series are is more accurate than the traditional maximum likelihood
LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1949

A B
N
May 2018 June 2018 July 2018 May 2018 June 2018 July 2018

August 2018 September 2018 August 2018 September 2018

Fig. 4 Monthly image composite results from May to September. A, Sentinel-2. B, Sentinel-1.

and decision tree classification methods, the algorithm is To test the differences in classification performance of
very popular in the field of remote sensing and has been the imaged crops during a single period, we distributed the
successfully applied in many recent classification studies images by month for classification to create a total of five
(Liaw and Wiener 2002; Díaz-Uriarte and De Andres 2006). groups. To test the timeliness of crop classification, we used
This article uses the RF classifier included in GEE to classify a distribution of images from May to the specified month for
the values of the decision trees and variables in each node. crop classification to create a total of four groups. Therefore,
In this study, we set the number of trees to 300 and the other this experiment involved 63 tests.
parameters to the default values.
2.9. Accuracy verification
2.8. Experimental design
In this paper, five evaluation indices are used. The overall
In order to evaluate the different band reflectance values, accuracy (OA), Kappa coefficient, user’s accuracy (UA),
vegetation indices, backscattering coefficient on crop producer’s accuracy (PA), and F1 score were calculated
classification, and the timeliness of crop classification, we using test points that were not included in model training
designed the following experiments. First, we divided the (Cohen 1960; Congalton 1991). The OA is the ratio of the
bands of Sentinel-2 into three categories: traditional bands number of correctly classified cells to the total number of
(visible bands and NIR band), red-edge bands and SWIR cells. The Kappa coefficient represents the proportion of
bands. To conduct the above evaluations, we designed the errors that are reduced by classification and completely
following seven experimental protocols. random classification. The PA is the conditional probability
Scheme 1: only the traditional bands (visible bands and that the classification result of a given place on the
NIR band) were used for classification. classification map is consistent with any random sample
Scheme 2: the traditional bands and SWIR bands were in the test data. The UA involves taking a random sample
used for classification. with the same conditional probability as the actual type of
Scheme 3: the traditional bands and red-edge bands ground from the classification results. The F1 score is the
were used for classification. harmonic mean of the UA and PA (Goutte and Gaussier
Scheme 4: the traditional bands, SWIR bands and red- 2005). Their formulas are listed below:
edge bands were used for classification. ∑ni=1pii
OA (%)= ×100 (5)
Scheme 5: the traditional bands, SWIR bands, red- N
edge bands and common vegetation indices were used for
classification. N∑ni=1pii–∑ni=1(pi+×p+i)
Kappa= (6)
Scheme 6: only the SAR data were used for classification. N 2–∑ni=1(pi+×p+i)
Scheme 7: all the optical and SAR data were used for pii
UA (%)= ×100 (7)
classification. pi+
1950 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

pii differences between the SWIR1, SWIR2, LSWI, VH and


PA (%)= ×100 (8)
p+i VV bands for rice and other crops types were the largest in
UA×PA May and June. The two main crops, corn and soybeans,
F1 score (%)= ×2 (9)
UA+PA differed the most in the red-edge 2, red-edge 3, NIR, red-
where n is the total number of columns of the confusion edge 4, SWIR 1 and SWIR 2 bands in July and August. The
matrix; Pii is the number of correct classifications of the two types of fallow land and other types of crops differed
upper crop-type sample in the ith row and ith column of the most from the other types in the NDVI, EVI, and GCVI in
confusion matrix, Pi+ and P+i are the total number of crop-type July and August.
samples in row i and column i, and N is the total number of
samples used for verification. 3.2. Accuracy verification

3. Results Crop classification with single-period images Here, we


use only the different schemes of the single-period images
3.1. Time-series profiles of the main crops for crop classification (Table  2). Table  2 shows that the
classification accuracies of Scheme 2 and Scheme 3 are
Fig. 5 shows the median time series of the different bands higher than that of Scheme 1 in all months, which proves
for the five types of training sites in Heilongjiang Province in that adding SWIR bands and red-edge bands can improve
2018 to illustrate the potential contributions of the different the classification accuracy when using single-period images
bands for crop classification. As shown in Fig.  5, the for crop classification. The classification accuracy of

Soybeans Corn Rice Other crops Fallow


0.15 0.15 0.20 0.20
Reflectance value

Reflectance value

Reflectance value

Reflectance value
0.15 0.15
0.10 0.10
0.10 0.10
0.05 0.05
0.05 0.05
Blue Green Red RE1
0 0 0 0
May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept.
0.5 0.6 0.6 0.6
Reflectance value
Reflectance value

Reflectance value

Reflectance value

0.4
0.4 0.4 0.4
0.3
0.2
0.2 0.2 0.2
0.1
RE2 RE3 NIR RE4
0 0 0 0
May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept.

0.4 0.3 1.0 1.0


Reflectance value

Reflectance value

0.3 0.8 0.8


0.2
Index value

Index value

0.6 0.6
0.2
0.4 0.4
0.1
0.1 0.2 0.2
SWIR1 SWIR2 EVI NDVI
0 0 0 0
May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept.

0.6 5 0 0
GCVI
0.4 4 –2
Backscattering

Backscattering

–5
Index value

Index value

coefficient

coefficient

VH VV
0.2 3 –10 –4
0 2 –15 –6
–0.2 1 –20 –8
LSWI
–0.4 0 –25 –10
May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept. May Jun. Jul. Aug.Sept.

Fig. 5 Median band reflectivity, vegetation indices and backscattering coefficients for the monthly composites from all the training
points. Blue, blue band; Green, green band; Red, red band; RE1, red-edge 1 band; RE2, red-edge 2 band; RE3, red-edge 3 band;
NIR, near infrared band; RE4, red-edge 4 band; SWIR 1, shortwave infrared 1; SWIR 2, shortwave infrared 2; EVI, enhanced
vegetation index; NDVI, normalized vegetation index; LSWI, land surface water index; GCVI, green chlorophyll vegetation index;
VH, VH polarization; VV, VV polarization.
LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1951

Table 2 Classification accuracy of the different schemes in the different months1)


May Jun. Jul. Aug. Sept.
Kappa Kappa Kappa Kappa Kappa
OA (%) OA (%) OA (%) OA (%) OA (%)
coefficient coefficient coefficient coefficient coefficient
Scheme 1 58.27 0.36 59.76 0.38 57.67 0.37 60.26 0.42 67.13 0.51
Scheme 2 67.03 0.49 66.24 0.48 69.72 0.55 71.12 0.63 72.61 0.64
Scheme 3 60.36 0.39 62.95 0.43 67.13 0.52 67.03 0.58 70.52 0.60
Scheme 4 66.24 0.48 68.03 0.51 72.11 0.59 74.40 0.63 74.70 0.63
Scheme 5 66.24 0.48 69.72 0.53 71.81 0.59 74.50 0.63 75.40 0.64
Scheme 6 51.59 0.27 57.37 0.34 48.01 0.20 51.10 0.27 42.13 0.12
Scheme 7 67.83 0.50 71.41 0.56 73.61 0.61 76.59 0.66 76.20 0.65
1)
OA, overall accuracy.

Scheme 2 is higher than that of Scheme 3 in all months, Overall accuracy (%) Kappa coefficient
which proves that the importance of SWIR bands is higher
100.00 1.00
than that of red-edge bands when using single-period

Overall accuracy (%)

Kappa coefficient
90.00 0.90
images for crop classification. For June, August and
September, the classification accuracy of Scheme 5 is 80.00 0.80
higher than that of Scheme 4, which proves that adding the 70.00 0.70
common vegetation indices in these months can improve 60.00 0.60
the crop classification accuracy. The classification accuracy
50.00 0.50
of Scheme 7 in all months was always higher than those of
1

7
the other band schemes, which means that when using a
e

e
m

m
he

he

he

he

he

he

he
single-period image for classification, adding radar images
Sc

Sc

Sc

Sc

Sc

Sc

Sc
to supplement the optical images can improve the accuracy
of crop classification. The highest classification accuracy Fig. 6 Classification accuracy of the different schemes of
and Kappa coefficient (76.59% and 0.66, respectively) were time-series images.
achieved by Scheme 7 when using August imagery for crop
classification. under different schemes. To achieve this goal, we set
Crop classification of the time-series images Here, May as the fixed start date and each month from May
we use time-series images from the different schemes for to September as the end dates. For each end date, the
crop classification. As shown in Fig.  6, the classification time-series data used for classification spans from May
accuracies of Scheme 2 and Scheme 3 are higher than that to a specific end date. Fig.  7 shows the change in the
of Scheme 1, which proves that adding the SWIR bands classification accuracy over time for the different schemes,
and the red-edge bands when using time-series images and Fig.  8 shows the rate of change in the classification
for crop classification can improve the crop classification accuracy over time for the different schemes. The schemes
accuracy. The classification accuracy of Scheme 2 is including the SWIR bands or red-edge bands (Scheme 2,
higher than that of Scheme 3, which proves that the Scheme 3, Scheme 4, Scheme 5 and Scheme 7)
SWIR bands were more useful than the red-edge bands have similar trends in their changes in accuracy, and the
when using time-series images for crop classification. classification accuracy increases from June to July by more
Adding the common vegetation indices and SAR data can than 15%. Schemes including SWIR bands can achieve an
improve the classification accuracy, but this improvement OA of more than 80% in July, and these schemes are more
is small (the difference between the lowest and the highest advantageous than the other schemes, mainly because
accuracies is 0.6%). When using the time-series images SWIR bands from July can distinguish between the two main
for crop classification, Scheme 7 had the best classification crops: corn and soybeans. The classification accuracy of
performance (89.74% and 0.85), and Scheme 6 had the Scheme 6 has a special trend over time. The classification
worst classification performance (71.12% and 0.56). accuracy increases from May to June by more than 15%,
mainly because the VH and VV polarizations in June can
3.3. Analysis of the effectiveness of crop classification distinguish between rice and the other types. Scheme 1
has the largest rate of change in the classification accuracy
For this analysis, we designed experiments to test the during the periods from July to August and from August to
variations in the crop classification accuracy over time September.
1952 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

3.4. Classification results respectively.

We used the most accurate scheme (Scheme 7), which 4. Discussion


has the highest OA and Kappa coefficient (Fig. 9), to map
the crop distribution in Heilongjiang Province with the GEE The very large amount of data, high computing power
platform. Of the distributions of the three main crops in and various machine learning algorithms available in the
Heilongjiang Province (corn, soybeans and rice), corn GEE platform are key to this research. For example, for
is mainly distributed in the southern part of Heilongjiang the traditional crop classification process, images must be
Province, soybeans are mainly distributed in the northwest downloaded first and then various preprocessing steps
of Heilongjiang Province, and rice is mainly distributed in must be performed. In this study, we directly included more
the Sanjiang Plain. Through remote sensing monitoring, the than 5 000 Sentinel-2 Level 1-C data and more than 400
areas planted with corn, soybeans and rice in Heilongjiang Sentinel-1 radar data from GEE for crop classification. If
Province in 2018 were 8 496 583, 3 947 725 and 4 848 971 ha, the traditional method was used, it might have taken several

Scheme 1 Scheme 2 Scheme 3 Scheme 1 Scheme 2 Scheme 3


Scheme 4 Scheme 5 Scheme 6 Scheme 4 Scheme 5 Scheme 6
100.00
0.25 Scheme 7
Scheme 7
Overall accuracy (%)

90.00
Rate of change in the

0.20
overall accuracy

80.00 0.15

70.00 0.10

60.00 0.05

50.00 0
May Jun. Jul. Aug. Sept. May–Jun. Jun.–Jul. Jul.–Aug. Aug.–Sept.

Fig. 7 Classification accuracy as a function of the different Fig. 8 The rate of change in the classification accuracy under
schemes. the different schemes.

Cropland
N
Non-cultivated land
Soybeans
Corn
Rice
Other crops
Fallow land

0 150 300
km

Fig. 9 Crop classification map of Heilongjiang Province, China.


LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1953

or even dozens of days, which is unacceptable in terms of (71.12% and 0.56) using only Sentinel-1 time-series data
both time delay and labor. In GEE, however, this process (Scheme 6). This finding indicates that the addition of
took only a few seconds. We spent most of our time writing Sentinel-1 radar data can improve the accuracy of crop
the code that was used to process the composite images classification, which is consistent with previous research
and then used RF classifiers for crop classification; the results; however, the degree of improvement is small (the
entire process from running the program to exporting the OA is less than 1%). Because the use of Sentinel-2 images
classification results took approximately 8 h. Moreover, in this study achieved high precision, it would be difficult to
the classification of different schemes can be processed in further improve the accuracy. The classification accuracy
parallel, which is a great advantage over traditional methods. using only Sentinel-1 data is disappointing, and all types
Balancing the temporal and spatial resolutions of satellite of classification results had low accuracies compared to
images is challenging in large-scale crop classification those from the other schemes (Fig. 10-A). The classification
studies. Our research shows that it is feasible to use results using Sentinel-1 images in areas with large plots
equal-synthesis images for crop classification. Our are not significantly different from those using Sentinel-2
results show that the highest accuracy (89.74% and 0.85) images (Fig. 10-B). However, the classification results using
for crop classification using time-series images is much Sentinel-1 images in areas with small plots are quite different
higher than the highest accuracy (76.59% and 0.66) using from the results using Sentinel-2 image classification;
single-period images. This finding proves that time-series the classification result of Sentinel-1 does not distinguish
information has a great influence on the accuracy of crop the boundaries of the parcels well. Although this study
classification. The Sentinel-1 and Sentinel-2 combination performed Lee filtering on the Sentinel-1 images, the noise
of time-series images (Scheme 7) achieved the highest in these images still affected the classification accuracy.
classification accuracy (89.74% and 0.85), which was In addition, we analyzed the effects of adding red-edge
slightly higher than the classification accuracy using only bands, SWIR bands, and common vegetation indices on
Sentinel-2 time-series data (Scheme 5) (89.34% and classification performance. The soybean classification
0.85) and much higher than the classification accuracy accuracy is poor when only traditional bands are used for

Cropland Cropland
A B
Non-cultivated land Non-cultivated land
Soybeans Soybeans
Corn Corn
N Rice N Rice
Other crops Other crops
Fallow land Fallow land

0 130 260 0 130 260


km km

Fig. 10 Comparison of the crop classification results when using Sentinel-1 and Sentinel-2. A, Sentinel-1. B, Sentinel-2.
1954 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

classification (F1 score=71.79%). Adding red-edge bands to ensure the smooth implementation of various agricultural
the traditional band can improve the classification accuracy policies in a timely fashion. This finding can also help the
of soybeans (F1 score=78.84%), as can adding SWIR relevant grain enterprises and agricultural cooperatives
bands (F1 score=84.97%) (Fig. 11). This finding indicates adjust their business strategies in a timely manner.
that both the red-edge bands and the SWIR bands have There are three main reasons for the crop classification
the ability to identify soybeans, but the SWIR bands have a errors in this paper. First, the quality of the monthly
stronger recognition ability (Cai et al. 2018). Furthermore, composite Sentinel-2 data is not ideal. Although we have
the addition of the SWIR bands can greatly improve monthly composite data, there are still some areas that are
the accuracy of corn classification and slightly improve not covered (Fig. 4), which may lead to errors. In addition,
the accuracy of rice classification, possibly because the the cloud mask algorithm for Sentinel-2 is not stable enough
SWIR bands are sensitive to moisture, and rice cultivation compared to that of Landsat 8, which may prevent some
is inseparable from water (Singha et al. 2019). Adding clouds from being removed (Claverie et al. 2018). Second,
common vegetation indices can improve the accuracy of the accuracy of the reference data is problematic. The
fallow land classification, possibly because the changes in crop classification training samples and validation samples
the vegetation indices for fallow land are very different from used in this paper are mostly from the insured plots of local
the changes in the vegetation indices for crops. insurance companies. Although the accuracy of these
Early access to crop classification maps can be beneficial. samples is very high, some crops with smaller distributions
Crop classification data are required for agricultural may still have errors. Therefore, we refer to these less
monitoring, such as growth monitoring and crop estimation. widely distributed crops as just “other crops”. In addition,
As shown in Fig. 12, the addition of the composite images we classify only crops within arable land, and the range of
from July can greatly improve the classification accuracy cultivated land we use is based on Landsat 8 images. The
of soybeans and increase the classification accuracy of difference in resolution may result in some fields, such as
corn and rice to an acceptable level (F1 score>80%). The field roads and small rivers, being unrecognized. Third,
classification accuracy of other crops and fallow land with the differences in the crop phenology in large-scale areas
smaller distribution areas is also improved. Our research is another source of error. Although our research area is a
shows that in Heilongjiang Province in northeastern China, provincial administrative district in China, both the overall
the combination of Sentinel-1 and Sentinel-2 data can be area and the cultivated area are larger than mid-sized
used to obtain crop classification maps with an OA greater countries in Europe. In such a large area, even for the same
than 80% as soon as the May, June, and July images type of crop, the sowing time and harvest time cannot be
are available. This finding will undoubtedly improve the guaranteed, which can cause errors.
convenience of subsequent agricultural management Large-scale crop classification maps have important
projects. Accurate crop classification maps available in early implications for many subsequent agricultural management
August will facilitate subsequent applications in the field of applications. For example, the effect of crop-planting
agricultural remote sensing. Furthermore, the government structure adjustment in Heilongjiang Province and the effect
can use these maps for the cultivation of regional crops to of crop rotation in Heilongjiang Province can be evaluated.

Soybeans Corn Rice


Scheme 1 Scheme 2 Scheme 3
Other crops Fallow
Scheme 4 Scheme 5 Scheme 6
100
Scheme 7
100 90
90 80
80
70
F1 score (%)

70
F1 score (%)

60
60
50 50
40 40
30 30
20 20
10 10
0
Soybean Corn Rice Other corps Fallow 0
May Jun. Jul. Aug. Sep.

Fig. 11 F1 scores for the different schemes. Fig. 12 Variations in the F1 scores over time under Scheme 7.
LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1955

In addition, crop classification maps are important for some Acknowledgements


crop growth monitoring models and agroecological models,
including crop yield estimation, soil and water profit and loss The paper was funded by the National Key R&D Program
assessment, greenhouse gas emission calculation, and of China (2017YFD0201803) and the Talent Recruitment
soil carbon stock assessment (Arief and Nafi 2018; Chen Project of Northeast Institute of Geography and Agroecology,
et al. 2018; Gao et al. 2018; Huang et al. 2018; Liu et al. Chinese Academy of Sciences.
2019). Developed countries such as the United States have
established comprehensive crop distribution databases, Declaration of competing interest
which have played a significant role in agricultural
management and related scientific research (Reitsma et al. The authors declare that they have no conflict of interest.
2016). In developing countries such as China, the production
of large-scale and high-resolution crop classification maps References
still faces many challenges due to the inadequacy of the
basic database (farm area, official crop-type verification Altieri M A, Funes-Monzote F R, Petersen P. 2012.
points, etc.). However, with the continuous launch of more Agroecologically efficient agricultural systems for
advanced, domestic high-resolution satellites (Gaofen-5 and smallholder farmers: contributions to food sovereignty.
Gaofen-6) and the completion of the registration of natural Agronomy for Sustainable Development, 32, 1–13.
resources in China, the database for the regular production Arief U M, Nafi A Y. 2018. An accurate assessment tool based
of large-scale crop classification maps is improving. The on intelligent technique for suitability of soybean cropland:
case study in Kebumen Regency, Indonesia. Heliyon, 4,
combination of multisource high-resolution images, GEE and
e00684.
deep learning algorithms can produce accurate and time-
Belgiu M, Csillik O. 2018. Sentinel-2 cropland mapping using
sensitive large-scale crop classification maps more easily
pixel-based and object-based time-weighted dynamic time
and quickly, providing support for more scientific agricultural warping analysis. Remote Sensing of Environment, 204,
management projects and related scientific research. 509–523.
Boryan C, Yang Z, Mueller R, Craig M. 2011. Monitoring US
5. Conclusion agriculture: the US department of agriculture, national
agricultural statistics service, cropland data layer program.
In this paper, GEE was used to generate monthly composite Geocarto International, 26, 341–358.
Sentinel-1 and Sentinel-2 images of the crop-growing Cai Y, Guan K, Peng J, Wang S, Seifert C, Wardlow B, Li Z.
period in Heilongjiang Province, China, and the main crops 2018. A high-performance and in-season classification
in this area were classified by an RF algorithm combined system of field-level crop types using time-series Landsat
data and a machine learning approach. Remote Sensing
with time-series data. We conclude that an RF algorithm
of Environment, 210, 35–47.
using monthly composite Sentinel-1 and Sentinel-2 imagery
Chen B, Xiao X, Li X, Pan L, Doughty R, Ma J, Dong J, Qin Y,
results can meet regional-scale crop classification needs.
Zhao B, Wu Z. 2017. A mangrove forest map of China in
The continuous development of the GEE processing 2015: Analysis of time series Landsat 7/8 and Sentinel-1A
platform makes it possible to generate annual crop imagery in Google Earth Engine cloud computing platform.
classification maps conveniently, quickly and accurately. ISPRS Journal of Photogrammetry and Remote Sensing,
The GEE processing platform simplifies the process of 131, 104–120.
crop classification, allowing researchers to devote more Chen D, Huang J, Jackson T J. 2005. Vegetation water content
energy to the improvement of algorithms without considering estimation for corn and soybeans using spectral indices
image preprocessing. The use of cloud processing derived from MODIS near- and short-wave infrared bands.
platforms such as GEE is important for the development Remote Sensing of Environment, 98, 225–236.
of agricultural remote sensing. Our classification of crops Chen S, Martin M P, Saby N P, Walter C, Angers D A, Arrouays
D. 2018. Fine resolution map of top- and subsoil carbon
in Heilongjiang Province in 2018 shows that corn is the
sequestration potential in France. Science of the Total
most widely distributed crop among the main crops in the
Environment, 630, 389–400.
area, followed by rice and soybeans. This information can
Cihlar J, Manak D, D’Iorio M. 1994. Evaluation of compositing
improve the accuracy of crop growth monitoring and crop algorithms for AVHRR data over land. IEEE Transactions
yield estimation. It is worth noting that the method was on Geoscience and Remote Sensing, 32, 427–437.
applied only in the northeastern part of China in the first Claverie M, Ju J, Masek J G, Dungan J L, Vermote E F, Roger
quarter of 2018 in this study. The feasibility of this method J C, Skakun S V, Justice C. 2018. The Harmonized Landsat
in other regions is still in need of further study. and Sentinel-2 surface reflectance data set. Remote
1956 LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957

Sensing of Environment, 219, 145–161. Sensing, 8, 362.


Clevers J G, Gitelson A A. 2013. Remote estimation of crop Jiang Q, Zhao Y, Wang Z, Fu Q, Wang T, Zhou Z, Dong Y. 2018.
and grass chlorophyll and nitrogen content using red-edge Simulating the evolution of the land and water resource
bands on Sentinel-2 and-3. International Journal of Applied system under different climates in Heilongjiang Province,
Earth Observation Geoinformation, 23, 344–351. China. Water, 10, 868.
Cohen J. 1960. A coefficient of agreement for nominal scales. Kussul N, Lavreniuk M, Skakun S, Shelestov A. 2017. Deep
Educational and psychological measurement, 20, 37–46. learning classification of land cover and crop types using
Congalton R G. 1991. A review of assessing the accuracy of remote sensing data. IEEE Geoscience Remote Sensing
classifications of remotely sensed data. Remote Sensing Letters, 14, 778–782.
of Environment, 37, 35–46. Kussul N, Lemoine G, Gallego F J, Skakun S V, Lavreniuk M,
Conrad C, Fritsch S, Zeidler J, Rücker G, Dech S. 2010. Per- Shelestov A Y. 2016. Parcel-based crop classification in
field irrigated crop classification in arid Central Asia using ukraine using Landsat-8 data and Sentinel-1A data. IEEE
SPOT and ASTER data. Remote Sensing, 2, 1035–1056. Journal of Selected Topics in Applied Earth Observations
Davidson A, Fisette T, Mcnairn H, Daneshfar B. 2017. Detailed Remote Sensing, 9, 2500–2508.
crop mapping using remote sensing data (crop data layers). Lee J S, Wen J H, Ainsworth T L, Chen K S, Chen A J. 2008.
In: Handbook on Remote Sensing for Agricultural Statistics: Improved sigma filter for speckle filtering of SAR imagery.
Handbook of the Global Strategy to Improve Agricultural IEEE Transactions on Geoscience and Remote Sensing,
and Rural Statistics (GSARS). Rome, Italy. pp. 91–117. 47, 202–213.
Davis K F, Rulli M C, Seveso A, D’Odorico P. 2017. Increased Liaw A, Wiener M. 2002. Classification and regression by
food production and reduced water use through optimized randomForest. R News, 2, 18–22.
crop distribution. Nature Geoscience, 10, 919. Liu C A, Chen Z X, Yun S, Chen J S, Hasi T, Pan H Z. 2019.
Díaz-Uriarte R, De Andres S A. 2006. Gene selection and Research advances of SAR remote sensing for agriculture
classification of microarray data using random forest. BMC applications: A review. Journal of Integrative Agriculture,
Bioinformatics, 7, 3. 18, 506–525.
Gao B, Huang T, Ju X, Gu B, Huang W, Xu L, Rees R M, Luo C, Liu H J, Fu Q, Guan H X, Kong F C. 2020. Mapping the
Powlson D S, Smith P, Cui S. 2018. Chinese cropping fallowed area of paddy fields on Sanjiang Plain of Northeast
systems are a net source of greenhouse gases despite China to assist water security assessments. Journal of
soil carbon sequestration. Global Change Biology, 24, Integrative Agriculture, 19, 1885–1896.
5590–5606. Mahdianpari M, Salehi B, Mohammadimanesh F, Homayouni S,
Gitelson A A, Kaufman Y J, Merzlyak M N. 1996. Use of a green Gill E. 2019. The wetland inventory map of newfoundland at
channel in remote sensing of global vegetation from EOS- a spatial resolution of 10 m using Sentinel-1 and Sentinel-2
MODIS. Remote Sensing of Environment, 58, 289–298. data on the Google Earth Engine Cloud Computing Platform.
Goutte C, Gaussier E. 2005. A probabilistic interpretation of Remote Sensing, 11, 43.
precision, recall and F-score, with implication for evaluation. Meng L, Liu H, Zhang X, Ren C, Ustin S, Qiu Z, Xu M, Guo D.
Advances in Information Retrieval, 3408, 345–359. 2019. Assessment of the effectiveness of spatiotemporal
Griffiths P, Nendel C, Hostert P. 2019. Intra-annual reflectance fusion of multi-source satellite images for cotton yield
composites from Sentinel-2 and Landsat for national- estimation. Computers Electronics in Agriculture, 162,
scale crop and land cover mapping. Remote Sensing of 44–52.
Environment, 220, 135–151. Potapov P V, Turubanova S A, Hansen M C, Adusei B, Broich
Hao P Y, Tang H J, Chen Z X, Meng Q Y, Kang Y P. 2020. M, Altstatt A, Mane L, Justice C O. 2012. Quantifying
Early-season crop type mapping using 30-m reference time forest cover loss in Democratic Republic of the Congo,
series. Journal of Integrative Agriculture, 19, 1897–1911. 2000–2010, with Landsat ETM+ data. Remote Sensing of
Hu Q, Wu W B, Song Q, Lu M, Chen D, Yu Q Y, Tang H J. Environment, 122, 106–116.
2017. How do temporal and spectral features matter in crop Reitsma K D, Clay D E, Clay S A, Dunn B H, Reese C. 2016.
classification in Heilongjiang Province, China? Journal of Does the US cropland data layer provide an accurate
Integrative Agriculture, 16, 324–336. benchmark for land-use change estimates? Agronomy
Huang Y, Chen Z, Tao Y U, Huang X, Gu X F. 2018. Agricultural Journal, 108, 266–272.
remote sensing big data: Management and applications. Rockström J, Williams J, Daily G, Noble A, Matthews N,
Journal of Integrative Agriculture, 17, 1915–1931. Gordon L, Wetterstrand H, DeClerck F, Shah M, Steduto
Huete A, Didan K, Miura T, Rodriguez E P, Gao X, Ferreira P. 2017. Sustainable intensification of agriculture for human
L G. 2002. Overview of the radiometric and biophysical prosperity and global sustainability. Ambio, 46, 4–17.
performance of the MODIS vegetation indices. Remote Singha M, Dong J, Zhang G, Xiao X. 2019. High resolution
Sensing of Environment, 83, 195–213. paddy rice maps in cloud-prone Bangladesh and Northeast
Inglada J, Vincent A, Arias M, Marais-Sicre C. 2016. Improved India using Sentinel-1 data. Scientific data, 6, 26.
early crop type identification by joint use of high temporal Tian F, Wu B, Zeng H, Zhang X, Xu J. 2019. Efficient
resolution SAR and optical image time series. Remote identification of corn cultivation area with multitemporal
LUO Chong et al. Journal of Integrative Agriculture 2021, 20(7): 1944–1957 1957

synthetic aperture radar and optical images in the Google 36, 1324–1338.
Earth Engine Cloud Platform. Remote Sensing, 11, 629. Xiao X, Boles S, Liu J, Zhuang D, Liu M. 2002. Characterization
Tucker C J. 1979. Red and photographic infrared linear of forest types in northeastern China, using multi-temporal
combinations for monitoring vegetation. Remote Sensing SPOT-4 VEGETATION sensor data. Remote Sensing of
of Environment, 8, 127–150. Environment, 82, 335–348.
Vuolo F, Neuwirth M, Immitzer M, Atzberger C, Ng W T. 2018. Yommy A S, Liu R, Wu S. 2015. SAR image despeckling using
How much does multi-temporal Sentinel-2 data improve refined Lee filter. 2015 7th International Conference on
crop type classification? International Journal of Applied Intelligent Human-Machine Systems and Cybernetics, 2,
Earth Observation Geoinformation, 72, 122–130. 260–265.
Waldhoff G, Lussem U, Bareth G. 2017. Multi-data approach You L, Wood S, Wood-Sichra U. 2009. Generating plausible
for remote sensing-based regional crop rotation mapping: crop distribution maps for Sub-Saharan Africa using a
A case study for the Rur catchment, Germany. International spatially disaggregated data fusion and optimization
Journal of Applied Earth Observation Geoinformation, 61, approach. Agricultural Systems, 99, 126–140.
55–69. Zhang X W, Liu J F, Qin Z Y, Qin F. 2019. Winter wheat
Wolfe R E, Roy D P, Vermote E. 1998. MODIS land data identification by integrating spectral and temporal
storage, gridding, and compositing methodology: Level 2 information derived from multi-resolution remote sensing
grid. IEEE Transactions on Geoscience Remote Sensing, data. Journal of Integrative Agriculture, 18, 2628–2643.

Executive Editor-in-Chief ZHANG Wei-li


Managing Editor SUN Lu-juan

You might also like