1 Predicting Poverty in Mali using Satellite Imagery

2
3 MOHAMMED BALLOUTI, University of Amsterdam
4
5 This paper looks at the use of satelite imagery in conjunction with Convolutional Neural Networks to predict poverty levels in Mali.
6 We find that although, the model performance is worse than other classification methods such as XGBoost, it is probable that higher
7
performance can be achieved if the model is slightly modified and the satellite imagery is improved. At the moment, we find that this
8
method is more cost-effective way of predicting poverty levels in difficult to reach areas.
9
10
Additional Key Words and Phrases: Convolutional Neural Networks, Poverty Detection, Machine Learning
11
12
13 ACM Reference Format:
14 Mohammed Ballouti. 2023. Predicting Poverty in Mali using Satellite Imagery. 1, 1 (February 2023), 7 pages. https://doi.org/10.1145/
15 nnnnnnn.nnnnnnn
16
17
18
1 INTRODUCTION
19
20 Poverty is one of the most fundamental problems that humanity has tried to and still is trying to solve. This problem is
21 especially difficult to eradicate in countries where we are unaware of the levels of poverty in the first place. Poverty
22
itself is best defined as the inability of a populace to meet the requirements of basic needs within a society. Therefore,
23
24
to measure this, we would need data on variables we simply do not have access. War-torn countries are quite often the
25 first victims of these kinds of problems. Mali, for example, is still suffering under a long civil war that started in 2012.
26 The recent developments in machine-learning-based methods have made it possible to still predict poverty levels in
27
poor countries while having limited data. One such possible solution involves the use of satellite imagery by using
28
29
Convolutional Neural Networks (CNN) to classify areas to a certain poverty level. This model combines computer vision
30 and machine learning to analyze satellite images and predict poverty levels. For this paper, we look at the situation in
31 Mali to try and see if we can predict poverty levels using satellite images and CNNs
32
Our research question is hence: How well can the relationship between satellite imagery data and poverty levels
33
34
in Mali be explained according to a CNN model? We find that comparing the CNN model with a baseline XGBoost
35 model, it is still somewhat lacking in accuracy and precision. However, we found that there is still room to improve
36 upon and conclude that this method is a cost-effective and feasible substitute for traditional poverty detection methods.
37
The remaining part of this paper proceeds as follows: Chapters 2 and 3 respectively provide the related works relevant
38
39
to this research and an explanation of the methods. Chapters 4 and 5 aim to explain the data, baseline and results of our
40 research. Finally, Chapters 6 and 7 deal with a discussion of the research and possible applications of the model.
41
42
Author’s address: Mohammed Ballouti, mohammed.ballouti@student.uva.nlUniversity of Amsterdam.
43
44
45
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not
46
made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components
47
of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to
48 redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.
49 © 2023 Association for Computing Machinery.
50 Manuscript submitted to ACM
51
52 Manuscript submitted to ACM 1
 2 Mohammed Ballouti

53 2 RELATED WORK
54
55
In order to circumvent the lack of reliable data to measure the state of economic disparities in poor countries, researchers
56 have been looking at the use of alternative sources to predict said disparities. In the last decade or so, we have seen
57 developments in new machine learning techniques such as Deep Learning and CNNs [1]. Recently, we have also seen
58
noticeable advancements in the detection of economic indicators with the use of geospatial data using these machine
59
60
learning models. One can think of data regarding night-time luminosity and day-light satellite imagery, the latter
61 of which is used in this paper. This kind of data is very beneficial when one considers the low costs involved with
62 retrieving the data from high-risk areas. There have been studies such as [3], which look at quantifiable features with
63
regards to the geospatial information such as the number of roads, hospitals etc. Another group, which this paper is
64
65
directly based on, looks at geospatial information which is correlated with economic activity in a certain region [2].
66 The researchers used a new multistep learning approach to overcome the low availability of poverty data in African
67 countries. This model can explain 55 to 75% of the variation in average household asset wealth and 37 to 55% in the
68
average household consumption. This method has also been used in other countries [6]
69
70
The early research regarding this topic used satellite imagery in conjunction with data on night luminosity (which
71 in this case was used as a substitute for economic development). This type of model can distinguish objects such as
72 buildings and roads and assign a certain economic value to the detected objects. One of the major downsides of using
73
nighttime luminosity in predicting poverty levels in Mali is that, in villages, there is little luminosity, to begin with
74
75
so the poverty estimation regarding images of villages will be far less accurate. For this paper, we hence look at the
76 International Wealth Index (IWI) as an economic indicator, since this is the first comparable asset-based wealth index
77 which works quite well as a wealth comparison index in developing countries like Mali [5].
78
79
80
81
82 3 METHODOLOGY
83
Our methodology is quite straightforward. We implement a CNN model where the satellite images are used as an input.
84
85
Neural networks are inspired by biological neural networks. CNN, however, is used when we want to control large
86 features in image recognition tasks such as ours. What happens is with the image data, we force the extraction of
87 the features inside an image to recognize the inputs. The image in figure 1 gives an example of this idea. In image
88
recognition tasks such as these, there are three parts, the input layer, where we input the data, the hidden layer, where
89
90
we apply the transformations before passing them on and finally, the output layer which consists of the number of
91 outputs, depending on if we have a regression or classification task.
92 Since we are working with a limited dataset, we have chosen to use five categories to define the IWI, namely,
93
𝑉 𝐸𝑅𝑌 𝐿𝑂𝑊 , 𝐿𝑂𝑊 , 𝑀𝐸𝐷𝐼𝑈 𝑀, 𝐻𝐼𝐺𝐻 and 𝑉 𝐸𝑅𝑌 𝐻𝐼𝐺𝐻 . The first step in our methodology is to locate the data points
94
95
from the DHS dataset onto the satellite images. Once we know where the IWI survey was conducted we can assign
96 a classification to a satellite image based on the IWI index. The other features of the DHS dataset will be used on a
97 different machine learning model which will perform as our baseline in this paper. The dataset is fine-tuned using
98
gradient descent and a cross-entropy loss function, the latter of which is given by:
99
100 𝐾
∑︁  
101 ˆ 𝑦) =
𝐿(𝑦, 𝑦𝑘 log 𝑦ˆ (𝑘 ) . (1)
102 𝑘=1
103
104 Manuscript submitted to ACM
 Predicting Poverty in Mali using Satellite Imagery 3

105
106
107
108
109
110
111
112
113
114
115
116
117
118
119 Fig. 1. Example of a Convolutional Neural Network
120
121
122
Learning rate 0.0001
123
Batch size: training 6
124
Step size 50
125
Gamma 0.5
126
Momentum 0.9
127
Table 1. The CNN regression model configuration
128
129
130
131
132
133 Here 𝑦 (𝑘 ) is equal to 0 or 1, indicating if class label 𝑘 is the correct classification. We fine-tune the model according
134 to the hyperparameters that are displayed in table 1. We use a pre-trained resnet18 model which we finetune over the
135
epochs.
136
137
138 4 EXPERIMENTS
139
4.1 Dataset
140
141 The dataset comes from two sources. The satellite images come from the "Satellite Images to predict poverty" on Kaggle.
142
This dataset contains more than 10,000 128x128 images from 4 Sub-Saharan countries along with the latitudes and
143
144
longitudes of each image. The data on poverty and the features used in the XGBoost model comes from the DHS dataset,
145 which comes from the Demographic and Health Surveys (DHS) Program, which is a U.S. project aimed at collecting and
146 developing accurate data on health and socio-economic data in developing countries. Since we can only classify the
147
poverty levels according to the places that DHS have looked at, our initial image dataset shrinks significantly. We use
148
149
1602 images for the model after removing the satellite image which does not contain any research conducted by DHS.
150
151 4.2 Evaluation Methods
152
153 Accuracy is one of the more famous metrics we can use for evaluating classification models.
154 𝑇𝑃 +𝑇𝑁
155
Accuracy =
𝑇𝑃 + 𝑇 𝑁 + 𝐹𝑃 + 𝐹𝑁
156 Manuscript submitted to ACM
 4 Mohammed Ballouti

157 XGBoost CNN

158
159
Accuracy 0.727829 0.524242
160
Precision 0.962264 0.647059
161
Recall 0.850000 0.897905
162 Table 2. Table showing the accuracy, precision and recall of the two models
163
164
165
166 The other metrics, we use are Precision, Recall and F1. Precision gives us the proportion of positive identifications
167 which were correct while Recall gives the proportion of actual positives that we identified correctly.
168
𝑇𝑃
169 Precision =
𝑇𝑃 + 𝐹𝑃
170
171
𝑇𝑃
172 Recall =
𝑇𝑃 + 𝐹𝑁
173
174 4.3 Competing methods
175
176 To compare our CNN model, we will look to use the data from DHS and run it through an eXtreme Gradient Boosting
177 (XGBoost) model. This is an advanced version of the Gradient Boosting algorithm with advanced speed and efficiency.
178 It is quite favourable because of its high accuracy in numerous applications of Kaggle competitions. The algorithm is
179
made up of Classification and Regression Trees (CART) which are used as base learners and are fitted to minimize a
180
181 certain loss function. This again is all done using gradient descent.
182
183 5 RESULTS AND ANALYSIS
184
The fine-tuning of the CNN model went quite smoothly. Figure 2 shows how the training and validation loss of the
185
186 model fine-tuning went. After around 10 it stabilised for both cases. The validation loss eventually shrinks more than
187 the training loss does and this could be because validation loss is calculated at the end of each epoch whereas training
188
loss is calculated during each epoch. In table 2, we can see the accuracy, precision and recall of the two models. We can
189
see that the CNN performs somewhat well but not as well as the XGBoost model.
190
191 On the next page, we find Figures 3 and 4. The first figure contains the confusion matrix for the CNN model. We
192 immediately see that the model has an issue with correctly detecting the 𝑉 𝐸𝑅𝑌 𝐿𝑂𝑊 category. This could be because
193
there were few images in that category to fine-tune our model on. The CNN model also has somewhat of a problem
194
correctly identifying the difference between 𝐿𝑂𝑊 and 𝑀𝐸𝐷𝐼𝑈 𝑀. We also see that 𝑉 𝐸𝑅𝑌 𝐻𝐼𝐺𝐻 is identified quite well
195
196 but that actually 𝐻𝐼𝐺𝐻 is more often misidentified as 𝑉 𝐸𝑅𝑌 𝐻𝐼𝐺𝐻 than correctly identified as 𝐻𝐼𝐺𝐻 . Figure 4 contains
197 a sample of the heatmap of the test images which shows concentrations of higher relative wealth in a few cities and a
198
clear drop-off everywhere outside of those cities.
199
200
6 REFLECTION AND FUTURE DIRECTION
201
202 In this paper, we proposed a prediction methodology to produce poverty predictions at the regional level using geospatial
203
data and machine learning algorithms. The study used a relatively small dataset with lower-resolution images. The
204
205
immediate improvements that can be made in this study are hence quite clear. We might also improve our models if we
206 increase our GPU capability. If we were to use deeper network architecture with an increase in batch size, we might
207 see an improvement in the evaluation metrics. CNN works worse than XGBoost but there is a noticeable difference
208 Manuscript submitted to ACM
 Predicting Poverty in Mali using Satellite Imagery 5

209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
Fig. 2. Test accuracy of CNN model
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
Fig. 3. Confusion matrix of the CNN model
251
252
253
254 between the two models. The latter can only work on survey data and can only make predictions about places where
255 the survey data was conducted. Conversely, our CNN model does not have this limitation. It learned using satellite
256
imagery together with the survey data but can make predictions using only satellite imagery so that it is quite easy to
257
258
use temporally. This is one of the big benefits of the use of CNN models. In short, further studies are needed with a
259 better dataset and increased technological capabilities to improve the model.
260 Manuscript submitted to ACM
 6 Mohammed Ballouti

261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
Fig. 4. Example of poverty heatmap using the classification of several test images. A darker shade of blue indicates more poverty.
288
289
290
7 APPLICATION IN BUSINESS
291
292 This model has the potential to be highly useful for aid organizations and governments in their efforts to alleviate
293 poverty, as it allows them to quickly and accurately predict poverty levels in a given area without the need for extensive
294
household surveys. As was said before, the dataset from DHS lacked the same kind of extensive outreach we see in
295
296
surveys conducted in richer countries. Though, this type of model can be updated regularly using new satellite images,
297 which provides a more up-to-date picture of poverty levels in a given region without the necessary costs involved. This
298 method can probably be extended to other research areas to, for example, improve agricultural field segmentation [4].
299
With a growing food problem in developing countries, it will become increasingly important to accurately classify land
300
301
use. From a business perspective, this innovation could increase yields through efficient land use.
302
303 8 CITATIONS AND BIBLIOGRAPHIES
304
REFERENCES
305
[1] Corinna Cortes, ND Lawarence, DD Lee, M Sugiyama, and R Garnett. Advances in neural information processing systems 28. In Proceedings of the
306
29th Annual Conference on Neural Information Processing Systems, 2015.
307
[2] Neal Jean, Marshall Burke, Michael Xie, W. Matthew Davis, David B. Lobell, and Stefano Ermon. Combining satellite imagery and machine learning
308
to predict poverty. Science, 353(6301):790–794, 2016.
309
[3] Kamwoo Lee and Jeanine Braithwaite. High-resolution poverty maps in sub-saharan africa. World Development, 159:106028, 2022.
310 [4] Furkat Safarov, Kuchkorov Temurbek, Djumanov Jamoljon, Ochilov Temur, Jean Chamberlain Chedjou, Akmalbek Bobomirzaevich Abdusalomov,
311 and Young-Im Cho. Improved agricultural field segmentation in satellite imagery using tl-resunet architecture. Sensors, 22(24):9784, 2022.
312 Manuscript submitted to ACM
 Predicting Poverty in Mali using Satellite Imagery 7

313 [5] Jeroen Smits and Roel Steendijk. The international wealth index (iwi). Social indicators research, 122:65–85, 2015.
314 [6] Potnuru Kishen Suraj, Ankesh Gupta, Makkunda Sharma, Sourabh Bikash Paul, and Subhashis Banerjee. On monitoring development using high
315 resolution satellite images. arXiv preprint arXiv:1712.02282, 2017.
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364 Manuscript submitted to ACM