Professional Documents
Culture Documents
89 system is capable of decoding data bit separating accurate data transmitting LED region
90 among various light-emitting sources present in the camera frame. Taking account of the
91 visual flickering, we have modulated the data stream by FSOOK technique, a low cost
92 and low complexity modulation technique, which encodes bits into different frequency
93 [26]. This modulated optical signal was transmitted through a wireless medium to the
94 receiver in the presence of other blinking LEDs. The scenario of how the proposed
95 intelligent camera receiver will work is depicted in Fig. 1. Based on our scheme, we
96 have developed a new data decoding algorithm for FSOOk based OCC Our designed
97 receiver first detects every LED region employing CNN. From each region, features are
98 extracted to train the SVM classifier. Using the trained classifier model, the camera can
99 successfully separate accurate regions for data decoding purposes. As a result, unwanted
100 LED regions will be removed which will result in low computational complexity and
101 mitigation of neighboring interference.
102 The remainder of this paper is structured as follows. Section II describes the SVM
103 classifier. The overall OCC system including modulation technique and algorithm for
104 decoding data intelligently from the recognized region is has been illustrated in Section
105 III. We have demonstrated how the data set has been prepared using extracted features
106 in Section IV. In Section V, we elaborately discussed the experimental results to justify
107 our scheme. Finally, we concluded our article in Section VI.
116 hyperplane. For a given training sample (x1 , y1 ), (x2 , y2 ), . . . .., (xm , ym ) where class
117 labels are represented by
θ T xi + b = 0, (2)
1 1
kθ k2 = θ T θ, (3)
2 2
which is subjected to
θ T xi + b ≥ +1 for yi = +1. (4)
T
θ xi + b ≤ +1 for yi = −1. (5)
The final nonlinear decision function can be obtained as follows
!
m
f (x) = sign ∑ αi θ xi + b .
T
(6)
i=1
Polynomial ( x T xi + c ) p
Sigmoid tanh(( x T xi + c) p )
Version May 22, 2021 submitted to Sensors 5 of 17
122 The commonly used kernel functions are Linear, Polynomial, sigmoid, the radial basis
123 function (RBF) which are briefly described in the following Table 1.
To generalize SVM classifier about complex data features, SVM uses a technique
called Kernel k(xi , x). The value k(xi , x) corresponds to ϕ(xi ).ϕ(x) which maps linearly
non-separable patterns into a higher dimension feature space. Finally, the decision
function can be modified as follows
!
m
f (x) = sign ∑ αi (ϕ(xi ).ϕ(x)) + b . (7)
i=1
126 In Fig. 3, the block diagram delineates the overall OCC architecture in addition to
127 the proposed classifier model. The structure mainly consists of two parts: the transmitter
128 part and the receiver part but unlike the traditional receiver system, the receiver part
129 includes some additional features that assist to differentiate the accurate data transmit-
130 ting LED over other LEDs. On the transmitter side, numerous LED regions are present.
131 The data are modulated by FSOOK modulation at a frequency level between 2-4 kHz
132 before sending to the receiver. While the modulation is performed, the LED blinking
133 should not be perceivable to the human eye. Since, the perceivable flickering rate is
134 up to 200 Hz [29]. This modulation frequency used in FSOOK can attenuate flickering
135 issue in a significant margin. With regards to the transmitting LEDs and other sources of
136 noise, they are not modulated in that particular frequency. The projection of all these
137 light-emitting sources is captured by the image sensor. Interference sources are dispelled
138 when a trained CNN model is used. All the possible LED sources are detected and
139 segmented using image processing techniques. Due to the rolling shutter effect of CMOS
140 based image sensor, each image frame gets striated of white and black shade. Afterward,
141 necessary geometrical features are extracted to classify and recognize accurate data
142 transmitting region from the range of all possible regions.
Version May 22, 2021 submitted to Sensors 6 of 17
144 It is a modulation scheme that encodes bit ‘1’ and bit ‘0’ with two different fre-
145 quencies known as mark and space frequency respectively [26]. The header and footer
146 is appended at the beginning and ending of of data bits respectively to identify the
147 data packets. In order to create distinction among header, footer and data bits, we use
148 synchronization bit pattern ‘111’ as the header and footer. Due to the rolling shutter
149 effect, we get sequence of bit pattern in each data packet depicted in Fig. 4.
151 The workflow of the proposed intelligent receiver system is depicted in Fig. 5.
152 The receiver system consists of CMOS based image sensor that collect optical signals
Version May 22, 2021 submitted to Sensors 7 of 17
153 transmitted through the optical channel. The captured image is now input to the
154 object classifier designed by CNN algorithm. These algorithms are eligible to extract
155 image features automatically and can learn from those features effectively. The CNN
156 recognizes all light source regions by performing the functions like reason proposal,
157 feature extraction, classification, etc. Several layers inside CNN namely convolutional
158 layers, pooling layers, and fully connected layers, perform those functions but the
159 convolutional layer is the main part of CNN. The convolutional layers are composed
160 of convolutional kernel which is used to generate proposed region and feature maps
161 from the original image. Convolutional kernel can be of many sizes. In our cases, we
162 have used 4 × 4 convolutional filter to extract the features. The convoluted output is
163 estimated by the following equation [30]
q −1 p −1
y= ∑ ∑ (w pq xa+1,b+ p + Θ), (0 ≤ a ≤ a f , 0 ≤ b ≤ b f ). (8)
q=0 p=0
164 Max pooling with a 2 × 2 filter further used to reduce the spatial size of the warped
165 images and computation in the network resulting in less prone to overfitting preserving
166 all features. This pooled feature map was flattened into one column while passing
167 through the FC layers. Here, FC layers tries to determine most correlated features of
168 a particular class. The non linear softmax function is used to classify LED from other
169 objects in Fc layer.
170 In Fig. 5, the detected LED regions with bounding boxes are the output of the CNN.
171 The bounding box objects are then finally processed to precisely determine the accurate
172 data transmitting LED and decode data as well. To make the receiver an intelligent
173 and robust retrieval system, the system needs to respond by differentiating accurate
174 LEDs as quickly as possible. Therefore, the selection of features is very important.
175 Extraction of appropriate features is very challenging since the objects are almost the
176 same. LED objects of the same type can have a different shape, stripe pattern based on
177 the communication distance, camera frame rate. mark and space frequency. To analyze
178 the different geometrical shapes, computation of the features from the contour line has
179 been considered significant distinguishable features [31]. Contours are the outline that
180 is designed using the edges of the object to represent the shape. They contain some
181 geometrical attributes that are effective to recognize and segment objects.
182 Based on the extracted input features, a trained SVM classifier detects accurate data
183 transmitting the LED region. Finally, from that region data bits are decoded accordingly.
184 The overall demodulation scheme of our proposed intelligent receiver is shown in
185 Algorithm 1.
187 where f pn∗q is nth region and bm∗n is the structuring element of which m, n a, b.
Parameter Value
LED 10 mm (diameter)
Camera exposure time 15.6 ms, 3.90 ms, 976.6 µs
Camera frame rate 30 fps
Camera image resolution 1280 ∗ 720
Mark and Space frequency 4000 kHz and 2000 kHz
Learning rate parameter 0.001
No. of epoch 30
Kernels RBF, linear, polynomial, sigmoid
188 In Fig. 6, we have presented a single frame where the shape of combined stripes
189 contour is shown. For each projected LED either data transmitter or unwanted LED, we
190 found this type of shape of the contour formed based on the combined stripe. Analyzing
191 the geometrical pattern of the contour, there is considerable change in the pattern when
192 the communication distance is varied or mark and space frequency has been changed.
193 So based on the contour pattern of the data transmitting LED region and other unwanted
194 LED regions, we have selected the following features and the features are elaborately
195 described in the next section.
Version May 22, 2021 submitted to Sensors 9 of 17
Figure 6. Example of a single frame where all the generated stripes are combined together and a
contour is formed.
197 where, σx p yq denotes moment weighting kernel. Different image properties lies in the
198 raw moments. In our case, we have denoted moment of order (0, 0) as area of the
199 combined stripes for data transmitting LED.
1
ST = , (11)
f tr
208 where, f represents mark or space frequency and tr is the time needed to read-out a
209 single pixel of the image.
219 curve of fewer points [34]. We have used this algorithm to relate the number of line
220 segments with the no. of stripes It is noticeable that with the variation of LED on-
221 off frequency and distance, getting accurate contour of the object is difficult. So, in
222 the counting of stripes error may present. Combining all stripes, we segmented the
223 combined contour using the Douglas-Peucker algorithm based on the corner point of
224 each stripe.
225 The scatter plot of the selected features has been shown in Fig. 7. In the figures,
226 the distribution of features has shown clear distinction among data transmitting LED
227 and unwanted LED which affirms our motive to introduce an SVM classifier in the
228 conventional receiver system.
Figure 7. Scatter plot of selected features (a) area vs. no. of stripes per LED region, (b) area vs.
perimeter of the combined stripes contour, and (c) area vs. no. of the line segment of the combined
stripes contour.
demonstrating receiver operating characteristic (ROC) curve and other key performance
parameters, such as precision, recall, F1 Score. These parameters are defined as
Figure 8. ROC curve of SVM classifier for (a) RBF kernel, (b) linear kernel, (c) polynomial kernel,
and (d) sigmoid kernel.
234 We have classified accurate data transmitting region from all the possible cap-
235 tured LED regions detected by CNN using the trained SVM classifier. Based on the
236 features that we have selected, the SVM classifier provides an accurate LED region for
237 demodulation. So, the other communication performances are greatly dependent on
238 the classification and resulted separated LED region. Therefore, to evaluate classifier
239 performance, we have shown the ROC curve for different Kernel functions, while the
240 communication distance was set to 30 cm and there were two LEDs among which one
241 LED was transmitting data actually, in Fig. 8. Beside, the exposure time Te was kept 976.6
242 µs. The area measured under the ROC curve known as ’AUC’ has been measured and the
243 measured ’AUC’ value is larger for RBF kernel than others and hence the classification
244 accuracy was also
Version May 22, 2021 submitted to Sensors 12 of 17
Figure 9. Comparison among precision, recall, F1 score varying exposure time (Te ).
245 Larger for the RBF kernel. In Table 3, we have presented overall accuracy and ’AUC’
246 measurement considering the different conditions. In Table 3, the recorded maximum
247 accuracy for the RBF kernel is 94.92% and the lowest accuracy for the sigmoid kernel is
248 79.87%. Moreover, we have also given a bar plot to show the comparison among key
249 performance parameters varying exposure time Te while the number of data transmitting
250 LED was one in Fig. 9. For the exposure time 976.6 µs, all the performance indexes are
251 high. However, the achieved performance has validated our scheme being capable of
252 recognizing accurate LED regions.
Table 3: Classifier performance (Accuracy and AUC) analysis with different no. of LED
No. of
Linear RBF Polynomial Sigmoid
LED
Accuracy
92.09 94.92 89.45 80.93
One (%)
AUC .89 .94 .87 .79
Accuracy
90.05 93.10 87.91 81.36
Two (%)
AUC .85 .91 .83 .81
Accuracy
87.76 89.91 88.13 79.87
Three (%)
AUC .84 .88 .85 .77
Version May 22, 2021 submitted to Sensors 13 of 17
Figure 10. Comparison of bit error rate (BER) among receiver using only SVM classifier, receiver
using only CNN classifier, and our proposed scheme.
254 After classifying accurate data transmitting LED, data bit has been decoded from
255 the cropped region using decoding algorithm described in the Algorithm 1. We have
256 decoded data and evaluated the communication performance of our proposed intelligent
257 receiver. According to our scheme, the receiver intelligently selects accurate LED regions
258 so the probability of interference from other sources or reflection from the surrounding
259 surface has been reduced. Fig. 10 depicts the measured BER performance against
260 communication distance for the receiver employing only SVM classifier, only CNN and
261 our proposed scheme. Significant reduction of BER has been observed when the SVM
262 classifier along with CNN is used. At the maximum distance of 60 cm, the BER increases
263 up to 7.9× 10−4 , which is below the FEC limit of 3.8 × 10−3 . At this point, only SVM
264 based receiver shows BER above than FEC limit whereas CNN based receiver is very
265 close to the FEC limit. It signifies the effectiveness of our proposed scheme.
Figure 11. BER analysis by (a) varying exposure time (Te ), and (b) varying no. of data transmitting
LED.
266 The BER performance varies based on exposure time. The effect of changing
267 exposure time Te on BER for a single data transmitting LED keeping two unwanted
268 LED regions has also appeared in Fig. 11(a). With the increasing of exposure time BER
269 shows enhancement in BER. This considerable enhancement in BER has appeared due
270 to the effect of interfering noise from other sources. Although with the increasing of the
Version May 22, 2021 submitted to Sensors 14 of 17
271 exposure time Te BER has increased, the maximum BER at the maximum distance is also
272 below the FEC limit in our scheme. We have also investigated BER performance varying
273 number of LED and it is shown in Fig. 11(b). When the no. of actual data transmitting
274 LED increases, efficient separation of LED regions have enormous influences on BER.
275 Therefore, a significant change in BER has been found when no. of data transmitting
276 LED has been increased from one to three. When the no. of data transmitting LED is
277 three, the BER has been achieved around 3.5 × 10−3 for exposure time Te = 976.6 µs.
Figure 12. Eye diagram of SVM assisted intelligent OCC receiver while exposure time (Te ) is 977.6
µs.
278 Furthermore, the eye diagram depicted in Fig. 12 at exposure time Te = 976.6 µs
279 has ensured the low ISI at the receiver side. During multiple data transmitting LED
280 region cases, if the receiver fails to differentiate distinct regions accurately, recovering
281 distinct pulse for a distinct bit can be difficult for the receiver. The large eye opening
282 affirms the low ISI presence at separated regions classified by the SVM classifier.
Figure 13. Data rate comparison varying no. of LED while exposure time (Te ) is 977.6 µs.
283 Since the SVM classifier separates LED regions and data is decoded from the
284 separated regions, it has a significant effect on data rate too. With the change in no. of
285 LED, data rate variation against communication distance has been shown in Fig. 12. For
286 a single LED, the maximum estimated data rate is 960 bps at distance 10 cm and 830 bps
Version May 22, 2021 submitted to Sensors 15 of 17
287 is the lowest data rate which is achieved at 60 cm for the three data transmitting LED.
288 The data rate can be augmented significantly using LED array with BER lower than FEC
289 limit.
Figure 14. BER against data rate comparison among our scheme based classifier, receiver using
only CNN classifier, and receiver using only SVM classifier.
290 Fig. 14 illustrates BER against data rate comparison while communication distance
291 was 50 cm for three different cases. At each case, two LED was in data transmitting
292 mood and one was unwanted region. For achieving 960 bps. BER recorded for SVM
293 based receiver is beyond the FEC limit while CNN based receiver shows very closeness
294 of FEC limit. Our scheme based receiver can decode data at 960 bps in BER of 3.5 × 10−3 .
295 This is due to the decoding of data from separated exact data transmitting LED region.
296 6. CONCLUSION
297 In this article, we proposed accurate data transmitting LED detection scheme for
298 reliable OCC communication. In the early receiving mechanism, only CNN has been
299 used to detect LED regions without considering unwanted LED regions or multiple
300 LED regions from multiple users. But, we have designed a scheme by which the
301 camera receiver can differentiate accurate data transmitting LED among unwanted
302 LEDs leveraging SVM classifier additionally for the first time. FSOOK modulation
303 scheme has been used to modulate data and send blinking of LED accordingly. We have
304 evaluated the overall performance elaborately demonstrating the ROC, and other key
305 performance parameters of the classifier, as well as other communication performance
306 features, such as BER, data rate, and ISI. The ROC of the SVM classifier affirms its efficacy
307 to accurately classify accurate LED. The BER analysis exhibits 10 times lower BER at
308 the maximum communication distance (60 cm). Besides, by eye diagram, we have
309 also investigated ISI of the decoded data from the recognized LED region. Moreover,
310 data rate against distance varying No. of LED have been observed for our proposed
311 scheme. In addition, BER against data rate is also discussed to ensure the necessity of our
312 proposed scheme. However, it is perceptible from the overall analysis that our scheme is
313 a very novel approach for reliable OCC communication. Based on our scheme, a new
314 frontier is expected to be outstretched for the OCC to deploy in future communication
315 technology.
316 Author Contributions: The contribution of the authors are as follows: conceptualization, M.H.R.;
317 methodology, M.H.R. and M.S.; software, M.H.R.; validation, M.S. and M.O.A.; formal analysis,
318 M.S. and M.K.H.; writing—original draft preparation, M.H.R.; writing—review and editing,
Version May 22, 2021 submitted to Sensors 16 of 17
319 M.K.H., M.S., and M.O.A.; supervision, Y.M.J.. All authors have read and agreed to the published
320 version of the manuscript.
321 Funding: “This work was supported by Institute for Information & communications Technology
322 Promotion (IITP) grant funded by the Korea government (MSIT) (No.2017-0-00824, Development of
323 Intelligent and Hybrid OCC-LiFi Systems for Next Generation Optical Wireless Communications)”.
References
1. Chow, C. W.; Yeh, C. H.; Liu, Y.; Liu, Y. F. Digital signal processing for light emitting diode based visible light communication.
IEEE Photon. Soc. Newslett. 2012, 26, 9–13.
2. C.-L. Liao,; Ho, C.-L.; Chang, Y.-F.; Wu, C.-H.; Wu, M.-C. High-speed light-emitting diodes emitting at 500 nm with 463-MHz
modulation bandwidth. IEEE Electron Device Lett. 2014, 35, 563–565.
3. Luo, P. et al. Experimental demonstration of RGB LED-based optical camera communications. IEEE Photonics J. 2015, 7, 1–12.
4. Le, N. T.; Hossain, M. A.; Jang, Y. M. A survey of design and implementation for optical camera communication. Signal Process.
Image Commun. 2017, 53, 95–109.
5. Yamazato T. et al. Image-sensor-based visible light communication for automotive applications. IEEE Commun. Mag. 2014, 52,
88–97.
6. Chowdhury, M. Z.; Hasan, M. K.; Shahjalal, M.; Hossan, M. T.; Jang, Y. M. Optical wireless hybrid networks: Trends, opportunities,
challenges, and research directions. IEEE Commun. Surv. Tutor. 2020, 22, 930–966.
7. Takai, I.; Harada, T.; Andoh, M.; Yasutomi, K.; Kagawa, K.; Kawahito, S. Optical vehicle-to-vehicle communication system using
LED transmitter and camera receiver. IEEE Photonics J. 2014, 6, 1–14.
8. Chowdhury, M. Z.; Hossan, M. T.; Shahjalal, M.; Hasan, M. K.; Jang, Y. M. A new 5G eHealth architecture based on optical camera
communication: An Overview, Prospects, and Applications. IEEE Consum. Electron. Mag. 2020, 9, 23–33.
9. Katz, M.; Ahmed, I. Opportunities and challenges for visible light communications in 6G. Proceedings of 2nd 6G Wireless Summit
(6G SUMMIT), Levi, Finland, 2020.
10. Marabissi, D.; Mucchi, L.; Caputo, S.; Nizzi, F.; Pecorella, T.; Fantacci, R.; Nawaz, T.; Seminara, M.; Catani, J. Experimental
Measurements of a Joint 5G-VLC Communication for Future Vehicular Networks. J. Sens. Actuator Netw. 2020, 9,32.
11. Pham, T. L.; Shahjalal, M.; Bui, V.; Jang, Y. M. Deep Learning for Optical Vehicular Communication. IEEE Access 2020, 8 ,
102691–102708.
12. Chen, Q. et al. Spaced color shift keying modulation for camera-based visible light communication system using rolling shutter
effect. Opt. Commun. 2019, 449, 19–23.
13. Roberts, R. D. Undersampled frequency shift ON-OFF keying (UFSOOK) for camera communications (CamCom). Proceedings of
22nd Wireless and Optical Communication Conference, Chongqing, China, 2013.
14. Liu, N.; Cheng, J.; Holzman, J. F. Undersampled differential phase shift on-off keying for optical camera communications. J.
Commun. Netw. 2017, 2, 47–56.
15. Monteiro, E.; Hranilovic, S. Design and implementation of color-shift keying for visible light communications. J. Lightwave Technol.
2014, 32, 2053–2060.
16. Rachim, V. P.; Chung, W.-Y. Multilevel intensity-modulation for rolling shutter-based optical camera communication. IEEE
Photonics Technol. Lett. 2018, 30, 903–906.
17. Islam, A.; Hossan, M. T.; Jang, Y. M. Convolutional neural networkscheme–based optical camera communication system for
intelligent Internet of vehicles. Int. J. Distrib. Sens. Netw. 2018, 14, 1550147718770153.
18. Yu, K.; He, J.; Huang, Z. Decoding scheme based on CNN for mobile optical camera communication. Appl. Optics 2020, 59,
7109–7113.
19. Hasan, M. K.; Chowdhury, M. Z.; Shahjalal, M.; Nguyen, V. T.; Jang, Y. M. Performance analysis and improvement of optical
camera communication. Applied Sciences 2018, 8, 2527.
20. Nagura, T.; Yamazato, T.; Katayama, M.; Yendo, T.; Fujii, T.; Okada, H. Improved decoding methods of visible light communication
system for ITS using LED array and high-speed camera. Proceedings of IEEE 71st Vehicular Technology Conference, Taipei,
Taiwan, 2010.
21. Chi, N.; Jia, J.; Hu, F.; Zhao, Y.; Zou, P. Challenges and prospects of machine learning in visible light communication. Journal of
Communications and Information Networks 2020, 5, 302–309.
22. Yuan, Y. et al. SVM-based detection in visible light communications. Optik 2017, 151, 55–64.
23. Li, J.; Guan, W. The optical barcode detection and recognition method based on visible light communication using machine
learning. Applied Sciences 2018, 8, 2425.
24. Niu, W., Ha, Y.; Chi, N. Support vector machine based machine learning method for GS 8QAM constellation classification in
seamless integrated fiber and visible light communication system. Sci. China Inf. Sci. 2020, 63, 202306.
25. Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of International
Conference on Learning Representation, virtual conference, 2021.
26. Yamga, G. M. M.; Ndjiongue, A. R.; Ouahada, K. Low complexity clipped Frequency Shift Keying (FSK) for Visible Light
Communications. Proceedings of IEEE 7th International Conference on Adaptive Science Technology (ICAST), Accra, Ghana,
2018.
Version May 22, 2021 submitted to Sensors 17 of 17
27. Bazi, Y.; Melgani, F. Toward an optimal SVM classification system for hyperspectral remote sensing images. IEEE Trans. Geosci.
Remote Sens. 2006, 44, 3374–3385.
28. Brereton, R. G.; Lloyd, G. R. Support vector machines for classification and regression. Analyst 2010, 135, 230–267.
29. Chavez-Burbano, P.; Rabadan, J.; Guerra, V.; Perez-Jimenez, R. Flickering-Free Distance-Independent Modulation Scheme for
OCC. Electronics 2021, 10, 1103.
30. Li, P.; Zhao, W. Image fire detection algorithms based on convolutional neural networks. Case Stud. Therm. Eng. 2020, 19, 100625.
31. Pedronette, D. C. G.; Torres, R. Shape retrieval using contour features and distance optimization. Proceedings of the International
Conference on Computer Vision Theory and Applications, Angers, France, 2010.
32. Vincent, L. Morphological transformations of binary images with arbitrary structuring elements. Signal Process. 1991, 22, 3–23.
33. Ashtputre, S.; Sharma, M.; Ramaiya, M. Image segmentation based on moments and color map. Proceedings of International
Conference on Communication Systems and Network Technologies, Gwalior, India, 2013.
34. Douglas, D. H.; Peucker, T. K. Algorithms for the reduction of the number of points required to represent a digitized line or its
caricature. Cartogr. Int. J. Geogr. Inf. Geovisualization 1973, 10, 112–122.