PROCEEDINGS OF SPIE

SPIEDigitalLibrary.org/conference-proceedings-of-spie

Adaptive RetinexNet and fusion

strategy for low-intensity image
enhancement

Xuehui Yin, Qiaoyu Chen, Ge Tu

Xuehui Yin, Qiaoyu Chen, Ge Tu, "Adaptive RetinexNet and fusion strategy
for low-intensity image enhancement," Proc. SPIE 12613, International
Conference on Computer Vision, Application, and Algorithm (CVAA 2022),
126130B (14 April 2023); doi: 10.1117/12.2673316

Event: International Conference on Computer Vision, Application, and

Algorithm (CVAA 2022), 2022, Chongqing, China

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use

 Adaptive RetinexNet and Fusion Strategy for Low-Intensity Image
Enhancement

Xuehui Yin1, Qiaoyu Chen1, Ge Tu1

1
School of Software Engineering, Chongqing University of Posts and Telecommunications,
Chongqing 400065, China
E-mail: ganseblumchenqy@163.com

ABSTRACT.

Aiming at the characteristics of low illumination image, such as low brightness, fuzzy detail information and obvious noise,
we propose an adaptive RetinexNet low illumination image enhancement method with the fusion strategy. Use the mutual
independence of each channel in the HSV color space model to enhance the brightness component; then use the correlation
coefficient to make the saturation component adaptively adjust with the brightness component; on the basis of UNet,
combine different areas of the illumination enhancement image to carry different levels of Noise, build a reflectivity noise
reduction model; a coarse enhanced image is generated through the above three steps; The camera response model is
obtained by combining the camera response function and the brightness conversion function, and the optimal exposure
rate is found to obtain the exposure rate map; finally, the overexposed image and the coarsely enhanced image are
combined with the original low-light image, and the image block decomposition is used for reconstruction and Fusion to
get the final low-light enhanced image. The experimental results show that the algorithm we adopt can ensure the texture
and details of the image while enhancing the contrast of the image.
Keywords- low-light image enhancement; Retinex theory; UNet; camera response model

1. INTRODUCTION
According to statistics, visual information accounts for 80% of the various information received by human beings. Vision
is the most important means of human perception, and images are the basis of vision, so image information is a very
important medium and way of information transmission. Whether it is in the field of home security, medical imaging,
remote sensing images, or other computing-as-vision fields, image enhancement plays a very important role and has a wide
range of applications [1]. Although digital cameras and other products are constantly being updated with the development
of technology, images taken under low-light conditions still exist, such as color distortion, underexposure, and poor
resolution, which are not conducive to observation. In order to meet the visual needs of users and provide high-quality
images for subsequent image information analysis and processing, we need to enhance low-light images.
With the continuous research of image enhancement algorithms, many algorithms have made good progress. The common
ones are low-light image enhancement based on histogram equalization, low-light image enhancement method based on
Retinex theory, and low-light image enhancement based on deep learning. In recent years, algorithms based on the retina-
cerebral cortex Retinex theory have gradually become a research hotspot. Common retinex calculations include: single-
scale Retinex algorithm [2]; multi-scale Retinex algorithm [3]; multi-scale Retinex with color restoration [4]; Lv et al. [5]
proposed a multi-branch low-light enhancement network MBLLEN by comprehensively considering enhancement
requirements such as image brightness, contrast, and noise reduction; Wang et al. [6] used a normalized flow to model the
complex distribution of positively exposed images. These algorithms can enhance low-light images, but from the research
status at home and abroad, the above-mentioned low-light enhancement methods have improved the image quality and
recognizability to a certain extent, but from the whole image enhancement process, they are not optimal methods, and there
are mainly the following problems:
(1) When low-illuminance image enhancement technology deals with non-uniform illumination images under complex
illumination conditions, the problems of over-enhancement and under-enhancement usually occur.
(2) There is noise in the dark area of the low-light image, and image enhancement technology will amplify the noise while
improving the contrast.

International Conference on Computer Vision, Application, and Algorithm (CVAA 2022)

edited by Hilal Imane, Proc. of SPIE Vol. 12613, 126130B · © 2023 SPIE
0277-786X · doi: 10.1117/12.2673316

Proc. of SPIE Vol. 12613 126130B-1

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 (3) The image enhancement technology cannot deal with the high-frequency detail information and noise well when
processing images with low signal-to-noise ratio.
This paper proposes an adaptive RetinexNet low-illuminance image enhancement method under the fusion strategy. For
images with complex structures such as hierarchical textures under non-uniform illumination conditions, considering the
organic combination of HSV space and convolutional neural network, and then through the camera Response Model [7],
explores low-light image enhancement methods with adaptive RetinexNet under the fusion strategy. The experimental
results show that the algorithm can effectively maintain the naturalness and detail features of the image while enhancing
the global contrast of the image.

2. RELATED WORK
2.1 RetinexNet illumination component enhancement algorithm of HSV color space
The main purpose of converting the original image from the RGB color space to the HSV space is to separate the color
component from the brightness component and saturation component, to ensure that the color component remains
unchanged, and to adjust the brightness component and saturation component. The mutual independence of each channel
is used to enhance the brightness component and change the problem of color distortion. Changes in image brightness will
cause changes in image contrast, resulting in color deviation in the enhanced image. The relative coefficient is used to
adaptively adjust the saturation of the image to maintain the contrast of the image.
s( x, y) = s( x, y) + t[v( x, y) − v( x, y)]   ( x, y) (1)

v ( x, y ) is the brightness of the pixel corresponding to the original image, v ( x, y ) is the brightness of the pixel after

enhancement, s ( x, y ) is the saturation of the pixel corresponding to the original image, s  ( x, y ) is the saturation of

the pixel after correction, and t is a proportional constant. t = 0.4,  ( x, y ) is the correlation coefficient between

v ( x, y ) and s ( x, y ) , n  n is the size of the neighborhood window W .

2.2 Using the UNet network to design a reflectance denoising model to study the removal of low-light image noise
under non-uniform illumination
According to prior knowledge, noise and color distortion mainly appear in weakly illuminated areas, that is, the distribution
of attenuation depends on the illumination distribution. Noise is complexly distributed in the reflection map, and the
distribution depends greatly on the illumination distribution, so it is used to assist reconstruction. The network diagram is
shown in Figure 1:

Figure 1. Albedo Restoration Network.

Its loss function is as follows:

LRR := Rˆ − Rh 2
2 ( )
−SSIM Rˆ , Rh +  Rˆ − Rh 2
2 (2)

where R̂ represents the restored reflection map. SSIM (..) is a structural similarity measure. The distance L2
between the restoration result R̂ and the target result Rh , and the last item makes the texture detail information
consistent.

Proc. of SPIE Vol. 12613 126130B-2

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 2.3 Using the camera response model to study the fusion method of multiple images
Nowadays, many image enhancement technologies will cause the brightness and color of the final enhanced result to be
distorted. At this time, we can consider the corresponding model of the camera. We can obtain a virtual exposure image
through this camera corresponding model, and use it as supplementary information of the original image to obtain a clearer
result. The camera response model is modeled as:
P = f (E) (3)

P is the pixel value, f is the nonlinear function CRF, E is the image irradiance.

We take two images in the same scene, namely P0 and P1 , and the mapping relationship between these two images is
as follows:

P1 = g ( P0 , k ) =  P0 f (kE ) =  f (E) (4)

k is the exposure rate, g is the brightness transfer function (BTF),  and  are the parameters related to the
exposure rate in the BTF model.

The solution of f (E) has the following two forms:

eb (1− E ) ,   1
a

f (E) =  c (5)
 E ,  = 1

The images captured by most cameras are non-linear, so here we only consider the case when   1 . Since the camera
response curve is generally fixed for a specific camera, the parameters of 𝑎 and 𝑏 can be obtained by least squares fitting
the curve. Based on this model, we can get the following formula:

g ( P, k ) = e
(
b 1− k a ) P( k )
a

(6)

Information entropy is generally used to represent the amount of information. The more average information in an image,
the greater the information entropy, and the better the quality of the image. We generally use the following formula to
express information entropy:
N
H ( B) = − pi  log 2 pi (7)
i =1

We need to fuse the initial image, the coarsely enhanced image, and the generated virtual overexposed image through the
method of image block decomposition:
P = cs +l (8)
where c is the block signal strength, s is the block structure, and l is the average strength of the block.
The brightness component is generally used as the lighting estimation of the scene, and the maximum brightness value is
used as a brightness map to estimate, so the lighting estimation is as follows:

cˆ = max ck (9)
1 k  K

We also need to consider the structure-aware smoothness of the image, which can be obtained by:

 c s
K

s= k =1 k
(10)
 s
K
k =1 k

Proc. of SPIE Vol. 12613 126130B-3

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 For the structure-aware correction of the initial illumination map, the key is the design of the weights, and we use a
weighted linear fusion mechanism:

ˆl =  k =1 ( k k ) k
L 0,l l
K

(11)
 k =1L ( k0 , lk )
K

Finally, we restore the stacked ŝ to the RGB channel, and then use the previously obtained ĉ and lˆ to reconstruct
according to the following formula.

P = cˆ  sˆ + lˆ (12)
The overall algorithm flow chart is shown in Figure 2:

Figure 2. Model Structure.

3. EXPERIMENTAL RESULTS AND ANALYSIS

The deep learning framework used in this experiment is TensorFlow 1.13 GPU, and the NumPy computing library and
PIL image processing library, the experimental software development environment is Pycharm2019 and python3.7.
3.1 Data set and parameter configuration
Dataset construction: The LOL dataset contains 500 low-light and normal-light image pairs of scenes such as campuses,
houses, and streets, among which 485 pairs are training data and 15 pairs are detection data. Select 1000 original images
from the RAISE dataset to synthesize low-light images, and adjust the illumination components to obey the uniform
distribution between (0, 1).
3.2 Discussion of experimental results
To verify the effectiveness of our algorithms, five mainstream enhancement methods were taken for comparison. These
five algorithms are LIME [8] algorithm, GLAD [9] algorithm, KinD [10] algorithm, SGM-Net [11] algorithm and
RetinexNet [12] method:

Proc. of SPIE Vol. 12613 126130B-4

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 Table 1. Comparison of LOL real data and other methods.
RetinexNet LIME GLAD SGM-Net KinD Ours
PSNR 16.7740 16.9662 20.11 20.06 20.7261 20.8105
SSIM 0.5994 0.6422 0.80 0.8158 0.8103 0.8217

Table 1 shows the comparison results with the other methods on real scenes from the LOL dataset. It can be seen from the
data that the method has a slight advantage over the other methods in terms of PANR and SSIM metrics, which indicates
that our model has better results for low illumination images. The visualisation results are given in the figure below and
can show the effectiveness of our method more visually.
Table 2. Visual comparison results.

Input GLAD LIME RetinexNet

SGM-Net KinD ours

For the image enhancement results in Table 2, we can see that these algorithms have obvious brightness improvements.
But for GLAD, LIME and RetinexNet, due to the excessive brightness enhancement, serious noise appeared, RetinexNet
also showed oil painting quality, the effect of SGM-Net’s brightness enhancement is not obvious but the details are
maintained well, KinD The result is too smooth even though the noise is removed.Compared with the original image, our
proposed model has a good performance overall, which can achieve better performance in brightness enhancement and
noise removal.

4. CONCLUSION
For low-illumination image enhancement, based on Retinex theory, after analysing the defects of existing image
enhancement, a low-illumination image enhancement method based on fusion strategy adaptive RetinexNet is designed.
On the LOL data set, this paper has done a comparative experiment with other classic methods, and the data shows that
the method of this paper is better than other methods on PSNR and SSIM indicators.

ACKNOWLEDGMENTS

This work was supported in part by the National Natural Science Foundation of China under Grant 61701060, in part by
the Scientific and Technological Research Program of Chongqing Municipal Education Commission under Grant
KJQN202000619.

Proc. of SPIE Vol. 12613 126130B-5

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
 REFERENCES

[1] Jebadass, J. (2022) "Low light enhancement algorithm for color images using intuitionistic fuzzy sets with
histogram equalization." Multimedia Tools and Applications. 81(6): 8093-8106.
[2] Brainard, H. and Brian, A. (1986) “Analysis of the retinex theory of color vision.” Journal of the Optical Society
of America A. 3(10): 1651-1661.
[3] Daniel, J. and Glenn, A. (1997) "A multiscale retinex for bridging the gap between color images and the human
observation of scenes." IEEE Transactions on Image Processing. 6(7): 965-976.
[4] Petro, A. Sbert, C. and Morel, J. (2014) "Multiscale retinex." Image Processing On Line, 71-88.
[5] Feifan, L. Feng, L. (2018) "MBLLEN: Low-light image/video enhancement using cnns" In: Machine Vision
Conference. British. 220(1): 4.
[6] Wang, Y. (2022) "Low-light image enhancement with normalizing flow." Proceedings of the AAAI Conference
on Artificial Intelligence. 36(3): 2604-2612.
[7] Ren, Y. Ying, Z. and Li, T. (2018) "LECARM: Low-light image enhancement using the camera response model."
IEEE Transactions on Circuits and Systems for Video Technology. 29(4): 968-81.
[8] Guo, X. Yu, L. and Haibin, L. (2016) "LIME: Low-light image enhancement via illumination map estimation."
IEEE Transactions on Image Processing. 26(2): 982-993.
[9] Mittal, A. Rajiv, S. and Alan, C. (2012) "Making a “completely blind” image quality analyzer." IEEE Signal
Processing Letters. 20(3): 209-212.
[10] Zhang, Y. Zhang, J. and Gou, X. (2019) "Kindling the darkness: A practical low-light image enhancer."
In:Proceedings of the 27th ACM international conference on multimedia. France. 1632-1640.
[11] Yang, W. Wang, W. Huang, H. (2021) "Sparse gradient regularized deep retinex network for robust low-light
image enhancement. " IEEE Transactions on Image Processing. 30: 2072-2086.
[12] Wei, C. Wang, W. Yang, W. (2018) "Deep retinex decomposition for low-light enhancement." Preprint
arXiv/1808.04560.

Proc. of SPIE Vol. 12613 126130B-6

Downloaded From: https://www.spiedigitallibrary.org/conference-proceedings-of-spie on 14 May 2023
Terms of Use: https://www.spiedigitallibrary.org/terms-of-use