Subscribe to DeepL Pro to translate larger documents.

Visit www.DeepL.com/pro for more information.

Chapter II. Automatic localization of cracks in marble using tools
image processing

Chapter II. AUTOMATIC LOCALIZATION OF A CRACK IN MARBLE USING

IMAGE PROCESSING TOOLS

II.1. GENERAL INTRODUCTION

Marble is a heterogeneous, anisotropic material with random textures due to the granular

composition of the rock and the inclusion of metal oxides. These patterns, known as veins,

vary both on the surface and at depth. Marble's heterogeneity ex- plains its brittle behavior,

caused by the presence of microcracks naturally in the material and the coalescence of these

cracks over time under the influence of external environmental con- tracts or stresses applied

to the material. These properties of marble can lead to plate fractures during machining

and/or material pull-outs at the entry and exit profiles of machined surfaces, thus limiting the

automation of the machining process.

Given this problem, the aim of this chapter is to automatically detect the location of surface

cracks in marble slabs using a simple tool compatible with the constraints of the marble

industry. This objective complements the results of the first chapter, which classify whether

marble is cracked or uncracked. Thus, after applying the classification (Chapter I) and finding

the crack location (Chapter II), an inspected marble slab will be identified as cracked or non-

cracked, and the crack location will be known. Crack location can then be used to optimize

the marble machining process and decide on the future use of the marble slab. Indeed,

integrating the location of the defect into the marble machining process would reduce the risk

of breakage of the machined slabs, the number of broken slabs, and can be used to adjust

the cutting parameters in the vicinity of the defect.

52
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Defect detection and pattern recognition on various material textures are widely studied in

scientific and industrial research. Detecting the position, geometry and depth of flaws informs

the user about the future use of the material, as these characteristics are linked to the degree

of material damage. Automatic crack detection is of interest in many industrial fields, such as

civil engineering (health monitoring of large infrastructures such as roads, bridges or

buildings)49,50 , aeronautics, textiles or the machining and manufacture of any other material

used in everyday life (steel, wood, stone, glass, etc.).

There are many automatic defect detection systems in the industry, based on a variety of

methods ranging from ultrasonic propagation and vibration measurement to image

processing and many others. To name but a few, Ground Penetrating Radar (GPR), or

georadar, is used to detect foreign bodies in the ground or in concrete slabs. The RPS

principle is based on the measurement of different electromagnetic wave reflections, each of

which corresponds to the presence of objects in the material inspected20 . The performance

of this tool depends on several parameters, such as relative dielectric permittivity, electrical

conductivity, and also the thickness of the material inspected. Similarly, in order to detect

cracks in difficult-to-access areas such as long-span bridges, it is possible to use drones

equipped with a camera51,52 .

For marble, it is interesting to locate the crack before machining and in this sense (Kardan et

al.; Sipko et al.) have used image processing techniques to detect impurities on the marble

surface and deduce the extent of damage to the plate53,54 . However, these studies are

limited to smooth-textured types of marble. What distinguishes marble from other materials is

its texture, which varies from one marble to another. It's a collection of randomly dis- tributed

veins of varying color and direction. Thus, the detection of cracks

53
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

on marble is not the same as on other materials (metals, tiles, medical images, concrete,

etc.) which share either a plain background or regularly spaced patterns. Consequently, the

crack for other materials will be different from the background and will not ap- pear to the

default texture patterns, so that a classification algorithm can easily extract the crack as a

foreign texture element55–58 . In marble, however, it is essential to distinguish a crack from a

vein, and this is where the contribution of this work lies. The aim of this chapter is therefore to

develop a texture-independent approach to crack detection. In order to meet the objectives of

this chapter, we first review the various image processing techniques available in the

literature (Part II.2), followed by an application to the detection of surface defects in marble

(Part II.3).

To assess the performance of the various crack detection methods studied in this chapter,

we propose to consider a set of three types of marble texture: a smooth texture, a

"moderately textured" texture and a "heavily textured" texture. In order to propose a definition

of these textures, we consider the grayscale histogram of a marble image. Thus, the marble

will be considered to have a smooth texture when the histogram has a single prominent

peak. A moderately loaded texture will have a histogram with two prominent peaks, one

corresponding to the background and the other to the patterns (veins) of the marble. The

heavily loaded texture will have a histogram with two main peaks and an additional lobe

spread out, reflecting a greater variety of gray levels. An example of each of these textures is

shown in (Ta- ble 3).

The performance of the crack detection and localization method will thus be eva- luated by its

ability to detect a crack on the three marble textures thus defined.

54
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Table 3: Illustrative examples of the three marble textures considered in this study.

Plain texture: the histogram of this texture shows a single peak.

The crack is of the order of 5 pixels

Medium-loaded texture: the histogram of this texture shows two main peaks which
coresspond respectively to background and texture.

The crack is of the order of 5 pixels

Heavily loaded texture: the histogram of this texture shows two main peaks, separated by
of varying amplitude and width. The histogram is also more spread out, reflecting a greater
wide variety of gray levels in the image.

55
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

The crack is of the order of 10 pixels

In this work, these three textures are used to apply image processing tools for crack location

detection.

II.2. STATE OF THE ART

Cracks are one of the most common types of imperfection found in materials, and have a

significant effect on the structural strength of the material59 . Mohan et al. have proposed

a fast and reliable method of surface defect analysis based on image processing

techniques, which can replace traditional, slower human inspection procedures. These

authors add that automatic defect detection is a fast-growing science, widely used in

various fields of scientific research and practice60 . The main difficulties encountered in

crack detection based on image processing techniques are caused by the random shape

and irregular size of cracks, as well as by di- versal disturbances linked to irregular

lighting conditions, shadows or the presence of stains in cracks.

There are two main families of approaches to defect detection using image processing

tools: those based on machine learning techniques, and those based on "classic" image

processing methods. These

Two approaches are described in sections II.2.1 and II.2.2.

56
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.2.1. Defect detection using machine learning algorithms :

Munawar et al. have defined four basic steps for implementing a crack detection method

based on machine learning. These include image acquisition, image pre-processing, labeling

of crack-related pixels and training of the learning model. These steps are illustrated in

Figure 22. This approach requires the geometry of the crack to be indicated to the algorithm,

which in turn requires the intervention of a human operator for the labelling stage61 . This is

what is known as supervised learning, where the characteristics of the entity (defect) being

sought are known from the outset.

Application of
Image Image pre- Labeling the machine
collection processing defective pixels learning model

Figure 22: General machine-learning crack detection methods

61

Machine learning (ML) and deep learning (DL) techniques have been widely used to

detect cracks in images of damaged materials. Among these machine learning

algorithms Munawar et al. listed the use of a range of algorithms such as GoogLe- Net,

Convolutional Neural Networks (CNN), Feature Pyramid Network (FPN), Random

Structured Forests, Support Vector Machine (SVM), MorphLink-C, Artificial Neural

Network (ANN), Deep Convolutional Encoder-Decoder or the combination of a CNN and

a Naive Bayes Con- volutional Neural Networks (NB-CNN) approach. Xin and Wang

added that traditional machine learning methods (such as multi-layer perceptrons,

support vector ma- chines, etc.) mainly use shallow structures.

to process a limited number of data samples, while the neural network

57
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

convolutional (CNN) has been widely used in image processing because it is able to

handle image classification and recognition problems with precision62 .

In general, the output of these algorithms are two classes of regions of the tested

material, with or without cracks. However, these algorithms are both computationally

time-consuming and require database labeling. Crack detection methods based on

learning techniques will not be con- sidered in this marble flaw detection study due to the

lack of a sufficiently large labelled database.

II.2.2. Defect detection using conventional image processing approaches :

Several crack detection applications based on image processing techniques can be found in

the literature61,63 . In general, these applications involve four steps: acquisition, image pre-

processing and processing, and extraction of crack attributes. Figure 23 shows the sequence

of these steps:

Figure 23: General image processing methods for crack detection 61,63

58
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Attribute extraction can then be used to characterize the crack. This makes it possible to

measure crack properties such as length, width, depth or crack density in the study zone.

Nguyen et al. have proposed crack characterization using crack properties extracted by

image processing and a classification algorithm63 .

Photographic systems, such as the multifunction road inspection device, can be used to

capture images, enabling digital images of the pavement surface to be captured with very

high resolution, whatever the lighting conditions of the pavement64 or using other camera

systems, as in the case of (Kao et al.) who used a camera (SONY DSC-RX0) to inspect

cracks on bridges51 . Similarly, Li et al. used an XIME MQ013MG-E2 camera with 1280×256

image resolution to classify wood defects65 . In this work, we will consider images of marble

slabs taken with a 13 MP (4160 x 3120 pixels) resolution cell phone camera.

The image acquisition system and pre-processing are highly dependent on the intended

application and the material conditions under which the image was taken. In this sense, the

filter used in this study to eliminate noise in the image is the Kuwahara filter66 . The

Kuwahara filtering process is essentially the division of a pixel grid into four overlapping sub-

grids, calculating a mean and variance for each. The output value is defined as the mean of

the sub-grid with the least variation, and this value will be assigned to the central pixel of

each region analyzed by the algorithm, and can be used in various ways in relation to the

division of its grids 67. According to Papari et al. let's take a grayscale image I (x, y) and a

square of length 2*a centered around a point

(x, y), which is partitioned into four identical square subregions (Figure 24) Q1 , Q2 , Q3 , Q4

determined as follows68 :

59
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Equation 9
⎛Q1(x, y)=[x, x+a] ×[x, y+a]
Q2 (x, y)=[x-a, +a] y, y+a]
⎨ Q3(x, y)=[x-a, x]×[y-a, y]
{ Q4 (x, y)=[x, x+a] ×[y-a, y]

Figure 24: Kuwahara filtering: (a) Q subregions: i = 1...4, on which local averages and
standard deviations are calculated. (b) The sub-region with the lowest standard
deviation (delimited by a thick line) determines the output of filter 68.

Let mi(𝑥, 𝑦)and si(𝑥, 𝑦) be the local mean and local standard deviation, respectively,

calculated over each subregion Qi(𝑥, 𝑦), 𝑖 = 1 ... 4. For a given point (x, y), the output of the

Kuwa- hara filter Ф (x, y) is defined as the local mean value mi(𝑥, 𝑦) corresponding to the ième

sub-region that provides the minimum value of the local standard deviation si(𝑥, 𝑦)69 . This

can be formulated as follows:

Equation 10
y)=(x, y)= ∑ m (x, y)× f (x, y)
Ф orf (x, 1, if si(x,y)=mink{sk(x,y)}
i i i {
0, otherwise
i

Next, we focus on the algorithmic part of the process, which extracts and characterizes a

crack. It is therefore assumed that the images considered to characterize the crack(s) in a

marble slab are of satisfactory quality. The following sections describe the various tools used

in the literature for image processing and crack attribute extraction.

60
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.2.2.1. Image processing tools

Cracks are difficult to identify from 2D images due to their random geometry70 . Numerous

image processing approaches have been tested in the literature, and this section presents

some of them classified into two categories: so-called

"statistical" and "spectral" approaches. Statistical methods include first-order methods when

the method depends on each pixel separately (threshold, edge detection filters, sliding

statistical moments on a thumbnail) and second-order methods when the method is based on

the relationship between two pixels characterized by a relative distance and direction (Co-

Occurrence Matrix (COM) descriptors). A higher-order statistical method is also used when

the detection technique seeks the relationship between more than two pixels. This is the

case, for example, with the local binary patterns (MBL) method, which determines whether a

pattern is found in different directions in the image71 . Secondly, the spectral approach, where

the detection result is based on texture regions (Fourier transform, Gabor filter and wavelet

transform).

II.2.2.1.1. Statistical approaches :

II.2.2.1.1.1. First-order statistical approaches :

Nihal et al. explained that texture analysis by first-order attributes is performed at the level of

individual pixels in the image region, and that these attributes are calculated from the

histogram of thumbnails72 :

II.2.2.1.1.1.1. Sliding moments

The first-order sliding statistical moments are :

✓ The average, noted Moy in Table 4 (gives the average gray level in a window (or
thumbnail) of the image).

61
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

✓ The variance, denoted Var in Table 4 (provides information on the dispersion and
variability of the image's gray level values).

✓ Skewness, noted as Ske in Table 4 (measures the deviation of the grayscale

distribution from a symmetrical distribution. For a deviation towards high values,

Skewness is positive).

✓ Kurtosis, noted Kur in Table 4 (characterizes the peak of the histogram so that the
lower the kurtosis, the rounder the peak of the histogram73 .

Table 4 describes the mathematical formulas for these attributes, where 𝑥𝑘 is the level value.

of pixel k and n corresponds to the total number of pixels in an image window73 . By

calculating these quantities over a sliding window, it is possible to obtain a map of the

image's local characteristics.

Table 4: Mathematical formulas for statistical moments.

Average (Equation 11) Variance (Equation 12)

𝒏 𝒏
𝟏 𝟏 𝟐
𝑴𝒐𝒚(𝒙) = ∑ 𝒙𝒌 𝑽𝒂𝒓(𝒙) = ∑(𝒙𝒊 - 𝑬(𝒙))
𝒏 𝒏 -𝟏
𝒌=𝟏 𝒊=𝟏

Skewness (Equation 13) Kurtosis (Equation 14)

𝒙 - 𝑬(𝒙) 𝒙 - 𝑬(𝒙)
𝑺𝒌𝒆(𝒙) = 𝑬[( )𝟑 ] 𝑲𝒖𝒓(𝒙) = 𝑬[( )𝟒 ]
√𝑽𝒂𝒓(𝒙) √𝑽𝒂𝒓(𝒙)

II.2.2.1.1.1.1. Grayscale thresholding

Threshold methods are the most conventional. They are simple to implement and do not

require a great deal of computing time. The image is a function of grayscale intensity. The

thresholding process divides the image pixels into two classes, C1 with

intensity values [1, ..., T] and C2 with intensity values [T +1, ..., L] where T is the level

62
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

threshold. The segmentation results in a binary image where all pixels assigned to class C1

(background) have a value of 0 (black) and all pixels assigned to class C2 (foreground) have

a value of 1 (white)74 .

Two types of thresholding can be defined: global thresholding and local thresholding. Global

thresholding considers a single threshold used globally. Kefali et al. have recommended the

use of this type of thresholding when the image has a relatively uniform background. Local

thresholding considers a threshold that depends on certain characteristics of the image

(contrast, for example). This makes it possible to obtain a different threshold for certain areas

of the image75 .

II.2.2.1.1.1.2. Edge detection filters

Numerous methods have been proposed for detecting contours in images, such as Robert's

gradient, Sobel's operators, Prewitt's operators, the Laplacian operator, etc.76 . A contour

detection filter consists in calculating the differences between horizontally and vertically

neighboring pixels, with each extreme value corresponding to a contour point. According to

Sidhom et al. the most widely used edge detection filters in the literature are based on a

gradient or Laplacian calculation77 . The first gradient method detects contours by looking for

local extrema of the gradient of the image intensity function, such as the Roberts, Prewitt and

Sobel operators, and the second Laplacian method looks for zero crossings in the second

derivative of the image to find contours, such as the Laplacian operator76,78 . To this we can

also add the Canny filter, invented by John Canny in 198679 , which represents a Sobel filter

preceded by Gaussian smoothing and followed by thresholding80 . Table 5 lists the

mathematical formulas for the three filters Sobel, Laplacian and Canny.

63
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Table 5: Mathematical formulas for Sobel, Laplacian and Canny filters

Let be the Sobel kernels along x and y :

Equation 15

-1 0 1
𝑥 𝑆= [-2 0 2] 𝑆𝑦
-1 0 1
1 2 1
The Sobel filter (Equation =[0 0 0]
16) for an image f(x,y) 81
-1 -2 -1
Let be the Sobel ∇𝑓 function:

Equation 16
𝜕𝑓 𝜕𝑓
∇𝑓 = [ ] = [𝑓 × 𝑆𝑥 𝑓 × 𝑆𝑦]
𝜕𝑥 𝜕𝑦

The Laplacian filter Let be the Gaussian function 2D :

(Equation 17) for an 𝑥2+𝑦2)⁄2𝜎2)
image f(x,y) 𝐺(𝑥, 𝑦) = 𝑒-((
=2 𝛛 𝛛 2
82 Let the Laplacian operator be: ∇𝛛𝑥2
2
+ 𝛛𝑦2

2
𝛛 𝐺(𝑥,𝑦)
The Laplacian filter is defined by =∇2 𝐺 = +
𝛛𝑥2
𝛛2𝐺(𝑥,𝑦)

𝛛𝑦2

Equation 17:
2+𝑦2)⁄2𝜎2]
2
∇𝐺 = [(𝑥2 + 𝑦2 - 2𝜎2 )⁄𝜎4 ] × 𝑒[−(𝑥

Where 𝜎 is the standard deviation.

The Canny filter (Equation Let the smoothed image be: 𝑓𝑙 (𝑥, 𝑦)
18) for an image f(x,y)
𝑙 𝑓(𝑥, 𝑦) = 𝐺(𝑥, 𝑦) × 𝑓(𝑥, 𝑦)
82
Let be the Canny function:

Equation 18

𝑙 𝜕𝑓𝜕𝑓𝑙
∇𝑓 = [ ] = [𝑓𝑙 × 𝑆𝑥 𝑓𝑙 × 𝑆𝑦]
𝜕𝑥 𝜕𝑦

64
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

where

-1 0 1
𝑥 𝑆= [-2 0 2] 𝑆𝑦
-1 0 1
1 2 1
=[0 0 0]
-1 -2 -1

The Canny filter is designed to be optimal as it is known to ensure good local edge

detection and have a low false positive rate 76,82,83.

II.2.2.1.1.2. Second-order statistical approaches

Second-order statistical attributes are Co-Occurrence Matrix (COM) attributes and were

invented by Professor Robert Haralick in 197384 . The OCM is obtained by calculating the

number of pairs of gray levels separated by a distance d in a given direction θ85 . Haralick et

al. proposed this formula for calculating the co-occurrence matrix84 (see equation 15):

Equation 19:
𝑁 𝑀
, 𝑠𝑖 𝐼(𝑛, 𝑚) = 𝑎 𝑒𝑡 𝐼(𝑛 + 𝑑𝑥, 𝑚 + 𝑑𝑦) = 𝑏
𝐺𝐿𝐶𝑀𝑑𝑥,𝑑𝑦 (𝑎, 𝑏) = ∑ ∑ {1
0, 𝑠𝑖𝑛𝑜𝑛
𝑛=1 𝑚=1

where a and b are the gray levels of the first and second pixel defining the displacement

vector (dx, dy) respectively. n and m correspond to the pixel coordinates in the image and (N

x M) to the image resolution. Figure 25 shows an example of how to calculate the co-

occurrence matrix of a grayscale image:

65
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 25: example of co-occurrence matrix calculation: the number of occurrences of

the pair (a=2, b=3) in blue is equal to 2 and the number of occurrences of the pair (a=3,
b=1) in red is equal to 1 86.
Haralick proposed a set of 14 statistical descriptors to describe tex- tural information from

OLS. The angular directions commonly used for θ are 0, 45, 90, and 135 degrees. The 5

Haralick descriptors most frequently used in the literature are energy, entropy, contrast,

correlation and homogeneity85 . Their definitions

are given below where 𝑃(𝑖, 𝑗) is the OLS element (i, j) and N is the number

grayscale56,86,87 :

✓ Energy: This parameter measures texture uniformity. It reaches high values when the
gray level distribution is constant or periodic:

Equation 20

N N

DH1 = ∑ ∑ P(i, j)2

i=1 j=1

✓ Contrast: Contrast measures local variations. If these variations are significant, the
contrast will be high:

Equation 21

66
 Chapter II. Automatic localization of a crack in marble with the help of the
image processing
𝑁 𝑁
𝑫𝑯𝟐 = ∑ ∑ 𝑃(𝑖, 𝑗) (𝑖 - 𝑗)2
𝑖=1 𝑗=1

✓ Entropy: Entropy measures image complexity. When the values of the co-occurrence
ma- trice are almost all equal, entropy is high. It's an indi-

cator, which reaches high values when the texture is completely random (with no

apparent structure). It is strongly correlated (inversely) with energy: Equation 22

𝑁 𝑁

𝑫𝑯𝟑 = - ∑ (𝑖, 𝑗) log 𝑃(𝑖, 𝑗)

𝑖=1 𝑗=1

✓ Correlation: This parameter determines whether certain columns of the matrix are
equal, i.e. whether there are linear dependencies in the image. Indeed,

the more uniformly the values are distributed in the co-occurrence matrix, the greater

the correlation. Correlation is not correlated with either energy or en- tropy:

Equation 23

𝑁 𝑁
(𝑖. 𝑗)𝑃(𝑖, 𝑗) - 𝜇𝑥𝜇𝑦
𝑫𝑯𝟒 = ∑
𝑥 𝜎𝜎𝑦
∑
𝑖=1 𝑗=1

Where 𝜇𝑥 , 𝜇𝑦 , 𝜎𝑥 and 𝜎𝑦 are respectively the means and deviations of 𝑃𝑥 and 𝑃𝑦 .

✓ Homogeneity or inverse differential moment: This parameter behaves differently from

contrast. In fact, the more homogeneous the texture and the more

the higher the inverse differential moment:

Equation 24

67
 Chapter II. Automatic localization of a crack in marble with the help of the
image processing
𝑁 𝑁
𝑃(𝑖, 𝑗)
𝑫𝑯𝟓 = ∑ ∑
𝟏 + (𝑖 - 𝑗)2
𝑖=1 𝑗=1

In this chapter, we will use these five descriptors. Indeed, these descriptors have been used

to detect textile defects57,58 , cracks in steel welds88 and also to detect defects on seabeds42 .

These descriptors will be applied to the three types of marble textures and the detection

result for each descriptor will be analyzed. The correlation relationship between the detection

results of these descriptors will also be discussed.

II.2.2.1.1.3. Higher-order statistical approaches :

In the same context of statistical approaches, the approach below is of an order of gran- deur

greater than two, meaning that this tool studies the relationship between more than two

pixels. This approach is a type of spatial modeling, i.e. it looks for spatial patterns in the

image, often in the form of a visual pattern or some form of spatial organization. According to

Porebski et al. the most widely used higher-order patterns in texture analysis and for defect

detection purposes are local binary patterns (MLBs)89 .

Ojala et al. introduced the MLB operator as a means of summarizing the local grayscale

structure of an image. The operator takes the local neighborhood around each pixel, restricts

neighboring pixels to the value of the central pixel and uses the resulting binary-valued image

patch as the local image descriptor90 . (Rani et al.; Tan et al.) added that MLB is a local

texture des- cripter with high discrimination and low computational complexity at a fine scale

that captures texture details91,92 . Atikah et al. explain that the main idea of the MLB approach

is to locally threshold the luminance of a pixel's surroundings to produce a binary pattern. If

the neighborhood (P) consists of 8 pixels, a total of 2^8 (256) different labels can be obtained

depending on the gray level of the central pixel and those of the neighboring pixels. The

central pixel of the image, a real number, which is the

68
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

value of the MLB pixel, is calculated by comparing its intensity value with those of its envi-

ronment. Figure 26 illustrates the choice of neighboring pixels considered by varying the

radius R or the number of points P93 .

Figure 26: Different configurations of the MLB calculation as a function of the number of points P and radius R 93

The pattern number is therefore calculated by comparing its value with those of its neighbors

according to the two parameters P and R according to the following equation (Equation 25)

and the thresholding of these calculations is done according to Equation 26)71 :

Equation 25
𝑷-𝟏

𝑃,𝑅 𝐿𝐵𝑃= ∑(𝒈𝒑 - 𝒈𝒄 )𝟐𝒑

𝒑=𝟎

Equation 26
𝟏, 𝒔𝒊 𝒙 ≥ 𝟎
()
𝒔 𝒙 = { 𝟎, 𝒔𝒊 𝒙 < 𝟎

Where 𝑔𝑐 is the gray value of the central pixel and 𝑔𝑝 is the value of its neighbors.

An example of MLB calculation for R=1 and P=8 is shown in Figure 27:

69
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 27: Illustration of the MLB basic operator 94

This technique has been used in the literature to detect defects on simple wood textures65 or

pavement95 . It will be used here to detect cracking on the three marble textures considered.

II.2.2.1.2. Spectral approach :

The spectral approach focuses on the frequencies of interactions between different regions

of an image. Methods used in this approach include the Fourier transform, Gabor filters and

the wavelet transform.

II.2.2.1.2.1. Fourier transform :

The Fourier transform TF was invented by Joseph FOURIER in 180796 , and consists in

converting the image from its spatial domain to its frequency domain. The Fourier transform

of an image of size (N x M) is defined by :

Equation 2785 :

𝐍 𝐌 𝟐𝛑𝐣(𝛍𝒊
- + 𝐯𝒋)
𝑭(𝝁,𝒗) = ∑ ∑ 𝑰(𝒊 , 𝒋) 𝐍 𝐌

𝐞
𝒊=𝟏 𝒋=𝟏

Where I (i, j) is the value of the image of size N x M in position (i, j). µ and 𝒗 are the
spatial frequencies of the image in the x and y directions.

70
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.2.2.1.2.2. Gabor filter:

The Gabor filter (FG) method is named after the English physicist Dennis Gabor (1900-

1979). These filters, which operate in a similar way to human visual processing, have the

advantage of being configurable in terms of frequency and orientation. Gabor filters have

been widely used in many image processing applications, such as optical character

recognition, iris recognition97 and fingerprint recognition98 . Mathematically, a Gabor filter is a

convolution of the Fourier transform of a harmonic function and the Fourier transform of a

Gaussian function99 . In the case of image processing (two dimensions), this means that the

Gabor function can be written as100 :

Equation 28:
(𝑖2+𝑗2)
)
𝐺1 (𝑖, 𝑗) = 𝑒𝑥𝑝 (2𝜎 ∗ cos(2𝜋𝑓(𝑖 ∗ 𝑐𝑜𝑠(𝜃) + 𝑗 ∗ sin(𝜃))
(𝑖2+𝑗2)
( 2𝜎 )
𝐺2 (𝑖, 𝑗) = 𝑒𝑥𝑝 ∗ sin(2𝜋𝑓(𝑖 ∗ 𝑐𝑜𝑠(𝜃) + 𝑗 ∗ sin(𝜃))

Where i and j are the position in the image, sigma is the variance of the Gaussian

envelope, theta is the angle in which the filter will be oriented and f defines the

frequency. The resulting image is the sum of 𝐺1 (𝑖, 𝑗) and 𝐺2 (𝑖, 𝑗).

II.2.2.1.2.3. Wavelet transform :

The wavelet transform (WT) was first proposed by Alfréd Haar in 1909. Mallat considers the

wavelet transform to be an effective characterization of the textural content of an image101 .

Porebski et al. explained that the wavelet transform is based on a multi-scale analysis of the

image in the sense that analysis windows of different sizes are used and numerous statistical

or co-occurrence attributes (Wavelet Statistical Features and Wavelet Co-occurrence

Features) can be extracted from the image.

71
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

color wavelet transforms of an image89 . Regniers et al. specified that the wavelet transform

decomposes the image using a series of functions created by translation and scaling of an

original function Ψ , called the mother wavelet This function is defined by85 :

Equation 29

𝟏 𝒊 -𝝁
𝝁,𝒔 𝜳(i) = 𝜳 )
( 𝒔
√𝒔

where s and µ define the scale factor and translation factor, respectively, of the parent

wavelet85 . The wavelet decomposition of an image f(𝑖, 𝑗) is obtained by the product of

convolution between image f(𝑖, 𝑗) and wavelet functions 𝛹𝜇,𝑠 85: Equation

30

+∞
𝟏 𝒊 -𝝁 𝒋 -𝝁
𝒄𝒊,𝒋 = ∫ 𝒇(𝒊, 𝒋) 𝜳( , ) 𝒅𝒊𝒅𝒋
√𝒔 𝒔 𝒔
-∞

This decomposition therefore calculates the wavelet coefficients 𝑐𝑖,𝑗 and also reconstructs the

image f (i, j) from the coefficients 𝑐𝑖,𝑗.

II.2.2.1.2.4. Review of the spectral approach

These three spectral tools have been used in the literature to detect defects in images of

various textures. Ralló et al., for example, use the Fourier transform to extract defects from

textile images102 . The Gabor filter has been used to detect defects in pavement images70

and steel sheet images103,104 . The wavelet transform was used by Ahmed et al. t o detect a

crack in images derived from ultrasonic measurements105 . These textures are relatively

simple. In this chapter, we will use these three tools to detect a crack in three types of marble

texture: smooth, moderately loaded and heavily loaded.

72
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.2.2.2. Attribute extraction

In general, after applying these crack detection tools, further operations are applied to the

detected contours to characterize the defect profile. For this step, classification algorithms

can be used to extract the defect from the rest of the40,54 texture. Alternatively, morphological

operations can be applied to the contours to bring out the characteristics of the defect

(length, area, perimeter, etc.).106

II.2.2.2.1. Classification of detected attributes

Classification algorithms play a major role in image processing and according to (Manjula et

al., 2017), these tools are used to classify objects detected in the image into different classes

based on the characteristics of the anomaly sought107 . We focus on crack detection in this

study.

There are several clustering algorithms, Wang et al. mention K-means clustering (K-means),

fuzzy C-means clustering (FCM), maximum entropy clustering (MEC), hierarchical clustering

(HC) as examples of algorithms used in the literature to distinguish a defect in an image59 .

These algorithms are suitable for different applications and the choice of classifier is based

on the algorithm's performance and the application domain. In this chapter, k-means

classification is used to extract the crack, as it is a classification algorithm that is highly

suitable for image processing and gives good results40,42,43,54,108 . The history and principle of

this algorithm has already been described in the first Chapter (section I.3.3.6.1) and we will

reuse this tool for the classification of detected attributes.

73
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.2.2.3. State-of-the-art conclusion

In conclusion, various tools are used in the literature to detect objects, con- tours and cracks

using machine learning and classical image processing. With regard to the machine learning-

based detection technique, several case studies detect the location of pavement cracks with

significant accuracy using deep neural convolution networks, given that the uniform texture of

the pavement does not have the same geometry as the crack discontinuities49,109–111 .

Similarly, machine learning algorithms are used in defect detection on other infrastructure

materials such as concrete decks50 , stone masonry112 . The texture of these example

materials is indeed uniform, highlighting the localization of the crack using the convolutional

neural networks used in both studies. Sipko et al. use automatic crack classification

algorithms on smooth-textured marble and manage to identify the crack as a different class

from the uniform texture of the two marble samples54 .

In relation to the aim of this chapter, pavement, concrete slabs and stone masonry have a

texture similar to the first smooth marble texture we studied, but we cannot use these

algorithms due to the difficulty of building a database rich enough to train this type of model.

Unlike the study by Sipko et al., whose classification algorithm is applied to a smooth marble

texture54 , this chapter aims to develop crack detection on other, more char- gerized textures

presenting both veins and cracks.

For the detection technique based on classical image processing techniques, the pavement

was inspected with a statistical approach in several studies63,106,113 and by the spectral

approach114 . Other case studies used a second-order statistical approach to locate the

defect on the input image of the inspected material such as textile57,58 ,

steel88 and seabeds42 . We noticed that the road, steel and textile textures

74
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

are simple, and the classic image processing approach succeeds in revealing the defect. In

connection with this chapter, these approaches will be applied not only to smooth-textured

marble, but also to two other types of medium- and highly-filled texture (see Ta- ble 3). The

main difficulty addressed in this work lies in the fact that the marbles under consideration (in

particular the moderately and heavily loaded marble textures) feature both randomly

distributed cracks and veins, and the marble cracks and veins may have very similar

geometric shapes and sometimes the same colors.

Thus, this study aims to apply different image processing approaches to three marble

textures and discuss the crack detection efficiency of these approaches in relation to the

texture loading rate.

II.3. MARBLE SURFACE CRACK DETECTION APPLICATION:

In what follows, the acquisition and pre-processing steps considered will be the same

whatever the marble texture under consideration. The images were taken with a mobile

c a m e r a , a n d pre-processed using a Kuwa- hara filter, which effectively reduces image

noise and preserves con- trusions66,115 . For example, if we take a statistical tool, the Canny

filter, we can see that the Kuwahara smoothing shown in Figure 28, will reduce the noise but

preserve the crack contours detected.

75
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 28: Kuwahara smoothing effect on edge detection, the Canny filter
The same can be said for one of the spectral tools, the wavelet transform, as shown in Figure

29.

Figure 29: Kuwahara smoothing effect on edge detection, the transform into
76
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

wavelets
For the "processing" stage, the various tools listed in section II.2.2.1 will be tested on each of

the marble textures. Finally, for attribute extraction, we will use K-means as an unsupervised

classification algorithm widely used in image processing and giving efficient results. Figure

30 below illustrates the marble surface crack detection process proposed in this work.

Figure 30: Procedure for detecting cracks in marble

II.3.1. Detecting surface cracks in plain-textured marble :

In this section, the various tools identified above will be tested in turn on plain-textured

marble to assess their ability to detect the presence of a crack. The tools considered here are

✓ First-order sliding moments: variance, mean, skewness and kurtosis.

✓ Canny, Sobel and Laplacian filters

✓ Two descriptors of the co-occurrence matrix: entropy and contrast. Homogeneity and
contrast behave respectively as the inverse of

77
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

entropy and energy, as pointed out by Regniers et al.116 . For this reason, these two

indicators will not provide any additional information for crack detection and will not be

presented here. Following tests on plain marble images, correlation does not seem to

provide any particular information on the presence of a crack and will not be presented

here.

✓ The binary local pattern (MLB) method

✓ Spectral approach tools: Fourier transform, Gabor filter and Wavelet transform.

II.3.1.1. Detection steps

In order to assess the ability of a detection tool to detect the presence of a crack in an image,

the detection tool will be applied to the pre-processed image and then a k-means

classification will be applied to the tool's output. A crack will be considered correctly detected

if one of the k-means clusters matches the crack in the image. The kink method presented in

section I.3.3.6.1 of the first chapter is used to fix the number of clusters, which is taken to

be equal to 3. To define a crack detection score, we manually define the faulty pixels in the

original image and compare the position of the pixels of the reference crack with those of the

crack detected by the image processing tool. The defective pixels defined in Figure 31

constitute the reference crack of the plain marble texture.

78
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 31: definition of defective pixels for untextured marble

Several indicators can be studied to look at the ability of detection tools to search for
the crack:

• Accuracy: accuracy is the ratio (VP+VN)/(VP+VN+FP+FN), the closer it is to

1, the better the test.

• Accuracy: Accuracy is the ratio of VP/(VP + FP). It corresponds to the proportion

of positive predictions that are actually correct.

• Balanced accuracy (binary case only): balanced accuracy is an indi- cator used

to assess the quality of a binary classifier. It is particularly useful when classes

are unbalanced.

• Sensitivity (also called True Positive Fraction or recall): proportion of positive

individuals actually detected by the classifier.

79
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

• Specificity (also called True Negative Fraction): proportion of negative individuals

actually detected by the test.

The calculation of each tool's ability to detect the faulty zone pixels defined in the figure

above is calculated in this study by the "accuracy" index, which is defined by Metz117 as the

proportion of correct predictions (both true positive and true ne- gative) among the total

number of cases examined. Sometimes, this parameter is referred to as the Rand index,

which presents a measure of similarity between two partitions of a set 118. Accuracy is used in

this study to calculate the rate of agreement between two partitions that are faulty zone

defined by the faulty pixel selection method and the result of the detection tool.

The score defining the detection accuracy of each tool is defined as the ratio between the

sum of true positives (VP) and true negatives (VN) and the sum of true positives (VP), true

negatives (VN) and false positives. This score is calculated using the equation below:

Equation 31 accuracy = VP+VN/(VP+ FP+VN)

Where

VP = what the tool declares as positive and this is the case: intersection (contours detected

by the detection tool and pixels associated with the crack zone).

FN = what the tool declares as negative but is not: what the tool fails to detect on the crack.

FP = false contours detected by the tool: what the tool declares as positive is not the case

VN = what the tool declares as negative, and this is the case.

80
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.3.1.2. Crack detection performance on plain marble texture using tools

image processing :

Similarly, the detection accuracy of each tool was calculated and the comparison between

the performance of the different tools for a plain textured marble is illustrated in Figure 32

below:

Figure 32: Crack detection performance of different tools for untextured marble.

The execution time of each tool used on a Core (TM) i5 pc with approximately 8.00 GB RAM

installed is also shown in Figure 32. It can be seen that, despite the accuracy of two OLS

descriptors (contrast and entropy) reaching 51.37% and 99.13% respectively, these

descriptors take up a lot of computing time in days (1.5 d and 0.9 d respectively). Also, the

sliding moments (mean, variance skewness and kurtosis) are a little slow (between 110 s

and 135 s) in execution time and reach a lower accuracy between 50% and 70%. First-order

sliding moments are a little more time-consuming than other tools (such as LBP, Gabor

...etc.), since they are calculated by scanning a thumbnail over the entire image, and this

takes time.

81
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

As for the other detection tools, they are much faster, and we find that the Canny filter, the

Gabor filter and the MLB are more effective at detecting cracks than the other tools,

exceeding 99% accuracy.

Table 6 shows the results of crack detection on a plain marble texture by the three most

efficient tools in terms of accuracy and computation time (Canny filter, Gabor filter and local

binary pattern). It can be seen that the Canny filter detects all the details of the crack and

even other details outside the crack with an intensity different from the image background.

The Gabor and MLB tools, on the other hand, detect the crack skeleton as a whole. The

performance of these two tools was slightly better than that of the Canny filter, as they

detected fewer false contours scattered on both sides of the crack. As a result, the

FP(Gabor) and FP(MLB) numbers are lower than FP(Canny).

Table 6: Crack detection on plain marble texture

Detection tool Extraction of the class defining the crack

Statistical
1st
approach
order 🡪 Canny

Performance=95.76

Execution time=30 seconds

82
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Approach
spectral 🡪 Gabor
filter

Performance=99.58

Execution time= 27 s

Higher-order
statistical
approach
🡪 MLB

Performance= 99.21

Run time= 25 s

So, we can see that all the tools cited in the state of the art are capable of detecting crack

location to varying degrees of accuracy. Nevertheless, OLS descriptors take a long time to

execute. First-order sliding moments (mean, variance, skewness and kurtosis) require a

relatively long computation time. The Canny filter, local thresholding, Laplace filter, Sobel

filter, MLB, Fourier transform, Gabor filter and wavelet transform are fast to execute, but the

Gabor filter, MLB tool and Canny filter detect the untextured marble crack better.

with respective accuracies of 99.58%, 99.21% and 95.76%.

83
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.3.2. Detection of surface cracks in moderately and highly loaded marble :

For the two types of moderately and heavily loaded marble textures, application of the list of

tools used previously (Canny, Laplacian, ..., Gabor filter, TO) detects both the crack and the

rib. If we take, for example, the Canny filter and local thresholding as tools belonging to the

first-order statistical approach, the OLS entropy as a tool belonging to the second-order

statistical approach, the MLB higher-order statistical tool and the wavelet transform as a tool

belonging to the spectral ap- proximate, the results of edge detection by these tools are

presented in Figure 33 for a moderately loaded texture and in Figure 34 for a heavily loaded

texture.

Figure 33: Edge detection for a moderately loaded texture using the statistical and
spectral approaches

84
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 34: Edge detection for a moderately loaded texture using the statistical and
spectral approaches
The detection results illustrate that the different tools detect both the crack and the ribs

present in two textures. The problem, then, is how to differentiate whether a contour belongs

to the crack or to the plate texture?

We'll take another look at the most accurate tools (the Canny filter, MLB and the Gabor filter)

for detecting cracks on these two types of texture.

It can be seen that, taken separately, the application of the different filters to medium and

highly textured marble does not correctly detect the crack. However, we note that the Canny

filter detects all the details in the image, i.e. the crack, the internal contours and the external

contours of the marble veins. The Gabor filter, on the other hand, detects the crack as well as

the outer contours of the marble veins. Finally, the MLB technique detects the crack and the

internal contours of the veins. This is illustrated in the following two examples

85
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

of cracked marble with medium and heavy loadings in Figure 35 and Figure 36 respectively.

Figure 35: Contour detection by (Gabor, MLB and Canny) of a moderately loaded texture

Figure 36: Contour detection by (Gabor, MLB and Canny) of a heavily loaded texture

86
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Based on these observations, we propose to combine the performances of these different

filters in order to extract the crack. The proposed approach is shown in Figure 37.

Figure 37: Crack extraction process for loaded marble textures

Indeed, for a texture containing random ribs, crack detection is more complex than for a plain

texture. To this end, we propose to extract the intersection between the three tools Gabor,

Canny and MLB to identify the common crack discontinuities detected by these tools.

II.3.2.1. Application o f Canny, MLB and Gabor intersection for medium marble
full-bodied and highly textured

II.3.2.1.1. Medium-filled marble texture :

The intersection of three Canny, MLB and Gabor tools detects the crack on lightly textured

marble, as shown in Figure 38.

87
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 38: Crack detection on lightly textured marble

The performance of this new detection tool is calculated in relation to the reference crack

defined manually by selecting the crack zone (see Figure 39).

Figure 39: Definition of the reference crack for the moderately loaded marble texture

In this case, the detection result is shown in Figure 40:

88
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 40: Intersection crack detection accuracy (Canny_Gabor_MLB) on a lightly textured marble slab

Detection accuracy is calculated according to Equation 31 and is 99.5%.

To validate this result, we will test the same method on two other images showing a crack in

the lightly textured marble plate. Table 7 shows the original image and the detection result

with corresponding accuracy.

89
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Table 7: Validation of the Canny, Gabor and MLB intersection method for others
images of lightly textured marble
Example 1: Lightly textured marble 2

Example 2: Lightly textured marble 2

90
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

So the intersection technique (Canny_Gabor_MLB) performs well, as it is fast in terms of

execution time and detects the crack with an accuracy of over 99%. We'll apply the same

technique to the heavily loaded marble texture.

II.3.2.1.2. Highly charged marble texture

Similarly, the intersection of these three tools detects discontinuities in the highly textured

marble crack Figure 41.

Figure 41: Crack detection on textured marble

In the same way as before, the pixels defining the reference crack of the heavily loaded

marble are defined manually, and Figure 42 shows the set of pixels defining this crack.

91
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Figure 42: Definition of the reference crack for the highly loaded marble texture

We then calculated the number of true positives, false positives, true negatives and false ne-

gative pixels detected by the technique (Canny_Gabor_MLB). The detection accuracy of this

tool on a lightly textured marble texture is of the order of 98%.

The method was also tested on two other images showing a crack in the highly textured

marble slab. Table 8 shows the original image and the detection result with corresponding

accuracy.

92
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

Table 8: Validation of the Canny, Gabor and MLB intersection method for other highly textured

marble images

Example 1

Example 2

We note that the method (Canny_Gabor_MLB) also detects the crack on heavily loaded

textures with an accuracy of over 98%.

93
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

II.4. CHAPTER CONCLUSION

In this chapter, we set out the steps involved in detecting surface cracks in marble in the form

of three types of texture with increasing loading rates (plain texture, moderately loaded

texture and heavily loaded texture).

The results of this chapter complement those of the first chapter by enabling the crack to be

located in the cracked plate. Firstly, a state-of-the-art study was carried out to identify the

different approaches to flaw detection, to describe the tools of each approach and to study

the applications of these tools in the literature. Secondly, a distinction was made between the

two main approaches: the statistical approach (first-order, second-order and higher-order)

and the spectral approach. The aim was to apply the different approaches to each texture

and conclude on the ability of the tool used to detect cracks for the three texture levels.

We conclude that the various tools used (Canny filter, local thresholding, Laplace filter, Sobel

filter, entropy, contrast, MLB, Fourier transform, Gabor filter and wavelet transform) are

capable of detecting cracks on a plain marble texture at variable accuracy rates, but the

Gabor filter, Canny filter and MLB tool remain more accurate and faster.

For moderately and heavily loaded textures, conventional image processing tools detect both

the crack and the marble veins. As a result, a new contour detection technique has been

proposed, based on the intersection of contours detected by the most efficient tools for the

case of plain-textured marble. It appears that the Canny filter detects the crack as well as the

texture's inner and outer contours, the Local Binary Patterns (LBP) tool detects the crack as

well as the texture's inner contours, and the Gabor filter detects the crack as well as the

texture's outer contours.

94
 Chapter II. Automatic localization of a crack in marble using the tools of
image processing

of the texture. The intersection of these three tools then extracts the crack from the rest of

the marble texture.

This new method was tested on the two base images of medium and highly loaded texture

(Table 3) and achieved a detection accuracy of over 98% compared to the manually selected

crack state by selecting the faulty pixels defining the crack.

However, we can further improve this application by retrieving an entire database of images

of healthy and damaged marbles and applying machine learning models to define the crack

discontinuities. Convolutional neural networks and auto-encoder models can be used, for

example, by defining the geometry of the crack discontinuity being sought as the target. A

second very interesting approach is to link and merge the detected crack discontinuities so

that the detection algorithm returns the crack median at the end.

Having identified in the first chapter whether a marble slab is cracked or not, and in the

second chapter the location of the crack on the slab, we now turn to the application of these

results to the machining of marble slabs. To this end, an experimental study is first carried

out on uncracked marble to determine the various machining de- faults that can occur.

Consequently, the third chapter describes an experimental study of the drilling of a non-

cracked "white Carrara" marble using a non-con- ventional tool: the abrasive water jet.

95