Privacy-Preserving Record Linkage Using
Bloom Filter
Shahidul Islam Khan Maikel Sutradhar
Associate Professor Department Of Computer Science &
Department Of Computer Science & Engineering.
Engineering. International Islamic University,
International Islamic University, Chittagong
Chittagong Email:michaelsutradhar@gmail.com
Email: nayeemkh@gmail.com
Abstract — In this modern era, privacy
preserving of an user information is becoming a
crucial issue. When two databases are linked up,
it’s very important case how to keep individual
user information secure due to the contain of
sensitive information about user. At this point of
current situation in Bangladesh it is really
important to get preserved their individual data,
when different organizations link their
databases. We used a technique for privacy
preserving record linkage which is called Bloom
Filter, it is a space saving probabilistic data
structure which is helped to identify individual
user uniquely and at the same time keep the user
information secure. Record linkage has various
challenges, including scalability of large
databases, accurate matching, classification,
privacy and confidentiality. By using this privacy
technique we can easily secure each and every
user records and then identify them from those
records. So that, the attacker cannot access these
information when the databases are linked for
various purposes. The objective of the paper is to
develop the security issue of an user with this
privacy technique in an efficient way when the
user information are linked.
Keywords— record linkage, bloom filter, privacy
preserving,confidentiality, classification, scalability.
1. INTRODUCTION
In today’s world, privacy is a great concern for the
individual users, who are connected to different
Electronic copy available at: https://ssrn.com/abstract=4237453
Emon Chowdhury
Department Of Computer Science
&
Engineering.
International Islamic University,
Chittagong
Email:emonchy35@gmail.com
technology, the most concerning issue is how to
protect the security of an user when different
organizations or individuals collect data from
different data sources.
Much of these records are about people or their
related information. Examples of the fields include
financial data such as shopping transactions,
telecommunication records or electronic health
records. In the above fields all these data are
sensitive and equally important. When we linkup
different data sources, we need to check the quality
of data and at the same time integrating data in a
secure way, so the users data remain safe from the
attackers or internal interference. It is the crucial
issue to identify a user uniquely from dissimilar
data sources. Record linkage is one of the
substantial areas of data integration. It is the task of
finding records in a data set that refers to the same
entity over diverse data sources (examples are data
files, books, websites and databases). So when we
do the record linkage task it may be possibility to
leak the highly sensitive information of an user
which is very harmful for an organization as well as
users also. For that various process and protocols
have been developed to protect the privacy of
individual and to maintain the security of data.
Our work is to secure the record linkage task and
at the same time keep the user information safe. For
this reason we used a privacy preserving technique
which is named as bloom filter. By using this
technique we are able to ensure security of an user
information and checked the same entity from
multiple datasets.
 2. BACKGROUND AND RELATED WORK
Privacy in record linkage was first explored in the
mid-1990s and since then privacy preserving record
linkage (PPRL) has been an vital research area. A
diversity of procedures have been created to allow
linkage using encoded or encrypted values of quasi-
identifying attributes over two or more databases.
Record linkage could be a broadly utilized
information pre-processing and information
cleaning task where the aim is to link and integrate
records that allude to the same entity from two or
multiple different databases. The record pairs (when
linking two databases) or record sets (when linking
more than two databases) are compared and
classified as ‘matches’ by a linkage model if they
are expected to allude to the same entity, or as ‘non-
matches’ if they are assumed to allude to different
entities. The frequent nonattendance of unique
entity identifiers over the databases to be linked
makes it impossible to utilize a straightforward
SQL-join, and therefore linkage requires advanced
comparisons between a set of QIDs (such as names
and addresses) that are commonly accessible within
the records to be linked. However, these QIDs
frequently contain individual data and so
uncovering or trading them for linkage is not
conceivable due to security and confidentiality
concerns.
Sean M. Randall, Anna M. Ferrante, James H.
Boyd, Jacqueline K. Bauer, James B. Semmens
adopted the bloom filter method for privacy
preserving record linkage [1] developed by Schnell
et.al [2] .The method appears robust and well-
developed, with a number of papers investigating its
security [3] and processing additions to its method
[4] Maintaining the Integrity of the Specifications.
The use of bloom filters was evaluated to
determine its suitability for conducting large scale
privacy preserving record linkage. Two datasets,
comprising in total over 26 million records, were
linked using this method, with results compared to
the linkage of unencrypted data. A probabilistic
linkage framework was adopted to allow large-scale
linkage to occur.
They adopt the SHA-1 hash function which is used
by Schnell et al [2] .They calculate approximately
Electronic copy available at: https://ssrn.com/abstract=4237453
700.000 hashes per second on their hardware. This
equates to a run time over of over 50 days to
complete a linkage of their smaller file. In two
datasets, the first one named WA contains
6,772,949 records and the NSW dataset contains
19,874,083 records. For WA unencrypted linkage
precision is 0.999, recall 0.981, f-measure 0.990 and
the result for encrypted linkage using bigrams is
precision 0.998, recall 0.981, f-measure 0.990. For
NSW unencrypted linkage precision is 0.986, recall
0.972, f-measure 0.979 and the result for encrypted
linkage using bigrams is precision 0.985, recall
0.970, f-measure 0.978.
Dinusha Vatsalan, Peter Christen discussed
efficient Bloom filter-based masking approaches for
numerical data [5] and develop a solution for the
privacy-preserving SPM (PP-SPM) problem. There
main contributions are (1) based on bloom filter
masking techniques that have shown to be
successful for approximate matching of string data
[3,6] , they propose novel Bloom filter masking
approaches for numerical data that have similar
characteristics as for string data ( i.e., they are
efficient ,effective and secure), (2) develop a
comprehensive framework for PP-SPM using
Bloom filter masking-based similarity calculations
for matching different types of data, (3) report on
an extensive empirical study of their framework on
three publicly available real-world datasets.
They evaluated the accuracy and privacy of their
proposed Bloom-filter based numerical data
matching approach for PP-SPM. They used 10,000
generated pairs of numerical values with uniform
random distribution for three attributes: age (integer
in the range [1, 101]), salary (integer in the range
[0,100,000]), and body weight (floating point in the
range [0,000,120,000]). They set the parameter
values for the length of Bloom filters l-1000, the
number of hash functions k-10.
Peter Christen, Thilina Ranbaduge, Dinusha
Vatsalan and Rainer Schnell discussed about precise
and fast cryptanalysis for bloom filter based
privacy-preserving record linkage [7], in their paper
they extend their recently proposed attack method
[8] which exploits how q-grams are hashed into
Bloom Filters. For each bit position, this method
identifies a set of possible and a set of not possible
q-grams and then re-identifies attribute values using
only the sets of possible q-grams , However as their
experiments these sets have low precision which
leads to low re-identification accuracy . In contrast,
their novel attack method uses the sets of not
possible q-grams which have higher precision. They
conducted experiments using three data sets from
two countries. The first is a pair of ‘NCVR’ datasets
(the database to be attacked) and (the plaintext
database). They extracted two subsets containing
224,073(2014) and 224,061 (2016) records,
respectively. The second pair of data sets ‘UKCD’
are census records collected from the years 1851
(the sensitive database) and 1861 (the plain-text
database). These data sets contain 17,034 (1851)
and 22,430(1861) records. For both data sets we use
the First name, Surname, and City (NCVR) or
Address (UKCD) attributes, respectively, as well as
the concatenation of pairs of these attributes. A
third data set was a voter database from Michigan
containing 7,408,330 records that they used as the
plain-text database available to an attacker in a
cross-state attack experiment.
Lailong Luo, Deke Guo, Richard T.B. Ma, Ori
Rottenstreich, and Xueshan Luo published a paper
named Optimizing Bloom Filter: Challenges,
Solutions, and Comparisons [9]. They survey the
existing variants from two dimensions, i.e.
performance and generalization. To improve the
performance, dozens of variants devote themselves
to reducing the false positives and easing the
implementation. Besides, tens of variants generalize
the BF framework in more scenarios by diversifying
the input sets and enriching the output
functionalities. In their survey, more than 60 up-to-
date designs are reviewed and qualitatively yet
systematically analyzed.
Despite its space-efficiency, BF still faces some
challenges related to false positives,
implementation, elasticity, and functionality to
some extent. To ease the potential challenges and
further improve the performance of BF, they
improved BF from four angles, i.e., techniques to
Electronic copy available at: https://ssrn.com/abstract=4237453
reduce the false positives, optimizations in a real
implementation, dedicated designs for diverse
datasets and proposals to enable more
functionalities.
Ibrahim Lazrig, Toan C. Ong, Indrajit Ray,
Indrakshi Ray, Xiaoqian Jiang, Jaideep Vaidya
published a paper named Privacy Preserving
Probabilistic Record Linkage Without Trusted
Third Party [10]. In their work, they focused on
PPRL across two datasets that contain records
belonging to the same entity that need to be linked
but a deterministic match is infeasible because of
the discrepancies in the values of the attributes
being matched. Their work is based on the use of
Bloom filters to perform probabilistic PPRL, and on
the concept of garbled circuits proposed by Yao that
allows two parties to securely compute any function
f(x, y) without sharing the x and y values. The total
time taken to link the 10k*10k datasets, was 2199
seconds (about 36 min) while the non-distributed
garbled circuit took 30283 seconds (8.5 hours).
The data fields included in both datasets are First
name, Last name, Social Security Number (SSN),
Date of birth and Identification (ID). In order to
validate the results, they use an identification
number (ID) for each record such that all records
that belong to the same individual have the same
identification number. Each dataset contains 10K
records and both datasets have 6K records in
common. A record which is in both datasets has the
same value in the ID field. Therefore, the values of
the ID field are used as the true answer to verify
linkage performance.
There are several task are executed over privacy
preserving record linkage but there are some
drawbacks of those task. Firstly, in most of the task
they are using such algorithm for privacy preserving
where cryptanalysis attack can be done due to use
of less secure hash functions. Secondly, increasing
the number of hashes reduce performance of Bloom
Filter. Thirdly, in most cases they use Bloom Filter
for string attributes or numerical attributes
seperately.
 3. METHODS
We used a privacy technique which is called Bloom Filter and we also used two party protocol for our
works. Because it can be consider more secure as they do not rely on the existence of third party and
requires greater amount of records [11]. Fig 3.1 shows the workflow of our methodology.

Figure 3.1 Workflow of our Methodology

Electronic copy available at: https://ssrn.com/abstract=4237453

3.1 Datasets
For our work, we used two individual datasets,
named as Dataset A and Dataset B. We have
collected the dataset from popular machine learning
website Kaggle. In dataset A it contains 1000
unique data for patient and we used three attributes
(patient_name, patient_gender, patient_dob) for our
work. Similarly, in dataset B it contains 500 data
where 400 data are duplicates from dataset A and
both of the datasets are complete.
3.2 Generalized the attribute (DOB)
As the attribute named patient_dob is in
(dd/mm/yyyy) format. So we generalized this by
using these steps.
 Convert patient_dob (dd/mm/yyyy) into age.
 Set the ages in particular ranges.
 For every particular ranges we use special
symbol.
3.3 Concatenating attributes
In this section, we are concatenating our attributes
such as patient_name, patient_gender, patient_dob
by using some special symbol.
For instance, patient_name + ‘$’ + patient _gender
+ ‘%’ + patient_dob
3.4 Element insertion in Bloom Filter
All the elements one by one will be added to
Bloom Filter 1 which is same for Bloom Filter 2.
Before inserting all the elements in Bloom Filters,
the bit size of bloom filter is initially set to 0.
3.5 Apply double hashing technique
In this section, we will discuss about double
hashing method and how our algorithm works. To
reduce the collisions we apply double hashing
technique to our record linkage task and for the
security purpose we encrypt all our inserted
elements in bloom filter. We used two secure hash
algorithm named as SHA224 and SHA256.
Algorithm 3.5.1 (Secure Hash Algorithm 224):
a)Variable = string which will be added to BF.
Electronic copy available at: https://ssrn.com/abstract=4237453
b)Encode the variable using variable.encode()
which converts string into bytes.
c)Apply hexdigest() function to the encoded data,
which produce 56 digit length value and represents
the data in a hexadecimal format.
d)Using int() function to represent the hexa value
into a decimal form.
Algorithm 3.5.2 (Secure Hash Algorithm 256):
a)Variable = string which will be added to BF.
b)Encode the variable using variable.encode()
which converts string into bytes.
c)Apply hexdigest() function to the encoded data,
which produce 64 digit length value and represents
the data in a hexadecimal format.
d)Using int() function to represent the hexa value
into a decimal form.
Now, the formula of double hashing technique [2]
is:
Double hashing = h1(x) + i*h2(x)
where h1 = first hash function (SHA256)
h2 = second hash function (SHA224)
x = value/records
and i ranges from 0 to k-1
where k is the number of hash function.
3.6 Compute Modulus of double hash
To compute the bit position of bloom filter, we
have to use the result of double hash and mod this
result by the size of bloom filter.
Bit position = (Double hashing) %
Bloom_Filter_Size
3.7 Set Bit Positions
Now, we set the bit positions for every element in
bloom filter. These positions in the bloom filter are
then changed to 1 .When all required elements are
added in this way, the bloom filter is completed and
ready for comparison. Fig 3.2 shows how the bloom
filter set bit positions for an elements.
 Number Matches Non- Average
of hash Matches Time(Second)
function
1 28 72 0.030
2 21 79 0.035
3 20 80 0.038
4 16 84 0.040
Figure 3.2 Bit Position of Bloom Filter
Table 4.1: Calculate Average time for hash
4. Result Analysis and Discussion functions.

4.1 Environment Setup

Number of Precision Recall Accuracy
Implementation requirements: The hash
implementation tools that are used in this research function
are listed below: 1 0.57 1 0.88
• Personal computer 2 0.76 1 0.95
• 64-bit Windows 7 operating system 3 0.8 1 0.96
• Python 3.7.0 4 1.0 1 1.0
• Anaconda 4.4.0
• Spyder Table 4.2: Calculate precision, recall, accuracy for
hash functions.
Working environment: The machine used for the
experiment was a Windows machine running 64-bit In test result 1, we measure the performance of our
Windows 7 operating system on a Core i5 2.2 GHz bloom filter for different hash function and see how
with 4GB RAM personal computer. The machine quickly it responds. We also measure the precision,
has a 4.00 GB RAM capacity and 750 GB hard-disk recall and accuracy for hash functions. The use of
space. more hash function will increase the accuracy and
decrease the false positive.

# Test Result 2:
4.2 Experimental Setup The size of bloom filter : 2200 bit
Dataset A : 1000 records
Dataset: We experimented on a patient admission Dataset B : 500 records (duplicate 400
dataset which we have downloaded from Kaggle records)
[12]. The dataset1 contains 1000 text data and
dataset2 contains 500 text data which are complete.
Number Matches Non- Average
of hash Matches Time(Second)
4.3 Result Analysis function
1 525 475 0.270
# Test Result 1: 2 478 521 0.300
The size of bloom filter: 1024 bit 3 458 542 0.350
Dataset A : 100 records 4 455 545 0.400
Dataset B : 100 records (duplicate 16 records)
Table 4.3: Calculate Average time for hash
functions.

Electronic copy available at: https://ssrn.com/abstract=4237453

 Number of Precision Recall Accuracy In test result 2, we have got precision 0.879, Recall
hash 1 and Accuracy 0.945 for using 4 hash functions.
function In both results, 4 is the optimal hash function, by
1 0.761 1 0.875 using this we can increase the performance of
2 0.835 1 0.921 bloom filter as well as control the false positives.
3 0.873 1 0.942
4 0.879 1 0.945 Result Comparison

Table 4.4: Calculate precision, recall, accuracy for Test result 1 (100):
hash functions.
In test result 2, for 1000 data we measure the No of hash Precision Recall Accuracy
performance of our bloom filter for different hash function
function and see how quickly it works for large 4 0.88 1 0.98
data. We also measure the precision, recall and
accuracy for hash functions in terms of records. As Table 4.7: Calculate precision, recall, accuracy for
same as test result 1, the use of more hash function 4 hash function.
will increase the accuracy and decrease the false
positive. In test result 1, we have got precision 0.88, Recall 1
and Accuracy 0.98 for using 4 hash functions.
Test result 1 (100):
Test result 2 (1000):
No of Precision Recall Accuracy
hash No of hash Precision Recall Accuracy
function function
4 1 1 1 4 0.83 1 0.92

Table 4.5: Calculate precision, recall, accuracy for Table 4.8: Calculate precision, recall, accuracy for
4 hash function. 4 hash function.

In test result 1, we have got precision 1, Recall 1 In test result 2, we have got precision 0.83, Recall 1
and Accuracy 1 for using 4 hash functions. and Accuracy 0.92 for using 4 hash functions.

Test result 2 (1000): By using same datasets for different hash

algorithms which have been used earlier is SHA1
No of Precision Recall Accuracy [1, 2] and MD5 [2] in our record linkage task and
hash we have got different result. So by our applied
function algorithms we have got better result and better
4 0.879 1 0.945 performance.

4.4 Discussion
Table 4.6: Calculate precision, recall, accuracy for Within the result of our observational ponder
4 hash function. appear the adequacy and proficiency of our privacy

Electronic copy available at: https://ssrn.com/abstract=4237453


preserving record linkage assignment using Bloom
Filter. We compare the result by using different
hash functions and show how it’s reflects on
the result such as time, precision, recall and
accuracy. We also described about the false positive
probability and show how it will change by
increasing the number of hash function. The use of
more hashes will decrease the false positive and
increase the performance and accuracy which are
greatly related to linkage quality and security of the
system.
5. Conclusions
By using Bloom Filter for privacy preserving
record linkage, we can easily encrypt and compare
the individual records between two or more datasets
and uniquely identify the records of user. To keep
safe user information from external interference or
attacker, we ensure security to the system by using
several process so that the organizations or
individuals can feel comfort to share their personal
information. We hoped, by our work we will keep
user information more secure and also checked the
linkage quality by using different parameters.
In the future, we wish to develop the privacy
preserving record linkage task by applying other
privacy based technique in terms of more
performance. And we would like to work on to
check the performance and accuracy of these
techniques when records are linked up during
record linkage process and also want to test how the
records will be secured in various format
6. References
[1] Randall, S. M., Ferrante, A. M., Boyd, J. H.,
Bauer, J. K., & Semmens, J. B. (2014). Privacy-
preserving record linkage on large real world
datasets. Journal of biomedical informatics, 50,
205-212
[2] Randall, S. M., Ferrante, A. M., Boyd, J. H.,
Bauer, J. K., & Semmens, J. B. (2014). Privacy-
preserving record linkage on large real world
datasets. Journal of biomedical informatics, 50,
205-212.
[3] Schnell, R., Bachteler, T., & Reiher, J. (2009).
Privacy-preserving record linkage using Bloom
Electronic copy available at: https://ssrn.com/abstract=4237453
filters. BMC medical informatics and decision
making, 9(1), 41.
[4] Kuzu, M., Kantarcioglu, M., Durham, E., &
Malin, B. (2011, July). A constraint satisfaction
cryptanalysis of bloom filters in private record
linkage. In International Symposium on Privacy
Enhancing Technologies Symposium (pp. 226-
245). Springer, Berlin, Heidelberg.
[5] Schnell, R., Bachteler, T., & Reiher, J. (2011).
A novel error-tolerant anonymous linking
code. German Record Linkage Center, Working
Paper Series No. WP-GRLC-2011-02.
[6] Vatsalan, D., & Christen, P. (2016). Privacy-
preserving matching of similar patients. Journal
of biomedical informatics, 59, 285-298.
[7] Durham, E. A. (2012). A framework for
accurate, efficient private record
linkage (Doctoral dissertation, Vanderbilt
University).
[8] Christen, P., Ranbaduge, T., Vatsalan, D., &
Schnell, R. (2018). Precise and fast
cryptanalysis for Bloom filter based privacy-
preserving record linkage. IEEE Transactions
on Knowledge and Data Engineering.
[9] Christen, P., Schnell, R., Vatsalan, D., &
Ranbaduge, T. (2017, May). Efficient
cryptanalysis of bloom filters for privacy-
preserving record linkage. In Pacific-Asia
Conference on Knowledge Discovery and
Data Mining (pp. 628-640). Springer, Cham.
[10] Luo, L., Guo, D., Ma, R. T., Rottenstreich, O.,
& Luo, X. (2018). Optimizing Bloom filter:
Challenges, solutions, and comparisons. IEEE
Communications Surveys & Tutorials, 21(2),
1912-1949.
[11] Lazrig, I., Ong, T. C., Ray, I., Ray, I., Jiang, X.,
& Vaidya, J. (2018, August). Privacy preserving
probabilistic record linkage without trusted third
party. In 2018 16th Annual Conference on
Privacy, Security and Trust (PST) (pp. 1-10).
IEEE.
[12] Vatsalan, D., Christen, P., & Verykios, V. S.
(2013). A taxonomoy of privacy preserving
record linkage techniques Information
Systems, 38(6), 946-969
[13] Randall, S. M., Ferrante, A. M., Boyd, J. H.,
Brown, A. P., & Semmens, J. B. (2016).
Limited privacy protection and poor sensitivity:
Is it time to move on from the statistical linkage
key-581?. Health Information Management
Journal, 45(2), 71-79.