Professional Documents
Culture Documents
ablebits.com /office-addins-blog/2016/03/02/identify-duplicates-excel/
The tutorial explains how to search for duplicates in Excel. You will learn a few formulas to identify duplicate
values or find duplicate rows with or without first occurrences. You will also learn how to count instances of each
duplicate record individually and find the total number of dupes in a column, how to filter out duplicates, and
more.
While working with a large Excel worksheet or consolidating several small spreadsheets into a bigger one, you
may find lots of duplicate rows in it. In one of our previous tutorials, we discussed various ways to compare two
tables or columns for duplicates.
And today, I'd like to share a few quick and effective methods to identify duplicates in a single list. These
solutions work in all versions of Excel 2016, Excel 2013, Excel 2010 and lower.
The easiest way to detect duplicates in Excel is using the COUNTIF function. Depending on whether you want to
find duplicate values with or without first occurrences, there's going to be a slight variation in the formula as
shown in the following examples.
Supposing you have a list of items in column A that you want to check for duplicates. These can be invoices,
product Id's, names or any other data.
Here's a formula to find duplicates in Excel including first occurrences (where A2 is the topmost cell):
=COUNTIF(A:A, A2)>1
Input the above formula in B2, then select B2 and drag the fill handle to copy the formula down to other cells:
As you can see in the screenshot above, the formula returns TRUE for
duplicate values and FALSE for unique values.
1/16
In case, you want an Excel formula to find
duplicates only, replace "Unique" with an empty
string ("") like this:
The formula will return "Duplicates" for duplicate records, and a blank cell for unique records:
As you can see in the following screenshot, this formula does not identity the first occurrence of " Apples" as
duplicate:
=IF(SUM((--EXACT($A$2:$A$8,A2)))<=1,"","Duplicate")
2/16
As shown in the screenshot below, it treats lowercase and uppercase as different characters (APPLES is not
identified as a duplicate):
The following screenshot demonstrates that the formula really locates only the rows that have identical values in
all 3 columns. For example, row 8 has the same order number and date as rows 2 and 5, but a different item in
column C, and therefore it is not marked as duplicate row:
To show duplicate rows without 1 st occurrences, make a little adjustment to the above formula:
3/16
How to count duplicates in Excel
If you want to know the exact number of identical records in your Excel sheet, use one of the following formulas to
count duplicates.
When you have a column with duplicated values, you may often need to know how many duplicates are there for
each of those values.
To find out how many times this or that entry occurs in your Excel worksheet, use a simple COUNTIF formula,
where A2 is the first and A8 is the last item of the list:
=COUNTIF($A$2:$A$8,
$A2)
As demonstrated in the following screenshot, the formula counts the occurrences of each item: "Apples" occurs 3
times, "Green bananas" - 2 times, "Bananas" and "Oranges" only once.
If you want to identify 1 st, 2 nd, 3 rd, etc. occurrences of each item,
use the following formula:
=COUNTIF($A$2:$A2, $A2)
Once the duplicate values are counted, you can hide unique values
and only view duplicates, or vice versa. To do this, apply Excel's
auto-filter as demonstrated in the following example: How to filter out duplicates in Excel.
The easiest way to count duplicates in a column is to employ any of the formulas we used to identify duplicates in
Excel (with or without first occurrences). And then you can count duplicate values by using the following
COUNTIF formula:
=COUNTIF(range, "duplicate")
Where "duplicate" is the label you used in the formula that locates duplicates.
4/16
In this example, our duplicate formula takes the following shape:
=COUNTIF(B2:B8, "duplicate")
Another way to count duplicate values in Excel by using a more complex array formula. An advantage of this
approach is that it does not require a helper column:
=ROWS($A$2:$A$8)-SUM(IF( COUNTIF($A$2:$A$8,$A$2:$A$8)=1,1,0))
Because it's an array formula, remember to press Ctrl + Shift + Enter to complete it. Also, please keep in mind
that this formula counts all duplicate records, including first occurrences:
5/16
To find the total number of duplicate rows, embed the COUNTIFS function instead of COUNTIF in the above
formula, and specify all of the columns you want to check for duplicates. For example, to count duplicate rows
based on columns A and B, enter the following formula in your Excel sheet:
=ROWS($A$2:$A$8)-SUM(IF( COUNTIFS($A$2:$A$8,$A$2:$A$8,
$B$2:$B$8,$B$2:$B$8)=1,1,0))
If you want to see all duplicates at a glance, use one of the formulas to find duplicates in Excel that better suits
your needs. Then select your table, switch to the Data tab, and click the Filter button. Alternatively, you can click
Sort & Filter > Filter on the Home tab in the Editing group.
6/16
Tip. To have filtering enabled automatically, convert your data to a fully-functional Excel table. Just select all data
and press the Ctrl + T shortcut.
After that, click the arrow in the header of the Duplicate column and check the "Duplicate row" box to show
duplicates. If you want to filter out, i.e. hide duplicates, select "Unique" to view only unique records:
And now, you can sort duplicates by the key column to group them for easier analysis. In this example, we can
sort duplicate rows by the Order number column:
7/16
How to filter duplicates by their occurrences
If you want to show 2 nd, 3 rd, or Nth occurrences of duplicate values, use the formula to count duplicate instances
we discussed earlier:
=COUNTIF($A$2:$A2, $A2)
Then apply filtering to your table and select only the occurrence(s) you want to view. For example, you can filter
the 2nd occurrences like in the following screenshot:
8/16
Select "is greater than" in the first box, type 1 in the box next to it, and click the OK button:
In a similar manner, you can show 2 nd, 3 rd and all subsequent duplicate occurrences. Just type the required
number in the box next to "is greater than".
To select duplicates, including column headers, filter them, click on any filtered cell to select it, and then press
9/16
Ctrl + A.
To select duplicate records without column headers, select the first (upper-left) cell, and press Ctrl + Shift +
End to extend the selection to the last cell.
Tip. In most cases, the above shortcuts work fine and select filtered (visible) rows only. In some rare cases,
mostly on very large workbooks, both visible and invisible cells may get selected. To fix this, use one of the above
shortcuts first, and then press Alt + ; to select only visible cells , ignoring hidden rows.
To clear duplicates in Excel, select them, right click, and then click Clear Contents (or click the Clear button >
Clear Contents on the Home tab, in the Editing group). This will delete the cell contents only, and you will have
empty cells as the result. Selecting the filtered duplicate cells and pressing the Delete key will have the same
effect.
To remove entire duplicate rows, filter duplicates, select the rows by dragging the mouse across the row
headings, right click the selection, and then choose Delete Row from the context menu.
To highlight duplicate values, select the filtered dupes, click the Fill color button on the Home tab, in the
Font group, and then select the color of your choosing.
Another way to highlight duplicates in Excel is using a built-in conditional formatting rule for duplicates, or
creating a custom rule specially tailored for your sheet. Experienced Excel users won't have any problem with
creating such a rule based on the formulas we used to check duplicates in Excel. If you are not very comfortable
with Excel formulas or rules yet, you will find the detailed steps in our next tutorial.
To copy duplicates, select them, press Ctrl + C, then open another sheet (a new or existing one), select the
upper-left cell of the range where you want to copy the duplicates, and press Ctrl + V to paste them.
To move duplicates to another sheet, perform the same steps with the only difference that you press Ctrl + X
(cut) instead of Ctrl + C (copy).
This is how you can identify duplicates in Excel using functions and built-in features. To better understand the
formulas discussed in this tutorial, feel free to download Identify Duplicates Sample Workbook.
This all-in-one tool can search for duplicate or unique values in a single column or compare two columns. It can
find, select and highlight duplicate records or entire duplicate rows, remove found dupes, copy or move them to
another sheet. I think an example of practical use is worth many words, so let's get to it.
To test the capabilities of our Duplicate Remover add-in, I've created a table with a few hundred rows that looks
like follows:
As you see, the table has a few columns. The first 3 columns contain the most relevant information, so we are
going to search for duplicate rows based solely on the data in columns A - C. To find duplicate records in these
columns, just do the following:
1. Select any cell within your table and click the Dedupe Table button on the Excel ribbon. After installing the
Duplicate Remover add-in, you will find this tool on the Ablebits Data tab, in the Dedupe group.
2. The smart add-in will pick up the entire table and ask you to specify the following two things:
Select the columns to check for duplicates (in this example, these are the Order no., Order date
and Item columns).
Choose an action to perform on duplicates. Because our purpose is to identify duplicate rows,
I've selected the Add a status column
11/16
Apart from adding a status column, an array of other options are available to you:
Delete duplicates
Color (highlight) duplicates
Select duplicates
Copy duplicates to a new worksheet
Move duplicates to a new worksheet
As you can see in the below screenshot, all of the rows that have identical values in the first 3 columns have
been located (first occurrences are not identified as duplicates).
12/16
If you want more options to dedupe your worksheets, use the Duplicate Remover wizard that can find duplicates
with or without first occurrences as well as unique values. The detailed steps follow below.
Depending on a particular sheet you are working with, you may or may not want to treat the first instances of
identical records as duplicates. One possible solution is using a different formula for each scenario, as we
discussed in How to identify duplicates in Excel. If you are looking for a fast, accurate and formula-free method,
try the Duplicate Remover wizard:
1. Select any cell within your table and click the Duplicate Remover button on the Ablebits Data tab, in the
Dedupe group. The wizard will run and the entire table will get selected.
2. On the next step, you are presented with the 4 options to
check duplicates in your Excel sheet:
For this example, let's go with the second option, i.e. Duplicates + 1st occurrences:
13/16
3. Now, select the columns where you want to check duplicates. Like in the previous example, we are
selecting the first 3 columns:
4. Finally, choose an action you want to perform on duplicates. As is the case with the Dedupe Table tool, the
14/16
Duplicate Remover wizard can identify, select, highlight, delete, copy or move duplicates.
Because the purpose of this tutorial is to demonstrate different ways to identify duplicates in Excel, let's
check the corresponding option and click Finish:
It only takes a fraction of a second for the Duplicate Remover wizard to check hundreds of rows, and deliver the
following result:
15/16
No formulas, no stress, no errors - always swift and impeccable results :)
If you are interested to try these tools to find duplicates in your Excel sheets, you are most welcome to download
a fully-functional trial version. Your feedback in comments will be greatly appreciated!
And if you like the tools, we will happily offer you the 15% discount, which we provide exclusively to our blog
readers. Just use the following coupon code on the order form: AB14-BlogSpo. It is valid for Duplicate
Remover as a separate product and as part of Ultimate Suite for Excel.
I thank you for reading and hope to see you on our blog next week!
16/16