How duplicate checking works
LeadConduit Classic provides a feature that automatically invalidates duplicate leads within a campaign in real-time. When this feature is enabled, LeadConduit Classic compares incoming leads against previously submitted good leads to see if the lead is a duplicate. Any duplicate lead that is received for a particular campaign, no matter what the source, will be automatically invalidated. If a lead is invalidated as a duplicate, LeadConduit Classic provides a real-time response to the source indicating this.
Duplicate checking is customizable. You can select the fields that are used for duplicate checking (the "primary key") and the window (number of days back) you want to use for looking for duplicates.
What is the primary key?
The primary key is the field or the combination of fields that are used for duplicate checking. By default, the email address field is used as the primary key. The primary key field(s) should be a unique value. Typically you will use a single field, such as a phone number or email address.
You can also select multiple fields to comprise the primary key, such as first name and last name (though a single-field primary key is generally more effective). If you are using a combination of fields as the primary key, then the combination of all of those fields must be exactly the same to be treated as a duplicate.
For example, assume you choose address, city, state, and zip as the primary key. You receive two leads with the following values for those fields:
123 Main St.,
These two leads would not be considered duplicates since the street address is slightly different.
Where do I configure the primary key?
Duplicate checking is configured separately for each individual campaign. You must be the campaign owner to configure duplicate checking. Simply go to the "Campaigns" tab and select the campaign you want to configure. Then click on the "edit duplicate checking" link in the "Acceptance Criteria" section.
Below, we have an example of the Primary Key and Duplicate Checking window.
It's important to select a unique identifier to act as your primary key, as mentioned above. Once you have selected a primary key, enter the number of days to check for duplicate leads. If you do not enter a value in the box, all historical data will be used to check for duplicate leads. However, you must select the check box in order for duplicate checking to be enabled.
The primary key and searching
Another way LeadConduit Classic uses the primary key is for lead searching. If you don't pick a primary key for a campaign, you won't be able to find leads in that campaign via the Lead Search page.
Is it possible to use multiple primary keys for duplicate checking?
While you can use multiple fields to define the primary key, you can't use multiple primary keys to check for duplicates in a campaign. In other words, you can't check for a duplicate if either the phone OR the email address is the same.
Workaround to use multiple primary keys for duplicate checking.
One workaround to duplicate check on multiple fields separately is to create a second campaign with a different primary key. For instance, if you would like to check for a duplicate on phone AND email, individually, the first campaign could check phone and the second campaign could check email.
Leads would be delivered from the first campaign to the second campaign, and the first campaign's success needs to depend on a successful delivery by the second campaign.
Can I change the primary key for a campaign?
Yes, you can change the primary key you are using for duplicate checking. However, this doesn't change the primary key for all of the leads that were previously collected. Using the new primary key, it will start checking against historical leads starting with the date you changed the primary key. If you were to switch back to the old key, it will still check against the old records that were originally collected using that key.
What is the duplicate checking window?
By default, all historical data for the campaign will be used for duplicate checking. However, you can specify a time period (or "window") specifying the number of days for which duplicate checking occurs. For example, if you set your campaign to check 7 days back, LeadConduit Classic will check for duplicates in that campaign in the last 7 days.
Can I change the duplicate checking window?
Yes, you can change the duplicate checking window and it will take effect immediately. By default, the dupe checking window is all historical data for the campaign. We retain all the duplicate checking data for a campaign for the life of the campaign.
What happens when the old data for my campaign is deleted or anonymized? Can I still check for duplicates against this data that has been purged?
Purging old data has a very large effect on duplicate checking. When you delete campaign data, you are no longer able to check new lead data against existing campaign data.
Does duplicate checking compare against bad leads?
No. When checking for duplicates, an incoming lead is compared against all other good leads (i.e., leads with status processing, pending, delivered or converted) within the specified window. If the primary key field value(s) for the incoming lead matches another lead in the campaign then the incoming lead is considered a duplicate.
If the original lead was bad (invalid, rejected or returned), then the incoming lead will be accepted, even though it is a duplicate. This allows publishers to re-submit leads (which may be necessary if they failed to provide a required field) without having them invalidated as duplicates.
When a lead is invalidated as a duplicate, can I see the original lead?
Yes, clicking on the lead ID will display the lead of which it is a duplicate (the original lead). This allows you to see if the original lead came from the same source or a different source.
With duplicate checking enabled, is it possible to still receive a duplicate lead?
Yes, since LeadConduit Classic processes leads concurrently, it is unable to detect duplicate leads in the rare case when two are submitted at exactly the same time. If this occurs, both leads will be sent to the recipient. If this ever happens, we recommend contacting the lead source that is submitting the leads to try to resolve this behavior.
How does lead anonymization affect duplicate checking?
De-duping is not affected by anonymization because the de-dupe table does not store actual field data in the first place. It stores a one-way hashed value of the primary key.
What's the error message provided for duplicate leads?
The reason will be "duplicate" and the event will read "Invalidated by LeadConduit".