While we've long had the ability to allow lead generators to deduplicate leads prior to submission using our pre-ping functionality in SuppressionList, this functionality requires real-time querying prior to real-time submission. We've found that sometimes lead generators have unique business processes which require identifying known duplicates prior to implementing these processes, ensuring they deliver the highest quality lead data.
To accommodate these unique workflows, you can build a flow allowing users to POST or upload a CSV to LeadConduit, check the client's SuppressionList for known duplicates, and return a CSV with the results for each line item using Batch File Delivery. Using this basic flow, duplicates can be checked quickly and easily without implementing a real-time query from a form.
This article assumes:
- You already have a SuppressionList for real-time pre-ping of duplicates. If not, you'll want to review the SuppressionList documentation for creating and populating a list.
1) Create a separate flow for your Batch De-duplication
- Add the Sources you want to offer Batch Deduplication
- Update the Fields section, removing all fields except for the field(s) you are deduplicating against (typically email or phone number)
2) Add the SuppressionList Marketplace Integration to your Flow
- You'll need the List Name from your SuppressionList where your duplicate list is maintained.
- Select the SuppressionList Query Item option from the Integration list
- Add the List Name and Field you are De-duplicating against to the appropriate sections
3) Add and Configure the Batch File Delivery Marketplace Integration to your Flow
- Select Batch File Delivery from the Add Delivery options
- Use the following recommended settings for your first Batch File Delivery
NOTE You will have to create a new batch file delivery configuration for each Vendor, but you will only need one step in your flow.
File Settings - Include Hour and Minute in the file name
Column Settings - Use the Select fields from Flow option and then pare it down to "email" or "phone_1" and "suppressionlist.query_id.found"
Delivery Schedule - Set to Every 15 minutes
Delivery Method - Select Email and add the email address(es) of the Vendor
4) Edit the Batch File Delivery Step to set Source Conditions
You need to add a conditional statement on the Batch File Configuration mapping so only leads from a specific Vendor are sent to the Batch File Delivery created for that Vendor.
Save your Flow!
5) Create more Batch File Configurations
- Switch to the Batches view under Libraries
- Find the Batch File Configuration you just created and Make a Copy
- Name the New Batch appropriately and step through the configuration. The only other setting you will need to change is the To field to the appropriate vendor email address(es).
- Save the new Batch File Configuration
6) Map the additional Batch File Configurations to the Flow
- Open and Edit your Batch De-Duplication flow, expand the Batch File Delivery Step, and click Edit Field Mappings
- Add a new field mapping line (or copy the existing Batch File Configuration line) and add the appropriate settings, so each Vendor/Source is delivering to the appropriate Batch File Configuration
Save the Flow!
7) Send the Submission Docs to Vendors
Now you can provide each vendor with Submission Docs wherein they can either upload a CSV or post to the LeadConduit endpoint. The Vendor will receive an email within 15 minutes with a CSV attached.
8) What Vendors Need to Know
The file Vendors receive will contain two columns:
- The primary key provided (Email or Phone Number)
- The SuppressionList Item Found result (true or false)
TRUE = DUPLICATE
FALSE = NOT A DUPLICATE