Skip to main content

Flow Lifecycle

Alex Wolfe

Flow Lifecycle

This guide traces the complete journey of a lead through LeadConduit, showing exactly how each concept participates in the processing lifecycle.

Overview

The flow lifecycle consists of five major phases:

  1. Configuration - Setting up your flow
  2. Lead Arrival - Receiving and authenticating
  3. Processing - Validating and enhancing
  4. Delivery - Sending to destinations
  5. Analysis - Tracking and optimizing

Phase 1: Flow Configuration

Before any lead can be processed, the flow must be configured with all necessary components.

Define Your Data Model

  • Choose standard fields for common data (email, phone, name)
  • Create custom fields for business-specific data
  • Each field references a type for parsing behavior

Configure Sources

Establish Business Logic

  • Define flow-level acceptance criteria using rules
  • Set up volume caps with conditional rules
  • Configure pricing tiers and conditions
  • Create suppression lists for duplicates/DNC

Design Processing Steps

Phase 2: Lead Arrival

When a vendor submits a lead, the arrival phase begins.

HTTP Request Reception

POST /flows/[flow-id]/sources/[source-name].in
Content-Type: application/json
Authorization: Bearer [api-key]

{
  "fname": "Sarah",
  "lname": "Chen",
  "email_addr": "sarah@example.com",
  "phone1": "(555) 123-4567"
}

Authentication

  • Integration validates the API key
  • Failed auth returns 401 immediately
  • No events generated for auth failures

Source Configuration Loading

The system loads:

Variable Initialization

Create the data structure that accumulates throughout processing:

{
  lead: {},        // Parsed lead data
  source: {},      // Source metadata
  flow: {},        // Flow metadata
  appended: {},    // Enhancement data
  destination: {}  // Delivery-specific data
}

Phase 3: Lead Processing

The core processing phase transforms and validates the lead.

Data Normalization

  1. Field Mapping: Apply source mappings

    • email_addr → email
    • phone1 → phone
    • fname + lname → name

  2. Type Parsing: Each type processes its fields

    • Phone: "(555) 123-4567" → {valid: true, area: "555", number: "5551234567"}
    • Email: Validates format, extracts domain
    • State: Normalizes to two-letter code

  3. Variable Population: Parsed data becomes available

    • Standard fields: lead.email, lead.phone.number
    • Custom fields: Direct access by name

Pre-Processing Gates

  1. Purchasing Decisions

    • TrustedForm Decisions runs if configured
    • Data available in trustedform_decisions namespace
    • Used by acceptance criteria and caps

  2. Acceptance Criteria

    • Flow-level rules evaluate first
    • Source-level rules evaluate second
    • First failure stops processing

  3. Suppression Lists

    • Check duplicates based on field combinations
    • Verify against DNC lists
    • Check existing customer lists

  4. Volume Caps

    • Evaluate cap rules to determine if lead counts
    • Check current usage against limits
    • Reject if cap exceeded

  5. Pricing Calculation

    • Source service → Source rules → Flow service → Flow rules
    • First layer returning a price wins
    • Purchase price recorded

Flow Step Execution

Steps process sequentially, each potentially modifying the lead's path:

  1. Filter Steps

    • Evaluate rules using current data
    • Can skip to specific step or stop flow
    • Generate filter events

  2. Enhancement Steps

    • Call external services via integrations
    • Append data to appended namespace
    • Data immediately available to subsequent steps

  3. Recipient Steps

  4. Destination Steps

Phase 4: Lead Delivery

Each destination receives the lead independently.

Data Transformation

Integration Execution

  • Integration module handles delivery
  • HTTP POST, SOAP, or custom protocol
  • Timeout and retry handling

Response Processing

  • Parse response using configured patterns
  • Extract success/failure indicators
  • Capture response data in appended namespace

Revenue Recording

  • On successful delivery, record sale price
  • Calculate profit (sale - purchase)
  • Update financial metrics

Phase 5: Analysis & Optimization

Post-processing provides visibility and insights.

Event Generation

Every significant action creates an event:

  • Source event: Summarizes entire journey
  • Filter events: Record routing decisions
  • Enhancement events: Track data appending
  • Destination events: Document delivery attempts

Real-Time Visibility

Events enable:

  • Live monitoring dashboards
  • Instant vendor feedback
  • Quick troubleshooting
  • Performance tracking

Business Intelligence

Reporting aggregates events into metrics:

  • Acceptance rates by source
  • Cost per qualified lead
  • Revenue by destination
  • Quality scores by vendor

Detailed Analysis

Exports provide row-level data for:

  • Reconciliation with vendors
  • Compliance documentation
  • Deep performance analysis
  • Custom analytics

Lifecycle Termination Points

Processing can end at various stages:

Pre-Processing Termination

  • Authentication Failed: Invalid API key
  • Acceptance Criteria Failed: Business rules not met
  • Suppression List Match: Duplicate or DNC
  • Volume Cap Exceeded: Limits reached
  • Hours of Operation: Outside business hours

Step Processing Termination

  • Filter Stop: Explicit flow termination
  • Integration Error: Critical failure
  • Timeout: Processing time exceeded

Normal Completion

  • All steps executed
  • Destinations attempted
  • Response sent to source
  • Events recorded

Data Availability Timeline

Understanding when data becomes available is crucial:

  1. Raw Submission: Unmapped vendor fields
  2. After Mapping: Standard field names applied
  3. After Parsing: Types create structured data
  4. After Enhancement: appended namespace populated
  5. In Destinations: All accumulated data available

Performance Considerations

Optimize Early Decisions

  • Put most restrictive rules first
  • Use simple field checks before complex logic
  • Filter early to reduce processing

Minimize External Calls

  • Batch enhancements when possible
  • Set appropriate timeouts
  • Handle failures gracefully

Efficient Data Access

  • Use parsed components (e.g., phone.area)
  • Leverage templates for transformations
  • Avoid redundant calculations

Troubleshooting the Lifecycle

Finding Where Leads Fail

  1. Check source events for outcomes
  2. Review rejection reasons
  3. Trace through filter events
  4. Examine enhancement failures

Common Failure Points

  • Missing required fields (check mappings)
  • Invalid data format (review types)
  • Failed acceptance criteria (examine rules)
  • Integration timeouts (check service status)

Using Events Effectively

  • Filter by source and time range
  • Look for patterns in failures
  • Track individual lead journeys
  • Monitor step durations

The flow lifecycle is the heartbeat of LeadConduit. Understanding each phase helps you build more effective flows and troubleshoot issues quickly.

Comments

0 comments

Please sign in to leave a comment.