Dedupe and normalize a contact list

When to use

• Several lists were merged and the same person appears three times with different casing.
• The user wants a clean master file before importing into a CRM or a sending tool.
• "Dedupe this", "normalise these emails", "which of these are info@ or Gmail?"

Before you start

1. Check the connection: npx ravn whoami (free).
2. Costs: the local clean-up is free. Flagging with validate_email is 1 lookup per unique address after dedupe, so dedupe first and you use fewer lookups.
3. Ask which columns identify a person (usually email; sometimes name plus company when emails are missing) and which row should win when duplicates disagree (newest, most complete, or a source column).

Steps

1. Normalise locally, writing to a new column email_normalized and leaving the original untouched: - Trim spaces, remove surrounding <> and mailto:, lowercase the whole address. - Remove obvious junk: blank cells, values without exactly one @, trailing punctuation. - Split cells that hold two addresses (a@x.com; b@x.com) into separate rows only if the user agrees.
2. Build a dedupe key from email_normalized. For matching only (not for sending), you may also strip a +tag from the local part and, for gmail.com and googlemail.com, remove dots before the @. Tell the user if you do this.
3. Group rows by key. Merge each group into one row: keep the winning row's values and fill its empty cells from the others. Add duplicate_count and merged_from (row numbers).
4. For rows with no email, dedupe on lowercased name plus company domain, and mark them match_type: name_company so the user can review.
5. Save the deduped file, then flag it with Ravn. Dry run, sample, full:

npx ravn enrich deduped.csv --tool validate_email --column email_normalized --dry-run
npx ravn enrich deduped.csv --tool validate_email --column email_normalized --limit 10 --out sample.csv
npx ravn enrich deduped.csv --tool validate_email --column email_normalized --out clean.csv

1. Over MCP, a deduped list of up to 1,000 can go in one validate_emails call, which already returns one verdict per unique address:

{ "emails": ["info@acme.com", "jane@gmail.com", "x7@mailinator.com"] }

1. Read the flags: rw_role_account, rw_free_provider, rw_disposable, rw_catch_all, plus rw_verdict. Add a single contact_flag column: disposable, role, free, invalid or blank.

Output

clean.csv with every original column plus:

• email_normalized, duplicate_count, merged_from, match_type
• rw_status (ok, not_found, error, dropped), rw_verdict, rw_risk_score, rw_reason
• rw_role_account, rw_free_provider, rw_disposable, rw_catch_all
• contact_flag

And a summary: rows in, rows out, duplicates merged, and counts of role, free, disposable and invalid.

Rules

• Never delete the user's original file. Write new files.
• Do not rewrite an address the person gave (for example removing Gmail dots) in the sending column. Those rules are for matching only.
• Keep opt-out and unsubscribe status when merging: if any duplicate opted out, the merged row is opted out. See /privacy#optout.
• Drop disposable, invalid and risky addresses from any send list.
• No FCRA uses. On a 429 fair_use_exhausted, stop and tell the human; on exit code 4, the 5-hour window is full and the run will wait unless --no-wait was set.

Example prompt

I merged three exports into contacts-all.csv and there are loads of duplicates. Clean up the emails, merge the dupes, and tag which ones are generic inboxes, Gmail, or throwaway addresses.