ScraperCity + Airtable

Create a new base in Airtable and add a table called Leads with these fields:

The Source field helps you track which ScraperCity scraper produced each lead. The Status field gives your sales team a simple workflow to manage outreach directly in Airtable.

Airtable uses Personal Access Tokens (PATs) for API authentication. Legacy API keys are no longer supported. To create a PAT:

1. Go to airtable.com/create/tokens (the Airtable Developer Hub).
2. Click + Create new token and give it a name like "ScraperCity integration".
3. Under Scopes, add data.records:read and data.records:write.
4. Under Access, select the specific base you just created.
5. Click Create token and copy it immediately - Airtable will only show it once.

Your Base ID is the string starting with app found in the URL when you open the base: airtable.com/appXXXXXXXX/...

Store both values as environment variables - never hard-code credentials in scripts.

export AIRTABLE_TOKEN="patXXXXXXXXXXXXXX"
export AIRTABLE_BASE_ID="appXXXXXXXXXXXXXX"

Here is the complete two-step flow using curl - query ScraperCity, then batch-create Airtable records:

# Step 1 - Pull leads from ScraperCity (any endpoint works the same way)
curl -X GET "https://app.scrapercity.com/api/v1/database/leads?title=CTO&country=United%20States&hasEmail=true&limit=10" \
  -H "Authorization: Bearer YOUR_SCRAPERCITY_KEY"

# Step 2 - Push a batch of up to 10 records to Airtable
curl -X POST "https://api.airtable.com/v0/YOUR_BASE_ID/Leads" \
  -H "Authorization: Bearer YOUR_AIRTABLE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "records": [
      {
        "fields": {
          "Name": "Jane Smith",
          "Email": "[email protected]",
          "Phone": "+15551234567",
          "Title": "CTO",
          "Company": "Acme Corp",
          "LinkedIn": "https://linkedin.com/in/janesmith",
          "Source": "Lead Database",
          "Status": "New"
        }
      }
    ]
  }'

The Airtable REST API returns a 200 with the created record IDs on success and a 422 if a required field is missing or formatted incorrectly.

For loading hundreds or thousands of leads, use pyairtable - the recommended Python client for the Airtable API. It handles batch chunking, rate-limit retries, and pagination automatically.

pip install pyairtable requests

Then create a script that pulls from ScraperCity and writes to Airtable in batches:

import os
import math
import time
import requests
from pyairtable import Api

# --- Credentials (set as environment variables) ---
SCRAPERCITY_KEY = os.environ["SCRAPERCITY_KEY"]
AIRTABLE_TOKEN  = os.environ["AIRTABLE_TOKEN"]
AIRTABLE_BASE   = os.environ["AIRTABLE_BASE_ID"]
AIRTABLE_TABLE  = "Leads"

# --- 1. Pull leads from ScraperCity Lead Database ---
def fetch_scrapercity_leads(title: str, country: str, limit: int = 100) -> list[dict]:
    url = "https://app.scrapercity.com/api/v1/database/leads"
    headers = {"Authorization": f"Bearer {SCRAPERCITY_KEY}"}
    params  = {"title": title, "country": country, "hasEmail": "true", "limit": limit}
    resp = requests.get(url, headers=headers, params=params)
    resp.raise_for_status()
    return resp.json().get("leads", [])

# --- 2. Map ScraperCity fields to Airtable fields ---
def map_lead(lead: dict) -> dict:
    return {
        "Name":     lead.get("name", ""),
        "Email":    lead.get("email", ""),
        "Phone":    lead.get("phone", ""),
        "Title":    lead.get("title", ""),
        "Company":  lead.get("company", ""),
        "LinkedIn": lead.get("linkedinUrl", ""),
        "Source":   "Lead Database",
        "Status":   "New",
    }

# --- 3. Batch-create records in Airtable (max 10 per call) ---
def push_to_airtable(leads: list[dict]) -> None:
    api   = Api(AIRTABLE_TOKEN)
    table = api.table(AIRTABLE_BASE, AIRTABLE_TABLE)
    records = [map_lead(l) for l in leads]

    chunk_size = 10
    total_chunks = math.ceil(len(records) / chunk_size)

    for i in range(0, len(records), chunk_size):
        chunk = records[i : i + chunk_size]
        table.batch_create(chunk)
        print(f"Pushed chunk {i // chunk_size + 1}/{total_chunks} ({len(chunk)} records)")
        # pyairtable retries 429s automatically, but a small sleep keeps
        # you well under the 5 req/sec per-base rate limit
        time.sleep(0.25)

# --- Run ---
if __name__ == "__main__":
    leads = fetch_scrapercity_leads(title="VP of Sales", country="United States", limit=500)
    print(f"Fetched {len(leads)} leads from ScraperCity")
    push_to_airtable(leads)
    print("Done.")

The batch_create() method accepts a list of field dictionaries and handles the 10-record-per-request limit internally. pyairtable automatically retries requests when it receives a 429 rate-limit response from Airtable.

If you run the same scrape multiple times, you will create duplicate rows unless you use Airtable's upsert feature. The Airtable REST API supports a performUpsert option on the PATCH endpoint that updates the matching record if one is found, or creates a new one if not. Use Email as the merge key since it is unique per contact.

# Upsert via raw REST (PATCH instead of POST)
curl -X PATCH "https://api.airtable.com/v0/YOUR_BASE_ID/Leads" \
  -H "Authorization: Bearer YOUR_AIRTABLE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "performUpsert": {
      "fieldsToMergeOn": ["Email"]
    },
    "records": [
      {
        "fields": {
          "Name": "Jane Smith",
          "Email": "[email protected]",
          "Title": "CTO",
          "Company": "Acme Corp",
          "Source": "Lead Database"
        }
      }
    ]
  }'

With pyairtable, use table.batch_upsert(records, key_fields=["Email"]) instead of batch_create(). The response includes separate createdRecords and updatedRecords arrays so you can log exactly what changed.