As an early-stage founder or business owner, there are many available ETL options today. If you're using Fivetran, once you surpass 500K monthly active rows (MARs) your bill can jump up over $100/mo and quickly compound to thousands per month. Here’s how to easily migrate over to a cheaper ETL tool (in this case, Airbyte) in under an hour.

Freeze your Fivetran account

This happens automatically if you hit Fivetran’s 500K MARs (the sum of rows synced to your database in a month) threshold for the second time. Billing info is prompted. Even if you reduce data sync volume after hitting the 500K limit, you’re obligated to pay ~$120/mo for 200k MARs on Fivetran’s ‘Starter’ plan. When switching to Airbyte, you pay ~$5/mo for 200k MARs (200 MB). $10/GB of data synced per mo + $15/million rows synced per mo.

Check how much data you have

Understand your sync volume by running a quick script (first below) in your SQL database (e.g. BigQuery, Metabase) measuring data across your schema. To add additional tables to the query, just add a new union all and select sum row below and customize for your desired table name. Take the sum of all rows to see volume in GBs.

To see a full list of your tables and columns, use the second script below.

select sum(size_bytes)/pow(10,9) as size from postgres.__TABLES__
union all
select sum(size_bytes)/pow(10,9) as size from hubspot.__TABLES__
union all
select sum(size_bytes)/pow(10,9) as size from shopify.__TABLES__
union all
select sum(size_bytes)/pow(10,9) as size from quickbooks.__TABLES__

--

select * from `database-name.postgres.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`
union all 
select * from `database-name.hubspot.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`
union all 
select * from `database-name.shopify.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`
union all 
select * from `database-name.quickbooks.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS`

Decide which data is business critical

Metafield and product event data irrelevant to core KPIs can be large and drive up cost (e.g. hubspot.product_properties; shopify.metafield). Leave these out of ETL pipelines if possible. You can always add in later if you’re unsure.

Setup your Airbyte account

Navigate to Airbyte’s website. Sign up with email using a shared team account (e.g. [email protected]). This allows you to (1) generate and store a secure password in a shared vault and (2) allow other stakeholders to access if needed.