Every Fivetran project eventually has the same meeting. Somebody from the business says the dashboard needs to be "real time", somebody from the data team says the connector already syncs every 15 minutes, and nobody writes down what either side actually means. Six weeks later there is a 1-minute sync schedule on nine connectors, a Snowflake bill that doubled, and a report that is still wrong because the upstream application only writes the field once a night.
This tutorial is about latency as an engineering decision: where the delay in a Fivetran pipeline actually comes from, what each sync frequency costs, how far you can push Fivetran toward near-real-time, and when the honest answer is that Fivetran is the wrong tool for that one table.
Where the latency actually lives
End-to-end freshness is not the sync frequency. It is the sum of five separate delays, and teams usually tune the one that matters least.
| Stage | Typical contribution | What controls it |
|---|---|---|
| Source commit to source log | Seconds | Source config: WAL flush, binlog, Oracle redo, Mongo oplog |
| Fivetran picks up the change | Up to one sync interval | Your sync frequency setting |
| Extract + load duration | Seconds to hours | Table volume, change volume, connector type |
| Destination MERGE / apply | Seconds to many minutes | Warehouse size, table size, clustering, MERGE strategy |
| Downstream transformation | Minutes to hours | dbt / Quickstart model schedule |
A connector on a 5-minute schedule whose sync takes 11 minutes to run is not a 5-minute pipeline. Fivetran will not start a new sync while the previous one is running, so the effective interval collapses to roughly the sync duration. Then dbt runs hourly on top, so the number the business sees is "up to 70 minutes old", not five.
Before changing anything, measure. The Fivetran Platform Connector gives you the raw material in your own warehouse:
-- Effective interval and duration per connection, last 7 days
select
connection_id,
count(*) as syncs,
avg(datediff('second', sync_start, sync_end)) as avg_sync_seconds,
max(datediff('second', sync_start, sync_end)) as max_sync_seconds,
avg(datediff('second',
lag(sync_start) over (partition by connection_id order by sync_start),
sync_start)) as avg_gap_seconds
from fivetran_log.sync_log_summary
where sync_start >= dateadd('day', -7, current_timestamp())
group by 1
order by avg_sync_seconds desc;
If avg_sync_seconds is close to avg_gap_seconds, your connector is already running back-to-back and tightening the schedule buys you nothing at all. That query ends a surprising number of arguments.
What sync frequency really costs
Fivetran's scheduler supports intervals from 1 minute up to 24 hours. The temptation is to move everything to the floor. Three things push back.
MAR usually does not change much, but it can. Monthly Active Rows count distinct primary keys changed in a month, so syncing the same updated row twice in an hour does not double-bill you. But syncing more often does capture intermediate states you previously collapsed: a row edited five times in an hour is one active row either way, while a row that bounces between states across hour boundaries can land in more monthly buckets. The bigger exposure is on API connectors with re-sync windows - ad platforms in particular, where every sync re-pulls a rolling attribution window. There, frequency multiplies work directly. We covered that pattern in attribution windows, re-syncs and the MAR they cost you.
Destination compute changes a lot. This is the real bill. Every sync is at minimum a staged file write plus a MERGE per changed table. On Snowflake, 96 syncs a day becomes 1,440 at one-minute cadence, each one waking a warehouse that bills a 60-second minimum. Small, frequent MERGEs against a large table also scan far more micro-partitions per changed row than one consolidated batch does. The arithmetic behind that is in what Fivetran costs you inside Snowflake; on BigQuery the equivalent trap is partition pruning and streaming-ish write patterns, covered in the BigQuery destination guide.
Source load changes for some connectors. CDC-based database connectors are cheap to poll frequently - they read a log position. API connectors are not: each sync spends its rate-limit budget again. A Salesforce connector on a 1-minute schedule will eat daily API call quota that the rest of the business also needs.
A rule of thumb that holds up in practice: tighten the schedule on log-based database connectors, leave API connectors at 15 minutes or slower unless there is a named business reason.
Tiering your connectors instead of tuning them one by one
Do not set frequency per connector by instinct. Define three tiers, write them down, and assign every connection to one.
- Tier 1 - operational (1 to 5 minutes). Log-based CDC on a handful of tables that drive an operational decision: fraud checks, inventory availability, on-call dashboards, reverse-ETL audiences that gate a customer-facing action. Expect to pay for it. Keep this list short enough to name every table in it.
- Tier 2 - analytical (15 to 60 minutes). The default. Nearly everything an analyst or a BI tool touches belongs here. Hourly is enough for almost every dashboard anyone has ever actually looked at before lunch.
- Tier 3 - reporting (6 to 24 hours). Finance systems, HR, slow-moving reference data, anything whose source only updates nightly. Syncing a nightly batch source every 15 minutes is pure waste; align the schedule to the source's own cadence instead, and use a daily start time just after it lands.
The tiering conversation is also where you discover the thing nobody said out loud: the "real-time" requirement is often one table, not one hundred. Splitting those tables into their own connection lets you run that connection hot and leave everything else cheap. Separate connections also isolate failure - a 1-minute connector that breaks does not block the hourly load of the other 200 tables.
Pushing Fivetran toward near-real-time, properly
If you genuinely need single-digit-minute freshness, these are the levers that work, in order of return.
1. Split the hot tables into a dedicated connection. Sync duration is dominated by the widest, busiest tables in the connection. Ten small operational tables in their own connection will finish in seconds; the same ten bundled with a 400-million-row fact table inherit its runtime.
2. Make sure CDC is genuinely incremental. Confirm the connector is reading the log and not falling back to a periodic full table scan - teleport/full-table modes exist and are quietly slow. Check that the source retains enough log history, that the replication slot is healthy, and that no table lacks a primary key (those tend to degrade to full comparisons). The Postgres CDC walkthrough covers the setup details.
3. Size the destination warehouse for the MERGE, not the extract. Frequent small syncs on an undersized warehouse queue. A larger warehouse that finishes in 20 seconds often costs less per day than a small one grinding for four minutes, because credits are consumed per second of uptime, not per unit of work.
4. Trigger transformations on sync completion, not on a clock. A 1-minute pipeline feeding an hourly dbt job is an hourly pipeline. Use Fivetran's sync-completion webhooks or the API to trigger the downstream run, or let an orchestrator wait on the connector. Orchestrating Fivetran syncs with Airflow and Dagster has the sensor and webhook patterns.
5. Shorten what runs downstream. Incremental dbt models on the hot tables only. A full-refresh model at the end of a 1-minute pipeline is a punchline.
6. Alert on freshness, not on failure. Once latency is a promise, measure it like one. Track timestamp_diff(now, max(_fivetran_synced)) per critical table and alert at your SLA threshold - that catches the silent case where syncs "succeed" but one table stopped changing. See the observability dashboard guide.
When Fivetran is the wrong tool for that one table
This is the part consultants are supposed to say out loud. Fivetran's model is scheduled micro-batch. It is excellent at it. It is not a streaming bus, and the floor is roughly one minute end-to-end plus your warehouse's apply time.
If the requirement is sub-second - a live fraud decision, a trading signal, an event-driven microservice - no amount of schedule tuning gets you there, and trying produces an expensive pipeline that still misses the SLA. In those cases the right architecture is usually:
- Kafka / Kinesis / Pub-Sub plus a streaming sink for the genuinely event-driven path, with Fivetran continuing to own the analytical copy of everything else.
- Reading the operational store directly for the one lookup that needs to be current, rather than replicating it faster.
- Warehouse-native streaming ingestion (Snowpipe Streaming and equivalents) for high-velocity event data, keeping Fivetran for the relational and SaaS sources it handles better than anything else.
A hybrid is normal and nothing to apologise for. Most mature stacks run Fivetran for 95% of sources on a 15-minute to hourly cadence, with one narrow streaming path for the handful of events that justify the operational overhead.
A latency SLA you can actually sign
Write it per data product, not per connector, and state all four numbers:
Order fact table. Freshness target: 95% of orders queryable within 10 minutes of commit, 99% within 30 minutes. Measured as
_fivetran_syncedminus sourceupdated_at. Source:ordersconnection, 5-minute schedule, dedicated connection. Downstream:fct_ordersincremental, triggered on sync completion. Breach alert: Slack#data-oncallat 30 minutes.
That paragraph is worth more than any schedule setting. It converts "real time" into a measurable number, names the owner, and gives you something to test against when someone asks for one minute instead of five - you can quote the cost of the change rather than argue about the word.
Quick checklist
- Measure current effective interval from
sync_log_summarybefore changing any schedule. - Assign every connection to a tier; justify each Tier 1 in one sentence.
- Split hot operational tables into their own connections.
- Verify CDC is log-based and primary keys exist on the hot tables.
- Trigger transformations on sync completion rather than a fixed clock.
- Alert on
_fivetran_syncedage per critical table. - Re-cost the destination warehouse after any frequency change, one week later.
SyncSpur tunes Fivetran pipelines for the freshness a business actually needs, without the bill that comes from guessing. If your pipelines feel slow, or fast and expensive, we can benchmark the current latency profile and give you a tiered schedule with the numbers attached - see performance tuning in Fivetran and Fivetran cost optimization & MAR management, or just get in touch.