Most Fivetran problems are not dramatic. The connector does not explode; it quietly falls behind. A Salesforce sync starts taking ninety minutes instead of nine, a schema change blocks a table, an OAuth token expires over a long weekend, and the first person to notice is a sales director staring at a dashboard that says yesterday when it should say this morning.
Email alerts help, but they are per-event and easy to filter into oblivion. What you actually want is the same thing you have for the rest of your stack: a table you can query, a dashboard you can look at, and alerts that fire on the conditions your business cares about rather than the ones Fivetran ships by default.
Fivetran gives you the raw material for free. The Fivetran Platform Connector syncs your account's own metadata and logs into your destination, where it is just SQL. This tutorial builds a working observability layer on top of it: freshness tracking, failure detection, cost attribution and alerting.
What the Platform Connector is
The Platform Connector (previously called the Fivetran Log connector) is a connector you create like any other, except its source is Fivetran itself. It lands in a schema — commonly fivetran_platform or fivetran_log — and it does not consume MAR. It is one of the few things in the platform that is genuinely free to run, and most accounts still do not have it enabled.
Set it up once per destination:
- In the dashboard, add a connector and search for Fivetran Platform.
- Choose the destination and a schema name. Use the same name in every environment so your SQL is portable.
- Start the initial sync. History goes back to whatever retention your account has; the log tables fill in quickly.
Set its sync frequency to 15 minutes. Alerting on metadata that is six hours stale defeats the purpose.
The tables that matter
There are a couple of dozen tables. In practice you will use five:
| Table | What it holds |
|---|---|
connection (or connector) | One row per connection: name, service, destination, paused flag, sync frequency |
destination | One row per destination in the account |
log | The event stream — sync starts, sync ends, errors, warnings, schema changes |
incremental_mar | Daily monthly-active-row counts per connection, schema and table |
transformation_runs | dbt/transformation execution history, if you run transformations in Fivetran |
The log table is the heart of it. Each row has a time_stamp, a connection_id, an event type (INFO, WARNING, SEVERE), a message_event (the machine-readable event name) and a message_data JSON blob whose shape depends on the event.
Two event names carry most of the value: sync_start and sync_end. A sync_end record's message_data includes the outcome, so a completed sync and a failed one are distinguishable without parsing prose.
Naming varies slightly by Fivetran version and destination — some accounts have
connectorandconnector_id, newer ones haveconnectionandconnection_id. Run aSELECT *against your own schema first and adjust the column names below to match. Everything else in the logic is the same.
1. A sync history view
Start by flattening the log into one row per sync attempt. This is the single model everything else builds on.
create or replace view analytics.fivetran_sync_history as
with starts as (
select
connection_id,
time_stamp as started_at,
row_number() over (
partition by connection_id order by time_stamp
) as seq
from fivetran_platform.log
where message_event = 'sync_start'
),
ends as (
select
connection_id,
time_stamp as ended_at,
json_extract_path_text(message_data, 'status') as status,
row_number() over (
partition by connection_id order by time_stamp
) as seq
from fivetran_platform.log
where message_event = 'sync_end'
)
select
c.connection_name,
c.connection_type as service,
d.destination_name,
s.started_at,
e.ended_at,
coalesce(e.status, 'RUNNING_OR_LOST') as status,
datediff('second', s.started_at, e.ended_at) as duration_seconds
from starts s
left join ends e
on s.connection_id = e.connection_id
and s.seq = e.seq
join fivetran_platform.connection c on c.connection_id = s.connection_id
join fivetran_platform.destination d on d.destination_id = c.destination_id;
json_extract_path_text is Snowflake/Redshift syntax; on BigQuery use json_value(message_data, '$.status'), on Databricks get_json_object(message_data, '$.status').
The RUNNING_OR_LOST bucket is deliberate. A sync that started and never ended is either in flight or died in a way that produced no end event, and the second case is exactly the failure mode that silently breaks a dashboard. Do not filter it out; that is the interesting row.
2. Freshness, per connection
Failures are obvious. Staleness is not, and staleness is what business users actually feel.
create or replace view analytics.fivetran_freshness as
select
connection_name,
service,
destination_name,
max(case when status = 'SUCCESSFUL' then ended_at end) as last_success_at,
datediff(
'minute',
max(case when status = 'SUCCESSFUL' then ended_at end),
current_timestamp()
) as minutes_since_success,
max(started_at) as last_attempt_at
from analytics.fivetran_sync_history
group by 1, 2, 3;
Now attach an expectation. Do not derive the threshold from the configured sync frequency alone — a connection scheduled hourly that takes fifty minutes to run is not late at seventy minutes. A workable rule is three times the schedule interval, floor of 90 minutes, then override the handful of connections that have a real SLA:
create or replace view analytics.fivetran_freshness_sla as
select
f.*,
coalesce(o.sla_minutes,
greatest(90, c.sync_frequency * 3)) as sla_minutes,
case
when f.minutes_since_success is null then 'NEVER_SUCCEEDED'
when f.minutes_since_success >
coalesce(o.sla_minutes, greatest(90, c.sync_frequency * 3))
then 'BREACHED'
else 'OK'
end as sla_status
from analytics.fivetran_freshness f
join fivetran_platform.connection c
on c.connection_name = f.connection_name
left join analytics.fivetran_sla_overrides o
on o.connection_name = f.connection_name;
fivetran_sla_overrides is a small seed table you maintain by hand — connection name, SLA minutes, owning team, whether it should page anyone. Keep it in your dbt project so it goes through code review. It is the cheapest way to encode "the finance close pipeline matters at 6am and the marketing sandbox does not."
3. Failure and error patterns
select
c.connection_name,
l.message_event,
count(*) as events,
max(l.time_stamp) as last_seen
from fivetran_platform.log l
join fivetran_platform.connection c using (connection_id)
where l.event = 'SEVERE'
and l.time_stamp >= dateadd('day', -7, current_date)
group by 1, 2
order by events desc;
Read this weekly rather than reacting to it hourly. Patterns show up that individual alerts hide: one connection producing sixty schema-change warnings a week is a source team shipping migrations you have not been told about, and that is a conversation, not an incident.
Watch for schema_change events in particular. If you have set connections to BLOCK_ALL schema change handling — as we recommend in the Terraform provider guide — new source tables land here as a notification rather than as a surprise line on your invoice.
4. Duration drift, before it becomes a failure
Syncs rarely fail without warning. They get slower first.
with daily as (
select
connection_name,
date_trunc('day', started_at) as day,
avg(duration_seconds) as avg_seconds,
count(*) as runs
from analytics.fivetran_sync_history
where status = 'SUCCESSFUL'
and started_at >= dateadd('day', -30, current_date)
group by 1, 2
)
select
connection_name,
avg(case when day >= dateadd('day', -7, current_date)
then avg_seconds end) as last_7d,
avg(case when day < dateadd('day', -7, current_date)
then avg_seconds end) as prior_23d,
round(
avg(case when day >= dateadd('day', -7, current_date) then avg_seconds end)
/ nullif(avg(case when day < dateadd('day', -7, current_date)
then avg_seconds end), 0), 2
) as ratio
from daily
group by 1
having ratio > 1.5
order by ratio desc;
Anything above 1.5 deserves a look. Common causes: a source table that grew past the point where the incremental key is selective, a replication slot falling behind, a warehouse that is now sharing a virtual warehouse with something noisy, or a connector that quietly switched from incremental to a full table sync. Our performance tuning work usually starts with exactly this query.
5. Cost attribution from incremental_mar
Because MAR is billed per connection, the log connector doubles as a cost dashboard.
select
c.connection_name,
m.schema_name,
m.table_name,
sum(m.incremental_rows) as mar_this_month
from fivetran_platform.incremental_mar m
join fivetran_platform.connection c using (connection_id)
where m.measured_date >= date_trunc('month', current_date)
group by 1, 2, 3
order by mar_this_month desc
limit 25;
Run it on the first working day of every month. The top of that list is almost always two or three tables nobody reads — an audit log, an event stream, a _history table replicated because it was easier than asking. Turning those off is the fastest cost win available, and the mechanics of why are in our connection-level MAR guide.
A month-over-month variance query is worth having too:
select
c.connection_name,
sum(case when date_trunc('month', m.measured_date)
= date_trunc('month', current_date) then m.incremental_rows end) as this_month,
sum(case when date_trunc('month', m.measured_date)
= date_trunc('month', dateadd('month', -1, current_date))
then m.incremental_rows end) as last_month
from fivetran_platform.incremental_mar m
join fivetran_platform.connection c using (connection_id)
group by 1
order by this_month desc;
Compare like-for-like: partial current month against full prior month will always look reassuring and always be wrong. Either normalise by day count or wait until month end.
6. Alerting that people do not mute
Metadata in a table is not an alert. Two mechanisms, used for different things:
Webhooks for events. Register an account-level webhook for sync_end and connection_failure and route it to your incident tool. This is the fast path — seconds, not minutes. Do it in code so every new connection inherits it.
Scheduled queries for conditions. Freshness breaches, duration drift and MAR spikes are states, not events, and no webhook will tell you about them. Run fivetran_freshness_sla every fifteen minutes from your scheduler and post breaches to Slack:
select connection_name, minutes_since_success, sla_minutes
from analytics.fivetran_freshness_sla
where sla_status in ('BREACHED', 'NEVER_SUCCEEDED')
order by minutes_since_success desc;
Three rules that decide whether anyone still reads these in six months:
- Route by owner, not to one channel. The overrides table already has the team column. Use it.
- Suppress paused connections. A deliberately paused staging connector is not an incident; join on
connection.paused = false. - Page for exactly one thing. Freshness breaches on connections marked as pager-worthy. Everything else is a daily digest. An alert that pages for a warning is an alert that gets muted, and the mute outlives the person who set it.
7. Putting it on a dashboard
Four tiles cover ninety percent of the need:
- Traffic light by connection —
sla_statusfrom the freshness view, sorted worst first. This is the tile that goes on the wall. - Sync success rate, last 7 days — successful syncs over total attempts, per connection.
- Duration trend — a line per connection over 30 days, log scale so a slow small connector and a slow big one both stay visible.
- MAR by connection, month to date — with last month as a reference band.
Build it wherever your users already are. Adding a fifth tool to the stack so people can watch the fourth tool is how observability projects die. If you would rather not build it yourself, this is standard work for our dashboard and report development team.
Limits worth knowing
- Log retention is finite. If you want multi-year trends, snapshot the log and MAR tables into your own history tables. A daily incremental model with a permanent target does it in a few lines.
- The Platform Connector syncs on a schedule. Alerting from it inherits that latency. Sub-minute detection needs webhooks.
- Field names drift between versions. Pin your models with tests — a dbt
not_nulltest onconnection_idin the sync history model will fail loudly the day a column is renamed, which is much better than a dashboard silently going empty. - It reports; it does not fix. Triggering re-syncs, resuming connections and rotating credentials happen through the REST API — see our Fivetran API development page if you want the remediation half automated too.
A one-week rollout
Day 1: enable the Platform Connector on every destination, 15-minute schedule. Day 2: build the sync history view and eyeball a week of data. Day 3: create the overrides seed table and agree SLAs with the three teams who complain most. Day 4: ship freshness alerts to Slack, routed by owner. Day 5: build the dashboard, then run the MAR query and cancel something.
That is a week of work that changes who finds out first when a pipeline breaks — you, instead of your CFO.
If you would like the whole layer built, tested and handed over with runbooks, our maintenance and support for Fivetran platforms team does this as a fixed-scope engagement. Get in touch with your destination type and connection count and we will tell you what it takes.