Before You Restart, Do You Know What the Failure Affected?

When a CDC pipeline fails, the pressure is to get replication moving again. But before choosing a recovery path, the team needs to understand what changed while the pipeline was down.

The source may have continued processing. The target may be increasingly stale. Downstream applications may have continued operating, stopped entirely, or taken compensating action. Required source history may still be available, or the normal recovery path may already be changing.

The question isn’t just: How do we restart?

It’s: What did this failure affect, and what does that mean for recovery?

FEMA continuity guidance describes business process analysis as mapping the workflows, activities, personnel expertise, systems, data, and facilities required to perform an essential function. That same dependency mindset is useful when assessing the downstream reach of a CDC interruption.*

A CDC incident impact assessment helps establish:

• The period and processing state affected
• The data that did not reach the target
• The downstream systems and business processes exposed
• The conditions that constrain available recovery options
• The open questions that must be resolved or explicitly managed

The goal isn’t to delay recovery. It’s to avoid choosing a recovery path based on an incomplete picture. 

What should I do immediately after a CDC replication job stops? 

Before deciding how to recover, determine what was affected. A CDC incident impact assessment helps establish the affected recovery window, the data and downstream consumers exposed, the size and significance of the resulting data gap, the conditions constraining available recovery paths, and the questions or risks that must be resolved before moving forward.

Only then can the team choose a recovery path based on evidence rather than assumptions.

Quick answer: What should a CDC incident impact assessment include?

A practical CDC incident impact assessment should answer five questions:

1. What is the affected recovery window?
2. Which data and downstream consumers were exposed?
3. How large and significant is the resulting data gap?
4. What conditions constrain the available recovery paths?
5. What unanswered questions or risks must be resolved or managed before recovery proceeds?

The sections that follow describe the process used to answer those questions. The goal isn’t to complete a checklist of activities. It’s to build enough evidence to choose an appropriate recovery path and establish the baseline for post-recovery validation.

The depth of the assessment should reflect the scope and risk of the incident. A contained interruption may require only a brief review, while a complex outage involving critical data, manual interventions, or significant downstream exposure will require deeper investigation. 

Define the incident window

Once the last trusted processing point is established, define the period the assessment needs to cover. The incident window usually extends from the last known trusted processing point through the interruption to the current target or recovery state. Depending on the incident, the window may include:

• The period before someone detected the problem
• The time replication was interrupted or degraded
• Continued source activity during the interruption
• Partial processing or retry attempts
• Target-side changes or interventions
• Initial restart or catch-up activity

Don’t assume that the time someone noticed the failure is the time the failure began.

Ask:

• When was replication last known to be operating as expected?
• When did processing begin slowing, failing, or stopping?
• When was the issue detected?
• Did the source continue processing transactions?
• Did capture, transport, or target apply continue partially?
• Did anyone attempt a restart or make configuration changes?
• Did the target receive any direct or compensating updates?
• What point represents the current state being assessed?

Why it matters

A five-minute interruption detected immediately is different from a five-minute interruption discovered hours later after downstream systems continued using stale data. The incident window establishes what the team must investigate. It also becomes the baseline for choosing a recovery path and validating the data afterward. 

Map the affected data

Not every table, file, dataset, or data domain within a replication stream carries the same risk. Start by identifying what was in scope when replication slowed or stopped. Then determine what actually changed during the incident window. For each affected object or data domain, consider:

Change activity

• Were inserts, updates, or deletes occurring?
• How much change activity accumulated?
• Were supported schema changes made?
• Did the same records change multiple times?
• Were high-value or business-critical transactions involved?

Business importance

• Does the data support reporting, settlement, claims, payments, inventory, customer service, fraud detection, or another critical process?
• Is the data subject to governance, audit, or regulatory requirements?
• How current must the target be for downstream use?

Data relationships

• Do multiple tables or files need to remain synchronized?
• Does a header-detail, parent-child, or other dependency matter?
• Would recovering only part of the affected data create an inconsistent state?

Target-side activity

• Did any process write directly to the target?
• Were records patched manually?
• Was data loaded from another source?
• Did a downstream team attempt a workaround?

Where related objects must remain consistent, group them according to those dependencies. Recovery and validation should reflect the relationships that matter to the application or business process, not simply alphabetical order or row counts.

The practical question to ask in every situation: “Which data matters most in this incident, and which relationships must be preserved during recovery?”

Applying the same response to every table can waste time on low-risk data while overlooking a small number of critical dependencies. 

Assess downstream exposure

A CDC failure doesn’t affect only the replication process. If the source continues operating, downstream applications may remain available while their data becomes increasingly stale. Other consumers may stop entirely. Some may continue processing partial or inconsistent information. For each downstream consumer, document:

What the consumer experienced

• Did the application stop receiving data?
• Did it continue operating on stale data?
• Did it receive only part of the expected information?
• Did users see an error, or did the problem remain hidden?
• Did an expected event or update fail to occur?

Whether compensating action occurred. During an extended interruption, teams may:

• Patch target data manually
• Load data from a backup or alternate source
• Temporarily query the system of record
• Pause a business process
• Reprocess transactions outside the normal workflow
• Override an automated decision

Those actions matter because they may alter the target or create conflicts with changes delivered later through replication.

What replay or catch-up may trigger

Some downstream systems can accept delayed changes without side effects. Others initiate business logic when an event arrives.

Depending on the application, replaying accumulated changes could trigger:

• Notifications
• Payments
• Inventory adjustments
• Workflow transitions
• Service requests
• Downstream integrations
• Duplicate actions

In many environments, this is fundamentally an idempotency question: can the consumer safely process a replayed event without creating unintended business consequences?

Why it matters

A technically recoverable pipeline can still create a broader business problem. The source may be healthy. The target may be available. But downstream users may be making decisions from stale or incomplete data, and recovery activity may trigger consequences outside the replication platform. 

Quantify the data gap

Once the incident window and affected scope are understood, describe the gap in terms useful to both technical and business stakeholders.

Technical measures may include:

• Elapsed time since the last trusted processing point
• Current replication lag
• Change or transaction backlog
• Inserts, updates, and deletes generated during the gap
• Source position compared with current processing position
• Processing or target-apply throughput
• Available catch-up capacity
• Relevant source-history availability
• Estimated catch-up time based on observed performance

Avoid treating estimated recovery time as a guarantee. Actual catch-up performance may vary based on source activity, network capacity, target pressure, infrastructure contention, change volume, and competing workload.

Business measures may include:

• Transactions not yet visible to a downstream system
• Reports or dashboards operating on stale information
• Business processes delayed or paused
• Decisions or actions taken using outdated data
• Customer or regulatory deadlines approaching
• Manual interventions performed during the gap
• Consumers requiring special handling during replay
• Financial, operational, service, or compliance exposure

Raw technical metrics help the operating team scope recovery. Business-impact measures help stakeholders understand why the incident matters and how recovery should be prioritized.

Why it matters

A backlog of 100,000 changes may be routine in one environment and business-critical in another. The size of the gap matters. So does what the gap contains and who depends on it. 

Confirm what recovery depends on

A gap can be measurable while the appropriate recovery path remains uncertain. Before choosing a restart, catch-up, targeted resynchronization, or broader reload, document the conditions that affect which options remain available.

Source-history availability: Determine what source information the supported recovery path depends on. That information could include:

• Active transaction logs
• Archived logs
• Journals
• Retained change history
• Restart or process state
• Source-specific log positions
• Platform-managed recovery information

If the required source history is no longer immediately available, confirm whether archived logs or other retained history can be retrieved and processed through a supported recovery path. If the necessary history is unavailable, recovery options may become more limited.

Don’t assume the presence of archived logs automatically means a particular recovery path is supported. Confirm how the platform uses that history for the specific source, target, and configuration.

Target state

Determine whether the target remains in a condition that supports recovery forward.

Ask:

• Did the target remain available?
• Were direct writes made?
• Were records patched manually?
• Did target structures or schemas change?
• Did any process partially reload or resynchronize data?
• Can the target be protected from additional changes during recovery if needed?

Consumer replay behavior: For consumers that initiate an action when data arrives, determine whether delayed or repeated events can be processed safely.

Ask:

• Can the consumer recognize already processed events?
• Could replay trigger duplicate actions?
• Are transaction identifiers or business keys available?
• Can notifications, payments, or workflows be suppressed temporarily?
• Does the consumer team need to coordinate the recovery?

Recovery objectives: Compare the estimated catch-up or recovery time with the objectives established for the affected systems.

Ask:

• How long can downstream users tolerate stale data?
• Is a business, reporting, settlement, or customer deadline approaching?
• Does estimated catch-up fit within the expected recovery window?
• If not, what alternative action or stakeholder decision is needed?
• If no formal recovery objective exists, document the business tolerance for the current incident instead of inventing a generic threshold.

Why it matters

Knowing the gap size doesn’t tell the team how to recover. The team also needs to understand what the supported recovery path requires, whether those conditions still exist, and what risks recovery itself may introduce. 

Document open questions and decision constraints

An impact assessment should make uncertainty visible, not hide it. Document questions such as:

• Is the restart position reliable?
• Is all required source history available?
• Did the target receive direct writes?
• Are any manual patches in conflict with source changes?
• Can accumulated events be replayed safely?
• Which data requires deeper validation?
• Which downstream teams need to approve resumption?
• Are any exceptions acceptable for a controlled recovery?

Open questions should be assigned and evaluated according to the risk they introduce. Some may block recovery. Others may be managed as explicit exceptions with defined owners, controls, and follow-up actions.

The objective isn’t to create artificial certainty. It’s to ensure the recovery decision reflects what is known, what is still unknown, and which assumptions the team is making. 

Produce the CDC incident impact summary

The assessment should conclude with a concise written summary that becomes the input to the recovery decision and the baseline for post-recovery validation. Include:

Incident window

• Last known trusted processing point
• Confirmed or estimated incident start
• Time the issue was detected
• Current assessment boundary
• Total elapsed period

Affected data

• Sources and targets involved
• Tables, files, datasets, or data domains affected
• Change volume during the gap
• Relevant dependencies
• Any direct target writes or manual changes

Downstream exposure

• Applications, reports, APIs, analytics, or processes affected
• Whether consumers stopped, continued on stale data, or received partial information
• Any compensating actions taken
• Replay or catch-up risks

Data-gap assessment

• Current lag or backlog
• Estimated catch-up effort
• Transactions or business events not yet reflected downstream
• Business deadlines or operational consequences

Recovery constraints

• Source-history availability
• Restart or process-state availability
• Target condition
• Consumer replay considerations
• Expected recovery window
• Platform or configuration limitations

Open questions

• What remains unknown?
• Which questions affect the recovery decision?
• Who owns each question?
• Which questions block recovery?
• Which may be managed as documented exceptions?

Recommended next decision

The impact assessment should not automatically prescribe a specific recovery path. It should provide enough information for the team to determine whether to:

• Resume from saved state
• Process retained historical changes
• Continue monitored catch-up
• Compare source and target data
• Resynchronize selected data
• Perform a broader resynchronization or reload
• Delay action until a critical question is resolved
• Escalate the decision to additional technical or business owners 

What a completed assessment looks like

The example below is illustrative. The names, values, workload, and recovery considerations are synthetic and don’t represent a recommended threshold, architecture, or recovery procedure.

Incident summary

Pipeline: Orders-to-warehouse replication stream 
Last trusted processing point: Confirmed target checkpoint corresponding to 2:14 a.m. 
Failure detected: 2:17 a.m. 
Assessment completed: 4:00 a.m. 
Current gap: Approximately 1 hour and 46 minutes

Affected data

Downstream exposure

Recovery constraints

• Required source history remains available.
• The target contains manual inventory updates made during the interruption.
• Delayed events may trigger customer notifications during catch-up.
• Order header and detail data must be recovered and validated together.
• Downstream owners need a clear communication and replay plan.

Open questions

• How should manual target updates be reconciled with source changes delivered during catch-up?
• Should delayed customer notifications be released, suppressed, or handled separately?
• What comparison will confirm order header and detail consistency after recovery?

Assessment conclusion

The available restart state and retained source history support further evaluation of a catch-up recovery path. Before recovery proceeds, the team must agree on how to handle manual target updates and delayed notifications.

The impact assessment doesn’t prove that recovery will succeed. It establishes the conditions the recovery decision must address.

Common CDC impact-assessment mistakes

Treating pipeline status as the incident boundary: A failed or degraded process may report an operational state that doesn’t correspond to the last change successfully reflected at the target. Use the most reliable source, platform, and target evidence available to establish the assessment boundary.

Measuring only elapsed downtime: Downtime alone doesn’t explain how much the target changed relative to the source, which data was affected, or which downstream systems continued operating. Measure both technical lag and downstream exposure.

Assuming change volume equals recovery time: Change volume is only one factor. Observed throughput, record size, source activity, target latency, infrastructure pressure, transformation complexity, and competing workloads also affect catch-up duration.

Stopping at the technical layer: An assessment that documents backlog and log position but ignores downstream consumers is incomplete. The business consequence determines urgency, sequencing, communication, and acceptable risk.

Ignoring compensating actions: Manual patches, alternate loads, workarounds, and direct target writes may change the recovery assumptions. Document them before replication begins delivering accumulated changes.

Hiding uncertainty: Unanswered questions don’t disappear when recovery starts. Document what remains unknown, determine which uncertainties block action, and manage the rest as explicit exceptions.

How the impact assessment fits the recovery workflow

The CDC incident impact assessment sits between detection and recovery execution.  

It supports:

The restart decision: The assessment establishes the failure scope, available processing state, source-history dependencies, target condition, and downstream considerations needed to choose a supported recovery path.

Read more: The Most Dangerous CDC Recovery Mistake Happens After the Failure

Catch-up monitoring: The expected backlog, source activity, downstream tolerance, and catch-up conditions documented in the assessment help the team interpret recovery progress.

Read more: Your CDC Pipeline Isn’t Down. Are You Sure It’s Healthy?

Post-recovery validation: The incident window, affected data, dependencies, and compensating actions identified in the assessment define what must be validated after catch-up.

Read more: The Pipeline Is Running Again. Can You Trust the Data?

Downstream communication: The consumer map and impact summary clarify who needs updates, which data remains restricted, and what evidence will support the eventual all-clear. The assessment converts an undefined failure into a bounded incident with a known scope, explicit constraints, and a clearer path to recovery.

Use the assessment after the incident

The impact assessment shouldn’t disappear when recovery begins. After the target has been validated and returned to use, revisit the assessment to identify what should change before the next interruption.

• Which information took too long to find?
• Which dependencies were missing or poorly documented?
• Which downstream teams learned about the issue too late?
• Which decisions depended on one specialist?
• Which manual interventions created reconciliation risk?
• Did the monitoring environment show the actual data and business impact?
• Did the platform provide enough restart, diagnostic, catch-up, and validation information?
• Should source-history retention or retrieval processes change?
• Which thresholds, runbooks, ownership assignments, or communication procedures should be updated?
• Could the same failure produce the same uncertainty again?

A single incident may expose a process gap. Repeated incidents may point to a larger monitoring, operating-model, training, or platform-capability problem.

The objective isn’t only to recover from this interruption. It’s to make the next response faster, clearer, and easier to defend. 

CDC incident impact assessment checklist

Before choosing a recovery path, confirm that the team has established:

• The last known trusted processing point
• The incident window
• The affected sources and targets
• The data changed during the gap
• Important relationships and dependencies
• Downstream systems and business processes exposed
• Whether stale or partial data was used
• Manual or compensating actions taken
• Current lag and backlog
• Required source-history availability
• Target-side changes or constraints
• Consumer replay or duplicate-action risks
• Expected recovery and catch-up requirements
• Open questions and assigned owners
• Questions that block recovery
• Exceptions being managed explicitly
• The evidence that post-recovery validation will require

The assessment is ready to inform the recovery decision when the team can explain what happened, what was affected, what recovery depends on, what remains uncertain, and what must be true before action proceeds. 

In definitiva

A CDC failure isn’t defined only by how long replication was unavailable. It’s defined by what changed during the gap, who was affected, what recovery options remain available, and what risks recovery itself may introduce.

The critical question isn’t simply: How do we get replication moving again?

It’s: Do we understand the impact well enough to choose the right recovery path?

A good impact assessment gives the team a shared understanding of:

• Where the incident begins
• What the incident affected
• Which conditions constrain recovery
• What questions remain unresolved
• What evidence will be needed afterward

That is the difference between restarting based on assumptions and recovering from a problem the team actually understands.

FEMA: Business Process Analysis and Business Impact Analysis