A Shipment Poller Must Commit Its Page Before Advancing Its Cursor

A shipment adapter retrieves a page of changes, saves the next cursor, then crashes before applying the page. On restart it resumes from the saved cursor. The service looks healthy, but the skipped changes never reached the local store.

The checkpoint should describe durable work, not the furthest HTTP request the adapter attempted.

The shipment API, records and worker protocol below are hypothetical. They are not a claim about a Zenith Eclipse deployment or any carrier's supported endpoints.

Establish what the provider actually promises

Before implementing polling, document whether the provider offers a snapshot, a change feed, an updated-since filter or only offset pagination. Ask how it represents deletions, late corrections, repeated records, expired tokens and changes made while pagination is in progress.

Microsoft Graph provides a useful primary example of a change-feed contract: its delta-query documentation distinguishes continuation links from completed-round delta links, treats state tokens as opaque, and warns about replays and synchronization resets. Those are Graph-specific semantics, not a shipping standard.

A carrier adapter must use its own documented completion signal. An empty page is not necessarily the end. A page with no records and a continuation token may still require another request.

Make page application and checkpoint advancement atomic

Maintain one synchronization state per authenticated account and query scope. Include the adapter version and filters in that identity: a token captured for one scope must not silently become the starting point for another.

An illustrative worker protocol is:

read checkpoint and generation
fetch page using the recorded continuation
validate the provider response
begin local transaction
  lock synchronization state
  require the same checkpoint and generation
  apply records using provider-specific revision rules
  record the page outcome
  advance continuation or completed-round checkpoint
commit

The HTTP request happens outside the database transaction. After the response arrives, the worker checks that another worker has not advanced the state. A stale worker discards its response or reconciles through the adapter's defined rules; it must not move the checkpoint backwards.

The state lock and generation check provide local fencing. A lease alone is insufficient if an expired worker can still commit after a replacement worker starts.

Decide what “applied” means for a bad record

If one record fails validation, do not log it and advance as though everything succeeded. Either roll back the page, or durably quarantine the rejected record with enough retained information for repair and replay.

The second choice needs an explicit policy: the checkpoint means every record is either applied or recoverably quarantined. It does not mean the local shipment view is complete. Show unresolved records in monitoring and downstream completeness indicators.

Zenith Eclipse's logistics control-tower coordination scope makes the operational distinction explicit: missing or outdated information is different from a confirmed delay. In this hypothetical adapter, expose both the last confirmed milestone and synchronization completeness. The service page does not promise continuous tracking or an existing software integration.

Do not allow a corrected duplicate to be discarded merely because its object ID already exists. Object identity, revision identity and delivery identity are different. A provider may return a partial update rather than a full replacement, so omitted fields must follow its documented meaning.

Recover without manufacturing certainty

A crash before commit should leave the old checkpoint. A crash after commit should leave the new checkpoint and corresponding local data. Re-fetching a page may happen; local application must tolerate the provider's documented replay behavior.

An expired token requires the provider's recovery procedure. For a full rebuild, a useful design is a separate synchronization generation that becomes current only after completion and reconciliation. Do not clear live shipment records simply because a fresh first page omits them.

If the API only offers updated-since polling, overlapping windows can reduce boundary misses but cannot guarantee completeness without assumptions about clock precision, late visibility and stable pagination. State that limitation and add periodic reconciliation where supported.

Test the checkpoint boundary

Inject failures before page validation, halfway through local application and immediately after commit. Include an empty continuation page, repeated objects, a partial update, a token reset and two workers competing for the same state.

For domain-specific fixtures, Zenith Eclipse's container rail freight scope identifies terminal receipt, train departure, border, arrival and release events as information becomes available. Model those as separate observations in a synthetic test; do not infer an unreported event from a later checkpoint.

Track last successful round completion separately from last HTTP success. A poller returning 200 responses can still be stuck on one page or accumulating quarantine records.

The goal is a restartable adapter whose checkpoint can be explained: everything before it has a durable local outcome, and everything unresolved remains visible.

---

About the publisher: Zenith Eclipse Co works across agricultural commodities, international logistics and machinery sourcing.