AI Batch Jobs: Reconcile Partial Results Before Retrying the Work

A thousand documents go into an overnight AI job. The batch finishes with several hundred useful results, some errors, and a few unresolved items. Resubmitting the entire input is easy. It can also pay for accepted work again and create conflicting versions of the same record.

The essential batch-processing skill is reconciliation: accounting for every requested item before deciding what happens next. A job status describes the container. Your application needs an item-level ledger that distinguishes a returned response, a validated result, a retryable failure, and work that has not yet been accounted for.

The provider documentation checked on October 20, 2026 makes this distinction practical. OpenAI's Batch guide describes output ordering, error files, and partial results after expiry. Claude's Message Batches guide has its own item outcomes and limits. The workflow below is original application design; the offline example is not a claim that these APIs share one response format.

Decide whether the work can wait

Batch processing fits independent work whose result can arrive later: classifying an authorized archive, running an evaluation suite, or extracting fields for a report due tomorrow. It is a poor substitute for an immediate interactive response when the user cannot proceed without it.

Define the latest useful completion time, not merely the provider's processing window. Leave time to retrieve files, validate outputs, review exceptions, and run any fallback. A provider deadline that coincides exactly with your business deadline leaves no recovery margin.

Check the selected model, endpoint, region, retention settings, and current account limits. These are provider-specific facts. A familiar synchronous request does not establish that the same feature, data policy, or response timing applies to its batch counterpart.

Create a manifest before submitting anything

The manifest is the list of work you intended to do. Each item needs an immutable logical identifier, an input version, a task or prompt version, and a destination record. Preserve the exact input or a suitable reference and integrity digest under your data policy.

A custom ID is a join key, not proof that the provider will deduplicate repeated submissions. Do not put personal names or confidential details into an identifier merely for convenience. Store the authorized mapping in your system.

Keep a stable logical key across attempts while assigning attempt records separately. This lets you answer both “which invoice is this?” and “which execution produced this output?” Combining those questions in one mutable status makes later investigation harder.

Match by identity, not output position

OpenAI explicitly warns that output line order can differ from input order and supplies custom_id for matching. It separates successful output lines from failed-request information in an error file. Its guide also explains that completed results remain available when a batch expires, with completed work charged.

The OpenAI FAQ separately describes cancellation and expiry returning already completed work. Thus “expired” does not mean “nothing happened,” and a cancellation request is not evidence that every in-flight operation instantly stopped.

Claude documents per-request outcomes including succeeded, errored, canceled, and expired. Its current documentation also imposes its own custom-ID character and length rules. Use an adapter for each provider instead of treating a normalized application record as the provider's wire response.

Before retries, obtain a terminal job state, retrieve all available result and error artifacts, and verify that retrieval and parsing completed. If a file download failed halfway through, “not seen locally” is not evidence that the original request never ran.

Separate transport success from acceptance

A response with HTTP status 200 may still contain a refusal, truncated content, an invalid schema, or a value that contradicts the source. Application acceptance should follow explicit checks appropriate to the task.

For a document extraction, retain missing-field markers and source references. Check quantities, currencies, and required columns. Do not silently convert an uncertain amount into zero just to complete the dataset. A well-formed response is useful evidence of delivery, not a certificate of correctness.

Classify failures carefully. A transient service problem may justify another attempt. A request with an unsupported parameter usually needs correction before resubmission. A content-quality failure may need human review or a different task design, rather than an identical retry.

Work an item-level reconciliation example

Suppose an original ten-item manifest receives seven accepted results, one permanent request error, one retryable service error, and no local outcome for one item. At a verified terminal reconciliation point, the next-action list contains two retry candidates: the retryable item and the unaccounted item. The permanent error goes to correction, not an automatic identical retry.

The following executable example consumes an application-normalized record shape. Its adapter is deliberately outside the example: real response bodies and errors must first be parsed and classified using the selected provider's schema and the application's acceptance rules.

def reconcile(expected_ids, records, *, terminal, retrieval_complete):
    expected = list(expected_ids)
    if (any(not isinstance(x, str) or not x for x in expected)
            or len(expected) != len(set(expected))):
        raise ValueError("manifest IDs must be unique nonempty strings")
    if not terminal or not retrieval_complete:
        raise ValueError("do not infer missing work before reconciliation")
    wanted = set(expected)
    seen = {}
    allowed = {"accepted", "retryable", "permanent", "review"}
    for record in records:
        key, outcome = record["custom_id"], record["outcome"]
        if key not in wanted or key in seen or outcome not in allowed:
            raise ValueError("unknown, duplicate, or invalid result")
        seen[key] = outcome
    missing = sorted(wanted - seen.keys())
    return {
        "accepted": sorted(k for k, v in seen.items() if v == "accepted"),
        "retry_candidates": sorted(missing + [k for k, v in seen.items()
                                  if v == "retryable"]),
        "permanent": sorted(k for k, v in seen.items() if v == "permanent"),
        "review": sorted(k for k, v in seen.items() if v == "review"),
        "missing": missing,
    }

expected = [f"item-{i:02d}" for i in range(10)]
records = [{"custom_id": key, "outcome": "accepted"}
           for key in reversed(expected[:7])]
records += [{"custom_id": expected[7], "outcome": "permanent"},
            {"custom_id": expected[8], "outcome": "retryable"}]
plan = reconcile(expected, records, terminal=True, retrieval_complete=True)
assert len(plan["accepted"]) == 7
assert plan["retry_candidates"] == ["item-08", "item-09"]
assert plan["permanent"] == ["item-07"]
assert plan["missing"] == ["item-09"]

Unknown IDs or duplicates stop this reducer rather than overwrite evidence. In production, conflicting records need investigation. The function does not prove that a missing result is unbilled, unprocessed, or safe to retry automatically; reconcile that uncertainty with provider status and artifacts before dispatch.

Calculate fallback economics on an equal workload

Use a separate synthetic scenario of 1,000 valid documents, each requiring one accepted result. Assume 800 finish in the batch and 200 expire without processing. After terminal reconciliation, the remaining 200 are completed synchronously. All outputs pass the same acceptance standard.

The hypothetical synchronous cost is $0.04 per document and batch cost $0.02 per completed document. Assume the unprocessed expired work is not charged in this example, and administration adds $3 per batch. These flat per-document costs simplify token billing; they are not current API rates.

The saving is $13, or 32.5%, while completing the same 1,000 documents. It is smaller than the discount on the 800 batched documents because fallback and administration remain.

Let x be the fraction completed at the batch rate. Under these assumptions, total cost is 20x + 40(1 − x) + 3, or 43 − 20x dollars. Beating the $40 synchronous baseline requires x > 0.15. At 40% completion, cost is $35; at zero completion, it is $43. This threshold belongs to this model, not to batch APIs in general.

Make retries bounded and writes idempotent

Set a retry budget and a latest dispatch time. Repeated transient failures should eventually become visible exceptions. Changing the prompt or input creates a new task version and should not quietly replace an accepted earlier result.

When persisting output, use a destination uniqueness rule tied to the logical item and task version, with an explicit policy for newer accepted attempts. This application-level rule can prevent duplicate writes even when external execution is repeated. It cannot guarantee exactly-once execution at the provider.

Avoid attaching irreversible business actions directly to unreviewed model output. A batch extraction can populate a reviewable record; sending a payment or modifying a customer account needs its own authorized workflow. Reconciliation should make the next step safer and more accountable.

Run a small pilot before scaling

Try a permitted sample containing easy inputs, difficult inputs, and deliberately invalid requests. Measure acceptance rate, unresolved items, retrieval completeness, review time, and final cost. Keep deadline failures visible rather than deleting them from the report.

For practice, add a duplicate accepted record to the example and confirm rejection. Then set retrieval_complete=False and explain why retries must stop. Finally, increase the fixed administration cost in the cost worksheet and recalculate the completion threshold. These tests connect a cheaper request rate to a workflow that actually finishes.

The document table audit offers concrete acceptance checks; the prompt caching worksheet covers a different optimization with different charging assumptions.

Sources and evidence limits

• OpenAI Batch guide, checked October 20, 2026: output matching, error artifacts, expiry, and charging for completed requests.
• OpenAI Batch FAQ: cancellation, status, and provider-specific processing context.
• Claude Message Batches documentation: its own result outcomes, identifiers, and limits.

The reducer was executed locally on synthetic records. No API batch, provider billing experiment, or third-party document upload was performed. Check current provider terms and limits before a real deployment.