Time-Series Validation: Make Label Availability and the Gap Visible

Reviewed October 6, 2026. Examples use Python 3.12.14 and scikit-learn 1.8.0. Dates, features, and label delays are synthetic. The gap is derived from a stated scenario, not a universal recommendation.

A time-series model trains on earlier rows and tests on later rows. The evaluation seems safe because the row order is chronological. Yet a training label summarizes an outcome that would not have been known when the first test prediction was made. Ordering the prediction origins did not establish the availability of their labels.

Temporal evaluation needs at least three clocks: when a prediction is made, when its inputs become available, and when its target outcome becomes observable. A splitter can arrange row positions; it cannot infer those clocks from your business process.

Practical rule: state the prediction schedule and information cutoff first, then derive training membership and any gap from availability. Validate folds against timestamps, fit transformations only on training data, and preserve a final period that is not used for model selection.

Define the prediction origin and horizon

The prediction origin is the time at which you would issue a prediction. The horizon describes the future outcome the prediction concerns. Label availability can occur after that horizon if outcomes arrive late or require processing.

In our scenario, each row represents one daily prediction origin. Its label becomes available two days later. The model must be trained strictly before the first test origin, so training labels available exactly at that origin are not allowed. A system permitting an update at the exact cutoff would have a different contract.

A two-day label delay is an assumption for this example. It is not automatically the same as a two-day forecast horizon. Record both when they differ, and account for processing or reporting delays when constructing a real evaluation dataset.

Inspect actual split membership

The TimeSeriesSplit reference documents gap as a count of samples excluded between training and test portions. It is not a calendar-day parameter. Equal-sized test folds span comparable durations only when the samples are appropriately equally spaced.

from datetime import date, timedelta
from sklearn.model_selection import TimeSeriesSplit

dates = [date(2026, 1, 1) + timedelta(days=i) for i in range(12)]
label_ready = [day + timedelta(days=2) for day in dates]
expected = [([0, 1, 2, 3], [6, 7]),
            ([0, 1, 2, 3, 4, 5], [8, 9]),
            ([0, 1, 2, 3, 4, 5, 6, 7], [10, 11])]
folds = []
for train, test in TimeSeriesSplit(n_splits=3, test_size=2, gap=2).split(dates):
    cutoff = dates[test[0]]
    assert max(label_ready[index] for index in train) < cutoff
    assert train[-1] < test[0]
    folds.append((train.tolist(), test.tolist()))
assert folds == expected
print(folds)

The first training period is smaller because the fold sizes, gap, and total sample count constrain membership. Do not assume every fold has a fixed-length training window. Use max_train_size when a bounded rolling history is the intended design, and document why it is appropriate.

Derive the gap rather than copying a number

Under the scenario's strict cutoff, the latest training origin must have its two-day-delayed label ready before the first test origin. Gap two satisfies that inequality for equally spaced daily rows. Gap one does not, because the latest label arrives exactly at the first test origin.

from sklearn.model_selection import TimeSeriesSplit

origins = list(range(12))
for gap in [0, 1, 2]:
    checks = []
    for train, test in TimeSeriesSplit(n_splits=3, test_size=2, gap=gap).split(origins):
        checks.append(int(train[-1]) + 2 < int(test[0]))
    if gap == 2:
        assert all(checks)
    else:
        assert not any(checks)
    print("gap", gap, "label availability checks", checks)

The result establishes this constructed sample-based availability rule. If labels arrive with variable delays, compare each label-ready timestamp to the actual cutoff instead of relying on a fixed row gap. Irregularly spaced events can make the same sample count represent very different calendar durations.

A gap also reduces available training evidence. Its size should follow the information and dependence assumptions rather than be maximized automatically. For overlapping targets, related observations, or entity-specific processes, a more specialized split design may be necessary.

A chronological split does not repair future-derived features

A feature calculated from a centered moving average can include observations after the prediction origin. A full-dataset normalization can incorporate future distribution information. A final revised measurement may differ from the preliminary value actually available at the time.

The splitter will still produce chronological indices, but the feature values can already contain future information. Construct features using the information snapshot available at each origin and fit learned preprocessing only on the training portion.

A lag feature also needs a boundary definition. If today's measurement is only published tomorrow, a one-row lag might still be unavailable at today's prediction time. Availability depends on the data collection process, not the appealing name of the column.

Match the test schedule to deployment

An evaluation that fits once at the start of a two-day test block answers a different question from one that refits every day using newly arrived labels. TimeSeriesSplit defines membership, not your operational retraining schedule. Specify whether the model stays fixed during a test block or updates at each permitted origin.

Forecasting several future steps from one origin differs from making repeated one-step predictions at later origins. If you use later test observations to construct features, that may be valid for a rolling one-step deployment but invalid for a multi-step forecast issued entirely at the initial origin.

State the horizon for each prediction and how outcomes are scored. A test_size of two means two records per fold; it does not itself define a two-day forecast horizon. Row count, calendar duration, prediction frequency, and target window are separate dimensions.

Compare models without repeatedly spending the final period

Use earlier chronological folds for method selection and preserve a later final evaluation period. Choosing features, gap policy, hyperparameters, and the best-looking metric after repeatedly inspecting the final period makes its score an optimistic development result.

Compare each candidate with a simple baseline under the same information availability and forecast schedule. A last-known-value baseline can be informative when that value was actually available; it becomes invalid if it uses a later revised observation or a target still in the future.

Report fold-level results, not just one pooled average. Different test periods can have different conditions. If you combine scores, state whether you weight by cases, periods, or another decision-relevant quantity. Missing targets must be audited instead of quietly altering the evaluation denominator.

Verify the timeline directly

Check that timestamps are sorted, duplicates have a stated tie policy, training origins precede test origins, and every training label is available at cutoff. Test the equality boundary explicitly. Inspect feature-ready timestamps where the collection process is delayed.

Then save the exact fold membership, model version, feature policy, horizon, and scoring population. The two examples here validate split and availability behavior without fitting a model; they make no claim about forecasting accuracy. That separation keeps the design evidence honest.

Sources and reproducibility

Primary references: scikit-learn 1.8 TimeSeriesSplit, cross-validation, and common pitfalls and preprocessing leakage. Both examples run independently with scikit-learn. Dates and two-day readiness are fixed teaching assumptions.