NOETRION

Choose a topic

LESSON 3 OF 9 · Python & data foundations
DATA WORKFLOW · 4 MIN READ

Read a CSV in Python without losing track of your data

Load a small file, inspect its columns, and handle the two common surprises: delimiters and missing values.

Reading a CSV is easy; knowing what your calculation includes is the real task. A file can contain quoted commas, blank scores, text identifiers, and values that cannot be converted. We will read a small assessment file and produce a summary that shows accepted and excluded records explicitly.

The examples use Python 3.12's standard library. Names and scores are synthetic. No personal records or external services are involved. Save the sample below as scores.csv beside your Python file.

Start with a file you can check by hand

student_id,name,score
001,"Ari, Jr.",80
002,Bela,90
003,Cici,
004,Deni,absent
005,Eka,100

The first row provides field names. Ari's name contains a comma inside quotes, so it still belongs to one field. Splitting each line with line.split(",") would mishandle that record. The csv module understands CSV quoting and should do the parsing.

Inspect raw values before calculating

import csv

with open("scores.csv", encoding="utf-8", newline="") as file:
    reader = csv.DictReader(file)
    for row in reader:
        print(row["student_id"], repr(row["name"]), repr(row["score"]))

DictReader associates each field with its header. In this ordinary reading mode, fields such as 80 remain strings. The reader does not know that one column represents an identifier and another represents a score. Preserve 001 as text; converting it to an integer would remove meaningful leading zeroes.

newline="" lets the CSV module handle newline conventions, including records containing quoted newlines. Specifying UTF-8 makes our encoding assumption explicit. A file from another system may use a different encoding, which you should establish rather than guess silently. The context manager closes the file when the block finishes.

Write a clear acceptance rule

For this example, an accepted record has a numeric score from 0 to 100, inclusive. Blank scores and unrecognized text are excluded with different reasons. This is our chosen assessment contract; another dataset may need another range or an explicit status column.

import csv
from statistics import mean

accepted = []
excluded = []
rows_seen = 0

with open("scores.csv", encoding="utf-8", newline="") as file:
    reader = csv.DictReader(file)
    required = {"student_id", "name", "score"}
    if reader.fieldnames is None or not required <= set(reader.fieldnames):
        raise ValueError("The file is missing required columns.")

    for row in reader:
        rows_seen += 1
        student_id = row["student_id"]
        raw = row["score"]
        if raw is None or not raw.strip():
            excluded.append((student_id, "missing score"))
            continue
        try:
            score = float(raw)
        except ValueError:
            excluded.append((student_id, "not numeric"))
            continue
        if not 0 <= score <= 100:
            excluded.append((student_id, "outside score range"))
            continue
        accepted.append(score)

assert len(accepted) + len(excluded) == rows_seen
print("Accepted:", len(accepted))
print("Excluded:", excluded)
print("Mean:", mean(accepted) if accepted else "No accepted scores")

For the sample, three scores are accepted: 80, 90, and 100. Their mean is 90. Cici has a missing score; Deni has a nonnumeric entry. Neither is automatically scored zero. The statistics documentation explains the mean calculation and its behavior on an empty dataset.

Reconcile the result with the source

Every sample record has an explained outcome
Record IDRaw scoreDecision
00180Accept
00290Accept
003BlankExclude: missing
004absentExclude: not numeric
005100Accept
Parse CSVCheck schemaValidate each scoreReconcile counts
Original teaching diagram: parsing and validation answer different questions.

The assertion checks an important invariant: every parsed record must end up in exactly one outcome. It catches an accidentally forgotten branch. It does not prove the input file is correct or the acceptance rule is appropriate. A well-balanced report can still summarize the wrong population.

Describe the denominator in your report: “Mean of three accepted scores; two records excluded.” Reporting only “mean 90” hides missingness. If missing scores are concentrated among students struggling with the assessment, that accepted-score mean may not represent the whole class.

Know this beginner reader's boundaries

The example checks required column names and score values. A production import should also decide what to do with duplicate headers, repeated IDs, extra fields, malformed rows, and an unexpected delimiter. This example deliberately does not claim to be a complete data-validation framework.

Do not use the physical line number as a guaranteed record count: a quoted field may span several lines. Keep the original file, document the encoding and delimiter, and retain the reason each record was excluded. If a file is sensitive, inspect it locally instead of pasting it into an unrelated online service.

Practice with a changed file

  1. Add an ID 006 with score 110. Predict accepted and excluded counts.
  2. Replace the blank score with 0. Explain why that changes the mean.
  3. Delete the score header. What should happen before any summary is printed?
Check your answers

A score of 110 is outside our range: three accepted and three excluded records. Replacing the original blank with zero gives four accepted scores and mean 67.5; it changes unknown into a measured value. A missing required header raises the schema error. A blank value and zero are different facts.

Sources and further reading

Source links checked October 13, 2026. Worked examples are original teaching examples; they are not research findings or product benchmarks.

Check your understanding

What type does csv.DictReader return for a score such as 82?

Choose one answer