Skip to content
Learn Python with BNHR

Stretch · Check and Store

On this page
  1. Overview
  2. S2.1 · The contract as a class
  3. S2.2 · The clean round as an Excel file, codes intact
  4. S2.3 · Several rounds, all or nothing
  5. S2.4 · A warning, written as a pandera check

Every exercise in 82_stretch_day2.py, with the solution and why it works. Each answer stays closed until you swear you tried it first. Opening one changes nothing in your notebook; your check still has to go green from your own code. Read the whole notebook, with its lessons.

Stretch S2.1. The contract as a class

The task

pandera can also write a schema as a class, one attribute per column, which some teams find easier to read. Write SurveyModel, a pa.DataFrameModel with the same column rules as your survey_schema for age, sex, hours and weight, and the each-person-once rule (class Config: unique = ["hhid", "member_no"]). Columns look like age: Series[float] = pa.Field(ge=0, le=120, nullable=True, coerce=True).

The check runs SurveyModel.validate(table, lazy=True) on a small table and compares which rows fail.

If the check is red

Row 3 is all missing values, which is allowed; rows 0 and 4 are the same person.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the check's hint above covers the usual slip. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Stretch S2.2. The clean round as an Excel file, codes intact

The task

Colleagues will ask for Excel. Write write_excel(clean, path): write the table to an .xlsx file so that, opened again, every code column (hhid, area_code, region, sex, employed) is still text, leading zeros included, and return the number of rows written. pandas writes text as text; the trap is reading it back without dtype=str, and Excel itself turning 0100105000 into a number when someone types over it. As a second sheet named about, write one row per column with the column's name and a description you choose.

The check writes a small table with your function and reads it back with dtype=str.

If the check is red

Two sheets, 'round' and 'about', with codes read back unchanged.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the check's hint above covers the usual slip. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Stretch S2.3. Several rounds, all or nothing

The task

load_round keeps one round all-or-nothing. A quarterly release may need several rounds to change together: if one fails, none should. Write load_rounds(con, table, rounds), where rounds is a dictionary {round_id: rows}: one transaction for all of them; on any error, roll back and raise. Return the total number of rows of those rounds in the table afterwards.

If the check is red

When the second round fails, the first must not change either: H0 must still be there.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the check's hint above covers the usual slip. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Stretch S2.4. A warning, written as a pandera check

The task

warnings_for compares weights in plain Python. Write it as pandera does it: weight_shift_check(previous_mean, limit) returns a pa.Check on the weight column that warns (raise_warning=True) when the column's mean differs from previous_mean by more than limit (0.10 = 10 %). A check on a whole column receives the column, a Series, and returns True (fine) or False.

The check validates two small tables with a schema using your check, and counts the warnings.

If the check is red

One warning for the second table, none for the first; and the validation must not fail.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the check's hint above covers the usual slip. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Something new

Identify results
LayerExplore a module
ModuleValues and names
AboutModule 2 of Python Essentials: numbers, text and true/false, names for them, f-strings and formats, and reading an error message. Its sections, its notebooks to read on the web, and the companion's solutions and explanations.
CoursePython Essentials, free