Skip to content
Learn Python with BNHR

Debugging drill: the report that goes wrong

On this page
  1. Overview
  2. D2.1 · A mean of means
  3. D2.2 · A join that multiplies rows
  4. D2.3 · Plots that drop out of the totals
  5. D2.4 · Dates read month-first
  6. D2.5 · Square meters mixed with hectares

Every exercise in 45_debugging_drill.py, with the solution and why it works. Each answer stays closed until you swear you tried it first. Opening one changes nothing in your notebook; your check still has to go green from your own code. Read the whole notebook, with its lessons.

Drill D2.1. A mean of means

The task

Why it matters: the headline of a survey report is a figure for everyone the survey stands for. A figure that gives a small province the same say as a large one describes no one.

The report has a table of each province's weighted rice yield, made by province_yields(rice), below; that table is right. Under it, weighted_yield(rice) should give the headline: the weighted average yield of all the rice plots of the seven provinces, each plot's yield counted as many times as its weight says.

Input → output: the rice table → the weighted average of yield_t_ha, weighted by weight, in tonnes per hectare, to two decimal places.

call should give
weighted_yield(rice) 4.5 (the report's 4.50 t/ha)

Tools: weighted means (module 4, section 3 (30_summarize_and_combine.py)).

Predict. The function averages the province table. Nueva Ecija has the highest yield and by far the largest weights. Will its answer be above or below the right one?

If the check is red: common causes

what you see the usual cause look at
4.25 The mean of the seven province yields: each province counts once. the weighted mean over the plots
4.35 The plain mean of the plots' yields: the weights are left out. multiply by weight, divide by its sum
A number far above 5 The weighted sum was divided by the number of plots, not by the sum of the weights. rice["weight"].sum()

Still stuck after ten minutes? Ask your table, then raise your hand.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the common causes above cover most red checks. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Drill D2.2. A join that multiplies rows

The task

Why it matters: a weighted total multiplies every plot by its household's weight. A plot counted twice adds its harvest twice, and nobody reading the total can tell.

rice_harvest_t(plots, households) should give the estimated rice harvest of all the farm households the survey stands for: each rice plot's production_kg times its household's weight, added up, in tonnes. The weights come from households with a join on hh_id.

Input → output: the plots and households tables → the weighted total rice harvest, in whole tonnes.

call should give
rice_harvest_t(plots, households) 3752516

Tools: joins and validate= (module 4, section 5 (30_summarize_and_combine.py)); duplicates (module 5, section 5 (40_cleaning_and_time.py)).

Predict. There are 1,034 rice plots. How many rows will the join give?

If the check is red: common causes

what you see the usual cause look at
A total a little too big Households in the table twice: their plots are joined, and counted, twice. drop_duplicates() before the join
MergeError: Merge keys are not unique in right dataset; not a many-to-one merge validate= found the households in twice, as it should. Now remove them. drop_duplicates() on the weights
KeyError: 'weight' Only hh_id was kept from households, or the weights were renamed on the way. households[["hh_id", "weight"]]

Still stuck after ten minutes? Ask your table, then raise your hand.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the common causes above cover most red checks. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Drill D2.3. Plots that drop out of the totals

The task

Why it matters: a table of totals by group should add up to the grand total. When it doesn't, a reader either loses trust in the table or, worse, doesn't notice.

The enumerators left crop blank for 14 plots. harvest_by_crop(plots) should give the harvest of the survey's plots by crop, in tonnes, with the plots whose crop was not recorded in a row of their own, its crop missing (NaN), so the rows add up to every plot's harvest.

Input → output: the plots table → one value per crop, in tonnes, plus one for the missing crop.

crop harvest (t)
Banana 2,627.40
Corn 2,044.36
Rice 3,840.65
Vegetables 1,046.39
NaN 64.79

Tools: grouping and missing keys (module 4, section 1 (30_summarize_and_combine.py)).

Predict. How many rows will the function give?

If the check is red: common causes

what you see the usual cause look at
Four rows, no NaN groupby dropped the plots with no crop. groupby's dropna=
A row named not recorded The blanks were filled with a label. The task keeps the missing key as NaN. dropna=False instead of fillna
Values in kilograms The division by 1,000 was lost. / 1000

Still stuck after ten minutes? Ask your table, then raise your hand.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the common causes above cover most red checks. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Drill D2.4. Dates read month-first

The task

Why it matters: the survey's fieldwork ran from October to December 2025. A table that shows interviews in March describes fieldwork that never happened.

The file writes each interview date day first: 19/12/2025 is 19 December 2025, as the codebook says. interviews_by_month(dates) should turn the texts into dates and count the interviews in each month.

Input → output: the interview dates as text → the number of interviews in each month, months in order.

call should give
interviews_by_month(interview_dates).to_dict() {Period('2025-10', 'M'): 381, Period('2025-11', 'M'): 384, Period('2025-12', 'M'): 425}

Tools: parsing day-first dates, and periods (module 5, section 3 (40_cleaning_and_time.py)).

Predict. 09/11/2025 and 19/12/2025 are two of the dates. Which month will the function put each one in?

If the check is red: common causes

what you see the usual cause look at
Months from January to December The dates are read month first. format=
Still wrong with dayfirst=True A format= is given too, and the format wins. the order of %d and %m
ValueError: time data "…" doesn't match format errors="coerce" was removed while the format is still wrong. The error names a date the format cannot read. format=

Still stuck after ten minutes? Ask your table, then raise your hand.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the common causes above cover most red checks. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Drill D2.5. Square meters mixed with hectares

The task

Why it matters: a yield in tonnes per hectare needs every area in hectares. A few areas in another unit move the average without making it look strange.

mean_yield(plots, crop) should give the average yield of one crop's plots, unweighted: each plot's harvest in tonnes divided by its area in hectares, then the mean of those yields, to two decimal places. In plots, 151 areas are in square meters (area_unit is "sqm"); 10,000 square meters make a hectare.

Input → output: the plots table and a crop → the mean yield of that crop's plots, in tonnes per hectare.

call should give
mean_yield(plots, "Rice") 4.35

Tools: one unit for a measure (module 5, section 2 (40_cleaning_and_time.py)).

Predict. Run Try it. Does its answer look like a rice yield?

If the check is red: common causes

what you see the usual cause look at
4.1 for rice Areas in square meters are used as hectares. area_unit, and divide those areas by 10,000
Thousands of tonnes per hectare Every area was divided by 10,000, hectares too. divide only where area_unit is "sqm"
0.0 or a tiny yield Areas multiplied by 10,000 instead of divided. 10,000 square meters make 1 hectare

Still stuck after ten minutes? Ask your table, then raise your hand.

The solution, and why it works

Try it yourself first: the check names the input that failed, and the common causes above cover most red checks. Open this when you have been stuck for ten minutes, or when your check is green and you want to compare.

Something new

Identify results
LayerExplore a capstone
CapstoneCapstone
AboutThe capstone of Arrays & DataFrames: in groups, build something that fixes a real problem with the farm survey or the prices, then show it working.
CourseArrays & DataFrames, free