Spatial Pipeline EngineeringMRV Automation

Designing Field Plot Sampling for Model Validation

Field plots are the most expensive data in a carbon project and the most frequently wasted. A network laid out for convenience — along roads, near the field station, in stands the team already knows — produces measurements that are perfectly accurate and nearly useless for validating a model, because the model’s errors do not live where the plots are. This guide covers designing a network that can support a validation claim, within ground truth alignment for carbon models in the spatial modeling and carbon stock validation stack.

The design question is narrower than it first appears, because a validation network has one job: to produce an unbiased estimate of the model’s error, with a stated precision, across the range of conditions the model will be applied to. That is a different job from estimating the stock itself, and it leads to a different layout. A network optimised for stock estimation concentrates effort where the stock is; a network optimised for validation concentrates effort where the model is uncertain, which is usually somewhere else entirely.

Three plot layouts over the same project, and what each can support Three panels showing the same project outline with different plot distributions. The first is convenience sampling: plots cluster along a road and near a field station, leaving most of the project unsampled; it can describe the stands visited and nothing else, and its bias is unquantifiable. The second is simple random sampling: plots scatter uniformly, giving an unbiased estimate overall but leaving rare strata with too few plots to say anything about, which is where model error is usually largest. The third is stratified random sampling with allocation weighted toward uncertain strata: plots are spread across all strata with extra effort in the regrowth and degraded classes, giving both an unbiased overall estimate and enough plots per stratum to detect a model that fails in one of them. A note reads that only the third supports a per-stratum error statement. Same budget, same number of plots, three different validation claims Convenience describes the stands visited bias present, magnitude unknowable supports: no validation claim Simple random unbiased overall estimate rare strata get 1–2 plots or none supports: one overall error figure Stratified, uncertainty-weighted unbiased with stratum weights extra effort where the model is weak supports: per-stratum error statements Only the third can answer “does the model work in regrowth?” — which is the question a verifier asks, because regrowth is where the credits are. The first two answer a question nobody asked.

Root Cause Analysis

Four design decisions determine whether a network can support a validation claim, and each one fails in a characteristic way when got wrong.

Stratification must follow the model’s uncertainty, not the landscape’s area. Allocating plots proportionally to stratum area is the instinctive choice and it is wrong for validation. The strata that matter are the ones where the model is least constrained — regrowth, degraded stands, transitional classes, the extremes of the biomass range — and those are usually small in area and large in leverage. Proportional allocation gives them two plots each and produces a validation that is silent about exactly the classes generating the credits.

Plot size must be commensurate with the model’s support. A 0.02 ha plot compared against a 30 m pixel is comparing a measurement of one small patch against a prediction about an area nine times larger, and the mismatch appears in the residuals as noise that has nothing to do with the model. The plot should be at least comparable to the pixel, and where that is impractical, several small plots should be aggregated to pixel support before comparison rather than compared individually.

Location accuracy has to be better than the pixel. A plot recorded with a handheld GPS under canopy can be off by fifteen metres or more, which under a 30 m pixel means the comparison is sometimes against the wrong pixel entirely. The resulting residual distribution has fat tails that look like model failure and are actually positional error, and no amount of model refitting removes them. Differential correction or a survey-grade receiver is the fix, and it is cheap relative to the cost of the plot itself.

Some plots must never enter the model. A network entirely consumed by calibration leaves nothing to validate with, and cross-validation on the calibration set answers a weaker question than an independent hold-out does. Reserving a share of plots before any modelling starts, chosen by the same design rather than by what is left over, is the difference between a validation and a goodness-of-fit statistic.

The common thread is that all four failures are locked in at design time and cannot be repaired by analysis. This is the rare part of an MRV pipeline where the engineering decision must be made before any data exists.

Diagnostic Pipeline / Pre-Flight Validation

Before committing a survey budget, check that the proposed design can deliver the precision the claim needs, and that each stratum will receive enough plots to say anything about. Both checks are arithmetic and both routinely reject a design that looked adequate.

import math
from dataclasses import dataclass

import structlog

log = structlog.get_logger()


@dataclass(frozen=True)
class Stratum:
    """One validation stratum with its area and its expected variability.

    `prior_cv` is the coefficient of variation expected within the stratum,
    from a pilot, a previous project, or literature. It is the number that
    drives allocation, and guessing it low is the usual way a design ends up
    underpowered.
    """
    stratum_id: str
    label: str
    area_ha: float
    prior_cv: float
    model_confidence: str        # high | medium | low


@dataclass(frozen=True)
class Allocation:
    stratum_id: str
    n_plots: int
    expected_half_width_pct: float
    adequate: bool


MIN_PLOTS_PER_STRATUM = 8
UNCERTAINTY_WEIGHT = {"high": 1.0, "medium": 1.6, "low": 2.5}


def neyman_allocation(
    strata: list[Stratum], total_plots: int, *, weight_by_uncertainty: bool = True
) -> dict[str, int]:
    """Allocate plots by area times variability, optionally weighted.

    Plain Neyman allocation optimises the precision of the overall mean.
    The uncertainty weighting deliberately departs from that: it buys
    precision where the model is least trusted, at a small cost to the
    overall figure. For a validation network that trade is correct.
    """
    weights: dict[str, float] = {}
    for s in strata:
        w = s.area_ha * s.prior_cv
        if weight_by_uncertainty:
            w *= UNCERTAINTY_WEIGHT[s.model_confidence]
        weights[s.stratum_id] = w

    total_w = sum(weights.values())
    raw = {k: total_plots * v / total_w for k, v in weights.items()}

    # Floor every stratum first, then distribute the remainder by largest
    # fractional part. Strata below the floor take priority over rounding.
    alloc = {k: max(MIN_PLOTS_PER_STRATUM, int(v)) for k, v in raw.items()}
    assigned = sum(alloc.values())

    if assigned > total_plots:
        raise ValueError(
            f"{len(strata)} strata at a floor of {MIN_PLOTS_PER_STRATUM} plots "
            f"each require {assigned} plots but only {total_plots} are budgeted; "
            "merge strata or raise the budget — do not drop the floor, because "
            "a stratum with four plots supports no statement about that stratum"
        )

    remainder = sorted(raw, key=lambda k: raw[k] - int(raw[k]), reverse=True)
    i = 0
    while assigned < total_plots:
        alloc[remainder[i % len(remainder)]] += 1
        assigned += 1
        i += 1

    return alloc


def assess_allocation(
    strata: list[Stratum], alloc: dict[str, int], *, target_half_width_pct: float
) -> list[Allocation]:
    """Expected precision per stratum, and whether it meets the target."""
    out: list[Allocation] = []
    for s in strata:
        n = alloc[s.stratum_id]
        half_width = 1.96 * s.prior_cv / math.sqrt(n) * 100
        adequate = half_width <= target_half_width_pct
        if not adequate:
            log.warning(
                "allocation.underpowered",
                stratum=s.stratum_id,
                n_plots=n,
                expected_half_width_pct=round(half_width, 1),
                target=target_half_width_pct,
                plots_needed=math.ceil((1.96 * s.prior_cv / (target_half_width_pct / 100)) ** 2),
            )
        out.append(
            Allocation(s.stratum_id, n, round(half_width, 1), adequate)
        )
    return out

The plots_needed figure in the warning is the number that changes budgets. A stratum with a coefficient of variation of 0.5 needs about a hundred plots to reach a ten percent half-width, and seeing that written down before the survey is far cheaper than discovering it after.

Plot size against pixel support, and the residual each mismatch produces Three cases drawn over a grid of thirty metre pixels. In the first, a small circular plot of about eight metres radius sits inside one pixel: the plot measures a fraction of what the pixel predicts, and the residual carries within-pixel variability that the model was never asked to capture. In the second, a plot comparable in size to the pixel aligns well, and the residual reflects model error alone. In the third, a plot sits across a pixel boundary because its recorded position is off by fifteen metres, and the comparison is partly against the neighbouring pixel, producing a large residual that looks like model failure. A panel notes that only the second case produces a residual that means what the analyst thinks it means. What the residual actually contains Two of these three residuals are dominated by something other than the model. 0.02 ha plot, 30 m pixel residual = model error + within-pixel variability the model never saw plot ≈ pixel, well located residual = model error this is the only usable case plot ≈ pixel, 15 m position error residual = model error + wrong pixel fat tails, mistaken for model failure

Deterministic Transformation Logic

Once the allocation is fixed, plot locations are drawn and the hold-out set is separated. Both steps must be reproducible from a seed, because a verifier asking how a plot came to be where it is deserves an answer better than that someone chose it.

from dataclasses import dataclass


@dataclass(frozen=True)
class PlotSite:
    plot_id: str
    stratum_id: str
    x: float
    y: float
    role: str                    # calibration | holdout
    draw_index: int


def draw_sites(
    strata_masks: dict[str, list[tuple[float, float]]],
    alloc: dict[str, int],
    *,
    seed: int,
    min_separation_m: float,
    holdout_fraction: float = 0.3,
) -> list[PlotSite]:
    """Draw plot locations per stratum and split calibration from hold-out.

    Two properties make this defensible. Locations come from a seeded
    generator, so the draw reproduces exactly. And the hold-out split happens
    here, before any modelling — assigning roles after the fact allows the
    hold-out to be chosen, however unintentionally, to flatter the model.
    """
    import random

    rng = random.Random(seed)
    sites: list[PlotSite] = []

    for stratum_id, n in sorted(alloc.items()):
        candidates = list(strata_masks[stratum_id])
        if len(candidates) < n * 4:
            raise ValueError(
                f"stratum {stratum_id} offers {len(candidates)} candidate "
                f"cells for {n} plots; the separation constraint will not be "
                "satisfiable — enlarge the stratum or reduce its allocation"
            )

        rng.shuffle(candidates)
        chosen: list[tuple[float, float]] = []
        for x, y in candidates:
            if len(chosen) == n:
                break
            if all(
                (x - cx) ** 2 + (y - cy) ** 2 >= min_separation_m ** 2
                for cx, cy in chosen
            ):
                chosen.append((x, y))

        if len(chosen) < n:
            raise ValueError(
                f"stratum {stratum_id}: only {len(chosen)} of {n} plots could "
                f"be placed at {min_separation_m} m separation"
            )

        n_holdout = max(2, round(len(chosen) * holdout_fraction))
        for i, (x, y) in enumerate(chosen):
            sites.append(
                PlotSite(
                    plot_id=f"{stratum_id}-{i:03d}",
                    stratum_id=stratum_id,
                    x=x,
                    y=y,
                    role="holdout" if i < n_holdout else "calibration",
                    draw_index=i,
                )
            )

    return sites


def aggregate_to_pixel_support(
    plot_values: list[float], plot_areas_ha: list[float], pixel_area_ha: float
) -> float | None:
    """Combine sub-pixel plots into one pixel-support observation.

    Returns None when the plots cover too little of the pixel to represent it.
    Returning None rather than a value is the point: a pixel with 12% coverage
    produces a comparison whose error is dominated by what was not measured.
    """
    covered = sum(plot_areas_ha)
    if covered / pixel_area_ha < 0.5:
        return None
    return sum(v * a for v, a in zip(plot_values, plot_areas_ha)) / covered

Assigning the hold-out role inside the draw, keyed on draw index rather than on anything measured, closes the most common quiet failure in this area. When the split is made later, it is nearly always made on data that has already been looked at, and the resulting validation is optimistic by an amount nobody can quantify.

Compliance Gating & Audit Trail Generation

The survey design is itself an auditable artefact, and it needs recording before the field season rather than reconstructed after it.

The stratification and its justification, including the prior coefficient of variation used for each stratum and where it came from. A design whose allocation depended on a guessed variability is fine; a design where the guess is undocumented cannot be assessed.

The draw seed, the candidate mask, and the resulting site list. Together these let anyone reproduce the locations exactly, which converts “why is there no plot in the north-west?” from a suspicion into a checkable fact about the mask.

The role assignment, timestamped before the model was fitted. This is what makes an independent validation independent, and it is worth a signed record rather than a field in a spreadsheet.

Plots that could not be visited, with reasons. Inaccessible plots are normal and they matter: if the unvisitable set correlates with steep terrain or remoteness, the realised sample is biased relative to the design, and the analysis needs to say so and where possible reweight. Silently substituting a nearby accessible plot reintroduces convenience sampling into a design built to avoid it.

Production Integration

The plot network is a long-lived asset and should be treated as one. Re-measuring the same plots over successive monitoring periods gives a paired comparison that is far more sensitive to change than two independent samples, and it turns the network into the project’s own growth model rather than merely a validation set. That argues for permanent, monumented plots and against a fresh draw each period.

Where the network feeds a model, keep the interfaces described in validating carbon models with field inventory data in Python, and note the interaction with correlation structure: plots spaced closer than the correlation range fitted in propagating spatial autocorrelation into uncertainty budgets carry less independent information than their count suggests, which is exactly what the minimum separation constraint above is protecting against.

One practical note on sequencing. The design depends on a stratification, the stratification usually depends on a preliminary map, and the preliminary map depends on a model that has not been validated yet. That circularity is unavoidable and it is handled by accepting that the first network validates a provisional stratification, and by revisiting the design once the first season’s data shows where the real variability sits. Designing as though the stratification were certain is what produces the network with two plots in the class that turns out to matter most.

The design sequence, and the one loop that has to close before fitting A five-step sequence. A preliminary map produces a provisional stratification. The stratification plus prior variability estimates produce an allocation. The allocation plus a seeded draw produce site locations, split immediately into calibration and hold-out roles. The field season produces measurements, with unvisited plots recorded rather than substituted. The calibration plots fit the model and the hold-out plots validate it. A feedback arrow returns from the field measurements to the stratification, labelled revise the strata after season one, with a note that this loop must close before the hold-out is opened, because revising strata using hold-out data destroys the independence the design was built for. Design, draw, measure, then fit — and keep the hold-out closed until the end Provisional stratification from a preliminary map Allocation area × variability × uncertainty weight Seeded draw + role assignment before any modelling Field season unvisited plots recorded, never substituted Fit, then validate hold-out opened once, at the end Revise the strata after season one — using calibration data only. Revising a design with hold-out data in hand converts an independent validation into a very expensive goodness-of-fit statistic. revise the strata after season one

Frequently Asked Questions

How many plots does a project actually need?

The count follows from the precision required and the variability present, not from a rule of thumb, and the arithmetic is unforgiving: halving the half-width requires four times the plots. A stratum with a coefficient of variation around 0.4 needs roughly sixty plots for a ten percent half-width and around two hundred and forty for a five percent one. Projects usually discover that the budget supports one target and not the other, and the useful response is to decide which strata deserve the tighter figure rather than spreading the difference evenly.

Should plot locations be shared with field teams in advance?

The coordinates, yes; the flexibility to move them, no. The most common way a good design degrades is a field team relocating a plot because the drawn location fell in difficult terrain or an awkward stand. Give teams a documented substitution protocol — a pre-drawn replacement from the same stratum, used in order — so that relocations remain part of the design rather than a judgement in the field. Record every substitution and the reason.

What is the right minimum separation between plots?

Far enough apart that they carry independent information, which means at least the correlation range of the variable being measured, and in practice a few hundred metres in most forest types. Closer plots are not useless — they are informative about fine-scale variability — but they should not be counted as independent replicates in the precision calculation. Where cluster designs are used deliberately, the analysis must use a cluster-aware estimator, not a simple one.

Can existing national forest inventory plots substitute for a project network?

Sometimes for calibration, rarely for validation. National inventories use their own plot design, their own measurement protocols, and often their own definitions of what counts as a tree, and those differences appear as an offset rather than as noise. They are also usually located on a grid designed for national estimates, which puts very few plots inside any one project. Where they are used, treat them as a distinct source with its own bias term rather than pooling them with project plots.

How should destructively sampled plots be handled?

As a separate, small, precious set used to check the allometry rather than the map. Destructive sampling gives the only direct biomass measurement available, and it is typically limited to a few dozen trees rather than plots. Its role is to validate or localise the allometric equations that convert diameter measurements into biomass — a step whose error is often larger than the remote sensing error and is frequently taken on faith from a published equation fitted somewhere else.

What happens when a stratum turns out not to exist in the field?

Record it, and do not redistribute its plots to the strata that were reachable. A stratum that the preliminary map predicted and the field team could not find is a finding about the map, and it changes the area weights used in every subsequent aggregation. Silently moving its plot budget elsewhere hides a map error and biases the resulting estimate, because the area attributed to a class nobody could find is still in the denominator.

Is a hold-out set worth thirty percent of an expensive survey?

Yes, and the alternative is worse than it sounds. Cross-validation on the calibration set answers whether the model interpolates within data it has seen, which is a real question but a softer one than whether it predicts locations it has not. For a claim that will be scrutinised by a verifier, an independent hold-out is the evidence that is actually persuasive, and thirty percent is a common share. Where the budget genuinely will not support it, reserve a smaller fraction — twenty percent — rather than nothing, and state the limitation.