Spatial Pipeline EngineeringMRV Automation

Preventing Scope 3 Double-Counting in Spatial Joins

A spatial join is the single most common place where a Scope 3 inventory silently inflates. This guide sits under GHG Protocol Scope 3 Spatial Mapping, the value-chain attribution discipline within the wider MRV Architecture & Carbon Accounting Fundamentals stack, and it addresses the failure that turns a technically correct join into an indefensible tonnage figure. When a geopandas.sjoin returns more rows than it received, or when two overlapping supplier catchments both claim the same hectare of activity, the emissions attached to those geometries are counted more than once — and the error is additive, so it survives every downstream sum untouched.

The mechanics of the join itself are covered in the step-by-step GHG Protocol Scope 3 geospatial calculation; this page is the troubleshooting companion for when that calculation returns a number that is quietly too large. Double-counting is rarely a crash. It is a left join fanning out on a many-to-many predicate, a buffer overlap nobody reconciled, or a boundary parcel intersecting three facilities and being attributed to all three at full weight. Each produces a plausible-looking table that no exception ever flags, which is exactly why detection has to be an explicit gate rather than an assumption — and why the corrected total must be reconciled against a mass balance before it enters the inventory.

Spatial join with a fan-out detector and area-apportionment gate Activity geometries and supplier catchments enter a spatial join. A fan-out detector compares row counts before and after the join and inspects per-key group sizes. If rows multiplied or geometries overlap, the flow routes to an area-apportionment gate that weights each match by intersection area over total area and runs a mass-balance check asserting that summed emissions before equal summed emissions after within tolerance. A clean one-to-one join bypasses apportionment. Both paths converge on a single deduplicated, audited emissions total. Activity geometries Supplier catchments Spatial join Area apportionment Clean 1:1 join Audited total sjoin · predicate w = A∩ / A_total rows unchanged deduplicated mass-balanced rows multiplied? yes / overlap no

Root Cause Analysis

Double-counting in a Scope 3 spatial join is a structural property of the join, not a data-entry mistake, and it arrives through four distinct mechanisms.

First, many-to-many predicates fan rows out. A geopandas.sjoin with how="left" and predicate="intersects" emits one output row for every left–right geometry pair that satisfies the predicate. If a single activity polygon intersects three supplier catchments, the activity’s emission value is replicated across three rows. A later groupby("supplier").sum() then attributes the full activity emission to each of the three suppliers, and the inventory total is inflated by a factor equal to the average match multiplicity. The join never errors; it simply returns more rows than it received, and unless that count is checked the inflation is invisible.

Second, overlapping catchments and buffers count the same activity twice. Supplier sourcing regions are frequently modelled as buffers around facilities or as Voronoi-style catchments, and these overlap wherever two facilities compete for the same territory. An activity falling in the overlap is legitimately linked to both suppliers, but if each link carries the full activity emission rather than a share, the overlap is double-counted. On a dense logistics network, catchment overlaps of 15–30% of total sourced area are routine, and counting them at full weight can inflate the attributed Scope 3 category by a comparable fraction.

Third, boundary parcels intersect multiple facilities. A parcel straddling an administrative or catchment boundary intersects every facility whose region touches it. intersects is permissive: a shared edge or a one-metre sliver counts as a full match. Without an area-weighted rule, a parcel that is 95% inside facility A and 5% inside facility B is attributed in full to both. Fourth, duplicate geometries — the same activity ingested twice from overlapping data extracts, or a self-intersecting multipolygon — introduce duplication before the join even runs, so the fan-out detector must also fingerprint geometries and keys, not just count rows.

Diagnostic Pipeline: Detecting Silent Fan-Out

Before trusting any aggregated tonnage, run the join under instrumentation that compares pre- and post-join cardinality, inspects per-key group sizes, tests the catchment layer for self-overlap with a unary_union, and fingerprints duplicate geometries and keys. Every area comparison here is performed in an equal-area CRS, because intersects counts and overlap areas computed in a geographic CRS such as EPSG:4326 are distorted by latitude and will misrank which overlaps matter. Getting that projection right is the same discipline enforced across the Scope 3 spatial mapping layer.

import geopandas as gpd
import pandas as pd
import structlog

log = structlog.get_logger()

# Equal-area projection so intersection areas and overlap ratios are metric-honest.
EQUAL_AREA_CRS = "EPSG:6933"  # World Cylindrical Equal Area


def diagnose_join(
    activity: gpd.GeoDataFrame,
    catchments: gpd.GeoDataFrame,
    activity_key: str = "activity_id",
    predicate: str = "intersects",
) -> dict:
    """Run a spatial join under instrumentation and report double-counting risk.

    Detects: row multiplication (pre/post counts + group sizes), overlapping
    catchments (unary_union area vs summed area), duplicate geometries, and
    duplicate join keys — none of which raise on their own.
    """
    issues: list[str] = []

    if activity.crs is None or catchments.crs is None:
        raise ValueError("untagged CRS; refusing to compute areas on a guessed datum.")

    act = activity.to_crs(EQUAL_AREA_CRS)
    catch = catchments.to_crs(EQUAL_AREA_CRS)

    # 1. Duplicate geometries / keys present before the join
    dup_geoms = int(act.geometry.duplicated().sum())
    dup_keys = int(act[activity_key].duplicated().sum())
    if dup_geoms or dup_keys:
        issues.append(f"pre_join_duplicates:geoms={dup_geoms},keys={dup_keys}")

    # 2. Row multiplication: a left join must not add rows
    n_in = len(act)
    joined = gpd.sjoin(act, catch, how="left", predicate=predicate)
    n_out = len(joined)
    fan_out = n_out / n_in if n_in else 1.0
    if n_out > n_in:
        issues.append(f"row_fan_out:{n_in}->{n_out} (x{fan_out:.2f})")

    # 3. Worst-case multiplicity for any single activity
    max_multiplicity = int(joined.groupby(activity_key).size().max())

    # 4. Catchment self-overlap: summed area vs dissolved area
    summed_area = float(catch.geometry.area.sum())
    union_area = float(catch.geometry.unary_union.area)
    overlap_frac = (summed_area - union_area) / summed_area if summed_area else 0.0
    if overlap_frac > 0.001:
        issues.append(f"catchment_overlap:{overlap_frac:.3f}")

    report = {
        "rows_in": n_in,
        "rows_out": n_out,
        "fan_out_ratio": round(fan_out, 3),
        "max_activity_multiplicity": max_multiplicity,
        "catchment_overlap_fraction": round(overlap_frac, 4),
        "safe_to_sum": not issues,
        "issues": issues,
    }
    if issues:
        log.warning("scope3.join.double_count_risk", **report)
    else:
        log.info("scope3.join.clean", **report)
    return report

A report with safe_to_sum=False must not be aggregated with a plain groupby().sum(). A non-unit fan_out_ratio or any catchment_overlap_fraction above zero means the emissions have to be apportioned, not replicated — which is the job of the transformation step below.

Deterministic Transformation Logic: Area-Weighted Apportionment

The fix is to replace the permissive intersects join with an overlay intersection that splits each activity across the catchments it touches, weighting every fragment by its share of the activity’s total area. The weight for a match is the intersection area divided by the total activity area, so the weights for any single activity sum to one and its emission is partitioned rather than duplicated. This conserves the inventory total by construction; the mass-balance gate then proves it, refusing to emit a result whose apportioned sum drifts from the input sum beyond a tight tolerance.

import geopandas as gpd
import numpy as np
import pandas as pd
import structlog
from datetime import datetime, timezone

log = structlog.get_logger()

EQUAL_AREA_CRS = "EPSG:6933"
MASS_BALANCE_TOL = 1e-6  # relative tolerance on conserved total emissions


def apportion_emissions(
    activity: gpd.GeoDataFrame,
    catchments: gpd.GeoDataFrame,
    emission_col: str = "emissions_tco2e",
    activity_key: str = "activity_id",
    supplier_key: str = "supplier_id",
) -> tuple[pd.DataFrame, dict]:
    """Area-weight Scope 3 emissions across overlapping catchments.

    Each activity's emission is split by (intersection area / total activity
    area), so shared hectares are partitioned, never counted twice. A mass-
    balance gate asserts the apportioned total equals the input total.
    """
    if activity.crs is None or catchments.crs is None:
        raise ValueError("untagged CRS; cannot apportion on a guessed datum.")

    act = activity.to_crs(EQUAL_AREA_CRS).copy()
    catch = catchments.to_crs(EQUAL_AREA_CRS).copy()

    total_before = float(act[emission_col].sum())
    act["_activity_area"] = act.geometry.area

    # Geometric intersection: one row per (activity, catchment) overlap fragment.
    parts = gpd.overlay(
        act[[activity_key, emission_col, "_activity_area", "geometry"]],
        catch[[supplier_key, "geometry"]],
        how="intersection",
        keep_geom_type=True,
    )

    # Weight = fragment area / whole-activity area  ->  weights per activity sum to 1.
    parts["_frag_area"] = parts.geometry.area
    parts["_weight"] = parts["_frag_area"] / parts["_activity_area"]
    parts["apportioned_tco2e"] = parts[emission_col] * parts["_weight"]

    # Activities that fall entirely outside every catchment lose emission mass;
    # recover the residual so the mass balance holds and the gap is auditable.
    attributed = parts.groupby(activity_key)["apportioned_tco2e"].sum()
    orig = act.set_index(activity_key)[emission_col]
    unattributed = float((orig - attributed.reindex(orig.index).fillna(0.0)).clip(lower=0).sum())

    result = (
        parts.groupby(supplier_key)["apportioned_tco2e"].sum().reset_index()
    )

    total_after = float(result["apportioned_tco2e"].sum()) + unattributed
    residual = abs(total_after - total_before) / total_before if total_before else 0.0

    # Mass-balance gate: apportionment must conserve total emissions.
    if residual > MASS_BALANCE_TOL:
        log.error("scope3.apportion.mass_balance_fail",
                  before=total_before, after=total_after, residual=residual)
        raise RuntimeError(
            f"mass balance broken: {total_before:.6f} -> {total_after:.6f} "
            f"(residual {residual:.2e} > tol {MASS_BALANCE_TOL:.0e})")

    manifest = {
        "method": "area_weighted_intersection_apportionment",
        "equal_area_crs": EQUAL_AREA_CRS,
        "total_before_tco2e": round(total_before, 6),
        "total_after_tco2e": round(total_after, 6),
        "unattributed_tco2e": round(unattributed, 6),
        "mass_balance_residual": residual,
        "generated_utc": datetime.now(timezone.utc).isoformat(),
        "compliance_standard": "GHG Protocol Scope 3 Category Attribution",
    }
    log.info("scope3.apportion.conserved", **manifest)
    return result, manifest

The unattributed_tco2e term is deliberate. Activities that overlap no catchment would otherwise vanish from the supplier rollup and silently break the balance; carrying them as an explicit residual keeps the total conserved and makes the coverage gap visible to a verifier rather than hiding it. The weight computation uses always_xy-consistent equal-area geometry throughout, so the fragment areas that drive the split are true metric areas and not longitude-stretched approximations.

Compliance Gating & Audit Trail Generation

Under the GHG Protocol, an emission source must be attributed once and only once across the value chain, and third-party assurance under ISO 14064-3 tests exactly this — the assessor recomputes totals and checks that boundary-straddling sources were not claimed twice. The mass-balance gate is the machine-checkable expression of that requirement: total_after equals total_before within tolerance, or the pipeline refuses to emit a figure. The manifest records the residual on every run, including the passing ones, so a slowly drifting upstream extract surfaces as a trend before it breaches tolerance.

The apportionment weights are themselves an audit artifact. Because each supplier’s share derives from a recorded intersection-area ratio in a named equal-area CRS, an auditor can reconstruct the split for any contested parcel without rerunning the whole pipeline. These outputs should be validated by the same automated checks that guard the rest of the inventory — the emissions data quality and validation gates that assert no supplier total exceeds its plausible sourced volume — and the fan-out ratio, catchment overlap fraction, and mass-balance residual all belong in that gate suite. When the corrected totals flow into disclosure, they must satisfy the traceability expectations set out in mapping CSRD ESRS E1 disclosures to spatial MRV outputs, which scrutinises value-chain figures for exactly this class of aggregation error.

Production Integration

Wire the diagnostic and apportionment routines into the Scope 3 pipeline as a fixed ingest → diagnose → transform → validate → export → submit sequence, so double-count prevention is a gate rather than a manual review step:

  1. Ingest. Load activity geometries and supplier catchments, confirm both carry machine-readable CRS tags, and reproject both to EPSG:6933 so every area and overlap is metric-honest.
  2. Diagnose. Run diagnose_join and read safe_to_sum. A clean one-to-one join with no catchment overlap may be summed directly; anything else routes to apportionment and logs a structlog warning with the fan-out ratio and overlap fraction.
  3. Transform. Call apportion_emissions to split each activity across the catchments it intersects by area weight, replacing any groupby().sum() on a fanned-out join.
  4. Validate. The mass-balance gate raises if the apportioned total drifts from the input total; treat the residual, fan-out ratio, and overlap fraction as monitored signals feeding the inventory-wide validation gates.
  5. Export. Serialize the deduplicated supplier totals with the audit manifest — CRS, method, before/after totals, unattributed residual — intact.
  6. Submit. Forward the conserved totals and their manifest to the value-chain rollup and, ultimately, disclosure. The end-to-end calculation that consumes these deduplicated figures is detailed in the step-by-step GHG Protocol Scope 3 geospatial calculation.

Treated this way, double-counting stops being a subtle statistical inflation discovered during assurance and becomes an assertion the pipeline enforces on every run: emissions are attributed by area, conserved by mass balance, and traceable to a recorded intersection weight.