Local empirical validation

This vignette describes how to run larger empirical checks without committing private or bulky rasters to the package repository. The package includes bundled hija_* example data for tests and documentation. Larger species examples, such as ASLO, should live outside the package and be passed to the scripts in tools/.

The local scripts are intentionally plain R scripts rather than package tests. That keeps GitHub Actions fast while still giving maintainers a reproducible way to validate realistic data on a local machine.

ASLO-Style Single Run

tools/validate-aslo-local.R runs one POPMAPS interpolation from an ASCII raster and an ancestry table.

Rscript tools/validate-aslo-local.R \
  "/path/to/aslo_avg.asc" \
  "/path/to/aslo.txt" \
  "local_validation/aslo"

Useful environment variables:

POPMAPS_ASLO_SURFACE=C \
POPMAPS_ASLO_SURFACE_VALUES=suitability \
POPMAPS_ASLO_NUM_SITES=15 \
POPMAPS_ASLO_NUM_TESTED=4 \
POPMAPS_ASLO_POPMOD=-0.05 \
Rscript tools/validate-aslo-local.R \
  "/path/to/aslo_avg.asc" \
  "/path/to/aslo.txt" \
  "local_validation/aslo"

The output directory contains run metadata, output summaries, and any exported rasters requested by the script. Keep these outputs out of package source unless they are intentionally small, public, and documented.

When a local run should produce map figures as well as rasters, save the full popmaps() result as an R object or rerun the final selected model and call write_popmaps_plot(). Use style = "manuscript" for manuscript-like grayscale probability maps, colored ancestry boundaries, and optional background habitat or suitability rasters.

Example Species Surface Comparisons

tools/compare-example-surfaces.R scans a directory of empirical examples and compares a point-derived geographic surface against a supplied suitability surface for each species.

Rscript tools/compare-example-surfaces.R \
  "../popmaps_test_data" \
  "local_validation/example_surface_comparison"

Each completed comparison writes:

• a ranked surface summary;
• a support table;
• the tuning grid used for each surface;
• diagnostic PNGs;
• optional full tuning-result CSVs.

These artifacts are the best place to ask whether surface comparison is telling us something biologically useful or simply returning the same parameter region for every species.

Example Species Tuning

tools/validate-example-tuning.R runs repeated spatial-block tuning across external empirical examples.

Rscript tools/validate-example-tuning.R \
  "../popmaps_test_data" \
  "local_validation/example_tuning"

After the tuning runs finish, summarize them:

Rscript tools/summarize-example-tuning.R \
  "local_validation/example_tuning"

The summary is intended to answer practical questions:

• which parameter combinations are best supported;
• whether support is sharp or broad;
• whether results are sensitive to spatial-block repeats;
• whether species with different biology select different distance-decay or empirical-site parameters.

The next useful local artifact is usually a small figure set for each species: one ancestry-probability map, one hard-boundary map, and one map per ancestry axis. Those maps help determine whether the selected tuning results produce a biologically interpretable surface or only improve a validation score.

Interpreting Results

Treat local empirical validation as evidence, not automation. Parameter tuning is useful when it changes conclusions, exposes broad uncertainty, or shows that different species or surfaces require different distance assumptions. If every species repeatedly supports the same parameter values, that suggests either a shared biological signal, an overly narrow tuning grid, or a validation design that is not sensitive to the biology of interest.