--- title: "Choosing and Checking a Genetic Covariance Response" author: "Bill Peterman" output: rmarkdown::html_vignette: number_sections: true toc: true self_contained: true mathjax: null pandoc_args: ["--mathml"] vignette: > %\VignetteIndexEntry{Choosing and Checking a Genetic Covariance Response} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 6, fig.height = 4.5) ``` **Goal.** Choose a genetic response, preserve its sample order, and check its mathematical properties before fitting. The [data-generation guide](data-generation.html) explains the landscapes, SLiM histories, sampling, and matched markers used in these examples. Open it in R with `vignette("data-generation", package = "terradish")`. ## What genetic quantity are you modeling? A genotype table needs a biological sampling unit and a documented transformation before it becomes a model response. This guide checks that transformation for the Ohio pond samples fitted in [`vignette("getting-started", package = "terradish")`](getting-started.html). The 24 ponds are demes (local breeding populations), each with 20 retained diploid animals genotyped at matched SNPs and microsatellites. Grouping animals by pond estimates relationships among ponds, while leaving them ungrouped asks about individuals and requires a different sampling and measurement assessment. | Available input | Response construction | Modeling question | |---|---|---| | SNP dosage, 0, 1, or 2 | Aggregate allele counts by pond, then construct covariance | How do standardized allele frequencies covary among ponds? | | Two microsatellite allele calls per locus | Expand allele labels into features, then group | How do multiallelic feature summaries covary among ponds? | | An existing genetic dissimilarity | Document its definition and dependence | How does pairwise genetic difference relate to resistance? | **Which response should you model?** - Use a **covariance response** with `measurement_model = wishart_covariance` when you have genotypes, can build the covariance with `cov_from_biallelic()` or `cov_from_genetic_data()`, and can defend a value of `nu`, the assumed number of independent genetic features. Its squared distances from `dist_from_cov()` give the same likelihood under `generalized_wishart`. - Use a **pairwise distance** with `measurement_model = mlpe` when you start from a distance such as $F_{ST}$ or $D_{PS}$, when the distance fails `check_distance_response()`, or when you cannot justify `nu`. MLPE (maximum likelihood population effects) regresses genetic distance on resistance distance, accounts for correlation between pairs that share a site, and estimates its own residual variance, so it needs no `nu`. [`vignette("mlpe-pairwise-distance", package = "terradish")`](mlpe-pairwise-distance.html) fits one and reads its summary. Passing the geometry check below shows that a response is a valid covariance, not that it follows a Wishart distribution. ## Preserve sampling order and marker meaning ```{r response} library(terradish) data(ohio_ponds) ponds <- ohio_ponds$ponds animals <- ohio_ponds$samples G <- ohio_ponds$snp stopifnot(identical(rownames(G), animals$individual), identical(rownames(ohio_ponds$microsat), animals$individual)) groups <- factor(animals$pond, levels = ponds$pond) # Counts and haploid totals must refer to the same ordered ponds. Y <- rowsum(G, groups, reorder = FALSE)[ponds$pond, , drop = FALSE] N <- 2 * as.numeric(table(groups)) stopifnot(all(N == N[1]), identical(rownames(Y), ponds$pond)) S_snp <- cov_from_biallelic(Y, N) / N[1] # Repeat counts identify alleles; they are not continuous SNP dosages. S_ms <- cov_from_genetic_data(ohio_ponds$microsat, groups = groups, input = "allele_calls", loci = ohio_ponds$microsat_loci, diagonal = "gower", normalize = "features") c(ponds = nrow(S_snp), haploid_copies_per_pond = N[1], snps = ncol(G), microsatellite_loci = ncol(ohio_ponds$microsat) / 2) ``` **How to read the output.** Each covariance is 24 by 24, although the genotype matrices have 480 rows. Every pond contributes 40 sampled allele copies at each SNP, so dividing the count covariance by the common `N[1]` gives a standardized frequency covariance. Do not apply that shortcut to unequal sample sizes (Getting Started gives the derivation). `normalize = "features"` puts the microsatellite covariance on a per-feature scale, and expanded allele features are not loci. SNPs and microsatellites differ in encoding and mutation process, so compare their fitted directions and predictive performance within each panel, not their covariance values or likelihoods. The panels share ancestry, so they are not ecological replicates. ## Check covariance and distance geometry A positive semidefinite covariance gives no negative variance for any linear combination of its entries. Centering across ponds removes the common mean direction, so expect a zero eigenvalue. Tiny negative values from floating-point rounding are not substantial violations. ```{r geometry} geometry <- function(S) { ev <- eigen(S, symmetric = TRUE, only.values = TRUE)$values c(symmetry_error = max(abs(S - t(S))), maximum_absolute_row_sum = max(abs(rowSums(S))), relative_minimum_eigenvalue = min(ev) / max(abs(ev))) } rbind(SNP = geometry(S_snp), microsatellite = geometry(S_ms)) # dist_from_cov() already returns squared distances; do not square them again. D <- dist_from_cov(S_snp) diagnostic <- check_distance_response(D) diagnostic[c("admissible", "relative_min_eigenvalue", "n_negative")] ``` **How to read the output.** Near-zero symmetry errors and row sums support the intended centered construction. `admissible = TRUE` means the squared distances have acceptable covariance geometry within numerical tolerance. It says nothing about marker filtering, relatedness, unequal sampling variance, or the biological adequacy of the measurement likelihood. The two representations satisfy $$D_{ij}=S_{ii}+S_{jj}-2S_{ij},\qquad H=I-\frac{1}{n}\mathbf{1}\mathbf{1}^{\mathsf T},\qquad B=-\frac12 HDH.$$ Here $n$ is the number of ponds, $H$ centers across ponds, and $B$ recovers the centered covariance. Confirm the round trip: ```{r roundtrip} H <- diag(nrow(D)) - 1 / nrow(D) B <- -0.5 * H %*% D %*% H c(maximum_roundtrip_error = max(abs(B - H %*% S_snp %*% H))) stopifnot(diagnostic$admissible, max(abs(B - H %*% S_snp %*% H)) < 1e-10) ``` **How to read the result.** A near-zero error confirms that the conversion keeps the site contrasts the Wishart interfaces use. Covariance input adds no information, so covariance and distance fits should agree numerically when response scaling, graph, sites, and `nu` match. Ranking the two representations as competing biological models answers no useful question. ## Inspect the response as a matrix ```{r covariance-map, fig.cap = "Between-pond SNP covariance in sampling order. Red cells are pond pairs whose allele frequencies depart from the mean in the same direction; blue cells, in opposite directions. The diagonal (pond variances) is left blank."} library(ggplot2) covariance_cells <- expand.grid(pond_i = seq_len(nrow(S_snp)), pond_j = seq_len(ncol(S_snp))) covariance_cells$covariance <- as.vector(S_snp) # Blank the diagonal: the much larger pond variances would wash out the colors. covariance_cells$covariance[covariance_cells$pond_i == covariance_cells$pond_j] <- NA limit <- max(abs(covariance_cells$covariance), na.rm = TRUE) ggplot(covariance_cells, aes(pond_i, pond_j, fill = covariance)) + geom_tile() + coord_equal() + theme_bw() + scale_fill_gradient2(low = "#2166AC", mid = "white", high = "#B2182B", limits = c(-limit, limit), na.value = "white", name = "Covariance") + labs(x = "Pond index", y = "Pond index") ``` **How to read the map.** The matrix should be symmetric about the diagonal, and an asymmetric pattern signals a row-ordering mistake. Red blocks mark ponds that share allele-frequency departures, usually because they are close or well connected. Negative values are expected after centering. After fitting, `plot(fit, type = "fit")` shows observed versus fitted covariance, as in Getting Started. ## Separate marker count from assumed information The fitted covariance is $$\Sigma=\tau E(\theta)+\exp(\sigma_{\mathrm{nugget}})I.$$ Here $E$ is the graph covariance, which changes with the conductance coefficients $\theta$, and $\tau \ge 0$ scales it. The diagonal nugget absorbs variation the graph misses and does not estimate population size or raster context. You supply `nu`, which weights the likelihood. The pond guides assume 40, which is neither the 2,500 SNPs nor the 40 allele copies sampled per pond. Linkage and shared history prevent any simple conversion from marker count to independent information. For a fixed response and model, multiplying `nu` does not move the theoretical optimum. It changes interior standard errors in proportion to $1/\sqrt{\nu}$ and changes the size of likelihood gains relative to the AIC penalty. Boundary estimates require separate treatment. Use `terradish_rescale_nu()` to report sensitivity and held-out ponds to check transfer, but neither estimates the correct `nu`. ## Adapt the construction to your data | Decision | What to record and check | |---|---| | Rare-allele filtering | Standardization can give rare variants large weight. Report the filter and examine sensitivity. | | Unequal sample sizes or missing calls | Record sampled allele totals by pond and locus; a common diagonal variance does not automatically correct unequal sampling variance. | | Grouped covariance diagonal | Use the coherent Gower construction. Replacing its diagonal with within-population variance changes the response and may make it indefinite. | | Existing distances | Confirm whether the function returns ordinary or squared distances. An arbitrary dissimilarity, including an $F_{ST}$ matrix, has no general Wishart justification. | | Independent prediction | Learn filters and frequency weights from training data if that is the target. The bundled examples condition on a panel and weights constructed from all retained animals. | Do not repair a failed geometry check by adjusting matrix entries until it passes. Ask whether the construction suits the question, and for generic pairwise dissimilarities, assess an MLPE model and its assumptions instead. For individual sampling, `groups = NULL` requests an individual-level covariance from `cov_from_genetic_data()`, as worked through in `vignette("individual-salamanders", package = "terradish")`. ## Summary of key functions | Function | Purpose | |---|---| | `cov_from_biallelic()` | Construct covariance from allele counts and sampled totals | | `cov_from_genetic_data()` | Construct covariance from numeric features or labeled alleles | | `dist_from_cov()` | Obtain matching squared distances | | `check_distance_response()` | Check squared-distance geometry | | `terradish_rescale_nu()` | Assess assumed information without re-estimating coefficients | ## See also - [`vignette("getting-started", package = "terradish")`](getting-started.html): fit and interpret both pond marker panels. - [`vignette("model-comparison", package = "terradish")`](model-comparison.html): information sensitivity and held-out prediction. - [`vignette("mlpe-pairwise-distance", package = "terradish")`](mlpe-pairwise-distance.html): fit a pairwise genetic distance with MLPE and read its summary. - [`vignette("ibe-ibr-workflow", package = "terradish")`](ibe-ibr-workflow.html): paired environmental differences versus covariance components. - [`vignette("simulation-design", package = "terradish")`](simulation-design.html): conditional study-design benchmarks, distinct from SLiM genetics. - [`vignette("individual-salamanders", package = "terradish")`](individual-salamanders.html): individual-level covariance from matched SNPs and microsatellites. - [`vignette("large-landscapes", package = "terradish")`](large-landscapes.html): `generalized_wishart` and solver choices on large rasters.