Urban pollination networks with stochastic block models

R companion of the slides (running example: Baldock et al. 2019, 4 UK cities)

Author

Sophie Donnet (INRAE, MIA Paris-Saclay)

Online version of this companion: https://sophiedonnet.github.io/UrbanNetworks/companion_slides_UK.html (course page, with the slides and the downloads: https://sophiedonnet.github.io/urban_networks.html).

NoteReferences

Data

  • Baldock, K. C. R., Goddard, M. A., Hicks, D. M., Kunin, W. E., Mitschunas, N., Morse, H., Osgathorpe, L. M., Potts, S. G., Robertson, K. M., Scott, A. V., Staniczenko, P. P. A., Stone, G. N., Vaughan, I. P. & Memmott, J. (2019). A systems approach reveals urban pollinator hotspots and conservation opportunities. Nature Ecology & Evolution, 3, 363–373. doi:10.1038/s41559-018-0769-y
  • Insect taxonomy completed with the compilation of Doré, M., Fontaine, C. & Thébault, E. (2021). Relative effects of anthropogenic pressures, climate, and sampling design on the structure of pollination networks at the global scale. Global Change Biology, 27(6), 1266–1280. doi:10.1111/gcb.15474

R packages

  • sbm: Chiquet, J., Donnet, S. & Barbillon, P. sbm: Stochastic Blockmodels. R package, CRAN · GitHub

  • colSBM: SBM for collections of networks. R package, GitHub. Method: Chabert-Liddell, S.-C., Barbillon, P. & Donnet, S. (2024). Learning common structures in a collection of networks. An application to food webs. The Annals of Applied Statistics, 18(2), 1213–1235. doi:10.1214/23-AOAS1831

    Bipartite networks: Lacoste, L., Barbillon, P. & Donnet, S. (2025). Common structure discovery in collections of bipartite networks: application to pollination systems. Preprint, arXiv:2512.01716 · HAL: hal-05399136

This document follows the sections of the slides and gives, for each figure, table or number of the slides, the R code that produces it.

The estimations are long (up to 1 h for a colSBM): they are cached in results_analysis/. Delete a file to re-estimate it.

The estimations can be launched with the scripts in the folder scripts/.

1 Setup

library(dplyr)
library(tidyr)
library(ggplot2)
library(patchwork)
library(sbm)       # SBM on one network
library(colSBM)    # SBM on a collection of networks (>= 0.4.11, GitHub GrossSBM/colSBM)
library(bipartite) # d' specialisation index
library(vegan)     # NODF

col_plant <- "#66C1BF"; col_poll <- "#FFA500"   # colours of the slides
cols_status <- c(Native = col_plant, `Non-native` = col_poll, Mixed = "grey55", unknown = "grey85")
cols_group <- c(Bee = "#F2D04B", Hoverfly = "#D45E00", `Other Diptera` = "#7A5195",
                Beetle = "#2F7F5F", `Butterfly / moth` = "#3E7CB1", `Other Hymenoptera` = "grey60")
theme_set(theme_minimal(base_size = 14))

# estimate once, then read the cached result
cached_fit <- function(file, expr) {
  if (file.exists(file)) return(readRDS(file))
  set.seed(1)
  fit <- expr
  saveRDS(fit, file)
  fit
}

2 Motivation: the data

Baldock et al. (2019), Nature Ecology & Evolution 3: 363–373. 4 cities (Bristol, Edinburgh, Leeds, Reading) × 9 land uses, 360 sites. One record = one insect caught on one flower. The networks are built from the supplementary files of the paper (in data_Baldock2019/, see its README.md for the choices made) by scripts/01_build_networks_Baldock2019.R, which writes data_Baldock2019/baldock2019_networks.rds.

baldock <- readRDS("data_Baldock2019/baldock2019_networks.rds")
head(baldock$records, 3)
# A tibble: 3 × 9
  city    landuse   site      plant           insect specimen status order group
  <chr>   <chr>     <chr>     <chr>           <chr>  <chr>    <chr>  <chr> <chr>
1 Bristol Allotment BR_ALT_01 Senecio jacoba… Lucil… BR_ALT_… Native Dipt… Othe…
2 Bristol Allotment BR_ALT_01 Bellis perennis Empis… BR_ALT_… Native Dipt… Othe…
3 Bristol Allotment BR_ALT_01 Origanum vulga… Sarco… BR_ALT_… Native Dipt… Othe…
nrow(baldock$records)   # number of visits
[1] 4921
# a network = a table plants x insects, counting the visits
net_city <- lapply(split(baldock$records, baldock$records$city),
                   function(d) unclass(table(d$plant, d$insect)))
all.equal(lapply(net_city, function(M) sum(M)), lapply(baldock$net_city, sum))
[1] TRUE
net_city <- baldock$net_city

Plant status (native / non-native) and insect groups:

plant_status <- setNames(baldock$plants$status, baldock$plants$plant)
plant_status[is.na(plant_status)] <- "unknown"
insect_group <- setNames(baldock$insects$group, baldock$insects$insect)

3 Introduction to networks

3.1 Graphs and matrices: the Bristol network

M <- net_city$Bristol           # counts: plants x insects
B <- 1 * (M > 0)                # binary incidence matrix
c(plants = nrow(M), insects = ncol(M), links = sum(B), visits = sum(M))
 plants insects   links  visits 
    193     180     662    1379 

Slide “Bristol: the count matrix” and “the weighted network”: the count matrix, and the distribution of the counts per pair (plant, insect).

pairs <- as.data.frame(as.table(M)) |> setNames(c("plant", "insect", "visits")) |> filter(visits > 0)
pairs |> slice_max(visits, n = 4)       # the largest counts
                plant                  insect visits
1 Solidago canadensis Reichertella geniculata     81
2    Senecio jacobaea    Eristalis arbustorum     41
3  Foeniculum vulgare        Lucilia sericata     41
4     Myrtus communis          Apis mellifera     33
mean(pairs$visits == 1)                 # share of the pairs seen once
[1] 0.723565
plotMyMatrix(log1p(M), dimLabels = c(row = "plants", col = "insects")) +   # log scale: counts 1 to 81
  ggplot(pairs, aes(visits)) + geom_histogram(binwidth = 1, fill = "grey40") +
  labs(x = "number of visits per pair (plant, insect)", y = "number of pairs")

The bipartite graph and the binarised incidence matrix:

plotweb(B, sorting = "dec", box_size = 0.03, higher_labels = rep("", ncol(B)), lower_labels = rep("", nrow(B)),
        higher_color = col_poll, lower_color = col_plant, link_color = "grey60", link_alpha = 0.3)

plotMyMatrix(B, dimLabels = c(row = "plants", col = "insects"))

3.2 Node-level descriptors

Degree (number of partners) and strength (number of visits), slides “Degree”, “Strength”, “Degree vs strength”.

nodes <- bind_rows(
  tibble(type = "plants", species = rownames(M), degree = rowSums(B), strength = rowSums(M)),
  tibble(type = "insects", species = colnames(M), degree = colSums(B), strength = colSums(M))) |>
  mutate(type = factor(type, levels = c("plants", "insects")))
nodes |> group_by(type) |> slice_max(degree, n = 3)   # Apis mellifera: 46 partners
# A tibble: 6 × 4
# Groups:   type [2]
  type    species           degree strength
  <fct>   <chr>              <dbl>    <dbl>
1 plants  Bellis perennis       38       61
2 plants  Taraxacum agg         27       55
3 plants  Ranunculus repens     22       46
4 insects Apis mellifera        46      162
5 insects Bombus pascuorum      44      118
6 insects Bombus terrestris     31       77
ggplot(nodes, aes(degree, fill = type)) + geom_bar() +
  facet_wrap(~type, scales = "free") + scale_fill_manual(values = c(col_plant, col_poll), guide = "none")

ggplot(nodes, aes(strength, fill = type)) + geom_histogram(bins = 25) + scale_x_log10() +
  facet_wrap(~type, scales = "free") + scale_fill_manual(values = c(col_plant, col_poll), guide = "none")

ggplot(nodes, aes(strength, degree, colour = type)) +
  geom_abline(linetype = "dashed", colour = "grey60") + geom_count(alpha = 0.7) +
  scale_x_log10() + scale_y_log10() + scale_colour_manual(values = c(col_plant, col_poll))

Specialisation index \(d'\) of the plants, native vs non-native (slide “Specialisation index \(d'\)”):

dprime <- tibble(plant = rownames(M), dprime = dfun(M)$dprime, status = plant_status[rownames(M)]) |>
  filter(status %in% c("Native", "Non-native"))
dprime |> group_by(status) |> summarise(n = n(), median = median(dprime))
# A tibble: 2 × 3
  status         n median
  <chr>      <int>  <dbl>
1 Native        59  0.453
2 Non-native    81  0.401
wilcox.test(dprime ~ status, data = dprime)

    Wilcoxon rank sum test with continuity correction

data:  dprime by status
W = 2587, p-value = 0.4057
alternative hypothesis: true location shift is not equal to 0
ggplot(dprime, aes(status, dprime)) + geom_boxplot(outlier.shape = NA) +
  geom_jitter(width = 0.1, alpha = 0.5, colour = col_plant) + labs(x = NULL, y = "d'")

3.3 Network-level descriptors

Table of the slide “Descriptors per city”: connectance \(C = L / (nm)\), interaction evenness \(E\), nestedness NODF.

network_metrics <- function(M) {
  B <- 1 * (M > 0); P <- M[M > 0] / sum(M)
  tibble(plants = nrow(B), insects = ncol(B), links = sum(B), visits = sum(M),
         C = round(mean(B), 3), E = round(-sum(P * log(P)) / log(sum(B)), 2),
         NODF = round(nestednodf(B)$statistic[["NODF"]], 1))
}
bind_rows(lapply(net_city, network_metrics), .id = "city") |> knitr::kable()
city plants insects links visits C E NODF
Bristol 193 180 662 1379 0.019 0.90 10.1
Edinburgh 137 129 425 1507 0.024 0.73 12.9
Leeds 155 126 573 1079 0.029 0.94 13.3
Reading 156 167 574 956 0.022 0.95 12.6

4 Introduction to stochastic block models

4.1 The simplest model: Erdős–Rényi

\(Y_{ij} \sim \mathcal{B}ern(p)\), \(\hat p = C\): the degrees would be Binomial.

p_hat <- mean(B); p_hat
[1] 0.01905584
er <- bind_rows(tibble(type = "plants", degree = 0:50, expected = nrow(B) * dbinom(0:50, ncol(B), p_hat)),
                tibble(type = "insects", degree = 0:50, expected = ncol(B) * dbinom(0:50, nrow(B), p_hat))) |>
  mutate(type = factor(type, levels = c("plants", "insects"))) |>
  filter(degree <= ifelse(type == "plants", max(rowSums(B)), max(colSums(B))))
ggplot(nodes, aes(degree)) + geom_bar(aes(fill = type)) +
  geom_line(data = er, aes(y = expected)) + facet_wrap(~type, scales = "free") +
  scale_fill_manual(values = c(col_plant, col_poll), guide = "none")

4.2 Bipartite SBM on simulated networks

“One model, many structures”: modular, core–periphery and nested structures simulated with sampleBipartiteSBM, then “What inference does”: the groups are recovered from a matrix in an arbitrary order.

set.seed(2)
structures <- list(
  Modular = list(alpha = matrix(c(.60, .05, .02, .03, .40, .05, .02, .04, .25), 3, byrow = TRUE),
                 pi = c(.2, .3, .5), rho = c(.25, .35, .4)),
  `Core-periphery` = list(alpha = matrix(c(.85, .45, .15, .30, .05, .01), 2, byrow = TRUE),
                          pi = c(.3, .7), rho = c(.15, .35, .5)),
  Nested = list(alpha = matrix(c(.95, .85, .70, .50, .85, .45, .15, .03, .60, .10, .02, .01), 3, byrow = TRUE),
                pi = c(.2, .3, .5), rho = c(.1, .2, .3, .4)))
sims <- lapply(structures, function(s)
  sampleBipartiteSBM(nbNodes = c(60, 90), blockProp = list(row = s$pi, col = s$rho),
                     connectParam = list(mean = s$alpha), model = "bernoulli",
                     dimLabels = c(row = "plants", col = "pollinators")))
wrap_plots(lapply(names(sims), function(s) plot(sims[[s]], type = "data") + ggtitle(s)))

Y <- sims$Modular$networkData
Y <- Y[sample(nrow(Y)), sample(ncol(Y))]          # arbitrary order, as observed
fit_sim <- estimateBipartiteSBM(Y, model = "bernoulli", estimOptions = list(verbosity = 0, plot = FALSE))
fit_sim$nbBlocks
row col 
  3   3 
plotMyMatrix(Y) + plot(fit_sim, type = "data")

4.3 Application to the UK data

One bipartite SBM per city, on the counts (Poisson) and on the binary networks (Bernoulli). estimateBipartiteSBM explores the numbers of groups \((K, L)\) and keeps the best ICL.

net_bin <- lapply(net_city, function(M) 1 * (M > 0))
sbm_opts <- list(verbosity = 0, plot = FALSE, nbCores = 4)
fit_bin <- cached_fit("results_analysis/results_city_SBM_bernoulli.rds",
  lapply(net_bin, estimateBipartiteSBM, model = "bernoulli",
         dimLabels = c(row = "plants", col = "insects"), estimOptions = sbm_opts))
fit_pois <- cached_fit("results_analysis/results_city_SBM_poisson.rds",
  lapply(net_city, estimateBipartiteSBM, model = "poisson",
         dimLabels = c(row = "plants", col = "insects"), estimOptions = sbm_opts))

Slide “ICL versus BIC”: the criteria along the explored models (Bristol, binary).

fit_bin$Bristol$storedModels |>
  mutate(K_plus_L = nbRowBlocks + nbColBlocks) |>
  pivot_longer(c(ICL, loglik), names_to = "criterion") |>
  ggplot(aes(K_plus_L, value, colour = criterion)) + geom_line() + geom_point() +
  scale_colour_manual(values = c(ICL = col_poll, loglik = "grey55"))

4.3.1 Bristol: SBM on the counts

fp <- fit_pois$Bristol
fp$nbBlocks
 plants insects 
      7       6 
# counts on a log scale, ordered by the estimated groups
plot_counts <- function(fit) plotMyMatrix(log1p(fit$networkData), dimLabels = c(row = "plants", col = "insects"),
  clustering = list(row = fit$memberships$plants, col = fit$memberships$insects))
plot_counts(fp)

The groups mostly sort species by their number of visits:

bind_rows(tibble(type = "plants", group = fp$memberships$plants, visits = rowSums(fp$networkData)),
          tibble(type = "insects", group = fp$memberships$insects, visits = colSums(fp$networkData))) |>
  mutate(type = factor(type, levels = c("plants", "insects"))) |>
  ggplot(aes(factor(group), visits, colour = type)) + geom_jitter(width = 0.15) +
  scale_y_log10() + facet_wrap(~type, scales = "free_x") +
  scale_colour_manual(values = c(insects = col_poll, plants = col_plant), guide = "none") + labs(x = "group")

4.3.2 Edinburgh: one insect drives the counts

Reichertella geniculata: 300 visits on Taraxacum at one site. The Poisson SBM with and without this insect:

reich <- "Reichertella geniculata"
baldock$records |> filter(insect == reich) |> count(site, plant, sort = TRUE) |> head(3)
# A tibble: 3 × 3
  site      plant                   n
  <chr>     <chr>               <int>
1 ED_OGS_09 Taraxacum agg         300
2 ED_ALT_09 Cirsium arvense       162
3 BR_ALT_04 Solidago canadensis    81
fit_pois_noR <- cached_fit("results_analysis/results_city_SBM_poisson_without_reichertella.rds",
  lapply(net_city, function(M) {
    M <- M[, colnames(M) != reich]
    M <- M[rowSums(M) > 0, colSums(M) > 0]          # plants visited only by this insect
    estimateBipartiteSBM(M, model = "poisson", dimLabels = c(row = "plants", col = "insects"),
                         estimOptions = sbm_opts)
  }))
c(with = fit_pois$Edinburgh$nbBlocks, without = fit_pois_noR$Edinburgh$nbBlocks)
    with.plants    with.insects  without.plants without.insects 
              7               7               4               6 
plot_counts(fit_pois$Edinburgh) + plot_counts(fit_pois_noR$Edinburgh)

4.3.3 Bristol: SBM on the binary network

fb <- fit_bin$Bristol
fb$nbBlocks
 plants insects 
      3       3 
round(fb$connectParam$mean, 2)        # alpha_kl
     [,1] [,2] [,3]
[1,] 0.28 0.31 0.11
[2,] 0.39 0.22 0.02
[3,] 0.14 0.03 0.00
lapply(fb$blockProp, round, 2)         # pi_k, rho_l
$plants
[1] 0.02 0.13 0.85

$insects
[1] 0.03 0.09 0.88
plot(fb, type = "data") + plot(fb, type = "expected") + plot(fb, type = "meso")

Species of the core groups (“Bristol: interpretation”):

core_p <- which.max(rowSums(fb$connectParam$mean)); core_i <- which.max(colSums(fb$connectParam$mean))
rownames(fb$networkData)[fb$memberships$plants == core_p]
[1] "Bellis perennis"       "Heracleum sphondylium" "Ranunculus repens"    
[4] "Taraxacum agg"        
colnames(fb$networkData)[fb$memberships$insects == core_i]
[1] "Apis mellifera"       "Bombus pascuorum"     "Bombus terrestris"   
[4] "Eristalis arbustorum" "Eristalis tenax"     
table(plants = fb$memberships$plants); table(insects = fb$memberships$insects)
plants
  1   2   3 
  4  25 164 
insects
  1   2   3 
  5  17 158 

Composition of the groups: plant status and insect group (“Bristol: composition of the groups”).

composition <- function(group, value, cols, title) {
  tibble(group = factor(group), value = factor(value, levels = names(cols))) |>
    ggplot(aes(group, fill = value)) + geom_bar(position = "fill") +
    scale_fill_manual(values = cols, name = NULL) + labs(y = NULL, title = title)
}
composition(fb$memberships$plants, plant_status[rownames(fb$networkData)], cols_status, "plants") +
  composition(fb$memberships$insects, insect_group[colnames(fb$networkData)], cols_group, "insects")

4.3.4 One SBM per city

tibble(city = names(net_city),
       Bernoulli = sapply(fit_bin, function(f) paste(f$nbBlocks, collapse = " x ")),
       Poisson = sapply(fit_pois, function(f) paste(f$nbBlocks, collapse = " x "))) |>
  knitr::kable()
city Bernoulli Poisson
Bristol 3 x 3 7 x 6
Edinburgh 3 x 3 7 x 7
Leeds 2 x 3 5 x 6
Reading 3 x 3 5 x 6
wrap_plots(lapply(names(fit_bin), function(cc) plot(fit_bin[[cc]], type = "data") + ggtitle(cc)), nrow = 2)

“Most species are seen in one city”:

n_cities <- bind_rows(
  tibble(type = "plants", species = unlist(lapply(net_city, rownames))),
  tibble(type = "insects", species = unlist(lapply(net_city, colnames)))) |>
  count(type, species, name = "n_cities") |> mutate(type = factor(type, levels = c("plants", "insects")))
table(n_cities$type, n_cities$n_cities)
         
            1   2   3   4
  plants  226  77  35  39
  insects 195  54  45  41
ggplot(n_cities, aes(n_cities, fill = type)) + geom_bar() + facet_wrap(~type) +
  scale_fill_manual(values = c(insects = col_poll, plants = col_plant), guide = "none")

5 Modelling collections of networks

5.1 Application to the city networks

estimate_colBiSBM fits the colBiSBM on the 4 binary networks, and the separate SBMs (sep_BiSBM) for comparison. Models: iid (same \(\alpha\), same proportions), pi / rho (free plant / insect proportions). About 30 min per model (scripts/fit_colSBM_cities.R).

col_fits <- lapply(c(iid = "iid", pi = "pi", rho = "rho"), function(model)
  cached_fit(paste0("results_analysis/results_city_colBiSBM_", model, ".rds"),
    estimate_colBiSBM(netlist = net_bin, colsbm_model = model, net_id = names(net_bin),
                      distribution = "bernoulli", nb_run = 3,
                      global_opts = list(nb_cores = max(1, parallel::detectCores() - 1),
                                         verbosity = 0, plot_details = 0))))

“Joint or separate models?”: the BICL of the joint models and of the separate SBMs.

bind_rows(
  tibble(model = "sep-BiSBM (one per city)",
         blocks = paste(sapply(col_fits$iid$sep_BiSBM$models, function(m) paste(m$Q, collapse = "x")), collapse = ", "),
         BICL = sum(col_fits$iid$sep_BiSBM$BICL)),
  tibble(model = paste0(names(col_fits), "-colBiSBM"),
         blocks = sapply(col_fits, function(f) paste(f$best_fit$Q, collapse = " x ")),
         BICL = sapply(col_fits, function(f) f$best_fit$BICL))) |>
  mutate(BICL = round(BICL)) |> knitr::kable()
model blocks BICL
sep-BiSBM (one per city) 3x3, 3x3, 2x3, 3x3 -8780
iid-colBiSBM 5 x 3 -8651
pi-colBiSBM 5 x 3 -8677
rho-colBiSBM 5 x 3 -8665

5.1.1 The common structure

\(\hat\alpha_{k\ell}\) shared by the 4 cities, with the block proportions \(\hat\pi_k\), \(\hat\rho_\ell\):

fit <- col_fits$iid$best_fit
round(fit$alpha, 2)
     [,1] [,2] [,3]
[1,] 0.35 0.51 0.19
[2,] 0.40 0.29 0.07
[3,] 0.35 0.17 0.02
[4,] 0.22 0.03 0.00
[5,] 0.04 0.04 0.01
lapply(fit$pi[[1]], round, 2)
[[1]]
[1] 0.01 0.04 0.12 0.49 0.34

[[2]]
[1] 0.04 0.13 0.84
plot(fit, type = "meso", values = TRUE)

“The same groups in every city”: each city reordered by the joint groups.

wrap_plots(lapply(seq_along(fit$net_id), function(m)
  plot(fit, type = "block", net_id = m) + ggtitle(fit$net_id[m])), nrow = 2)

The joint groups of each species in each city:

groups <- extract_nodes_groups(fit) |>
  mutate(type = ifelse(node_type == "row", "plant", "insect"),
         value = ifelse(type == "plant", plant_status[node_name], insect_group[node_name])) |>
  rename(city = network, species = node_name, block = cluster)
head(groups)
  city                species block node_type  type   value
1    1        Acanthus mollis     4       row plant unknown
2    1 Achillea filipendulina     5       row plant unknown
3    1     Alliaria petiolata     5       row plant  Native
4    1    Allium ampeloprasum     4       row plant   Mixed
5    1            Allium cepa     4       row plant   Mixed
6    1         Allium ursinum     5       row plant   Mixed

“Joint modelling: more statistical power”: number of species of each plant group per city (P1: 1 to 3 species per city).

groups |> filter(type == "plant") |> count(city, block = paste0("P", block)) |>
  pivot_wider(names_from = block, values_from = n, values_fill = 0)
# A tibble: 4 × 6
  city     P1    P2    P3    P4    P5
  <chr> <int> <int> <int> <int> <int>
1 1         1     7    25   100    60
2 2         3     3    11    68    52
3 3         1     8    20    65    61
4 4         2     8    17    86    43

“The 4 cities: composition of the groups” (a species seen in 2 cities counts twice):

with(filter(groups, type == "plant"), composition(paste0("P", block), value, cols_status, "plants: status")) +
  with(filter(groups, type == "insect", !is.na(value)),
       composition(paste0("I", block), value, cols_group, "insects: group"))

“The same composition as in Bristol?”: the Bristol groups (separate SBM) vs the joint groups.

g_bristol <- filter(groups, city == "Bristol")
table(separate = fb$memberships$plants[match(g_bristol$species[g_bristol$type == "plant"], rownames(fb$networkData))],
      joint = g_bristol$block[g_bristol$type == "plant"])
< table of extent 0 x 0 >
table(separate = fb$memberships$insects[match(g_bristol$species[g_bristol$type == "insect"], colnames(fb$networkData))],
      joint = g_bristol$block[g_bristol$type == "insect"])
< table of extent 0 x 0 >

5.1.2 Does a species keep its group across cities?

For each species seen in 2 cities and each pair of cities: its group in the two cities.

pairs_cities <- inner_join(groups, groups, by = c("type", "species"), relationship = "many-to-many") |>
  filter(city.x < city.y)
pairs_cities |> group_by(type) |> summarise(same_group = mean(block.x == block.y))
# A tibble: 2 × 2
  type   same_group
  <chr>       <dbl>
1 insect      0.8  
2 plant       0.522
with(filter(pairs_cities, type == "plant"), table(pmin(block.x, block.y), pmax(block.x, block.y)))
   
      1   2   3   4   5
  1   6   8   1   0   0
  2   0  16  18   4   8
  3   0   0  21  58  29
  4   0   0   0 116  73
  5   0   0   0   0  58
with(filter(pairs_cities, type == "insect"), table(pmin(block.x, block.y), pmax(block.x, block.y)))
   
      1   2   3
  1  19  18   4
  2   0  61  65
  3   0   0 268

“The insects that change group”:

changing <- groups |> filter(type == "insect") |> group_by(species) |>
  filter(n() > 1, n_distinct(block) > 1) |> mutate(mean_block = mean(block)) |> ungroup()
c(changing = n_distinct(changing$species),
  seen_in_several_cities = sum(table(groups$species[groups$type == "insect"]) > 1))
              changing seen_in_several_cities 
                    28                    140 
ggplot(changing, aes(city, reorder(species, -mean_block), fill = paste0("I", block))) +
  geom_tile(colour = "white") + geom_text(aes(label = paste0("I", block)), size = 3) +
  scale_fill_manual(values = c(I1 = "#D45E00", I2 = "#F5A26B", I3 = "#FBE3D0"), guide = "none") +
  labs(x = NULL, y = NULL) + theme(axis.text.y = element_text(face = "italic"))

“Ecological messages” and “A nested backbone”: the core groups hold species seen in many cities, the peripheral groups species seen in 1 or 2 cities.

groups |> left_join(n_cities |> mutate(type = sub("s$", "", type)), by = c("type", "species")) |>
  group_by(type, block) |>
  summarise(n = n(), seen_in_4_cities = round(mean(n_cities == 4), 2), .groups = "drop")
# A tibble: 8 × 4
  type   block     n seen_in_4_cities
  <chr>  <int> <int>            <dbl>
1 insect     1    20             1   
2 insect     2    76             0.75
3 insect     3   506             0.17
4 plant      1     7             1   
5 plant      2    26             0.77
6 plant      3    73             0.45
7 plant      4   319             0.18
8 plant      5   216             0.18
groups |> filter(type == "plant", block <= 2) |> count(block, species, sort = TRUE) |> head(10)
   block                   species n
1      2         Ranunculus repens 4
2      1           Bellis perennis 3
3      1             Taraxacum agg 3
4      2      Rubus fruticosus agg 3
5      2          Senecio jacobaea 3
6      2           Cirsium arvense 2
7      2     Heracleum sphondylium 2
8      2      Leucanthemum vulgare 2
9      2 Scorzoneroides autumnalis 2
10     1     Heracleum sphondylium 1

5.2 Application to the land uses

One network per land use, pooled over the 4 cities (“The 9 land-use networks”):

net_lu <- lapply(baldock$net_landuse, function(M) 1 * (M > 0))
tibble(land_use = names(net_lu), visits = sapply(baldock$net_landuse, sum),
       plants = sapply(net_lu, nrow), insects = sapply(net_lu, ncol),
       connectance = round(sapply(net_lu, mean), 3)) |>
  arrange(desc(visits)) |> knitr::kable()
land_use visits plants insects connectance
Allotment 1657 180 155 0.024
Garden 1060 237 145 0.019
Cemetery 540 50 120 0.043
Other greenspace 530 25 62 0.072
Nature reserve 467 52 113 0.037
Park 385 42 100 0.051
Verge 207 28 72 0.061
Pavement 47 20 22 0.075
Manmade surface 28 12 15 0.117

clusterize_bipartite_networks looks for a partition of the networks into groups, each modelled by its own colBiSBM (divisive algorithm, score = sum of the BICL of the groups). The scores of all the steps (the trees of the slides) are printed during the fit with verbosity = 1; the result keeps the final partition.

partition_table <- function(clu) {
  bind_rows(lapply(clu$partition, function(p) tibble(
    group = paste(p$best_fit$net_id, collapse = ", "),
    blocks = paste(p$best_fit$Q, collapse = " x "), BICL = round(p$best_fit$BICL, 1)))) |>
    mutate(score = sum(BICL))
}
opts_clu <- list(nb_cores = max(1, parallel::detectCores() - 1), verbosity = 1, plot_details = 0)

5.2.1 The 9 land uses (\(\pi\rho\)-colBiSBM)

About 1.5 h (scripts/fit_colSBM_landuse.R).

clu9 <- cached_fit("results_analysis/results_landuse_colBiSBM_clustering_pirho.rds",
  clusterize_bipartite_networks(netlist = net_lu, colsbm_model = "pirho", net_id = names(net_lu),
                                nb_run = 3, global_opts = opts_clu))
clu9$clustering_history
  Allotment Cemetery Garden Manmade surface Nature reserve Other greenspace
1         1        1      1               1              1                1
2         1        1      1               2              1                1
  Park Pavement Verge
1    1        1     1
2    1        1     1
partition_table(clu9) |> knitr::kable()
group blocks BICL score
Allotment, Cemetery, Garden, Nature reserve, Other greenspace, Park, Pavement, Verge 4 x 4 -8659.7 -8727.1
Manmade surface 1 x 1 -67.4 -8727.1

5.2.2 Without the 3 small land uses (iid-colBiSBM)

Other greenspace, Pavement and Man-made surface removed. About 10 min (scripts/fit_colSBM_landuse6_clustering.R).

landuses6 <- c("Allotment", "Cemetery", "Garden", "Nature reserve", "Park", "Verge")
clu6 <- cached_fit("results_analysis/results_landuse6_colBiSBM_clustering_iid.rds",
  clusterize_bipartite_networks(netlist = net_lu[landuses6], colsbm_model = "iid", net_id = landuses6,
                                nb_run = 3, global_opts = opts_clu))
clu6$clustering_history
  Allotment Cemetery Garden Nature reserve Park Verge
1         1        1      1              1    1     1
2         1        2      1              2    2     2
partition_table(clu6) |> knitr::kable()
group blocks BICL score
Allotment, Garden 3 x 4 -5291.8 -8159.8
Cemetery, Nature reserve, Park, Verge 3 x 3 -2868.0 -8159.8
fits6 <- lapply(clu6$partition, function(p) p$best_fit)
names(fits6) <- ifelse(sapply(fits6, function(f) "Cemetery" %in% f$net_id), "greenspaces", "gardened")

5.2.3 Two different structures

Graphon view (“Two different structures: graphon view”) and \(\hat\alpha\) of each group:

plot(fits6$greenspaces, type = "graphon") + ggtitle("Cemetery, Nature reserve, Park, Verge") +
  plot(fits6$gardened, type = "graphon") + ggtitle("Allotment, Garden")

lapply(fits6, function(f) round(f$alpha, 2))
$gardened
     [,1] [,2] [,3] [,4]
[1,] 0.47 0.35 0.16 0.03
[2,] 0.36 0.07 0.02 0.00
[3,] 0.00 0.07 0.04 0.01

$greenspaces
     [,1] [,2] [,3]
[1,] 0.57 0.83 0.22
[2,] 0.37 0.02 0.10
[3,] 0.09 0.00 0.02
plot(fits6$greenspaces, type = "meso", values = TRUE) | plot(fits6$gardened, type = "meso", values = TRUE)

5.2.4 Greenspaces: a core–periphery

g_green <- extract_nodes_groups(fits6$greenspaces) |>
  mutate(type = ifelse(node_type == "row", "plant", "insect"),
         value = ifelse(type == "plant", plant_status[node_name], insect_group[node_name]))
# plants of the two core groups (all land uses pooled), with their status
g_green |> filter(type == "plant", cluster <= 2) |> distinct(cluster, node_name, value) |> arrange(cluster)
   cluster                 node_name  value
1        1           Bellis perennis Native
2        1     Heracleum sphondylium Native
3        1             Taraxacum agg Native
4        2     Anthriscus sylvestris Native
5        2         Crepis capillaris Native
6        2      Hypochaeris radicata Native
7        2      Leucanthemum vulgare Native
8        2         Ranunculus repens Native
9        2 Scorzoneroides autumnalis Native
10       2          Senecio jacobaea Native
11       2             Taraxacum agg Native
12       2           Cirsium arvense Native
13       2          Ranunculus acris Native
14       2      Rubus fruticosus agg Native
15       2   Chamerion angustifolium Native
16       2     Heracleum sphondylium Native
17       2          Trifolium repens Native
# insect groups: composition
g_green |> filter(type == "insect") |> count(cluster, value) |> group_by(cluster) |>
  mutate(p = round(n / sum(n), 2)) |> select(-n) |> pivot_wider(names_from = value, values_from = p, values_fill = 0)
# A tibble: 3 × 8
# Groups:   cluster [3]
  cluster   Bee Hoverfly `Other Diptera` Beetle `Butterfly / moth`
    <int> <dbl>    <dbl>           <dbl>  <dbl>              <dbl>
1       1  0.39     0.37            0.24   0                  0   
2       2  0.07     0.15            0.59   0.08               0.02
3       3  0.15     0.23            0.37   0.17               0.04
# ℹ 2 more variables: `Other Hymenoptera` <dbl>, `NA` <dbl>

5.2.5 Gardens: bee flowers vs fly flowers

g_gard <- extract_nodes_groups(fits6$gardened) |>
  mutate(type = ifelse(node_type == "row", "plant", "insect"),
         value = ifelse(type == "plant", plant_status[node_name], insect_group[node_name]))
g_gard |> filter(type == "plant") |> count(cluster, value) |>
  pivot_wider(names_from = value, values_from = n, values_fill = 0)
# A tibble: 3 × 5
  cluster Mixed Native `Non-native` unknown
    <int> <int>  <int>        <int>   <int>
1       1     1     21           25       3
2       2    16     30           71      43
3       3    19     47           87      54
# insects in I2 in one land use and in I3 in the other
g_gard |> filter(type == "insect") |> group_by(node_name) |>
  filter(n() == 2, n_distinct(cluster) == 2) |> ungroup() |> count(network, cluster)
# A tibble: 7 × 3
  network cluster     n
  <chr>     <int> <int>
1 1             1     1
2 1             2     5
3 1             3     9
4 1             4     4
5 2             2     3
6 2             3     9
7 2             4     7

6 Session

packageVersion("colSBM"); packageVersion("sbm")
[1] '0.4.11'
[1] '0.4.7'