Optimize the Silhouette Width of Hierarchical Clustering Solutions

Performs a parallelized grid search to find the number of clusters maximizing the overall silhouette width of the clustering solution (see utils_cluster_silhouette()). When method = NULL, the optimization also includes all methods available in stats::hclust() in the grid search. This function supports parallelization via future::plan() and a progress bar generated by the progressr package (see Examples).

Usage

utils_cluster_hclust_optimizer(d = NULL, method = NULL)

Arguments

d

(required, matrix) distance matrix typically resulting from distantia_matrix(), but any other square matrix should work. Default: NULL

method

(optional, character string) Argument of stats::hclust() defining the agglomerative method. One of: "ward.D", "ward.D2", "single", "complete", "average" (= UPGMA), "mcquitty" (= WPGMA), "median" (= WPGMC) or "centroid" (= UPGMC). Unambiguous abbreviations are accepted as well.

Value

data frame

See also

Other internal_dissimilarity_analysis: utils_block_size(), utils_cluster_kmeans_optimizer(), utils_cluster_silhouette(), utils_importance_df_to_wide()

Examples

#parallelization and progress bar
#for large datasets, parallelization accelerates cluster optimization
future::plan(
  future::multisession,
  workers = 2 #set to parallelly::availableWorkers() - 1
)

#progress bar
# progressr::handlers(global = TRUE)

#weekly covid prevalence
#in 10 California counties
#aggregated by month
tsl <- tsl_initialize(
  x = covid_prevalence,
  name_column = "name",
  time_column = "time"
) |>
  tsl_subset(
    names = 1:10
  ) |>
  tsl_aggregate(
    new_time = "months",
    fun = max
  )

if(interactive()){
  #plotting first three time series
  tsl_plot(
    tsl = tsl_subset(
      tsl = tsl,
      names = 1:3
    ),
    guide_columns = 3
  )
}

#compute dissimilarity matrix
psi_matrix <- distantia(
  tsl = tsl,
  lock_step = TRUE
) |>
  distantia_matrix()

#optimize hierarchical clustering
hclust_optimization <- utils_cluster_hclust_optimizer(
  d = psi_matrix
)

#best solution in first row
head(hclust_optimization)
#>   clusters   method silhouette_mean
#> 1        2  average       0.3175009
#> 2        2 mcquitty       0.3175009
#> 3        2   median       0.3175009
#> 4        5   ward.D       0.3080265
#> 5        5  ward.D2       0.3080265
#> 6        5 complete       0.3080265

#disable parallelization
future::plan(
  future::sequential
)