Parallel Computing in R Exercises: 15 Practice Problems
Fifteen practice problems on parallel computing in R: mclapply, future, furrr, doParallel, parallel maps. Hidden solutions.
library(future)
library(furrr)
library(parallel)
library(foreach)
library(future.apply)
library(tibble)Exercise 1: detectCores
Difficulty: Beginner.
Show solution
parallel::detectCores()Exercise 2: mclapply (unix)
Difficulty: Intermediate.
Show solution
parallel::mclapply(1:4, function(x) { Sys.sleep(0.5); x^2 }, mc.cores = 2)Exercise 3: parLapply with cluster
Difficulty: Advanced.
Show solution
cl <- parallel::makeCluster(2)
parallel::parLapply(cl, 1:4, function(x) x^2)
parallel::stopCluster(cl)Exercise 4: future multisession
Difficulty: Intermediate.
Show solution
future::plan(future::multisession, workers = 2)
f <- future::future({ Sys.sleep(0.5); 42 })
future::value(f)Exercise 5: furrr future_map
Difficulty: Intermediate.
Show solution
future::plan(future::multisession, workers = 2)
furrr::future_map_dbl(1:4, ~ { Sys.sleep(0.5); .x^2 })Exercise 6: future_map_dfr
Difficulty: Intermediate.
Show solution
future::plan(future::multisession, workers = 2)
furrr::future_map_dfr(1:3, ~ tibble::tibble(n = .x, sq = .x^2))Exercise 7: Globals: pass variables
Difficulty: Advanced.
Show solution
future::plan(future::multisession, workers = 2)
k <- 10
furrr::future_map_dbl(1:5, ~ .x * k)Exercise 8: Set seed reproducibly
Difficulty: Advanced.
Show solution
future::plan(future::multisession, workers = 2)
furrr::future_map_dbl(1:5, ~ runif(1), .options = furrr::furrr_options(seed = 123))Exercise 9: doParallel + foreach
Difficulty: Advanced.
Show solution
cl <- parallel::makeCluster(2)
doParallel::registerDoParallel(cl)
foreach::foreach(i = 1:4, .combine = c) %dopar% { Sys.sleep(0.3); i^2 }
parallel::stopCluster(cl)Exercise 10: future plan sequential
Difficulty: Beginner. Reset to sequential.
Show solution
future::plan(future::sequential)Exercise 11: Compare serial vs parallel
Difficulty: Advanced.
Show solution
slow <- function(x) { Sys.sleep(0.5); x^2 }
future::plan(future::multisession, workers = 4)
serial_time <- system.time(lapply(1:8, slow))
parallel_time <- system.time(furrr::future_map(1:8, slow))
list(serial = serial_time["elapsed"], parallel = parallel_time["elapsed"])Exercise 12: parSapply
Difficulty: Intermediate.
Show solution
cl <- parallel::makeCluster(2)
parallel::parSapply(cl, 1:5, function(x) x^2)
parallel::stopCluster(cl)Exercise 13: Caret parallel via doParallel
Difficulty: Advanced.
Show solution
cl <- parallel::makeCluster(2)
doParallel::registerDoParallel(cl)
# train() with method="rf" auto-parallelizes when registered
parallel::stopCluster(cl)Exercise 14: future apply
Difficulty: Intermediate.
Show solution
future::plan(future::multisession, workers = 2)
future.apply::future_sapply(1:5, function(x) x^2)Exercise 15: Stop cluster cleanly
Difficulty: Beginner.
Show solution
cl <- parallel::makeCluster(2)
# ... do work ...
parallel::stopCluster(cl)What to do next
- R-Performance-Optimization-Exercises (shipped), single-thread first.
- purrr-Exercises (shipped), serial map functions before parallel.