How can I use parallel so that it preserves the list of data frames

0 votes

The function my_function takes parameter stored in a data frame. parameters and take one extra parameter as another data frame independently in indf

library(tidyverse)

my_function <- function (x=NULL,y=NULL,z=NULL, indf=NULL) {
 out <- (x * y *z )
 out * indf
}


parameters <- tribble(
  ~x, ~y, ~z,
  5,     1,  1,
  10,     5,  3,
  -3,    10,  5
)

indf <- tribble(
  ~A, ~B, ~C,
  100,     10,  1,
  1000,     300,  3,
  20,    10,  5
)


parameters %>% 
  pmap(my_function, indf=indf)

The output shows the list of data frames

#> [[1]]
#>      A    B  C
#> 1  500   50  5
#> 2 5000 1500 15
#> 3  100   50 25
#> 
#> [[2]]
#>        A     B   C
#> 1  15000  1500 150
#> 2 150000 45000 450
#> 3   3000  1500 750
#> 
#> [[3]]
#>         A      B    C
#> 1  -15000  -1500 -150
#> 2 -150000 -45000 -450
#> 3   -3000  -1500 -750

When I run the above function with parallel package using the following code:

library(parallel)
parameters %>% 
  lift(mcmapply, mc.cores = detectCores() - 1)(FUN = my_function, indf=indf)

The following matrix is produced.

     [,1]  [,2] [,3]
[1,]  500  1500 -150
[2,] 5000 45000 -450
[3,]  100  1500 -750

How can I implement parallel so that it produces a list of data frames like the initial output?

Apr 4, 2018 in Data Analytics by DataKing99
• 8,130 points
34 views

1 answer to this question.

0 votes

You can use pmap as follows:

nc <- max(parallel::detectCores() - 1, 1L)

par_pmap <- function(.l, .f, ..., mc.cores = getOption("mc.cores", 2L)) {
  do.call(
    parallel::mcmapply, 
    c(.l, list(FUN = .f, MoreArgs = list(...), SIMPLIFY = FALSE, mc.cores = mc.cores))
  )
}

library(magrittr)

parameters %>% 
  par_pmap(my_function, indf = indf, mc.cores = nc)

# [[1]]
#      A    B  C
# 1  500   50  5
# 2 5000 1500 15
# 3  100   50 25
# 
# [[2]]
#        A     B   C
# 1  15000  1500 150
# 2 150000 45000 450
# 3   3000  1500 750
# 
# [[3]]
#         A      B    C
# 1  -15000  -1500 -150
# 2 -150000 -45000 -450
# 3   -3000  -1500 -750
answered Apr 4, 2018 by kappa3010
• 2,020 points

Related Questions In Data Analytics

0 votes
1 answer

How can I list all the data sets available in all R packages?

You can use the below line of ...READ MORE

answered Sep 7, 2018 in Data Analytics by zombie
• 3,690 points
38 views
0 votes
2 answers
0 votes
1 answer

How to create a list of Data frames?

Basically all we have to do is ...READ MORE

answered Apr 9, 2018 in Data Analytics by DeepCoder786
• 1,700 points
68 views
0 votes
1 answer

How to subset data so that it contains only columns whose names match a condition

You can use grepl on the names ...READ MORE

answered Apr 26, 2018 in Data Analytics by darklord
• 6,140 points
31 views
0 votes
4 answers

How to remove NA values with dplyr::filter()

Can we create a alist as below ...READ MORE

answered Aug 5 in Data Analytics by anonymous
6,216 views
0 votes
1 answer
0 votes
1 answer

How to join two tables (tibbles) by *list* columns in R

You can use the hash from digest ...READ MORE

answered Apr 5, 2018 in Data Analytics by kappa3010
• 2,020 points
54 views
0 votes
1 answer

Join list of data.frames using map() call

You can use Reduce set.seed(24) r1 <- map(c(5, 10, 15), ...READ MORE

answered Apr 5, 2018 in Data Analytics by darklord
• 6,140 points
19 views
0 votes
1 answer

How to join a list of data frames using map()

You can use reduce set.seed(24) r1 <- map(c(5, 10, ...READ MORE

answered Apr 11, 2018 in Data Analytics by kappa3010
• 2,020 points
44 views
0 votes
1 answer