---
title: "Translating the Excel mapping commands to R"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Translating the Excel mapping commands to R}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

```{r setup}
library(datadaptor)
```

When a `mapping` object is created,

```{r}
mapping_file <- system.file("extdata", "mapping.xlsx", package = "datadaptor")
spss_file <- system.file("extdata", "mtcars_labelled.sav", package = "datadaptor")
m <- Mapping$new(spss_file, mapping_file)
```

the command blocks are translated to a list structure `m$cmd` containing intermediate 
results. 

## Excel sheets parsed

When the `mapping_file` is passed to construct a `Mapping` object `m`, the
commands of all sheets whose names start with one of the following strings will be
parsed:

```{r}
command_block_classes$sheet |>
  na.omit() |>
  unique()
```

## Parse the sheets to an R list object

The sheets are parsed to a named list `m$cmd$sheet_data_raw` with dataframes.
This is what it looks like for the  `"Free1"` sheet of the Excel file:

```{r}
m$cmd$sheet_data_raw$Free1
```

## Raw command table

The whole information in `m$cmd$sheet_data_raw` is then joint to a dataframe
```{r}
m$cmd$df_cmd_raw
```

## Generating `"command_block"`s

Each line in `m$cmd$df_cmd_raw` is translated to a `command_block()`. The whole
resulting list of all lines comprised in a `command_blocks()` object. These
objects have their own printing method. See here for the 3 first elements:

```{r}
m$cmd$command_blocks[1:3]
```

But underneath, it is a list where each element has a field `args`.
For instance, the third element looks like this:
```{r}
names(m$cmd$command_blocks[[3]])
m$cmd$command_blocks[[3]]$args
```

## Overview command table

The whole information is put to `m$cmd_tbl`:

```{r}
m$cmd_tbl
```


## `args` overview table

Lets have a look at a table with an example of the types of the objects
in `args` for each of the `"command_block"` subclasses:

  * `x`, `y`: Character strings of variable names in the data set (or to be created).
  * `varlab` Character string containing a variable label
  * `v`: numeric() (or character()) vector; value(s) of a value label
  * `vallab`: character() vector; value label(s)
  * `id_list`: a list of ids in the data,
  * valid R character vectors of R expressions: 
    * `ex`: an expression, 
    * `ex_cond`: a conditional expression, 
    * `ex_fun`: an existing function name
    * `filepath`: a filepath


<!-- <details> -->
<!--   <summary>Click here to show intermediate steps to calculate `args_tbl2`! </summary> -->

<!-- ```{r} -->
<!-- cdbs <- m$cmd_tbl -->
<!-- # Generate a `command_blocks` object with all unique types of -->
<!-- # `command_block`s  in the mapping file -->
<!-- (unique_cmd_tbl <- m$cmd_tbl[!duplicated(cdbs$action), ]) -->
<!-- # Generate/print a list of all args sub-elements: -->

<!-- args_overview <- setNames( -->
<!--   purrr::map(unique_cmd_tbl$command_blocks, "args"), -->
<!--   unique_cmd_tbl$action -->
<!-- ) -->
<!-- args_overview[1:3] -->


<!-- # Show a table of all `arg`s, and an example of their type & length -->
<!-- # for all command block types `cmd`: -->
<!-- args_tbl <- tibble::tibble( -->
<!--   cmd = unique_cmd_tbl$action, -->
<!--   example = args_overview) %>% -->
<!--   tidyr::unnest_longer(example, indices_to = "args") %>% -->
<!--   .[c(1, 3, 2)] -->

<!-- print(args_tbl, n = 1000) -->

<!-- args_tbl2 <- args_tbl %>%  -->
<!--   dplyr::group_by(arg_class = args %>%  -->
<!--     stringr::str_remove("\\d$") %>%  -->
<!--     ifelse(. == "filepath", "ex_filepath", .) %>%  -->
<!--     stringr::str_remove("(?<=ex)_.*") %>%  -->
<!--     stringr::str_remove("s$")   -->
<!--   ) %>%  -->
<!--   dplyr::group_by(cmd) %>% -->
<!--   dplyr::mutate(args = list(args))  %>% -->
<!--   tidyr::pivot_wider(names_from = arg_class, -->
<!--                      values_from = example, -->
<!--                      values_fn = list) %>% -->
<!--   dplyr::relocate(y, varlab, .after = x)  -->
<!-- args_tbl2 -->
<!-- ``` -->

<!-- </details> -->

<!-- The following table shows an example of a subset of the columns arguments passed for each of the -->
<!-- command blocks: -->

<!-- ```{r} -->
<!-- # This is a hack to print the named arguments.  -->
<!-- # If you use RStudio, the dataframe is best shown with: -->
<!-- # View(args_tbl2) -->
<!-- args_tbl2 %>%  -->
<!--   dplyr::mutate( -->
<!--     dplyr::across( -->
<!--       x:id,  -->
<!--       ~ paste(names(.x), .x, sep = "; ", collapse = "; ") %>% -->
<!--         stringr::str_remove_all( -->
<!--           'list\\(|\\)|c\\(|; NULL|"|^; ' -->
<!--         ) -->
<!--       ) -->
<!--     ) %>%  -->
<!--   # show a subset of the columns: -->
<!--     dplyr::select(-x, -id_list, -id, -args) %>%  -->
<!--     print(n=222) -->
<!-- ``` -->



## Apply command blocks

These command blocks can then be applied to the data with 

```{r}
m$modify_data()
```

Underneath the hood, this function calls `apply_commands()` and passes it the arguments `args` (see above).

You can then access the data with
```{r}
m$dat_mod
```


<details>
  <summary>Click here to show whole list structure print output of `m$cmd`!</summary>
```{r}
m$cmd
```
  
</details>