Functions and tidy evaluation

Lesson 9 — R Programming

Lesson 9 of 13 Intermediate to advanced ~170 min

Learning objectives

Write functions with sensible arguments, defaults and return values
Explain lazy evaluation, promises and why rlang exists
Capture arguments with quosures and inject them with !!, !!! and { }
Convert between strings and symbols, and use the .data and .env pronouns
Use dynamic dots, name injection with :=, and rlang conditions
Use the whole apply family correctly, including why sapply() is unsafe
Choose between base *apply and purrr, and know the mapping between them
Validate arguments and fail with useful messages

When to write a function

The rule of three: if you have copy-pasted a block twice, make it a function. The real trigger is subtler — write a function as soon as you find yourself editing the same logic in two places and worrying you missed one.

# Repeated three times with different columns
adsl$age_cat    <- cut(adsl$AGE, c(-Inf, 18, 65, Inf), labels = c("<18","18-64",">=65"))
adsl$weight_cat <- cut(adsl$WEIGHT, c(-Inf, 60, 90, Inf), labels = c("low","mid","high"))

# Once
categorise <- function(x, breaks, labels) {
  cut(x, breaks = c(-Inf, breaks, Inf), labels = labels, right = FALSE)
}

Anatomy

calculate_bmi <- function(weight, height, units = c("cm", "m")) {
  units <- match.arg(units)

  height_m <- switch(units, cm = height / 100, m = height)
  weight / height_m^2
}

calculate_bmi(70, 175)            #> 22.86
calculate_bmi(70, 1.75, "m")      #> 22.86
calculate_bmi(70, 175, "inches")
#> Error in match.arg(units) : 'arg' should be one of "cm", "m"

match.arg() with a character-vector default gives you validated choices and uses the first as the default. Use it whenever an argument has a fixed set of allowed values.

Return values

R returns the last evaluated expression. Use explicit return() only for early exits:

safe_divide <- function(x, y) {
  if (y == 0) {
    return(NA_real_)      # early exit — explicit return is right here
  }
  x / y                   # normal path — no return() needed
}

invisible() returns a value without printing it, which is what you want for functions called for side effects:

write_output <- function(data, path) {
  readr::write_csv(data, path)
  cli::cli_alert_success("Wrote {nrow(data)} rows to {.path {path}}")
  invisible(data)         # allows piping to continue
}

adsl |> write_output("out.csv") |> nrow()

Lazy evaluation

Arguments are not evaluated until used:

f <- function(x, y) {
  x * 2       # y is never touched
}
f(5, stop("boom"))
#> [1] 10      <- no error!

This enables defaults that reference other arguments:

summarise_var <- function(x, label = deparse(substitute(x)), digits = 2) {
  sprintf("%s: %.*f (%.*f)", label, digits, mean(x, na.rm = TRUE),
          digits, sd(x, na.rm = TRUE))
}

age <- c(45, 52, 38)
summarise_var(age)
#> [1] "age: 45.00 (7.00)"

It also causes a classic bug in loops that create functions:

fns <- list()
for (i in 1:3) {
  fns[[i]] <- function() i
}
fns[[1]]()
#> [1] 3       <- all three see the final value of i

# Force evaluation
fns <- lapply(1:3, function(i) { force(i); function() i })
fns[[1]]()
#> [1] 1

Functions that take column names

This is where R functions get awkward, and it is worth understanding why rather than memorising the syntax.

dplyr uses data masking: you write filter(df, age > 65), not filter(df, df$age > 65). The bare name age is not an object in your workspace — it is a column, resolved inside the data frame. That is convenient interactively and it breaks the moment you wrap it in a function.

# Does not work
summarise_by <- function(data, group_var, value_var) {
  data |>
    summarise(mean = mean(value_var), .by = group_var)
}
summarise_by(adsl, ARM, AGE)
#> Error in `summarise()`:
#> i In argument: `mean = mean(value_var)`.
#> Caused by error:
#> ! object 'ARM' not found

The function received the expression AGE, passed it along unevaluated as value_var, and summarise() then looked for a column literally called value_var. The rlang package exists to solve exactly this.

How R function arguments actually work

R arguments are promises: an unevaluated expression plus the environment to evaluate it in. Nothing is computed until the argument is used.

f <- function(x) {
  substitute(x)          # the EXPRESSION the caller typed
}
f(AGE + 1)
#> AGE + 1

class(f(AGE + 1))
#> [1] "call"

substitute() is base R’s way of capturing the expression. It works, but it loses the environment, which matters when the expression refers to local variables. rlang captures both together.

Quosures

A quosure is an expression plus its environment. enquo() captures an argument as one.

library(rlang)

f <- function(x) {
  enquo(x)
}

q <- f(AGE + 1)
q
#> <quosure>
#> expr: ^AGE + 1
#> env:  global

quo_get_expr(q)     #> AGE + 1
quo_get_env(q)       #> <environment: R_GlobalEnv>

Why the environment matters:

threshold <- 65

make_filter <- function(data, condition) {
  cond <- enquo(condition)
  dplyr::filter(data, !!cond)
}

local({
  threshold <- 40                    # a DIFFERENT threshold, locally
  make_filter(adsl, AGE > threshold) # uses 40, because the quosure remembers
})

The quosure carries the environment where threshold means 40. substitute() would have lost that and found the global 65 instead. This is why rlang is built on quosures rather than bare expressions.

`!!` — unquote

!! (pronounced “bang-bang”) takes a quosure or expression and injects it into the surrounding code, replacing the name with what it holds.

summarise_by <- function(data, group_var, value_var) {
  gv <- enquo(group_var)
  vv <- enquo(value_var)

  data |>
    summarise(mean = mean(!!vv, na.rm = TRUE), .by = !!gv)
}

summarise_by(adsl, ARM, AGE)
#> # A tibble: 3 x 2
#>   ARM      mean
#>   <chr>   <dbl>
#> 1 Placebo  75.2

Use rlang::qq_show() to see exactly what gets built — this is the single most useful debugging tool for tidy evaluation:

f <- function(var) {
  v <- enquo(var)
  qq_show(summarise(data, m = mean(!!v)))
}
f(AGE)
#> summarise(data, m = mean(^AGE))

`{ }` — the shorthand

{ var } is exactly !!enquo(var). It is the form to use in almost all cases.

summarise_by <- function(data, group_var, value_var) {
  data |>
    summarise(
      n    = n(),
      mean = mean({{ value_var }}, na.rm = TRUE),
      sd   = sd({{ value_var }},   na.rm = TRUE),
      .by  = {{ group_var }}
    )
}

summarise_by(adsl, ARM, AGE)
summarise_by(adsl, c(ARM, SEX), AGE)      # works with multiple grouping vars

Read { x } as “take what the user typed and put it here”. It works in any data-masking argument: filter(), mutate(), group_by(), arrange(), summarise(), count(), and aes() in ggplot2.

When you need enquo() rather than { }

{ } embraces at the point of use. If you need the captured expression itself — to inspect it, modify it, or use it more than once in a way that { } cannot express — capture it explicitly.

plot_var <- function(data, var) {
  v <- enquo(var)

  ggplot(data, aes(x = !!v)) +
    geom_histogram() +
    labs(x = as_label(v),                      # the expression as text
         title = paste("Distribution of", as_label(v)))
}
plot_var(adsl, AGE)

as_label() turns the quosure back into a string for the axis label. You cannot do that with { } alone.

`!!!` — splice

!!! injects a list of expressions as separate arguments.

vars <- exprs(AGE, WEIGHT, HEIGHT)      # a list of expressions

adsl |> select(!!!vars)
# equivalent to select(adsl, AGE, WEIGHT, HEIGHT)

# Building several mutations programmatically
spec <- list(
  AGE_MONTHS = expr(AGE * 12),
  BMI        = expr(WEIGHT / (HEIGHT / 100)^2)
)

adsl |> mutate(!!!spec)

This is the mechanism behind metadata-driven derivation — see Metadata-driven programming.

Expressions without environments

When you do not need the environment, expr() and enexpr() are lighter than quosures.

Function	Captures	From
`expr()`	One expression	What you write
`exprs()`	Several expressions	What you write
`enexpr()`	One expression	A function argument
`enexprs()`	Several	`...`
`quo()` / `enquo()`	Expression + environment	As above
`quos()` / `enquos()`	Several, with environments	As above

expr(AGE + 1)                 #> AGE + 1
exprs(AGE, SEX)               #> list of 2 expressions

f <- function(x) enexpr(x)
f(AGE + 1)                    #> AGE + 1    (no environment attached)

Rule of thumb: use quosures (enquo, { }) for anything evaluated in user data, and expressions (enexpr, expr) for code you are constructing or inspecting.

Strings and symbols

Column names often arrive as character strings — from a configuration file, a specification spreadsheet, or a Shiny input. Strings and bare names are different things, and converting between them is a common need.

sym("AGE")                    #> AGE          — a symbol
syms(c("AGE", "SEX"))         #> list of 2 symbols

as_name(sym("AGE"))           #> [1] "AGE"    — symbol back to string
as_label(expr(AGE + 1))       #> [1] "AGE + 1" — any expression to string

ensym() captures an argument that may be given either way:

flexible <- function(data, var) {
  v <- ensym(var)             # accepts AGE or "AGE"
  data |> summarise(mean = mean(!!v, na.rm = TRUE))
}

flexible(adsl, AGE)           # bare name
flexible(adsl, "AGE")         # string — both work

The `.data` and `.env` pronouns

For strings, the cleanest approach is the .data pronoun. No conversion needed.

summarise_by_name <- function(data, group_col, value_col) {
  data |>
    summarise(
      n    = n(),
      mean = mean(.data[[value_col]], na.rm = TRUE),
      .by  = all_of(group_col)
    )
}

summarise_by_name(adsl, "ARM", "AGE")

Context	Strings	Bare names
Data masking (`mutate`, `filter`)	`.data[[x]]`	`{ x }`
Selection (`select`, `across`)	`all_of(x)` / `any_of(x)`	`{ x }`

all_of() errors if a name is missing; any_of() silently ignores missing names. Use all_of() unless absence is genuinely acceptable.

.env is the mirror image — it forces a name to resolve in the environment, not the data:

AGE <- 100                    # a variable that shadows a column name

adsl |> filter(AGE > .env$AGE)       # column AGE vs the variable AGE
adsl |> filter(.data$AGE > .env$AGE) # explicit on both sides

.data is a pronoun, not a data frame

.data                         # outside a verb: error
names(.data)                  # does not work
.data[["AGE"]] |> mean()      # does not work outside a masking context

.data only exists inside a data-masking argument. It is not an object you can inspect, subset outside a verb, or pass around. If you need the actual data, use the data frame.

Naming outputs with `:=`

To name a new column after the input, you need := (walrus) and glue syntax.

add_change <- function(data, value_var, baseline_var) {
  data |>
    mutate("{{ value_var }}_chg" := {{ value_var }} - {{ baseline_var }})
}

adlb |> add_change(AVAL, BASE)
#> new column AVAL_chg

# With a string
add_flag <- function(data, col, threshold) {
  data |>
    mutate("{col}_high" := .data[[col]] > threshold)
}

adlb |> add_flag("AVAL", 40)
#> new column AVAL_high

englue() gives you the same string outside a mutate():

describe <- function(data, var) {
  label <- englue("Summary of {{ var }}")
  cat(label, "\n")
  data |> summarise(mean = mean({{ var }}, na.rm = TRUE))
}
describe(adsl, AGE)
#> Summary of AGE

Selection helpers

For arguments that select columns rather than compute on them, tidyselect semantics apply and { } passes them through unchanged.

round_columns <- function(data, cols, digits = 2) {
  data |> mutate(across({{ cols }}, \(x) round(x, digits)))
}

adlb |> round_columns(c(AVAL, BASE, CHG))
adlb |> round_columns(where(is.numeric))
adlb |> round_columns(starts_with("A"))
adlb |> round_columns(all_of(numeric_cols))     # a character vector

All the tidyselect helpers work: starts_with(), ends_with(), contains(), matches(), num_range(), where(), everything(), last_col(), all_of(), any_of().

Dynamic dots

... forwards any number of arguments. rlang adds useful behaviour on top.

plot_endpoint <- function(data, param, ...) {
  data |>
    filter(PARAMCD == param) |>
    ggplot(aes(AVISITN, AVAL, ...)) +
    geom_line()
}

plot_endpoint(adlb, "ALT", colour = TRT01A, group = USUBJID)

Capturing and inspecting them:

f <- function(...) {
  dots <- enquos(...)          # a named list of quosures
  names(dots)
  length(dots)
  dots
}

g <- function(...) {
  args <- list2(...)           # evaluates them, supports !!! splicing
  args
}

g(a = 1, !!!list(b = 2, c = 3))
#> $a [1] 1
#> $b [1] 2
#> $c [1] 3

list2() is list() plus splicing support and trailing-comma tolerance. Use it whenever a function takes ....

Catching argument typos:

summarise_safe <- function(data, var, ...) {
  check_dots_empty()           # error if anything extra was passed
  data |> summarise(mean = mean({{ var }}))
}

summarise_safe(adsl, AGE, na.rm = TRUE)
#> Error: `...` must be empty.
#> x Problematic argument:
#> * na.rm = TRUE

Without that check, a misspelled argument name is silently swallowed by ... — a genuinely nasty class of bug because the function appears to work.

Related: check_dots_used() errors if a forwarded argument was never consumed.

rlang conditions

rlang provides the error, warning and message functions that cli builds on.

abort("Something went wrong")                     # error
warn("Something looks suspicious")                # warning
inform("Progress update")                         # message
signal("Custom event", class = "my_event")        # arbitrary condition

The structured form, with a bulleted message and metadata:

validate_adsl <- function(data) {
  required <- c("USUBJID", "TRT01P", "SAFFL")
  missing  <- setdiff(required, names(data))

  if (length(missing) > 0) {
    abort(
      message = c(
        "`data` is not a valid ADSL dataset.",
        "x" = paste0("Missing variable(s): ", paste(missing, collapse = ", ")),
        "i" = "Check the ADSL specification."
      ),
      class   = "adsl_missing_vars",       # so callers can handle it
      missing = missing                    # metadata on the condition
    )
  }
  invisible(data)
}

Because the condition is classed and carries data, a caller can respond programmatically rather than parsing the message text:

tryCatch(
  validate_adsl(bad_data),
  adsl_missing_vars = function(cnd) {
    message("Deriving ", length(cnd$missing), " missing variable(s)")
    derive_missing(bad_data, cnd$missing)
  }
)

Argument-error helpers that produce consistent messages:

f <- function(x, type) {
  if (!is.numeric(x)) {
    abort(c("`x` must be numeric.",
            "x" = paste0("You supplied ", obj_type_friendly(x), ".")))
  }
  type <- arg_match(type, c("mean", "median", "sum"))
  switch(type, mean = mean(x), median = median(x), sum = sum(x))
}

f(1:10, "men")
#> Error in `f()`:
#> ! `type` must be one of "mean", "median", or "sum".
#> i Did you mean "mean"?

arg_match() is match.arg() with a spelling suggestion — worth preferring.

Useful rlang utilities

# Default for NULL — also in base R since 4.4
attr(x, "label") %||% "no label"

# Call a function with arguments in a list
exec("mean", 1:10, na.rm = TRUE)
exec(mean, !!!list(x = 1:10, na.rm = TRUE))

# Name a vector in one step
set_names(c(1, 2, 3), c("a", "b", "c"))
set_names(1:3, letters[1:3])
set_names(c("ALT", "AST"))             # names from the values themselves

# Type predicates — stricter and clearer than base
is_character(x, n = 1)        # a length-1 character vector
is_bare_numeric(x)
is_scalar_integerish(x)
is_empty(x)
is_named(x)
has_name(df, "USUBJID")

# Argument checks that produce standard messages
check_required(var)
check_number_whole(n, min = 1)
check_bool(verbose)
check_character(cols)

cli or rlang for messages?

They layer: rlang::abort() is the condition machinery, and cli::cli_abort() wraps it to add glue interpolation, pluralisation and semantic markup ({.var}, {.path}, {.val}).

Use cli_abort() for user-facing errors. Use rlang::abort() directly when you need a custom class with metadata and do not need the formatting — or pass class = to cli_abort(), which forwards it.

The apply family

Base R’s *apply functions apply a function over a structure and collect the results. They predate purrr by two decades, appear throughout existing code, and have no dependencies — all three are good reasons to know them properly.

The family, at a glance:

Function	Input	Returns	Notes
`lapply()`	List or vector	Always a list	Predictable; the workhorse
`sapply()`	List or vector	Simplified — varies	Convenient, type-unstable
`vapply()`	List or vector	The type you declare	Type-safe; prefer over `sapply`
`apply()`	Matrix or array	Vector, matrix or list	Operates on rows or columns
`mapply()`	Several vectors	Simplified	Multivariate `sapply`
`Map()`	Several vectors	Always a list	Multivariate `lapply`
`tapply()`	Vector + factor	Grouped result	Precursor to `group_by`
`rapply()`	Nested list	Recursive	Rare
`eapply()`	Environment	List	Rare

`lapply()` — always a list

lapply(1:3, function(x) x^2)
#> [[1]]
#> [1] 1
#> [[2]]
#> [1] 4
#> [[3]]
#> [1] 9

The return is a list of the same length as the input, every time, regardless of what the function returns. That predictability is its virtue.

# Over the columns of a data frame — a data frame IS a list of columns
lapply(adsl, class)
#> $USUBJID
#> [1] "character"
#> $AGE
#> [1] "numeric"

# Over files
lapply(files, read.csv)

# Extra arguments pass through after the function
lapply(list(c(1, NA, 3), c(4, 5, NA)), mean, na.rm = TRUE)
#> [[1]]
#> [1] 2
#> [[2]]
#> [1] 4.5

`sapply()` — simplifies, unpredictably

sapply() is lapply() followed by an attempt to simplify the result.

sapply(1:3, function(x) x^2)
#> [1] 1 4 9                        — simplified to a vector

sapply(adsl, class)
#>     USUBJID         AGE
#> "character"   "numeric"           — a named character vector

Convenient. The problem is that the return type depends on the data:

f <- function(x) if (x > 1) c(x, x) else x

sapply(1:3, f)
#> [[1]] 1        — a LIST, because lengths differ
#> [[2]] 2 2
#> [[3]] 3 3

sapply(2:3, f)
#>      [,1] [,2]
#> [1,]    2    3   — a MATRIX, because lengths match
#> [2,]    2    3

sapply(integer(0), f)
#> list()           — an empty LIST

Three different return types from the same function, decided by the input. In a script you are watching, fine. Inside a function whose output feeds something else, it is a latent bug.

The empty-input case is the one that bites

sapply(character(0), nchar)
#> list()

A pipeline that works all year produces a list() instead of a numeric vector the first time a study has no records for a domain — and the error surfaces somewhere else entirely.

sapply(..., simplify = FALSE) is just lapply(). If you want simplification and safety, use vapply().

`vapply()` — type-safe

vapply() takes a template describing one return value, and errors if any iteration does not match.

vapply(adsl, class, character(1))
#>     USUBJID         AGE
#> "character"   "numeric"

vapply(1:3, function(x) x^2, numeric(1))
#> [1] 1 4 9

# The template is the contract
vapply(1:3, function(x) c(x, x), numeric(1))
#> Error in vapply(1:3, function(x) c(x, x), numeric(1)) :
#>   values must be length 1, but FUN(X[[1]]) result is length 2

vapply(character(0), nchar, integer(1))
#> integer(0)              — correct type even when empty

The FUN.VALUE template:

Template	Guarantees
`numeric(1)`	One double per element
`integer(1)`	One integer
`character(1)`	One string
`logical(1)`	One `TRUE`/`FALSE`
`numeric(2)`	Exactly two doubles — result is a 2×n matrix
`character(3)`	Three strings each

Named templates name the result rows:

vapply(
  split(adsl$AGE, adsl$ARM),
  function(x) c(mean = mean(x), sd = sd(x)),
  c(mean = 0, sd = 0)
)
#>      Placebo   Drug A
#> mean   75.21    74.38
#> sd      8.59     7.89

Use vapply() anywhere sapply() appears inside a function. It is the same amount of typing plus a template, and it converts a silent type change into an immediate error.

`apply()` — over matrix margins

apply() is for matrices and arrays, and MARGIN selects the dimension.

m <- matrix(1:6, nrow = 2)
m
#>      [,1] [,2] [,3]
#> [1,]    1    3    5
#> [2,]    2    4    6

apply(m, 1, sum)        # MARGIN = 1 -> rows
#> [1]  9 12

apply(m, 2, sum)        # MARGIN = 2 -> columns
#> [1]  3  7 11

apply(m, c(1, 2), \(x) x * 10)    # both margins, element-wise

The margin you name is the one retained; the other is collapsed. With MARGIN = 1 you get one result per row.

Returning several values gives a matrix — transposed from what people usually expect:

apply(m, 2, range)
#>      [,1] [,2] [,3]
#> [1,]    1    3    5     — results are COLUMNS, not rows
#> [2,]    2    4    6

apply() on a data frame coerces to a matrix

df <- data.frame(id = c("001", "002"), age = c(45, 52))

apply(df, 1, function(row) row["age"])
#>  age  age
#> "45" "52"        — STRINGS, not numbers

A matrix holds one type. apply() converts the data frame first, and a single character column turns every value into a string. Arithmetic then silently fails or produces nonsense.

For row-wise work on a data frame use one of:

df |> rowwise() |> mutate(total = sum(c(a, b, c)))     # dplyr
df |> mutate(total = pmap_dbl(list(a, b, c), sum))     # purrr
rowSums(df[, c("a", "b", "c")])                        # vectorised, best

rowSums(), rowMeans(), colSums() and colMeans() are also much faster than apply() for those specific operations.

`mapply()` and `Map()` — several inputs

mapply(function(x, y) x + y, 1:3, c(10, 20, 30))
#> [1] 11 22 33

Map(function(x, y) x + y, 1:3, c(10, 20, 30))
#> [[1]] 11
#> [[2]] 22
#> [[3]] 33

Map() is mapply(SIMPLIFY = FALSE) — always a list, so it is the predictable one. mapply() simplifies, with the same unpredictability as sapply().

A realistic use — reading files with per-file arguments:

Map(
  function(path, skip) read.csv(path, skip = skip),
  files,
  skip_rows
)

# MoreArgs for arguments that are the same every time
mapply(rep, 1:3, 3:1, MoreArgs = list(), SIMPLIFY = FALSE)

`tapply()` — grouped summaries

The ancestor of group_by() |> summarise().

tapply(adsl$AGE, adsl$ARM, mean)
#>  Placebo   Drug A   Drug B
#>    75.21    74.38    75.67

tapply(adsl$AGE, list(adsl$ARM, adsl$SEX), mean)     # two factors -> a matrix

Related base grouping tools:

split(adsl, adsl$ARM)                 # a list of data frames
split(adsl$AGE, adsl$ARM)             # a list of vectors

aggregate(AGE ~ ARM, data = adsl, FUN = mean)        # formula interface
by(adsl, adsl$ARM, function(d) summary(d$AGE))

# split + lapply is a very common base-R idiom
adsl |> split(adsl$ARM) |> lapply(\(d) mean(d$AGE))

For anything analytical, dplyr is clearer — see Data manipulation with dplyr. These matter when reading existing code or working without dependencies.

`do.call()` — arguments from a list

Not an apply function, but constantly used alongside them.

do.call(sum, list(1, 2, 3))             #> [1] 6
do.call("rbind", list_of_dataframes)    # bind a list into one frame
do.call(paste, c(list_of_strings, sep = "-"))

The classic use is combining a list of results:

results <- lapply(files, read.csv)
combined <- do.call(rbind, results)

# Modern equivalents
combined <- dplyr::bind_rows(results)
combined <- purrr::list_rbind(results)

bind_rows() matches columns by name and fills gaps with NA; do.call(rbind, ...) matches by position and errors on a mismatch. Prefer bind_rows().

Functionals: `Reduce`, `Filter`, `Map`, `Find`

Reduce(`+`, 1:5)                        #> [1] 15
Reduce(`+`, 1:5, accumulate = TRUE)     #> [1]  1  3  6 10 15
Reduce(function(a, b) merge(a, b, by = "USUBJID"), list_of_dfs)

Filter(\(x) x > 2, 1:5)                 #> [1] 3 4 5
Filter(is.numeric, adsl)                # keep numeric columns

Find(\(x) x > 2, 1:5)                   #> [1] 3    — the first match
Position(\(x) x > 2, 1:5)               #> [1] 3    — its index

Reduce() with a merge function is the base-R way to join a list of data frames — the equivalent of purrr::reduce(dfs, left_join, by = "USUBJID").

A performance note

A widely repeated claim is that apply functions are faster than loops. They are not, in general — they are loops written in C around an R function call, and the R function call dominates.

x <- runif(1e6)

# Loop
system.time({ out <- numeric(length(x)); for (i in seq_along(x)) out[i] <- x[i] * 2 })
#> ~0.4s

# sapply
system.time(out <- sapply(x, \(v) v * 2))
#> ~1.2s     — slower

# Vectorised
system.time(out <- x * 2)
#> ~0.005s   — 80x faster than the loop

The real speedup comes from vectorisation, not from apply. Choose apply functions for clarity — they state “apply this to each element” more directly than a loop with an index — and reach for a vectorised operation when speed matters.

Where apply genuinely helps performance: it pre-allocates the result, so it avoids the quadratic growth of result <- c(result, x) in a loop.

`purrr`

purrr is the tidyverse’s answer to the apply family: consistent naming, type-stable returns, and better handling of multiple inputs and errors.

The `map` family

library(purrr)

map(1:3, \(x) x^2)          # always a list, like lapply
map_dbl(1:3, \(x) x^2)      #> [1] 1 4 9
map_int(1:3, \(x) x * 2L)
map_chr(letters[1:3], toupper)
map_lgl(1:3, \(x) x > 1)
map_vec(dates, \(d) d + 1)  # any vector type, including Date

The typed variants are vapply() with a better interface — they check the type and error immediately:

map_dbl(1:3, \(x) as.character(x))
#> Error in `map_dbl()`:
#> i In index: 1.
#> Caused by error:
#> ! Can't coerce from a string to a double.

Note the error names the index that failed, which vapply() does not.

Several inputs

map2(x, y, \(a, b) a + b)
map2_dbl(c(1, 2, 3), c(10, 20, 30), \(a, b) a * b)
#> [1] 10 40 90

pmap(list(a = 1:2, b = 3:4, c = 5:6), \(a, b, c) a + b + c)
pmap_chr(list(x = c("a", "b"), n = 1:2), \(x, n) strrep(x, n))
#> [1] "a"  "bb"

# pmap over a data frame's rows — each column becomes an argument
params <- tibble(path = files, skip = skip_rows)
pmap(params, \(path, skip) read_csv(path, skip = skip))

imap(list(a = 1, b = 2), \(x, nm) paste(nm, x))       # value and name
#> $a "a 1"
#> $b "b 2"

pmap() over a data frame of parameters is a genuinely useful pattern: one row per job, one column per argument.

Side effects

walk(files, \(f) file.remove(f))         # returns input invisibly
walk2(data_list, paths, write_csv)
iwalk(results, \(x, nm) saveRDS(x, paste0(nm, ".rds")))

Use walk when you want the side effect, not the return value. It makes the intent explicit.

Filtering and reshaping lists

keep(1:10, \(x) x > 5)              #> [1]  6  7  8  9 10
discard(1:10, \(x) x > 5)           #> [1] 1 2 3 4 5
compact(list(1, NULL, 2, NULL))     #> drops NULLs

keep(adsl, is.numeric)              # numeric columns only

list_c(list(1:2, 3:4))              #> [1] 1 2 3 4
list_rbind(list_of_tibbles)         # stack into one tibble
list_rbind(list_of_tibbles, names_to = "source")
list_cbind(list_of_tibbles)

pluck(nested, "a", 1, "b")          # deep extraction, NULL if absent
pluck(nested, "a", 99, .default = NA)
modify(1:3, \(x) x * 2)             # same type as input
map_depth(nested, 2, \(x) x * 2)    # map at a given nesting level

Reduce and accumulate

reduce(1:5, `+`)                            #> [1] 15
reduce(list_of_dfs, left_join, by = "USUBJID")
reduce(1:5, `+`, .init = 100)               #> [1] 115
accumulate(1:5, `+`)                        #> [1]  1  3  6 10 15
reduce2(x, y, f)                            # reduce with a second input

reduce() with left_join is the idiomatic way to merge a list of domains.

Error handling

safe_read <- safely(read_csv)
res <- safe_read("missing.csv")
res$result      #> NULL
res$error       #> <simpleError ...>

ok_read <- possibly(read_csv, otherwise = NULL)
quiet_read <- quietly(read_csv)       # captures output, messages and warnings

# Over many files, collecting both outcomes
out    <- map(files, safely(read_csv))
data   <- out |> map("result") |> compact()
errors <- out |> map("error")  |> compact()

This is the pattern for a batch job that must process forty files and report on the three that failed, rather than stopping at the first.

Anonymous function syntax

map_dbl(x, function(v) mean(v, na.rm = TRUE))   # verbose
map_dbl(x, \(v) mean(v, na.rm = TRUE))          # base R 4.1+ — preferred
map_dbl(x, ~ mean(.x, na.rm = TRUE))            # purrr formula
map_dbl(x, mean, na.rm = TRUE)                  # pass arguments directly

Prefer \(x): it is base R, works with the apply family too, and named arguments make it readable. The ~ .x form is shorter for one-liners and still very common in existing code. In map2/pmap formulas the arguments are .x, .y and ..1, ..2, ..3.

Base and purrr side by side

Task	Base R	purrr
List out	`lapply(x, f)`	`map(x, f)`
Double out	`vapply(x, f, numeric(1))`	`map_dbl(x, f)`
Character out	`vapply(x, f, character(1))`	`map_chr(x, f)`
Two inputs	`Map(f, x, y)`	`map2(x, y, f)`
Many inputs	`mapply(f, ...)`	`pmap(list(...), f)`
With names	—	`imap(x, f)`
Side effects	`for` / `invisible(lapply())`	`walk(x, f)`
Keep matching	`Filter(f, x)`	`keep(x, f)`
Drop `NULL`	`x[!sapply(x, is.null)]`	`compact(x)`
Accumulate	`Reduce(f, x)`	`reduce(x, f)`
Running total	`Reduce(f, x, accumulate = TRUE)`	`accumulate(x, f)`
Combine rows	`do.call(rbind, x)`	`list_rbind(x)`
Catch errors	`tryCatch()` per call	`safely()` / `possibly()`
Deep extract	`x[["a"]][[1]]`	`pluck(x, "a", 1)`

Which to use

Base R when the code must have no dependencies — a package with a minimal Imports, a validated environment with an approved package list, or a utility script. vapply() and Map() are perfectly good tools.

purrr when you already have the tidyverse. The consistency is worth real money on a large codebase: the suffix tells you the return type, the error messages name the failing index, and safely()/possibly() have no concise base equivalent.

Do not mix them arbitrarily within one file. Pick one per project and be consistent.

Validating arguments

Fail early, and say what is wrong:

library(cli)

derive_bmi <- function(data, weight_var, height_var, height_units = c("cm", "m")) {
  if (!is.data.frame(data)) {
    cli_abort("{.arg data} must be a data frame, not {.obj_type_friendly {data}}.")
  }
  height_units <- match.arg(height_units)

  w <- data |> dplyr::pull({{ weight_var }})
  h <- data |> dplyr::pull({{ height_var }})

  if (!is.numeric(w)) cli_abort("{.arg weight_var} must be numeric.")
  if (!is.numeric(h)) cli_abort("{.arg height_var} must be numeric.")

  if (any(h <= 0, na.rm = TRUE)) {
    n_bad <- sum(h <= 0, na.rm = TRUE)
    cli_abort(c(
      "Height must be positive.",
      "x" = "Found {n_bad} non-positive value{?s}.",
      "i" = "Check the units — are these metres recorded as centimetres?"
    ))
  }

  h_m <- if (height_units == "cm") h / 100 else h
  dplyr::mutate(data, BMI = w / h_m^2)
}

cli_abort() gives you glue interpolation, pluralisation ({?s}), semantic markup ({.arg}, {.path}, {.val}) and a bulleted structure with x for the problem and i for the hint. It costs nothing extra and makes errors diagnosable by someone who did not write the function.

For quick internal checks, stopifnot() is fine:

stopifnot(
  "data must be a data frame" = is.data.frame(data),
  "weights must be positive"  = all(w > 0, na.rm = TRUE)
)

Common mistakes

Mistake	Consequence	Fix
Bare column name in a function	“object not found”	`{ }`
String column name with `{ }`	Treated as a literal string	`.data[[x]]` / `all_of()`
`substitute()` instead of `enquo()`	Loses the environment	`enquo()` / `{ }`
Using `.data` outside a masking context	Error — it is a pronoun	Only inside a verb
`"{col}_new" = ...` instead of `:=`	Syntax error	`:=` for computed names
Typo silently swallowed by `...`	Function appears to work	`check_dots_empty()`
`sapply()` inside a function	Return type varies with the data	`vapply()` with a template
`sapply()` on empty input	Returns `list()`	`vapply()`
`apply()` on a mixed data frame	Coerces every value to character	`rowwise()`, `pmap()`, `rowSums()`
Wrong `MARGIN` in `apply()`	Aggregated the wrong way	The named margin is retained
`mapply()` for predictable output	Simplifies unpredictably	`Map()`
`do.call(rbind, ...)` on ragged frames	Errors or misaligns	`bind_rows()` / `list_rbind()`
Expecting apply to be faster than a loop	No gain	Vectorise instead
`map()` when you need a vector	List where you expected numbers	`map_dbl()` etc.
No argument validation	Cryptic errors deep in the call stack	`cli_abort()` at the top
Functions with side effects	Untestable, order-dependent	Return a value
Very long argument lists	Unusable	Pass a config list or a data frame

Exercise 8.1 — A reusable summary function

Write summary_stats(data, var, by) that returns n, mean, SD, median, Q1, Q3, min and max of var, grouped by by. It must work with bare column names, and by must be optional.

Show solution

library(dplyr)

summary_stats <- function(data, var, by = NULL) {
  data |>
    summarise(
      n      = sum(!is.na({{ var }})),
      nmiss  = sum(is.na({{ var }})),
      mean   = mean({{ var }}, na.rm = TRUE),
      sd     = sd({{ var }}, na.rm = TRUE),
      median = median({{ var }}, na.rm = TRUE),
      q1     = quantile({{ var }}, 0.25, na.rm = TRUE),
      q3     = quantile({{ var }}, 0.75, na.rm = TRUE),
      min    = min({{ var }}, na.rm = TRUE),
      max    = max({{ var }}, na.rm = TRUE),
      .by    = {{ by }}
    )
}

summary_stats(adsl, AGE)
summary_stats(adsl, AGE, by = ARM)
summary_stats(adsl, AGE, by = c(ARM, SEX))

.by = {{ by }} handles NULL naturally — summarise(.by = NULL) is ungrouped, which is exactly the desired default.

One refinement for real use: min() and max() of an all-NA vector return Inf/-Inf with a warning. Guard it:

      min = if (all(is.na({{ var }}))) NA_real_ else min({{ var }}, na.rm = TRUE),

Exercise 8.2 — Metadata-driven derivation

Given a specification tibble with columns source, target and factor, write apply_conversions(data, spec) that creates each target column as source * factor. Column names arrive as strings.

Show solution

library(dplyr); library(purrr); library(cli)

apply_conversions <- function(data, spec) {
  missing_cols <- setdiff(spec$source, names(data))
  if (length(missing_cols) > 0) {
    cli_abort(c(
      "Specification refers to column{?s} not present in {.arg data}.",
      "x" = "Missing: {.var {missing_cols}}"
    ))
  }

  reduce(
    seq_len(nrow(spec)),
    function(dat, i) {
      src <- spec$source[i]
      tgt <- spec$target[i]
      fct <- spec$factor[i]
      mutate(dat, "{tgt}" := .data[[src]] * fct)
    },
    .init = data
  )
}

spec <- tibble::tribble(
  ~source,   ~target,     ~factor,
  "WEIGHT",  "WEIGHT_LB",  2.20462,
  "HEIGHT",  "HEIGHT_IN",  0.393701
)

adsl |> apply_conversions(spec) |> select(USUBJID, WEIGHT, WEIGHT_LB)

reduce() threads the data frame through one mutate() per specification row. The alternative — building a list of expressions and splicing with !!! — is more idiomatic tidyeval but harder to read:

apply_conversions2 <- function(data, spec) {
  exprs <- set_names(
    map2(spec$source, spec$factor, \(s, f) rlang::expr(.data[[!!s]] * !!f)),
    spec$target
  )
  mutate(data, !!!exprs)
}

Both are correct. Choose based on who has to maintain it.

Exercise 8.3 — Batch import with error collection

Write import_all(dir) that reads every CSV in a directory, adds a source column, combines them, and returns a list with data and failures (a tibble of file name and error message) rather than stopping on the first failure.

Show solution

library(purrr); library(dplyr); library(readr); library(fs)

import_all <- function(dir) {
  files <- dir_ls(dir, glob = "*.csv")

  if (length(files) == 0) {
    cli::cli_warn("No CSV files found in {.path {dir}}")
    return(list(data = tibble(), failures = tibble()))
  }

  results <- map(files, safely(\(f) read_csv(f, show_col_types = FALSE)))

  ok  <- map(results, "result")
  err <- map(results, "error")

  data <- ok |>
    compact() |>
    list_rbind(names_to = "source") |>
    mutate(source = path_file(source))

  failures <- tibble(
    file    = path_file(names(err)),
    message = map_chr(err, \(e) if (is.null(e)) NA_character_ else conditionMessage(e))
  ) |>
    filter(!is.na(message))

  cli::cli_alert_info(
    "Imported {length(compact(ok))}/{length(files)} file{?s}; {nrow(failures)} failure{?s}."
  )

  list(data = data, failures = failures)
}

res <- import_all("data/raw")
res$failures
#> # A tibble: 1 x 2
#>   file      message
#>   <chr>     <chr>
#> 1 lb_v2.csv Column `RESULT` can't be converted from character to double

The design choice worth noting: returning a list with both successes and failures, rather than warning and discarding. A batch job that silently skips three of forty files is worse than one that crashes — at least the crash is visible. This pattern gives you both the data and an auditable record of what did not load.

list_rbind() requires the tibbles to have compatible columns; if the files genuinely differ, list_rbind() fills with NA and you should check names() consistency explicitly before combining.

Exercise 8.4 — A tidyeval function that accepts either form

Write tabulate_by(data, var, by, pct = TRUE) producing counts and optional percentages of var within by. It must accept either bare column names or strings for both var and by, name the percentage column after var, and label the output with the variable name.

Show solution

library(dplyr)
library(rlang)

tabulate_by <- function(data, var, by, pct = TRUE) {
  # ensym() accepts AGEGR1 or "AGEGR1" and yields a symbol either way
  var_sym <- ensym(var)
  by_sym  <- ensym(by)

  var_name <- as_name(var_sym)
  by_name  <- as_name(by_sym)

  # Fail early and clearly if the columns are absent
  missing <- setdiff(c(var_name, by_name), names(data))
  if (length(missing) > 0) {
    cli::cli_abort(c(
      "Column{?s} not found in {.arg data}.",
      "x" = "Missing: {.var {missing}}",
      "i" = "Available: {.var {head(names(data), 8)}}"
    ))
  }

  out <- data |>
    count(!!by_sym, !!var_sym, name = "n")

  if (pct) {
    out <- out |>
      mutate(
        "{var_name}_pct" := round(100 * n / sum(n), 1),
        .by = !!by_sym
      )
  }

  structure(out, variable = var_name, grouped_by = by_name)
}

# Bare names
tabulate_by(adsl, AGEGR1, TRT01A)
#> # A tibble: 9 x 4
#>   TRT01A  AGEGR1     n AGEGR1_pct
#>   <chr>   <chr>  <int>      <dbl>
#> 1 Placebo <65       14       16.3
#> 2 Placebo 65-80     42       48.8
#> 3 Placebo >80       30       34.9

# Strings — identical result
tabulate_by(adsl, "AGEGR1", "TRT01A")

# Mixed, which also works
tabulate_by(adsl, AGEGR1, "TRT01A", pct = FALSE)

attr(tabulate_by(adsl, AGEGR1, TRT01A), "variable")
#> [1] "AGEGR1"

Six things doing the work

ensym() rather than enquo(). ensym() accepts a bare name or a string and returns a symbol in both cases. enquo() would capture the string "AGEGR1" as a literal character value, and !! would then inject a string where a column name is needed. Try it to see the difference:

f_quo  <- function(v) { q <- enquo(v); qq_show(count(data, !!q)) }
f_sym  <- function(v) { s <- ensym(v); qq_show(count(data, !!s)) }

f_quo("AGEGR1")   #> count(data, ^"AGEGR1")    — a string literal
f_sym("AGEGR1")   #> count(data, AGEGR1)       — a column name

as_name() converts the symbol to a string so it can be used in the error message, the glue name and the attribute. as_label() would also work here but is for arbitrary expressions; as_name() is the right one for a symbol.

!! rather than { }. Because the symbol was captured into a variable, it must be injected with !!. { var_sym } would try to capture var_sym itself — the local variable, not what it holds.

:= with "{var_name}_pct". A computed name on the left of a mutate() assignment requires :=; = is a syntax error. The glue braces interpolate the string.

.by = !!by_sym inside mutate() makes the percentages sum to 100 within each group rather than across the whole table. Dropping .by is the most common error in this kind of function and produces plausible-looking wrong numbers.

The structure() call attaches the variable name as metadata, so a downstream table program can label the row block without being told separately.

One refinement worth knowing. If you wanted to accept several variables, switch to ensyms() and splice:

tabulate_many <- function(data, ..., by) {
  vars <- ensyms(...)
  by_sym <- ensym(by)
  data |> count(!!by_sym, !!!vars, name = "n")
}
tabulate_many(adsl, AGEGR1, SEX, by = TRT01A)

Exercise 8.5 — Replace unsafe sapply() calls

This function works in testing and fails in production. Identify every unsafe construct, then rewrite it twice — once in base R, once with purrr.

summarise_domains <- function(paths) {
  data <- lapply(paths, read.csv)
  n_rows <- sapply(data, nrow)
  n_cols <- sapply(data, ncol)
  types  <- sapply(data, function(d) sapply(d, class))
  ids    <- sapply(data, function(d) unique(d$USUBJID))
  data.frame(
    file = basename(paths),
    rows = n_rows,
    cols = n_cols,
    n_subjects = sapply(ids, length)
  )
}

Show solution

Five problems

sapply(data, nrow) returns list() if paths is empty, so the data.frame() call then fails with a confusing length error rather than returning zero rows.
sapply(d, class) is doubly unsafe. class() can return more than one value (a POSIXct column returns c("POSIXct", "POSIXt")), so the inner sapply returns a character vector for some files and a list for others. The outer sapply then cannot simplify consistently.
sapply(d, function(d) unique(d$USUBJID)) returns a vector when all files have the same number of subjects and a list when they do not — a matrix in the first case, a list in the second.
No error handling. One unreadable file aborts all forty.
read.csv with inferred types means a column is numeric in one file and character in another, as covered in Importing data.

Base R rewrite

summarise_domains <- function(paths) {
  stopifnot(is.character(paths))

  if (length(paths) == 0) {
    return(data.frame(
      file = character(0), rows = integer(0), cols = integer(0),
      n_subjects = integer(0), status = character(0),
      stringsAsFactors = FALSE
    ))
  }

  read_one <- function(p) {
    tryCatch(
      read.csv(p, colClasses = "character"),
      error = function(e) NULL
    )
  }

  data <- lapply(paths, read_one)
  ok   <- !vapply(data, is.null, logical(1))

  n_subjects <- vapply(data, function(d) {
    if (is.null(d) || !"USUBJID" %in% names(d)) NA_integer_
    else length(unique(d$USUBJID))
  }, integer(1))

  data.frame(
    file       = basename(paths),
    rows       = vapply(data, \(d) if (is.null(d)) NA_integer_ else nrow(d), integer(1)),
    cols       = vapply(data, \(d) if (is.null(d)) NA_integer_ else ncol(d), integer(1)),
    n_subjects = n_subjects,
    status     = ifelse(ok, "OK", "READ FAILED"),
    stringsAsFactors = FALSE
  )
}

Every sapply() is now a vapply() with a template, so the type is fixed regardless of the data — and integer(1) forces each branch to return an integer, which is why the NA is written NA_integer_ rather than bare NA (a logical).

purrr rewrite

library(purrr)
library(dplyr)
library(readr)

summarise_domains <- function(paths) {
  stopifnot(is.character(paths))

  empty <- tibble(
    file = character(), rows = integer(), cols = integer(),
    n_subjects = integer(), status = character(), error = character()
  )
  if (length(paths) == 0) return(empty)

  results <- map(paths, safely(\(p) read_csv(p, col_types = cols(.default = "c"))))

  tibble(
    file   = basename(paths),
    result = map(results, "result"),
    error  = map_chr(results, \(r) if (is.null(r$error)) NA_character_
                                   else conditionMessage(r$error))
  ) |>
    mutate(
      rows = map_int(result, \(d) if (is.null(d)) NA_integer_ else nrow(d)),
      cols = map_int(result, \(d) if (is.null(d)) NA_integer_ else ncol(d)),
      n_subjects = map_int(result, \(d) {
        if (is.null(d) || !"USUBJID" %in% names(d)) NA_integer_
        else dplyr::n_distinct(d$USUBJID)
      }),
      status = if_else(is.na(error), "OK", "READ FAILED")
    ) |>
    select(file, rows, cols, n_subjects, status, error)
}

summarise_domains(c("data/dm.csv", "data/ae.csv", "data/broken.csv"))
#> # A tibble: 3 x 6
#>   file        rows  cols n_subjects status      error
#>   <chr>      <int> <int>      <int> <chr>       <chr>
#> 1 dm.csv       306    24        306 OK          NA
#> 2 ae.csv      1847    31        218 OK          NA
#> 3 broken.csv    NA    NA         NA READ FAILED could not open file

summarise_domains(character(0))
#> # A tibble: 0 x 6         — correct columns, zero rows

The separate column-types question

The original types object tried to summarise column classes and could not, because the result is ragged. That belongs in its own long-format function rather than a cell in a wide table:

column_types <- function(paths) {
  map(paths, \(p) {
    d <- read_csv(p, col_types = cols(.default = "c"), n_max = 0)
    tibble(file = basename(p), column = names(d),
           type = map_chr(d, \(x) class(x)[1]))
  }) |>
    list_rbind()
}

class(x)[1] takes only the first class, which is what makes map_chr() safe here — and n_max = 0 reads the header only, so this is fast even on large files.

Which rewrite to prefer

The purrr version is better for this job: safely() has no concise base equivalent, the error message is captured per file rather than discarded, and map_int() names the failing index when something does go wrong. The base version is the one to use in a package with a minimal dependency footprint, and vapply() makes it just as type-safe — only more verbose.

Recap

R arguments are promises — an expression plus an environment
A quosure captures both; that is why enquo() beats substitute()
{ x } is exactly !!enquo(x); !! injects one, !!! splices many
rlang::qq_show() shows you the code that tidy evaluation actually builds
Strings: .data[[x]] to compute, all_of(x) to select, sym() to convert
:= with glue syntax for computed column names; englue() outside mutate()
list2() for ..., check_dots_empty() to catch argument typos
abort(class =, metadata =) for errors callers can handle programmatically
lapply() always returns a list; vapply() enforces a type; sapply() guesses
sapply() returns a list on empty input — never use it inside a function
apply() on a data frame coerces to character; use rowwise() or rowSums()
Apply functions buy clarity, not speed — vectorise for performance
purrr suffixes declare the return type and errors name the failing index

Next: Error handling and debugging.

When to write a function

Anatomy

Return values

Lazy evaluation

Functions that take column names

How R function arguments actually work

Quosures

!! — unquote

{ } — the shorthand

!!! — splice

Expressions without environments

Strings and symbols

The .data and .env pronouns

Naming outputs with :=

Selection helpers

Dynamic dots

rlang conditions

Useful rlang utilities

The apply family

lapply() — always a list

sapply() — simplifies, unpredictably

vapply() — type-safe

apply() — over matrix margins

mapply() and Map() — several inputs

tapply() — grouped summaries

do.call() — arguments from a list

Functionals: Reduce, Filter, Map, Find

A performance note

purrr

The map family

Several inputs

Side effects

Filtering and reshaping lists

Reduce and accumulate

Error handling

Anonymous function syntax

Base and purrr side by side

Validating arguments

Common mistakes

`!!` — unquote

`{ }` — the shorthand

`!!!` — splice

The `.data` and `.env` pronouns

Naming outputs with `:=`

`lapply()` — always a list

`sapply()` — simplifies, unpredictably

`vapply()` — type-safe

`apply()` — over matrix margins

`mapply()` and `Map()` — several inputs

`tapply()` — grouped summaries

`do.call()` — arguments from a list

Functionals: `Reduce`, `Filter`, `Map`, `Find`

`purrr`

The `map` family