xportr

Lesson 10 — Clinical Programming with R

Lesson 10 of 12 Advanced ~70 min

Learning objectives

Apply types, lengths, labels, formats and ordering from a specification
Understand every SAS transport V5 constraint and why it exists
Write conformant XPT files with strict checking
Diagnose the errors xportr raises
Fit xportr into the ADaM pipeline

The problem `xportr` solves

An R data frame and a SAS transport file are different things. R has no concept of a variable length, a SAS format, or a dataset label. Transport V5 requires all three, plus a long list of constraints:

Constraint	V5 limit
Dataset name	8 characters
Variable name	8 characters, uppercase, letter or `_` first
Variable label	40 characters
Dataset label	40 characters
Character value	200 bytes
Numeric	8-byte float only
Encoding	ASCII
Variable order	Must be specified

Applying these by hand across forty datasets is tedious and error-prone. xportr applies them from a specification and checks the result.

install.packages("xportr")
library(xportr)

The pipeline

Six functions, applied in order, then the write:

adsl |>
  xportr_type(spec)     |>   # coerce types to match the spec
  xportr_length(spec)   |>   # set the SAS length attribute
  xportr_label(spec)    |>   # apply variable labels
  xportr_order(spec)    |>   # reorder columns
  xportr_format(spec)   |>   # apply SAS display formats
  xportr_df_label(spec) |>   # apply the dataset label
  xportr_write("data/submission/adsl.xpt", strict_checks = TRUE)

The order matters: xportr_length() should follow xportr_type() because lengths apply to the final type, and xportr_order() before xportr_write() because the write is what fixes the order in the file.

The specification

xportr accepts a data frame with specific column names, or a metacore object:

spec <- tibble::tribble(
  ~dataset, ~variable,  ~type,      ~length, ~label,                       ~format,  ~order,
  "ADSL",   "STUDYID",  "character", 20,     "Study Identifier",            NA,       1,
  "ADSL",   "USUBJID",  "character", 30,     "Unique Subject Identifier",   NA,       2,
  "ADSL",   "SUBJID",   "character", 10,     "Subject Identifier",          NA,       3,
  "ADSL",   "AGE",      "numeric",    8,     "Age",                         NA,       4,
  "ADSL",   "AGEU",     "character",  6,     "Age Units",                   NA,       5,
  "ADSL",   "SEX",      "character",  1,     "Sex",                         NA,       6,
  "ADSL",   "TRTSDT",   "numeric",    8,     "Date of First Exposure",      "DATE9.", 7,
  "ADSL",   "TRT01P",   "character", 30,     "Planned Treatment for Period 1", NA,    8
)

Configure the column names if yours differ:

options(
  xportr.variable_name = "variable",
  xportr.type_name     = "type",
  xportr.length        = "length",
  xportr.label         = "label",
  xportr.format        = "format",
  xportr.order_name    = "order",
  xportr.domain_name   = "dataset",
  xportr.df_label      = "label"
)

Or pass domain = explicitly, which is clearer:

adsl |> xportr_type(spec, domain = "ADSL")

Each function

`xportr_type()`

Coerces columns to match the specification:

adsl <- adsl |> xportr_type(spec)
#> ✔ Variable type mismatches found. 2 variables coerced.
#>   AGE:    character -> numeric
#>   SITEID: numeric   -> character

The messages are the point. A variable arriving as the wrong type usually means an upstream derivation is wrong, and xportr_type() silently fixing it hides the problem. Read them.

adsl |> xportr_type(spec, verbose = "warn")   # or "message", "stop", "none"

Use verbose = "stop" in a production pipeline: a type mismatch should fail the build.

`xportr_length()`

Sets the SAS length attribute:

adsl <- adsl |> xportr_length(spec)
#> ✔ Variable lengths missing from metadata. 1 variable set to default.

Length is not truncation

xportr_length() sets the attribute. If a value is longer than the specified length, the truncation happens at write time — silently.

adsl$DCSREAS[1]
#> [1] "Adverse event leading to discontinuation of study treatment"   # 59 chars
# spec says length 40

Check before applying:

check_lengths <- function(data, spec, domain) {
  s <- dplyr::filter(spec, dataset == domain)

  purrr::map(names(data), function(v) {
    if (!is.character(data[[v]])) return(NULL)
    spec_len <- s$length[s$variable == v]
    if (length(spec_len) == 0) return(NULL)
    actual <- max(c(0L, nchar(data[[v]], type = "bytes")), na.rm = TRUE)
    if (actual > spec_len) {
      tibble::tibble(variable = v, spec = spec_len, actual = actual,
                     example = data[[v]][which.max(nchar(data[[v]], "bytes"))])
    }
  }) |> purrr::compact() |> purrr::list_rbind()
}

Either truncate deliberately (and document it) or increase the specified length. Never let it happen at write time.

`xportr_label()`

adsl <- adsl |> xportr_label(spec)
#> ✔ All variables have labels.

Errors on labels over 40 characters:

Error: Length of variable label must be 40 characters or less.
  TRT01P: "Planned Treatment for Period 1 - Full Description" (48 characters)

Shortening a label is a specification change, not a code change — update the spec, then regenerate the define.xml so the two agree.

`xportr_order()`

adsl <- adsl |> xportr_order(spec)
#> ✔ 3 variables not in the specification moved to the end.
#>   TEMP_FLAG, WORK_VAR, DEBUG_N

Those three variables should not be in the output at all. Combine with metatools::drop_unspec_vars() to remove them (lesson 5).

`xportr_format()`

Applies SAS display formats — mostly dates:

adsl <- adsl |> xportr_format(spec)

spec |> filter(!is.na(format))
#>   variable  format
#>   TRTSDT    DATE9.
#>   TRTEDT    DATE9.
#>   TRTSDTM   DATETIME20.
#>   AVAL      8.2

An R Date becomes a numeric with a DATE9. format — which is exactly how SAS represents it, and how a reviewer’s SAS session will display it.

`xportr_df_label()`

adsl <- adsl |> xportr_df_label(spec)
attr(adsl, "label")
#> [1] "Subject-Level Analysis Dataset"

`xportr_write()`

xportr_write(
  adsl,
  path = "data/submission/adsl.xpt",
  metadata = spec,
  domain = "ADSL",
  strict_checks = TRUE
)

strict_checks = TRUE turns every violation into an error rather than a warning. Use it. A warning in a build log is a warning nobody reads.

Diagnosing errors

Error in `xportr_write()`:
! The following variable names are more than 8 characters:
  TREATMENT_START, TREATMENT_STOP

Cause. V5 name limit. Fix: rename in the derivation and the specification. Note that truncation would produce a collision here — both become TREATMEN — which is why xportr refuses rather than truncating.

Error: Variable lengths exceed the maximum for SAS V5 transport:
  DCSREAS: 247 bytes (maximum 200)

Cause. A character value over 200 bytes. Fix: truncate deliberately, or split across two variables and describe it in define.xml. Truncating verbatim medical text needs a documented decision.

Error: Non-ASCII characters found in:
  AETERM (14 records), INVNAM (3 records)

Cause. V5 is ASCII. Fix:

find_non_ascii <- function(data) {
  purrr::imap(data, function(x, nm) {
    if (!is.character(x)) return(NULL)
    bad <- which(!is.na(x) & grepl("[^\x01-\x7F]", x))
    if (length(bad) == 0) return(NULL)
    tibble::tibble(variable = nm, n = length(bad),
                   examples = paste(head(unique(x[bad]), 3), collapse = " | "))
  }) |> purrr::compact() |> purrr::list_rbind()
}

find_non_ascii(adae)
#> # A tibble: 2 x 3
#>   variable     n examples
#>   <chr>    <int> <chr>
#> 1 AETERM      14 Kopfschmerzen mit Übelkeit | Diarrhée
#> 2 INVNAM       3 Müller | Sørensen

adae$AETERM <- iconv(adae$AETERM, "UTF-8", "ASCII//TRANSLIT")

Document the transliteration. Silently changing a verbatim term is a data integrity issue, and //TRANSLIT behaves differently on different platforms — check what actually changed:

changed <- tibble::tibble(before = original, after = transliterated) |>
  dplyr::filter(before != after) |>
  dplyr::distinct()

Error: Variable names must not start with a number:
  1STDOSE

Fix: rename to FSTDOSE in the derivation and the spec.

In the full pipeline

#-------------------------------------------------------------------------------
# Program: ad_adsl.R
#-------------------------------------------------------------------------------

library(metacore); library(metatools); library(admiral)
library(xportr); library(dplyr); library(here)

# --- 1. Specification -------------------------------------------------------
meta <- spec_to_metacore(here("metadata", "adam_spec.xlsx"))
spec <- select_dataset(meta, "ADSL")

# --- 2. Derive --------------------------------------------------------------
adsl <- dm |>
  mutate(TRT01P = ARM, TRT01A = ACTARM) |>
  derive_vars_merged(...) |>
  derive_var_trtdurd() |>
  mutate(SAFFL = if_else(!is.na(TRTSDT), "Y", "N"))

# --- 3. Conform to the spec -------------------------------------------------
adsl <- adsl |>
  drop_unspec_vars(spec) |>
  check_variables(spec) |>
  check_ct_data(spec, na_acceptable = TRUE) |>
  order_cols(spec) |>
  sort_by_key(spec)

# --- 4. Pre-flight checks ---------------------------------------------------
stopifnot(nrow(check_lengths(adsl, spec$var_spec, "ADSL")) == 0)
stopifnot(nrow(find_non_ascii(adsl)) == 0)
stopifnot(all(nchar(names(adsl)) <= 8))

# --- 5. Apply transport metadata and write ---------------------------------
adsl |>
  xportr_type(spec, verbose = "stop")   |>
  xportr_length(spec, verbose = "stop") |>
  xportr_label(spec, verbose = "stop")  |>
  xportr_order(spec, verbose = "stop")  |>
  xportr_format(spec)                   |>
  xportr_df_label(spec)                 |>
  xportr_write(here("data", "submission", "adsl.xpt"), strict_checks = TRUE)

# --- 6. Save the R version for downstream programs -------------------------
saveRDS(adsl, here("data", "adam", "adsl.rds"))

Steps 3–5 are identical for every dataset. Factor them out:

# R/finalise.R
finalise_adam <- function(data, meta, dataset, submission_dir = here("data", "submission")) {
  spec <- select_dataset(meta, dataset)

  data <- data |>
    drop_unspec_vars(spec) |> check_variables(spec) |>
    check_ct_data(spec, na_acceptable = TRUE) |>
    order_cols(spec) |> sort_by_key(spec)

  # Pre-flight
  bad_len <- check_lengths(data, spec$var_spec, dataset)
  if (nrow(bad_len) > 0) {
    cli::cli_abort(c("{dataset}: value{?s} exceed the specified length.",
                     stats::setNames(sprintf("%s: spec %d, actual %d",
                                             bad_len$variable, bad_len$spec, bad_len$actual),
                                     rep("x", nrow(bad_len)))))
  }
  bad_ascii <- find_non_ascii(data)
  if (nrow(bad_ascii) > 0) {
    cli::cli_abort("{dataset}: non-ASCII in {.var {bad_ascii$variable}}")
  }

  data |>
    xportr_type(spec, verbose = "stop")   |>
    xportr_length(spec, verbose = "stop") |>
    xportr_label(spec, verbose = "stop")  |>
    xportr_order(spec, verbose = "stop")  |>
    xportr_format(spec)                   |>
    xportr_df_label(spec)                 |>
    xportr_write(file.path(submission_dir, paste0(tolower(dataset), ".xpt")),
                 strict_checks = TRUE)
}

Every ADaM program then ends with one line:

finalise_adam(adsl, meta, "ADSL")

Verifying the output

# Read it back and check
check <- haven::read_xpt("data/submission/adsl.xpt")

tibble::tibble(
  variable = names(check),
  label    = purrr::map_chr(check, ~ attr(.x, "label") %||% NA_character_),
  type     = purrr::map_chr(check, ~ class(.x)[1]),
  format   = purrr::map_chr(check, ~ attr(.x, "format.sas") %||% NA_character_)
)

attr(check, "label")
#> [1] "Subject-Level Analysis Dataset"

# Compare to the source
diffdf::diffdf(adsl, check, keys = "USUBJID")

The round-trip comparison is worth doing on the first dataset of every study: it surfaces any silent truncation before forty files have been written.

Then run Pinnacle 21 or CORE on the whole submission package.

Common mistakes

Mistake	Consequence	Fix
`strict_checks = FALSE`	Non-conformant files with warnings	Always `TRUE`
Ignoring type-coercion messages	Upstream derivation bug hidden	`verbose = "stop"`
Not checking lengths first	Silent truncation at write	Pre-flight check
Non-ASCII characters	Write fails or file is non-conformant	`iconv()`, documented
Variable names over 8 characters	Write fails; truncation would collide	Rename in derivation and spec
Applying `xportr` before dropping working variables	Extra variables in the output	`drop_unspec_vars()` first
Not reading the file back	Truncation undetected until Pinnacle 21	Round-trip and compare

Exercise 10.1 — Diagnose and fix a failing write

xportr_write() fails with three errors. Explain and fix each.

Error: Variable names longer than 8 characters: TREATMENT_START, RANDOMISATION_DT
Error: Character values exceed 200 bytes: AETERM (247), DCSREAS (218)
Error: Non-ASCII characters found in: INVNAM (7 records)

Show solution

Error 1 — variable names

long_names <- names(adae)[nchar(names(adae)) > 8]
long_names
#> [1] "TREATMENT_START"  "RANDOMISATION_DT"

Truncation is not a fix: TREATMENT_START and TREATMENT_STOP both truncate to TREATMEN. Rename to standard ADaM names in both the derivation and the specification:

adae <- adae |> rename(TRTSDT = TREATMENT_START, RANDDT = RANDOMISATION_DT)

And check for collisions across the whole dataset:

check_name_collisions <- function(data) {
  short <- toupper(substr(names(data), 1, 8))
  dup   <- unique(short[duplicated(short)])
  if (length(dup) > 0) {
    cli::cli_abort(c(
      "Variable names collide when truncated to 8 characters.",
      "x" = "Stem{?s}: {.val {dup}}",
      "i" = "Affected: {.var {names(data)[short %in% dup]}}"
    ))
  }
  invisible(TRUE)
}

The specification must be updated too, or xportr_label() will then fail because the renamed variables have no spec entry.

Error 2 — character values over 200 bytes

over_limit <- adae |>
  summarise(across(where(is.character),
                   ~ max(nchar(.x, type = "bytes"), na.rm = TRUE))) |>
  pivot_longer(everything(), names_to = "variable", values_to = "max_bytes") |>
  filter(max_bytes > 200)
#>   variable max_bytes
#>   AETERM         247
#>   DCSREAS        218

adae |> filter(nchar(AETERM, type = "bytes") > 200) |> pull(AETERM) |> head(1)
#> [1] "Elevated alanine aminotransferase and aspartate aminotransferase with
#>     associated right upper quadrant abdominal discomfort reported by the
#>     investigator as possibly related to study drug administration following
#>     the week 8 visit"

Three options, in order of preference:

Option A — split across two variables. Preserves all the text.

adae <- adae |>
  mutate(
    AETERM1 = substr(AETERM, 1, 200),
    AETERM2 = substr(AETERM, 201, 400)
  )

Add both to the spec and describe the split in define.xml. This is the correct approach for verbatim text that must not be lost — and it is what SUPPQUAL’s QNAM1/QNAM2 convention exists for.

Option B — truncate with a marker. Acceptable for DCSREAS if the full text is available elsewhere.

truncate_marked <- function(x, n = 200, marker = "...") {
  ifelse(nchar(x, type = "bytes") > n,
         paste0(substr(x, 1, n - nchar(marker)), marker),
         x)
}

adae$DCSREAS <- truncate_marked(adae$DCSREAS)

Document it in define.xml and record how many values were affected.

Option C — query the data. A 247-character AE verbatim term is unusual and may be a data entry problem — several events entered in one field, for instance. Raise it with data management before engineering around it.

Always record what changed:

truncated <- tibble::tibble(
  variable = "DCSREAS",
  usubjid  = adae$USUBJID[nchar(adae$DCSREAS, "bytes") > 200],
  original = adae$DCSREAS[nchar(adae$DCSREAS, "bytes") > 200]
)
readr::write_csv(truncated, "output/qc/truncated_values.csv")

Error 3 — non-ASCII characters

non_ascii <- adae |>
  filter(grepl("[^\x01-\x7F]", INVNAM)) |>
  distinct(INVNAM)
#>   INVNAM
#> 1 Müller
#> 2 Sørensen
#> 3 Fernández

adae <- adae |>
  mutate(INVNAM_ORIG = INVNAM,
         INVNAM = iconv(INVNAM, "UTF-8", "ASCII//TRANSLIT"))

adae |> filter(INVNAM != INVNAM_ORIG) |> distinct(INVNAM_ORIG, INVNAM)
#>   INVNAM_ORIG INVNAM
#> 1 Müller      Muller
#> 2 Sørensen    Sorensen
#> 3 Fernández   Fernandez

Two cautions:

//TRANSLIT behaves differently across platforms and locales. On some systems ø becomes o, on others "o" or ?. Verify the output and, for reproducibility, use an explicit mapping instead:

transliterate <- function(x) {
  map <- c("ä"="a","ö"="o","ü"="u","ß"="ss","ø"="o","å"="a","æ"="ae",
           "é"="e","è"="e","ê"="e","á"="a","à"="a","í"="i","ó"="o","ú"="u",
           "ñ"="n","ç"="c","Ä"="A","Ö"="O","Ü"="U","Ø"="O","Å"="A","É"="E")
  for (i in seq_along(map)) x <- gsub(names(map)[i], map[i], x, fixed = TRUE)
  x
}

Investigator names are usually not in submission datasets at all — check whether INVNAM should be there. If it is a leftover working variable, drop_unspec_vars() removes the problem entirely.

Prevention

Run all three checks before xportr_write(), not after:

preflight_xpt <- function(data, spec, domain) {
  check_name_collisions(data)
  stopifnot("Variable names must be <= 8 characters" = all(nchar(names(data)) <= 8))

  bad_len <- check_lengths(data, spec, domain)
  if (nrow(bad_len)) cli::cli_abort("Values exceed spec length: {.var {bad_len$variable}}")

  bad_ascii <- find_non_ascii(data)
  if (nrow(bad_ascii)) cli::cli_abort("Non-ASCII in {.var {bad_ascii$variable}}")

  cli::cli_alert_success("{domain}: pre-flight checks passed")
  invisible(TRUE)
}

Failing at the top of the program with a clear message beats failing at the write with a stack trace, and it catches the problem while the derivation code is still in front of you.

Recap

xportr_type → length → label → order → format → df_label → write
strict_checks = TRUE and verbose = "stop" in production
xportr_length() sets the attribute; truncation happens silently at write — check first
Names over 8 characters cannot be truncated safely; collisions are silent
V5 is ASCII; transliterate deliberately with an explicit mapping and record what changed
Drop unspecified variables before applying xportr
Read the XPT back and diffdf against the source, at least once per study

Next: Pharmaverse workflows.

The problem xportr solves

The pipeline

The specification

Each function

xportr_type()

xportr_length()

xportr_label()

xportr_order()

xportr_format()

xportr_df_label()

xportr_write()