Recently, I was working on creating parameterized reports for all of the schools in the division where I work. The basic idea was to provide school leadership teams with individualized reports on several (common) key metrics that they could use to both 1) reflect on the previous year(s) and 2) set goals for the upcoming year(s).
The beauty of parameterized reporting via RMarkdown is that you can build a template report, define some parameters that will vary within each iteration of the report, and then render several reports all from a single template along with a file that will loop (or purrr::walk()) through the parameters. (If you want to learn more about parameterized reporting, the always-incredible Alison Hill has a recent-ish tutorial on them that you can find here). In my case, this meant creating one template for all 65+ schools and then looping through a function that rendered the report for each school. Sounds great, right?
See, what had happened was…
This workflow is great…when it works. Except it doesn’t always. This isn’t to say that {rmarkdown} mysteriously breaks or anything, but rather that when you create these reports using real (read: usually messy) data, and when you’re trying to present a lot of data in a report, the probability that one of your iterations throws an error increases. This is especially true when you work in a school division and the integrity of your data has been absolutely ravaged by COVID during the past ~18 months. When this happens, instead of watching the text zip by on your console as all of your reports render like they’re supposed to, you end up hunting through the data for each individual school wondering why calculating a particular metric threw an error. Which is like, not nearly as much fun.
So what can we do about this?
Fortunately, we can get around this by making “safe” versions of our functions. What exactly that means will vary from function to function and from use case to use case, but generally it means wrapping a function in another function that can facilitate error handling (or prevent errors from occuring). In some cases, it might mean using purrr::safely() or purrr::possibly() to capture errors or provide default values to the functions. In other cases, it might mean writing your own wrapper (which is what I’ll demonstrate below) to deal with errors that pop up. Regardless of the exact route you go, the goal here is to prevent errors that would otherwise stop your document(s) from rendering.
Let’s see this in action.
Setup
I’m not actually going to create “real” parameterized reports here, but I’ll illustrate the principle using data from {palmerpenguins} and some ggplots. First, I’ll load some packages and set some options and whatnot, plus also take a peek at the penguins data we’ll be using.
library(palmerpenguins) #data on penguinslibrary(tidyverse) #you all know what this islibrary(eemisc) #personal ggplot themeslibrary(harrypotter) #colorslibrary(reactable)herm <- harrypotter::hp(n =1, option ="HermioneGranger")opts <-options(ggplot2.discrete.fill =list( harrypotter::hp(n =3, option ="HermioneGranger"), harrypotter::hp(n =7, option ="Always") ))theme_set(theme_ee())glimpse(penguins)
If you’re not familiar with this dataset, it contains data on a few hundred penguins, and you can learn more here. One feature of this dataset is that it has data on three different species of penguins: Adelie, Gentoo, and Chinstrap. So, let’s imagine we wanted to provide separate reports for each species of penguin. To do this, let’s first divide our data up into separate dataframes to emulate a potential workflow of creating parameterized reports.
Great, so now we have a separate dataframe for each penguin species.
Create a Plot
Now, imagine we want to create a ggplot to include in each of our reports. To illustrate this, let’s just do a scatterplot of flipper length by bill length for Adelie penguins. This is a fairly basic scatterplot, but it works for our purposes.
Good so far, right? Our next step might be to define a function to make this plot. Although this is a fairly basic plot, writing a function still saves us a little bit of typing, and it will prove useful later when we need to wrap it, so let’s go ahead and write our function.
make_penguin_plot <-function(df) { title <-as.character(unique(df$species))ggplot(df, aes(x = flipper_length_mm, y = bill_length_mm)) +geom_point(color = herm) +labs(title = title )}make_penguin_plot(split_df[[1]])
So what if something goes wrong?
Let’s imagine, now, that we don’t have measurements of Gentoo penguins’ bill lengths. Maybe the bill-length-measuring machine was broken on the one day we were going to take their measurements. Or maybe all of them were especially bite-y and wouldn’t let us measure their bills (I have no clue if penguins actually bite).
#dropping the bill_length measurement from the Gentoo datasplit_df[[3]] <- split_df[[3]] %>%select(-bill_length_mm)
Now what happens when we try to make our penguin plot?
make_penguin_plot(split_df[[3]])#produces message 'Error: Column `bill_length_mm` not found in `.data`'
Oh no! An error! In this contrived example here, I’m just showing you the error message produced, but if you’re actually rendering a report, this will stop your report from rendering, which isn’t great.
If this happens to you, you have a few options. You might create a separate report template just for Gentoo penguins, although this doesn’t seem ideal, because it defeats the point of having a template if you need to make separate ones every time an exception pops up. You could drop this metric from your main report template if the data seems problematic (which is a good thing to investigate). You could potentially use purrr::possibly() or purrr::safely() if you have a default value you want to use.
Another option is to write your own little wrapper to make your function “safe”, which I’ll show below.
Wrap that function!
The best part here is that this is, in the very specific case, it’s fairly straightforward. I’m just going to check the names of the variables in the data I’m passing into the function to see if flipper length and bill length are present in the data, then execute make_penguin_plot() if they are and print out “Whoops!” if they’re not.
safe_penguin_plot <-function(df) { nms <-names(df)if (all(c("flipper_length_mm", "bill_length_mm") %in% nms)) {make_penguin_plot(df) } elseprint("Whoops! Looks like you don't have all of the data you need for this plot!")}safe_penguin_plot(split_df[[1]])
safe_penguin_plot(split_df[[3]])
[1] "Whoops! Looks like you don't have all of the data you need for this plot!"
Generalize your functions using {rlang}
You can also imagine generalizing this to accept other variables. This requires some quoting/unquoting and diving into {rlang}, which is something I’ve been trying to learn lately:
gen_penguin_plot <-function(df, xvar, yvar) { title <-as.character(unique(df$species)) x <-enexpr(xvar) #captures xvar as an expression y <-enexpr(yvar) #captures yvar as an expressionggplot(df, aes(x =!!x, y =!!y)) +geom_point(color = herm) +labs(title = title )}gen_penguin_plot(split_df[[1]], bill_length_mm, bill_depth_mm)
There’s a little bit more work in making the more generalized version “safe” that has to do with handling the quoted expressions/environments, especially since we’re passing them into another function:
safe_gen_plot <-function(df, xvar, yvar) { nms <-names(df) vec <-c(deparse(enexpr(xvar)), deparse(enexpr(yvar))) x <-enquo(xvar) y <-enquo(yvar)if (all(vec %in% nms)) {gen_penguin_plot(df, xvar =!!x, yvar =!!y) } elseprint("Whoops! Looks like you don't have all of the data you need for this plot!")}safe_gen_plot(split_df[[1]], bill_length_mm, bill_depth_mm)
[1] "Whoops! Looks like you don't have all of the data you need for this plot!"
Conclusion
By implementing a simple (or less simple, depending on how generalizable you want your function to be) wrapper function, we can replace errors with a message to be displayed when rendering a report. I can’t emphasize how much time this approach has saved me when creating parameterized reports, especially since our data has gotten so wonky due to COVID and this provides a flexible way to handle all of this craziness.
Hope this helps others who might be in similar positions!
