Grab screenshots when outputs update

[schloerke]

How hard would it be to take a small screenshot of the app state at key time points, like at the idle state? Or on the shiny:value event (that's when an output was updated, right?)
It'd be super awesome to see a small screengrab of the app when you page through the idle points

cc @gadenbuie

[schloerke]

With the app authors running their apps in their own browsers, I don't know if this is possible.

If we had full control, let's say shinytest, then it might be possible.

[schloerke]

Closing for now

[gadenbuie]

Here's an idea. Using html-to-image, we grab a screenshot of the app state and send it back to the server.

library(shiny)
# turn off autoreload so added screenshots don't trigger reload
options(shiny.autoreload = FALSE)

# using the old faithful example app...
app <- new.env()
source(system.file("examples", "01_hello", "app.R", package = "shiny"), local = app)

we can use html-to-image to grab a PNG which we then send back to the server for select Shiny events.

ui <- tagList(
  app$ui,
  tags$script(type = "module", HTML("
import { toPng } from 'https://cdn.skypack.dev/html-to-image';

const page = document.querySelector('body .container-fluid');

function screenGrabApp (shinyEventName) {
  toPng(page)
    .then(dataURL => Shiny.setInputValue('screenshot', {event: shinyEventName, data: dataURL}))
    .catch(console.error)
}

['value', 'idle', 'outputinvalidated'].forEach(function(name) {
  $(document).on('shiny:' + name, function() { screenGrabApp(name) })
})
"))
)

On the server we decode the base64 dataURL and save it with a timestamp.

server <- function(input, output) {
  app$server(input, output)

  observeEvent(input$screenshot, {
    event <- input$screenshot$event
    time <- sub("[.]", "", strftime(Sys.time(), "screenshots/%F_%H%M%OS3"))
    path <- sprintf("%s_%s.png", time, event)
    message(path)
    data <- sub("data:image/png;base64,", "", input$screenshot$data, fixed = TRUE)
    data <- base64enc::base64decode(data)
    writeBin(data, path)
  })
}

Run the app in a browser (not the IDE because it blocks the remote dependency from skypack) and boom.

dir.create("screenshots", showWarnings = FALSE)
shinyApp(ui, server, options = list(port = 6543))