How to measure the scalability of all ways to deploy a Shiny app?

Question

There a few approachs to deploy a Shiny app:

With Heroku or Using ShinyProxy, Traefik and Docker Swarm or Shinyapps.io or Shiny Server or Replace Shiny UI by shiny.semantic

Is well known that ShinyServer is very expensive and Shinyapps.io i'ts not scalable:

Article

But compare all the approachs, how to measure which is more cost efficient?

Answer 1

I have hosted apps using the three approaches listed in the table. On the basis of cost = time + money:

1. shinyapps.io: by far the cheapest, so long as it meets your needs.
2. shiny-server: takes longer to setup, requires a server, but fewer limitations.
3. ShinyProxy: takes even longer to setup, requires a server, experience with Docker, very capable.

The most cost-efficient option will be the first option in this list that meets all of your needs: (1) shinyapps.io, (2) shiny-server (free), (3) ShinyProxy, (4) Shiny-server Pro.

Memory: Shiny-server can share memory between multiple instances of the same app while each instance of a ShinyProxy app is completely independent. Many instances of an app that demands a lot of memory will drain your server resources faster if it is hosted using ShinyProxy vs Shiny-server.

CPU: Multiple instances of the same app will share the same R instance and processor thread in Shiny-server. So many concurrent users may experience a slowdown of a compute-heavy app when hosted using Shiny-server. In ShinyProxy, each user will get their own Docker container/app which could be distributed between multiple processors, supporting better scaling of compute-heavy apps.

Authentication: Authentication only supported by ShinyProxy (or Shiny-server Pro)

Shiny-server Pro: (paid) allows you to optimize between CPU and memory, provides authentication, user monitoring, etc., but includes a subscription cost.

Actually measuring/testing app scalability: Use shiny load test (see https://rstudio.github.io/shinyloadtest/ for more details).