ValueError: Missing staticfiles manifest entry on Heroku with Docker, django-pipeline, whitenoise

Question

I am trying to deploy a Django project on Heroku using Docker, django-pipeline, and whitenoise. The container builds successfully, and I see that collectstatic generates the expected files inside container-name/static. However, upon visiting any page I receive the following 500 error:

ValueError: Missing staticfiles manifest entry for 'pages/images/favicons/apple-touch-icon-57x57.png'

Below are my settings.py, Dockerfile, and heroku.yml. Since I'm also using django-pipeline, one thing I'm unsure of is what setting to use for STATICFILES_STORAGE? I tried

STATICFILES_STORAGE = 'pipeline.storage.PipelineStorage'

but that lead to the file paths 404ing.

Any advice is appreciated. Thank you.

#settings.py
...
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env.bool("DJANGO_DEBUG", default=False)

ALLOWED_HOSTS = ['.herokuapp.com', 'localhost', '127.0.0.1']

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.admindocs",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "django.contrib.sites",
    # Third-party
    "allauth",
    "allauth.account",
    "debug_toolbar",
    "django_extensions",
    "pipeline",
    "rest_framework",
    "whitenoise.runserver_nostatic",
    "widget_tweaks",
    # Local
    ...
]
MIDDLEWARE = [
    "debug_toolbar.middleware.DebugToolbarMiddleware",
    "django.middleware.security.SecurityMiddleware",
    "whitenoise.middleware.WhiteNoiseMiddleware",
    ...
]

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.1/howto/static-files/

STATIC_URL = "/static/"
# STATICFILES_DIRS = [str(BASE_DIR.joinpath("code/static"))]
STATIC_ROOT = str(BASE_DIR.joinpath("static"))

MEDIA_URL = "/media/"

MEDIA_ROOT = str(BASE_DIR.joinpath("media"))

# django-pipeline config
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"
DEBUG_PROPAGATE_EXCEPTIONS = True
STATICFILES_FINDERS = (
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
    "pipeline.finders.PipelineFinder",
)
...

# Dockerfile
# Pull base image
FROM python:3.8

# Set environment variables and build arguments
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get install -y nodejs build-essential

# Set working directory
WORKDIR /code
COPY . /code/
RUN npm install sass --dev
RUN npm install yuglify --dev
RUN npm install
RUN mkdir static
RUN mkdir staticfiles

# Install dependencies
COPY Pipfile Pipfile.lock /code/
# Figure out conditional installation of dev dependencies
# Will need to remove --dev flag for production
RUN pip install pipenv && pipenv install --system --dev

# heroku.yml
setup:
  addons:
  - plan: heroku-postgresql
build:
  docker:
    web: Dockerfile
release:
  image: web
  command:
    - python manage.py collectstatic --noinput
run:
  web: gunicorn config.wsgi

UPDATE

Based on ENDEESA's response to this similar SO post, I updated my settings to the following, since my static files are stored inside pages/static/pages:

STATIC_URL = "/static/"
STATICFILES_DIRS = [str(BASE_DIR.joinpath("pages/static"))]
STATIC_ROOT = str(BASE_DIR.joinpath("static"))

I also noticed that my top-level urls.py file included the following line:

urlpatterns += staticfiles_urlpatterns()

As I understand it, this is useful for serving static files in development, but should not be used in production, so I moved it to only be added if DEBUG is True. But alas, the error persists.

More mysteriously, when I run

python manage.py findstatic <file-path> --verbosity 2

the file is found:

Found 'images/favicons/apple-touch-icon-57x57.png' here:
  /code/pages/static/images/favicons/apple-touch-icon-57x57.png
  /code/pages/static/images/favicons/apple-touch-icon-57x57.png
Looking in the following locations:
  /code/pages/static
  /code/static
  /usr/local/lib/python3.8/site-packages/django/contrib/admin/static
  /usr/local/lib/python3.8/site-packages/debug_toolbar/static
  /usr/local/lib/python3.8/site-packages/django_extensions/static
  /usr/local/lib/python3.8/site-packages/rest_framework/static

So why am I still getting ValueError: Missing staticfiles manifest entry?

Answer 1

SOLVED

At long last, I came to the following solution. My main issues were:

• the static files appeared to be collected during the release command, but the actual STATIC_ROOT dir was empty in my container. I'm not sure why. My solution was to NOT run collectstatic as a release command in heroku.yml, and instead do so in Dockerfile.
• NOTE: in order to collectstatic in my Dockerfile I needed to set a default for all environment variables, including SECRET_KEY, the latter for which I did using get_random_secret_key() from django.core.management.utils. Thank you to Ryan Knight for illustrating this here.
• In addition to my settings.py needing a default secret key, my final static files settings were as shown below.
• Since I'm using django-pipeline, my js files weren't loading correctly with whitenoise storage options. I wound up using pipeline.storage.PipelineStorage instead.
• It turned out I did not need to set STATICFILES_DIRS at all. Previously I was setting it as:

STATICFILES_DIRS = [
    str(BASE_DIR.joinpath("pages/static")),
    str(BASE_DIR.joinpath("staticfiles")),]

Both were unnecessary because app_name/static/app_name is the default place Django will look for static files already, and I wasn't actually storing additional non-app-specific files in root/staticfiles. So I removed this setting.

• In heroku.yml I removed the release command for collectstatic.
• On the Settings page of my Heroku app's admin, I added a config variable for DISABLE_COLLECTSTATIC, set to 1.

# Dockerfile
# Pull base image
FROM python:3.8

# Set environment variables and build arguments
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt-get install -y nodejs build-essential

# Set working directory
WORKDIR /code
COPY . /code/
RUN npm install sass --dev
RUN npm install yuglify --dev
RUN npm install

# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system

# Collect static files here instead of in heroku.yml so they end up in /code/static, as expected in the app
RUN python manage.py collectstatic --noinput

# settings.py
...
from django.core.management.utils import get_random_secret_key
SECRET_KEY = env("DJANGO_SECRET_KEY", default=get_random_secret_key())
...
STATIC_URL = "/static/"
STATIC_ROOT = str(BASE_DIR.joinpath("static"))
STATICFILES_STORAGE = 'pipeline.storage.PipelineStorage'
DEBUG_PROPAGATE_EXCEPTIONS = True
STATICFILES_FINDERS = (
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
    "pipeline.finders.PipelineFinder",
)
...

# heroku.yml
setup:
  addons:
  - plan: heroku-postgresql
build:
  docker:
    web: Dockerfile
release:
  image: web
run:
  web: gunicorn config.wsgi

Project structure, in case it's helpful:

config
  settings.py
  ...
pages
  static
    pages
      scss
      js
      images
static
Dockerfile
heroku.yml
docker-compose.yml
...

Best of luck to anyone else battling the deployment gods. May the odds be ever in your favor, and don't give up!

Answer 2

This works! Comment out STATICFILES_STORAGE and type the code below.

#STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"

WHITENOISE_USE_FINDERS = True
WHITENOISE_MANIFEST_STRICT = False
WHITENOISE_ALLOW_ALL_ORIGINS = True