How to ignore predefined variables in a template file?

Question

I have a automatically generated file with variables. I want to replace these variables. Therefore, I use templatefile. It is working unless the file contains a predefined variable like ${__org.name}, which should not be resolved by Terraform.

I want to ignore predefined variables like ${__org.name}, but templatefile has no feature like this, see The template provider parses every variable, even those that are undeclared..

So I tried to work around it by replaceing the predefined variable with the escaped predefined variable (see Escape Sequences), but I get an error:

Invalid value for "vars" parameter: invalid template variable name "__org.name": must start with a letter, followed │ by zero or more letters, digits, and underscores.

Code

resource "grafana_dashboard" "test" {
  config_json = templatefile("test.json",
    {
      "DS_MYSQL" = grafana_data_source.mysql.uid
      "__org.name" = "$${__org.name}"
  })
}

Generated file

{
  "__inputs": [
    {
      "name": "DS_MYSQL",
      "label": "MySQL",
      "description": "",
      "type": "datasource",
      "pluginId": "mysql",
      "pluginName": "MySQL"
    }
  ],
  "__elements": {},
  "__requires": [
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "10.0.3"
    },
    {
      "type": "datasource",
      "id": "mysql",
      "name": "MySQL",
      "version": "1.0.0"
    }
  ],
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": null,
  "links": [],
  "liveNow": false,
  "panels": [],
  "refresh": "",
  "schemaVersion": 38,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {},
        "datasource": {
          "type": "mysql",
          "uid": "${DS_MYSQL}"
        },
        "definition": "SELECT id AS __value, name AS __text FROM tenant WHERE name = '${__org.name}'",
        "hide": 0,
        "includeAll": false,
        "multi": false,
        "name": "tenant",
        "options": [],
        "query": "SELECT id AS __value, name AS __text FROM tenant WHERE name = '${__org.name}'",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "DUR test",
  "uid": "c66eafb7-1254-45f3-afdd-a20f1b7cf2a4",
  "version": 3,
  "weekStart": ""
}

Remark

I think it is a bug, that templatefile tries to resolve variables in the file, which are not allowed as an argument, see templatefile Function:

Variable names must each start with a letter, followed by zero or more letters, digits, or underscores.

However, I need a solution for my problem.

Research

• I tried to use replace (only for the predefined variable) before templatefile, but there is no string version, see Introduce a template() function that works on string content.

• I read format Function, but there is no support for ${} syntax.

Question

How to ignore predefined variables in a template file?

Answer 1

If you wish to generate literal sequences that conflict with Terraform's template syntax then you must use escape sequences to explain to Terraform which parts should be taken literally and which should be evaluated.

Backslash sequences are not interpreted as escapes in a heredoc string expression. Instead, the backslash character is interpreted literally.

Heredocs support two special escape sequences that do not use backslashes:

Sequence	Replacement
$${	Literal ${, without beginning an interpolation sequence.
%%{	Literal %{, without beginning a template directive sequence.

For your template in particular, that means you should write $${__org.name} instead of just ${__org.name}. Terraform will then remove the extra $ and take the remainder of the sequence literally, causing ${__org.name} to appear directly in the template result.

Answer 2

I found some workarounds:

• Use replace instead of templatefile

  You could replace all non predefined variables with replace Function.

  resource "grafana_dashboard" "test" {
    config_json = replace(file("test.json"), "$${DS_MYSQL}", grafana_data_source.mysql.uid)
  }
  

  Disadvantages:

  • If you have more than one variable, you have to chain the replace function, like

    replace(replace(file("test.json"), "$${VAR1}", "value1"), "$${VAR2}", "value2")
    

• Escape predefined variables with a script before running Terraform

  You could escape all predefined variables with sed.

  $ sed 's/${__org.name}/$${__org.name}/g' generated.json > test.json 
  $ terraform apply
  

  Disadvantages:

  • The escaping is not done with Terraform.