Terraform state mv commands in powershell fail using names with spaces

Question

We have an issue where when we try to move items using tf state mv it wil parse incorrectly in powershell.

As you can see below, the terraform CLI interpets the space as a new item and fails the state mv call

If we run the command in command prompt with modified escaping it will work, we would like to find a way to make it work in powershell possibly with custom escaping.

Repro steps in PS:

Terraform init
Terraform plan
Terraform apply
Terraform state mv 'local_file.file[\"b c\"]' 'local_file.file[\"e f\"]'

Command in PS (fails) terraform state mv 'local_file.file[\"b c\"]' 'local_file.file[\"e f\"]'

Command in CMD terraform state mv "local_file.file[\"b c\"]" "local_file.file[\"e f\"]"

Output

2020/06/14 23:59:22 [INFO] Terraform version: 0.12.23 2020/06/14 23:59:22 [INFO] Go runtime version: go1.12.13 2020/06/14 23:59:22 [INFO] CLI args: string{“C:\Windows\terraform.exe”, “state”, “mv”, “local_file.file[“b”, “c”]”, “local_file.file[“e”, “f”]”} 2020/06/14 23:59:22 [DEBUG] Attempting to open CLI config file: C:\Users\p120b60.CORPADDS\AppData\Roaming\terraform.rc 2020/06/14 23:59:22 [DEBUG] File doesn’t exist, but doesn’t need to. Ignoring. 2020/06/14 23:59:22 [INFO] CLI command args: string{“state”, “mv”, “local_file.file[“b”, “c”]”, “local_file.file[“e”, “f”]”} Exactly two arguments expected.

main.tf

provider “azurerm” {

version = “=2.13.0”

skip_provider_registration = “true”

features {}

}

locals {

filenames = toset([“a”, “b c”])

}

terraform {

backend “local” {

path = "./terraformconfig.tfstate"
}

}

resource “local_file” “file” {

for_each = local.filenames

filename = each.value

}

TF version Terraform v0.12.23

provider.azurerm v2.13.0 provider.local v1.4.0

Answer 1

When you run non-PowerShell programs with PowerShell (which includes Terraform), you need to contend both with PowerShell's parsing of the command line and the final program's parsing of the command line.

Terraform on Windows expects to receive a command line string following the Microsoft C++ command line processing patterns, which means:

• Arguments are delimited by white space, which is either a space or a tab.
• A string surrounded by double quotation marks ("string") is interpreted as a single argument, regardless of white space contained within. A quoted string can be embedded in an argument.
• A double quotation mark preceded by a backslash (\") is interpreted as a literal double quotation mark character (").
• Backslashes are interpreted literally, unless they immediately precede a double quotation mark.
• If an even number of backslashes is followed by a double quotation mark, one backslash is placed in the argv array for every pair of backslashes, and the double quotation mark is interpreted as a string delimiter.
• If an odd number of backslashes is followed by a double quotation mark, one backslash is placed in the argv array for every pair of backslashes, and the double quotation mark is "escaped" by the remaining backslash, causing a literal double quotation mark (") to be placed in argv.

As you've seen, Terraform writes out in its early logs the result of the above processing rules, which occur before Terraform's own code starts running.

The challenge in your case is that the above rules are written under the assumption of running a program via normal Windows mechanisms such as the CreateProcess function, where all of the arguments are passed as a single string in the lpCommandLine argument. But that's not how PowerShell works...

---

PowerShell, by virtue of being a scripting language in its own right, has its own parser that it uses to interpret the command line. When working with PowerShell's own cmdlets that's perfectly fine and reasonable, because cmdlets expect to receive their arguments as distinct PowerShell values that result from that parsing.

The problem arises when you use PowerShell to run a program that isn't a PowerShell cmdlet: PowerShell will parse the command line using its own rules, then detect that the command line is running a non-PowerShell program, and thus have to try to reconstruct a single command line string using the separate values it already parsed.

This means that when we run normal programs using PowerShell we need essentially to trick PowerShell into constructing a string that complies with the Microsoft C++ command line processing patterns described above, because PowerShell itself is the one constructing that final string.

A very heavy way to avoid these problems is to prevent PowerShell from interpreting the arguments itself at all, by manually launching the program using the Start-Process cmdlet, which takes the lpCommandLine value as a single string argument:

& Start-Process -FilePath "terraform" -ArgumentList "import `"\`"null_resource.things[\\\`"thing one\\\`"]\`"`" 800778739314064726"

PowerShell uses a backtick as its escape character, so the above is using backticks to escape quotes to force PowerShell to take them literally, and then using backslashes so that once PowerShell is finished parsing the result is a string that the normal command line processing rules should be able to interpret.

There are less extreme ways to get this same result if you are willing to study the PowerShell parsing rules and external-program-launching behavior in detail and figure out how to get PowerShell to construct a suitable string itself, but it's usually easier to just use the normal Windows command interpreter -- which is designed to be compatible with the way normal Windows programs expect command line arguments -- and save PowerShell for situations where you are primarily using PowerShell cmdlets, and thus PowerShell's more advanced command line parsing is helpful rather than a hindrance.