How to generate and apply git patches correctly in Powershell (vs bash)?

Question

Please, observe the following short scenario (this is in Powershell):

PS> git diff -U3 -r -M HEAD -- .\Metadata\LegacyTypeModules\xyz.Web.Main.draft.json | Out-File -Encoding ascii c:\temp\1.diff

PS> git apply --cached C:\temp\1.diff
error: patch failed: Metadata/LegacyTypeModules/xyz.Web.Main.draft.json:69
error: Metadata/LegacyTypeModules/xyz.Web.Main.draft.json: patch does not apply

This fails because the last line in the file does not end with CRLF:

However, the same exact commands work when run in bash:

$ git diff -U3 -r -M HEAD -- Metadata/LegacyTypeModules/xyz.Web.Main.draft.json > /c/Temp/2.diff

$ git apply --cached /c/Temp/2.diff

P11F70F@L-R910LPKW MINGW64 /c/xyz/tip (arch/1064933)

The difference between the two patches is:

So the problem seems to happen because Powershell terminates each line going through the pipe with CRLF whereas bash preserves the original line endings.

I understand why this happens - Powershell operates with objects and the objects are strings excluding the EOL characters. When writing to file Powershell converts objects to strings (in the case of strings the conversion is a nop) and uses the default EOL sequence to delimit the lines.

Does it mean Powershell cannot be used at all in EOL sensitive scenarios?

Answer 1

Indeed:

• PowerShell, up to v7.3.x, invariably decodes output from external programs as text (using [Console]::OutputEncoding).

• It then sends the decoded output line by line through the pipeline, as lines become available.

• A file-output cmdlet such as Out-File then invariably uses the platform-native newline sequence - CRLF on Windows - to terminate each (stringified) input object when writing to the target file (using its default character encoding (or the one specified via -Encoding), which is technically unrelated to the encoding that was used to decode the external-program output).

In other words:

• In Windows PowerShell and PowerShell (Core) up to v7.3.x, PowerShell pipelines (and redirections) do not support passing raw binary data through.

• They do, however, in PowerShell (Core) v7.4+ (see this answer).

  • Note that you'll have to use > rather than Out-File in order to capture the raw byte output from an external program such as git, just like in Bash.
  • In other words: in v7.4+, your Bash command can be used as-is in PowerShell.

Workarounds for v7.3-:

• Manually join and terminate the decoded output lines with LF newlines ("`n"), and write the resulting multi-line string as-is (-NoNewLine) to the target file, as shown in zdan's helpful answer.

• In this simple case it is easiest to delegate to cmd.exe /c, given that cmd.exe's pipelines and redirections are raw byte conduits:

cmd /c @'
git diff -U3 -r -M HEAD -- .\Metadata\LegacyTypeModules\xyz.Web.Main.draft.json > c:\temp\1.diff
'@

^{Note the use of a here-string for readability and ease of any embedded quoting (n/a here).}

Answer 2

A standard diff (also known as a patch) terminates lines with LF line endings. That's because that's what POSIX specifies for the output of diff. All lines must contain an LF line ending.

When a CR precedes the LF in a patch, it is considered part of the content to be patched. Therefore, the patch in your situation likely doesn't apply because the old content is being listed as having CRLF line endings, which it does not.

Unfortunately, PowerShell is completely broken in this regard and its pipelines corrupt data passing through them. This is especially true for any sort of binary data. If you're using any sort of tool designed to run on Unix like Git, you'll need to avoid using PowerShell's pipes.