accented french characters with powershell core 7.1.2 or 7.2 don't work

Question

On powershell core (7.1.2) or 7.2, I don't have the accented french characters "é" or "è" or "à", I have this:

Liste des mises à jours déjà installées dans la dernière mise à jour

It works on powershell windows 5.1. the text is like this:

Liste des mises à jours déjà installées dans la dernière mise à jour

I use the IDE visual studio code in the 2 cases.

Whats is the solution please? I have already tried to change the encoding : utf8 or utf 16 for example with the french language pack

I tried this code for example, but it doesn't work

#affiche la tentative de mise à jour"
function affichetentative {
 
    #nom du serveur
    $b=HOSTNAME.EXE
    #chemin  où  est lecalisé le fichier de log du serveur
    $path="C:\LOGS\log_$b.txt"
    #affichage du texte ci-dessous dans un fichier de log
    $c=Get-Date
    $a="Liste des mises à jours déjà installées dans la dernière mise à jour le {0} " -f  $c
    ADD-content -path $path -Encoding utf8BOM -value  $a  
        
    }
 affichetentative 

I have this output in my file:

Liste des mises à jours déjà installées dans la dernière mise à jour le 16/02/2021 22:13:41

even if i configure utf8BOM or UTF-16LE on visual studio code, it doesn't work

Answer 1

The implication is that your file is UTF8-encoded, but without a BOM.

• Caveat: If you use Add-Content to append to an existing file (that isn't empty), PowerShell matches the (possibly inferred) existing encoding and ignores an -Encoding argument.

While PowerShell (Core) 7+ reads such files correctly, Windows PowerShell does not, because it assumes ANSI encoding in the absence of a BOM; this applies both to files read explicitly with Get-Content and implicitly read source-code files.

Any file you want both PowerShell editions to interpret correctly by default should be UTF8-encoded with a BOM (or UTF16-LE-encoded, which PowerShell calls Unicode, which always has a BOM).

The pitfall is that modern editors such as Visual Studio Code create BOM-less files by default, because UTF8 is assumed to be the default encoding nowadays, and because some utilities, notably those with a Unix heritage, do not expect a BOM and may even misinterpret as data.

• However, as filimonic points out, you can configure Visual Studio Code to create UTF8 files with BOM by default (optionally just for PowerShell source-code files) - see the docs.

---

From Windows PowerShell, the problem can be demonstrated as follows:

# Use a .NET API directly to create a test file.
# .NET APIs create BOM-*less* UTF-8 files by default.
[IO.File]::WriteAllText(
  "$PWD/test.txt",
  'Liste des mises à jours déjà installées dans la dernière mise à jour'
)

# Now read it with Get-Content on Windows PowerShell, which
# results in MISINTERPRETATION.
# Note: In PowerShell (Core) 7+, this works correctly.
Get-Content test.txt

You'll get:

Liste des mises à jours déjà installées dans la dernière mise à jour

because the UTF8 encoding was misinterpreted as the active ANSI code page's encoding.

By passing -Encoding utf8 to Get-Content, you could avoid the problem.