.NET Core build OS affects json output encoding

Question

I just noticed that our api responses were returning some invalid characters. In debugging this issue I realized that the invalid characters were only being returned if the project was built on linux runners on Azure DevOps.

Using a minimal api as example:

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.MapGet("/weatherforecast", () =>
{
    return "Anulação";
});

app.Run();

Built on ubuntu-latest, if I run dotnet TestJson.dll on my Windows 11 machine I get (also ran this ubuntu publish output on my ubuntu 22.04 wsl and the results were the same):

Build on windows-latest, if I run dotnet TestJson.dll on my Windows 11 machine I get:

In summary:

• project built on ubuntu-latest runners on Azure DevOps or ubuntu wsl on my Windows 11 machine results in the invalid chars being print
• project built on windows-latest runners on Azure DevOps or on my Windows 11 machine with either dotnet or Visual Studio prints the correct chars

I've also been directed to this issue on github actions (which are similar to Azure DevOps pipelines) and I've generated the pt_PT.UTF-8 locale files and I've set up all locale options as such (now my WSL is totaly in Portuguese and printing the correct chars such as ç and ã):

LANG=pt_PT.UTF-8
LANGUAGE=pt_PT.UTF-8
LC_CTYPE="pt_PT.UTF-8"
LC_NUMERIC="pt_PT.UTF-8"
LC_TIME="pt_PT.UTF-8"
LC_COLLATE="pt_PT.UTF-8"
LC_MONETARY="pt_PT.UTF-8"
LC_MESSAGES="pt_PT.UTF-8"
LC_PAPER="pt_PT.UTF-8"
LC_NAME="pt_PT.UTF-8"
LC_ADDRESS="pt_PT.UTF-8"
LC_TELEPHONE="pt_PT.UTF-8"
LC_MEASUREMENT="pt_PT.UTF-8"
LC_IDENTIFICATION="pt_PT.UTF-8"
LC_ALL=pt_PT.UTF-8

But still the api returns the invalid characters. I have looked at maybe configuring System.Text.Json Encoder but I don't understand why this happens.

Answer 1

After two days of debugging and testing various combinations, what fixed it was setting the encoding on the *.cs files of my solution to "UTF-8". They were in "Western European" by default on my installation. I guess that's why building on Windows worked and Linux not (maybe Linux does not know this encoding).

I used VS2022 with Tools->Customize->Commands, Menu bar: File and then added the command "Advanced Save Options" to my File menu list. Then I was able to choose the encoding for saving the file.

After this change the problem was fixed.