Converting special characters in TStringList

Question

I am using Delphi 7 and have a routine which takes a csv file with a series of records and imports them. This is done by loading it into a TStringList with MyStringList.LoadFromFile(csvfile) and then getting each line with line = MyStringList[i].

This has always worked fine but I have now discovered that special characters are not picked up correctly. For example, Rue François Coppée comes out as Rue François Coppée - the accented French characters are the problem.

Is there a simple way to solve this?

Answer 1

Your file is encoded as UTF-8. For instance consider the ç. As you can see from the link, this is encoded in UTF-8 as 0xC3 0xA7. And in Windows-1252, 0xC3 encodes à and 0xA7 encodes §.

Whether or not you can handle this easily using your ANSI Delphi depends on the prevailing code page under which your program runs.

• If you are using Windows 1252 then you will be fine. You just need to decode the UTF-8 encoded text with a call to UTF8Decode.
• If you are using a different locale then life gets more difficult. Those characters may not be present in your locale's character set and in that case you cannot represent them in a Delphi string variable which is encoded using the prevailing ANSI charset. If this is the case then you need to use Unicode.

If you care about handling international text then you need to either:

• Upgrade to a modern Delphi which has Unicode support, or
• Stick to Delphi 7 and use WideString and the TNT Unicode components.

Answer 2

Probably it's not in UTF8 encoding. Try to convert it:

Text := UTF8Encode(Text);

Regards,