Flutter Unicode Apostrophe In String

Question

I'm hoping this is an easy question, and that I'm just not seeing the forest due to all the trees.

I have a string in flutter than came from a REST API that looks like this: "What\u0027s this?"

The \u is causing a problem.

I can't do a string.replaceAll("\", "\") on it as the single slash means it's looking for a character after it, which is not what I need.

I tried doing a string.replaceAll(String.fromCharCode(0x92), "") to remove it - That didn't work.

I then tried using a regex to remove it like string.replaceAll("/(?:\)/", "") and the same single slash remains.

So, the question is how to remove that single slash, so I can add in a double slash, or replace it with a double slash?

Cheers

Jase

Answer 1

When the string is read, I assume what's happening is that it's being interpreted as literal rather than as what it should be (code points) i.e. each character of \0027 is a separate character. You may actually be able to fix this depending on how you access the API - see the dart convert library. If you use utf8.decode on the raw data you may be able to avoid this entire problem.

However, if that's not an option there's an easy enough solution for you.

What's happening when you're writing out your regex or replace is that you're not escaping the backslash, so it's essentially becoming nothing. If you use a double slash, that solve the problem as it escapes the escape character. "\\" => "\".

The other option is to use a raw string like r"\" which ignores the escape character.

Paste this into https://dartpad.dartlang.org:

  String withapostraphe = "What\u0027s this?";

  String withapostraphe1 = withapostraphe.replaceAll('\u0027', '');
  String withapostraphe2 = withapostraphe.replaceAll(String.fromCharCode(0x27), '');

  print("Original encoded properly: $withapostraphe");
  print("Replaced with nothing: $withapostraphe1");
  print("Using char code for ': $withapostraphe2");

  String unicodeNotDecoded = "What\\u0027s this?";
  String unicodeWithApostraphe = unicodeNotDecoded.replaceAll('\\u0027', '\'');
  String unicodeNoApostraphe = unicodeNotDecoded.replaceAll('\\u0027', '');
    String unicodeRaw = unicodeNotDecoded.replaceAll(r"\u0027", "'");

  print("Data as read with escaped unicode: $unicodeNotDecoded");
  print("Data replaced with apostraphe: $unicodeWithApostraphe");
  print("Data replaced with nothing: $unicodeNoApostraphe");
  print("Data replaced using raw string: $unicodeRaw");

To see the result:

Original encoded properly: What's this?
Replaced with nothing: Whats this?
Using char code for ': Whats this?
Data as read with escaped unicode: What\u0027s this?
Data replaced with apostraphe: What's this?
Data replaced with nothing: Whats this?
Data replaced using raw string: What's this?

Answer 2

I found the issue. I was looking for hex 92 (0x92) and it should have been decimal 92.

I ended up solving the issue like this...

String removeUnicodeApostrophes(String strInput) {
    // First remove the single slash.
    String strModified = strInput.replaceAll(String.fromCharCode(92), "");
    // Now, we can replace the rest of the unicode with a proper apostrophe.
    return strModified.replaceAll("u0027", "\'");
}