Ruby - Microsoft Translator unexpected token error

Question

I'm using the following method to translate a simple word from English to Russian by calling:

translate("hello")

This is my method:

def translate(text)

    begin

        uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{@appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
        page = HTTParty.get(uri).body
        show_info = JSON.parse(page) # this line throws the error

    rescue

        puts $!

    end

end

The JSON output:

{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}

The error:

unexpected token at '{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}'

Not sure what it means by unexpected token. It's the only error I'm receiving. Unfortunately I can't modify the JSON output as it's returned by the API itself.

UPDATE:

Looks like the API is returning some illegal characters (bad Microsoft):

'´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}'

Full error:

C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse': 743: unexpected token at '´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched
OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}' (JSON::ParserError)
        from C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse'
        from trans.rb:13:in `translate'
        from trans.rb:17:in `<main>'

Answer 1

Try ensuring UTF-8 encoding and stripping any leading BOM indicators in the string:

# encoding: UTF-8
# ^-- Make sure this is on the first line!

def translate(text)
  begin
    uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{@appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
    page = HTTParty.get(uri).body
    page.force_encoding("UTF-8").gsub!("\xEF\xBB\xBF", '')
    show_info = JSON.parse(page) # this line throws the error
  rescue
      puts $!
  end
end

Sources:

• Ruby 1.9's String
• Wikipedia: Byte order mark
• Using awk to remove the Byte-order mark