Why does the string change when using python split?

Question

test_str = "Question: The cryptocurrency Bitcoin Cash (BCH/USD) settled at 1368 USD at 07:00 AM UTC at the Bitfinex exchange on Monday, April 23. In your opinion, will BCH/USD trade above 1500 USD (+9.65%) at anу timе bеfore Арril 28? Indicаtоr: 60.76%"

print(test_str)
print(test_str.split('before '))

This the output I get after spliting

"['Question: The cryptocurrency Bitcoin Cash (BCH/USD) settled at 1368 USD at 07:00 AM UTC at the Bitfinex exchange on Monday, April 23. In your opinion, will BCH/USD trade above 1500 USD (+9.65%) at an\xd1\x83 tim\xd0\xb5 b\xd0\xb5fore \xd0\x90\xd1\x80ril 28? Indic\xd0\xb0t\xd0\xber: 60.76%']"

Demo: https://repl.it/repls/VitalOrganicBackups

Answer 1

The problem is caused by a mix of Latin and Cyrillic characters. They print exactly the same in most policies, but are still different characters and do have different codes.

The output in the question is for Python 2.7 (what original question asker used) but it is easy to have equivalent behaviour in Python 3:

>>> print(test_str.encode('UTF8'))
b'Question: The cryptocurrency Bitcoin Cash (BCH/USD) settled at 1368 USD at 07:00 AM UTC at the Bitfinex exchange on Monday, April 23. In your opinion, will BCH/USD trade above 1500 USD (+9.65%) at an\xd1\x83 tim\xd0\xb5 b\xd0\xb5fore \xd0\x90\xd1\x80ril 28? Indic\xd0\xb0t\xd0\xber: 60.76%'

The unicodedata module helps to better understand what actually happens:

>>> for i in b'\xd1\x83\xd0\xb5\xd0\x90\xd1\x80\xd0\xbe'.decode('utf8'):
    print(i, hex(ord(i)), i.encode('utf8'), unicodedata.name(i))

у 0x443 b'\xd1\x83' CYRILLIC SMALL LETTER U
е 0x435 b'\xd0\xb5' CYRILLIC SMALL LETTER IE
А 0x410 b'\xd0\x90' CYRILLIC CAPITAL LETTER A
р 0x440 b'\xd1\x80' CYRILLIC SMALL LETTER ER
о 0x43e b'\xd0\xbe' CYRILLIC SMALL LETTER O

So the original text contains cyrillic letters and for comparisons, they are not the same of their latin equivalent, even if they print the same. The problem has nothing to do with split but is just a poor original string.