Importing a text file gives error

Question

I have a text file that has the following data:

5298    10036   4   360 8
6128    11947   2   385 7
9472    18930   0   233 4
5056    9790    1   293 6

I read this file using the following code:

file1 = open("test.txt","r")
lines = file1.readlines()       
BF=[map(float, line.split()) for line in lines]

This gives me the following error:

could not convert string to float: ÿþ5

Why do I see this error?

Update:

print lines 

shows:

['\xff\xfe5\x002\x009\x008\x00\t\x001\x000\x000\x003\x006\x00\t\x004\x00\t\x003\x006\x000\x00\t\x008\x00\r\x00\n', '\x006\x001\x002\x008\x00\t\x001\x001\x009\x004\x007\x00\t\x002\x00\t\x003\x008\x005\x00\t\x007\x00\r\x00\n', '\x009\x004\x007\x002\x00\t\x001\x008\x009\x003\x000\x00\t\x000\x00\t\x002\x003\x003\x00\t\x004\x00\r\x00\n', '\x005\x000\x005\x006\x00\t\x009\x007\x009\x000\x00\t\x001\x00\t\x002\x009\x003\x00\t\x006\x00\r\x00\n', '\x001\x005\x000\x006\x004\x00\t\x003\x000\x001\x006\x000\x00\t\x001\x00\t\x003\x001\x002\x00\t\x008\x00']

Answer 1

You have a utf-16 BOM, this is 0xFE 0xFF which is interpreted as ÿþ, you need to open the file and pass the encoding.

file1 = open("test.txt","r", encoding = "utf-16")

As you using python 2 you could try this:

import io
file1 = io.open("test.txt","r", encoding = "utf-16")

Answer 2

import io
file1 = io.open("test.txt","r",encoding='utf-16')
lines = file1.readlines()
BF=[map(float, line.split()) for line in lines]
print BF

Result:

[[5298.0, 10036.0, 4.0, 360.0, 8.0], [6128.0, 11947.0, 2.0, 385.0, 7.0], [9472.0, 18930.0, 0.0, 233.0, 4.0], [5056.0, 9790.0, 1.0, 293.0, 6.0]]