How to open an ascii-encoded file as UTF8?

Question

My files are in US-ASCII and a command like a = file( 'main.html') and a.read() loads them as an ASCII text. How do I get it to load as UTF8?

The problem I am tring to solve is:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xae' in position 38: ordinal not in range(128)

I was using the content of the files for templating as in template_str.format(attrib=val). But the string to interpolate is of a superset of ASCII.

Our team's version control and text editors does not care about the encoding. So how do I handle it in the code?

Answer 1

A solution working in Python2:

import codecs
fo = codecs.open('filename.txt', 'r', 'ascii')
content = fo.read()  ## returns unicode
assert type(content) == unicode
fo.close()

utf8_content = content.encode('utf-8')
assert type(utf8_content) == str