Get python getaddresses() to decode encoded-word encoding

Question

msg = \
"""To: =?ISO-8859-1?Q?Caren_K=F8lter?= <[email protected]>, [email protected]
Cc: "James =?ISO-8859-1?Q?K=F8lter?=" <[email protected]>
Subject: hello

message body blah blah blah

"""

import email.parser, email.utils
import itertools


parser = email.parser.Parser()
parsed_message = parser.parsestr(msg)

address_fields = ('to', 'cc')
addresses = itertools.chain(*(parsed_message.get_all(field) for field in address_fields if parsed_message.has_key(field)))
address_list = set(email.utils.getaddresses(addresses))


print address_list

It seems like email.utils.getaddresses() doesn't seem to automatically handle MIME RFC 2047 in address fields.

How can I get the expected result below?

actual result:

set([('', '[email protected]'), ('=?ISO-8859-1?Q?Caren_K=F8lter?=', '[email protected]'), ('James =?ISO-8859-1?Q?K=F8lter?=', '[email protected]')])

desired result:

set([('', '[email protected]'), (u'Caren_K\xf8lter', '[email protected]'), (u'James \xf8lter', '[email protected]')])

Answer 1

The function you want is email.header.decode_header, which returns a list of (decoded_string, charset) pairs. It's up to you to further decode them according to charset and join them back together again before passing them to email.utils.getaddresses or wherever.

You might think that this would be straightforward:

def decode_rfc2047_header(h):
    return ' '.join(s.decode(charset or 'ascii')
                   for s, charset in email.header.decode_header(h))

But since message headers typically come from untrusted sources, you have to handle (1) badly encoded data; and (2) bogus character set names. So you might do something like this:

def decode_safely(s, charset='ascii'):
    """Return s decoded according to charset, but do so safely."""
    try:
        return s.decode(charset or 'ascii', 'replace')
    except LookupError: # bogus charset
        return s.decode('ascii', 'replace')

def decode_rfc2047_header(h):
    return ' '.join(decode_safely(s, charset)
                   for s, charset in email.header.decode_header(h))

Answer 2

Yeah, the email package interface really isn't very helpful a lot of the time.

Here, you have to use email.header.decode_header manually on each address, and then, since that gives you a list of decoded tokens, you have to stitch them back together again manually:

for name, address in email.utils.getaddresses(addresses):
    name= u' '.join(
        unicode(b, e or 'ascii') for b, e in email.header.decode_header(name)
    )
    ...