This takes a string or unicode string in unknown encoding, tries to
guess the encoding and to replace Latin-1 characters with something
equivalent in 7-bit ASCII. Decoding an unknown string is based on
heuristics. This function may return complete garbage. The function
returns a plain ASCII string, making a best effort to convert Latin-1
characters into ASCII equivalents. It does not just strip out the Latin-1
characters. All characters in the standard 7-bit ASCII range are
preserved. In the 8th bit range all the Latin-1 accented letters are
converted to unaccented equivalents. Most symbol characters are converted
to something meaningful. Anything not converted is deleted.
Adapted from
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/251871
|