Python UnicodeDecodeError utf-8 codec can t decode byte 0xa0 in position 10 invalid start byte

Question

Unable to import this file it shows an error. My code was:

import pandas as pd
a = pd.read_csv("filename.csv")

Answer 1

You have to use the encoding as latin1 to read this file as there are some special character in this file, use the below code snippet to read the file. Try this:

import pandas as pd
data=pd.read_csv("C:\\Users\\akashkumar\\Downloads\\Customers.csv",encoding='latin1')
print(data.head())

Hope it helps!!

If you need to know more about Python, It's recommended to join Python course today.

Thanks!

Comments

worked for me also thanks

---

OMG! Thank you so much. I was stuck on this for a while!! :D

---

Thank you!!! that additional encoding='latin1' entirely fixes this issue

---

Thanks a lot. I was really stuck with this problem. 

Alternatively we can use "encoding='unicode_escape'" with the same effect.

import pandas as pd
data=pd.read_csv("C:\\Users\\akashkumar\\Downloads\\Customers.csv",encoding=''unicode_escape')
print(data.head())

---

worked for me as well. thanks very much indeed

Answer 2

tl;dr / quick fix

1. Don't decode/encode willy nilly.
2. Don't assume your strings are UTF-8 encoded.
3. Try to convert strings to Unicode strings as soon as possible in your code.
4. Fix your locale: How to solve UnicodeDecodeError in Python 3.6?
5. Don't be tempted to use quick reload hacks.

Ready to unlock the power of data? Join our Data Science with Python Course and gain the skills to analyze, visualize, and make data-driven decisions.

Answer 3

str = unicode(str, errors='replace')

or

str = unicode(str, errors='ignore')

Note: This will strip out (ignore) the characters in question returning the string without them.

For me this is ideal case since I'm using it as protection against non-ASCII input which is not allowed by my application.

Alternatively: Use the open method from the codecs module to read in the file:

import codecs
with codecs.open(file_name, 'r', encoding='utf-8',
                 errors='ignore') as fdata:

Answer 4

Python bytes decode() function is used to convert bytes to string object. Both these functions allow us to specify the error handling scheme to use for encoding/decoding errors. The default is 'strict' meaning that encoding errors raise a UnicodeEncodeError.

The UnicodeDecodeError normally happens when decoding an str string from a certain coding. Since codings map only a limited number of str strings to unicode characters, an illegal sequence of str characters will cause the coding-specific decode() to fail