Error Pandas read csv low memory and dtype options

+1 vote

When calling

df = pd.read_csv('somefile.csv')

I get:

/Users/Niraj/anaconda/envs/py27/lib/python2.7/site-packages/pandas/io/parsers.py:1130: DtypeWarning: Columns (4,5,7,16) have mixed types. Specify dtype option on import or set low_memory=False.

Why is the dtype option related to low_memory, and why would making it False help with this problem?

Jun 22, 2020 in Python by kartik
• 37,490 points
1,986 views

1 answer to this question.

0 votes

Hello @kartik,

Try this:

dashboard_df = pd.read_csv(p_file, sep=',', error_bad_lines=False, index_col=False, dtype='unicode')

According to the pandas documentation:

dtype : Type name or dict of column -> type

As for low_memory, it's True by default and isn't yet documented. I don't think its relevant though. The error message is generic, so you shouldn't need to mess with low_memory anyway.

Hope this is helpful!

Thank You!!

answered Jun 22, 2020 by Niroj
• 82,560 points

Related Questions In Python

0 votes
1 answer

Python error "TypeError: Cannot compare types 'ndarray(dtype=int64)' and 'str'"

Hey @Ashish, change the emotion_map to the ...READ MORE

answered May 30, 2019 in Python by Mir
3,513 views
0 votes
1 answer

When I create and remove files rapidly on windows using python I get WindowsError (Error 5)

Here's the short answer: disable any antivirus or ...READ MORE

answered Aug 31, 2018 in Python by charlie_brown
• 7,780 points
806 views
0 votes
1 answer

How to Pivot pandas for removing of some headers and renaming of some indexes?

Solution is add parameter values to pivot, then add reset_index for column ...READ MORE

answered Sep 27, 2018 in Python by Priyaj
• 58,100 points
9,018 views
–1 vote
1 answer

Python pandas axis error

When you are applying sum() you don't ...READ MORE

answered Jan 8, 2019 in Python by Omkar
• 69,090 points
838 views
0 votes
1 answer

How to create Pandas series from numpy array?

Hi. Refer to the below command: import pandas ...READ MORE

answered Apr 1, 2019 in Python by Pavan
1,517 views
0 votes
1 answer
0 votes
1 answer

How to create Pandas series from dictionary?

Here's a sample script: import pandas as pd import ...READ MORE

answered Apr 1, 2019 in Python by Prateek
784 views
0 votes
2 answers
0 votes
2 answers

Python Pandas error: AttributeError: 'DataFrame' object has no attribute 'rows'

Try this: data=pd.read_csv('/your file name', delim_whitespace=Tru ...READ MORE

answered Dec 10, 2020 in Python by anonymous
• 82,560 points
59,678 views
0 votes
1 answer

Error:urllib and “SSL: CERTIFICATE_VERIFY_FAILED”

Hello @kartik, Insert this code at the beginning ...READ MORE

answered Jun 5, 2020 in Python by Niroj
• 82,560 points
1,034 views