How can I replace values with none in a dataframe using pandas

+2 votes

I'm learning pandas and I came across this problem.

[code]



df.replace('-', None)
[/code]
and this returns a pretty strange result
[code]
0
0   - // this isn't replaced
1   3
2   2
3   5
4   1
5  -5
6  -1
7  -1 // this is changed to `-1`...
8   9
[/code]
what exactly am I doing wrong?

Apr 3, 2018 in Python by aryya
• 7,450 points
118,950 views

4 answers to this question.

+2 votes
Best answer

Actually in later versions of pandas this will give a TypeError:

df.replace('-', None)
TypeError: If "to_replace" and "value" are both None then regex must be a mapping

You can do it by passing either a list or a dictionary:

In [11]: df.replace('-', df.replace(['-'], [None]) # or .replace('-', {0: None})
Out[11]:
      0
0  None
1     3
2     2
3     5
4     1
5    -5
6    -1
7  None
8     9

But I recommend using NaNs rather than None:

In [12]: df.replace('-', np.nan)
Out[12]:
     0
0  NaN
1    3
2    2
3    5
4    1
5   -5
6   -1
7  NaN
8    9
answered Aug 13, 2018 by bug_seeker
• 15,520 points

selected Oct 12, 2018 by Omkar
+1 vote

I found the solution using replace with a dict the most simple and elegant solution:

df.replace({'-': None})

You can also have more replacements:

df.replace({'-': None, 'None': None})

And even for larger replacements, it is always obvious and clear what is replaced by what - which is way harder for long lists, in my opinion.

answered Oct 12, 2018 by findingbugs
• 4,780 points
Thanks a lot.

p.s.- I just sign up
0 votes

Steps to Replace Values in Pandas DataFrame

  1. Step 1: Gather your Data. To begin, gather your data with the values that you'd like to replace. ...
  2. Step 2: Create the DataFrame. ...
  3. Step 3: Replace Values in Pandas DataFrame.
answered Dec 16, 2020 by Gitika
• 65,910 points
0 votes

Steps to replace NaN values:

  1. For one column using pandas: df['DataFrame Column'] = df['DataFrame Column'].fillna(0)
  2. For one column using numpy: df['DataFrame Column'] = df['DataFrame Column'].replace(np.nan0)
  3. For the whole DataFrame using pandas: df.fillna(0)
  4. For the whole DataFrame using numpy: df.replace(np.nan0)
answered Dec 16, 2020 by Roshni
• 10,520 points

Related Questions In Python

0 votes
1 answer

How to replace values with None in Pandas data frame in Python?

Actually in later versions of pandas this ...READ MORE

answered Aug 30, 2018 in Python by Priyaj
• 58,090 points
11,137 views
0 votes
1 answer

How can I lookup hostname using the IP address with a timeout in Python?

Good question. I actually was stuck with ...READ MORE

answered Feb 6, 2019 in Python by Nymeria
• 3,560 points
2,214 views
0 votes
1 answer

How do I create a dataframe using a dictionary in pandas?

Hi @Hannah, You need to define your dictionary ...READ MORE

answered Nov 18, 2019 in Python by Eric
583 views
0 votes
1 answer

How can I define a multidimensional array in python using ctype?

Here's one quick-and-dirty method: >>> A = ((ctypes.c_float ...READ MORE

answered Oct 9, 2018 in Python by aryya
• 7,450 points
4,351 views
0 votes
2 answers

How can I rename multiple files in a certain directory using Python?

import os from optparse import OptionParser, Option class MyOption ...READ MORE

answered Jul 29, 2020 in Python by The real slim shady
4,418 views
0 votes
0 answers

how do sort the columns in a dataframe using the pandas?

can you give an example using dataframe? READ MORE

May 6, 2019 in Python by Waseem
• 4,540 points
450 views
0 votes
5 answers

how to exit a python script in an if statement

Instead of using the normal UTF-8 encoding, ...READ MORE

answered Jul 4, 2023 in Python by bodymist
• 140 points
348,135 views
0 votes
1 answer

Create an empty list in python with certain size

Try this instead: lst = [None] * 10 The ...READ MORE

answered Aug 2, 2018 in Python by bug_seeker
• 15,520 points
27,748 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP