How can I replace values with none in a dataframe using pandas

Question

I'm learning pandas and I came across this problem.

[code]

df.replace('-', None)

[/code]

and this returns a pretty strange result

[code]

0
0   - // this isn't replaced
1   3
2   2
3   5
4   1
5  -5
6  -1
7  -1 // this is changed to `-1`...
8   9

[/code]

what exactly am I doing wrong?

Answer 1

Actually in later versions of pandas this will give a TypeError:

df.replace('-', None)
TypeError: If "to_replace" and "value" are both None then regex must be a mapping

You can do it by passing either a list or a dictionary:

In [11]: df.replace('-', df.replace(['-'], [None]) # or .replace('-', {0: None})
Out[11]:
      0
0  None
1     3
2     2
3     5
4     1
5    -5
6    -1
7  None
8     9

But I recommend using NaNs rather than None:

In [12]: df.replace('-', np.nan)
Out[12]:
     0
0  NaN
1    3
2    2
3    5
4    1
5   -5
6   -1
7  NaN
8    9

Answer 2

I found the solution using replace with a dict the most simple and elegant solution:

df.replace({'-': None})

You can also have more replacements:

df.replace({'-': None, 'None': None})

And even for larger replacements, it is always obvious and clear what is replaced by what - which is way harder for long lists, in my opinion.

Comments

Thanks a lot.

p.s.- I just sign up

Answer 3

Steps to Replace Values in Pandas DataFrame

1. Step 1: Gather your Data. To begin, gather your data with the values that you'd like to replace. ...
2. Step 2: Create the DataFrame. ...
3. Step 3: Replace Values in Pandas DataFrame.

Answer 4

Steps to replace NaN values:

1. For one column using pandas: df['DataFrame Column'] = df['DataFrame Column'].fillna(0)
2. For one column using numpy: df['DataFrame Column'] = df['DataFrame Column'].replace(np.nan, 0)
3. For the whole DataFrame using pandas: df.fillna(0)
4. For the whole DataFrame using numpy: df.replace(np.nan, 0)