Creating an empty Pandas DataFrame then filling it

0 votes

I'd like to iteratively fill the DataFrame with values in a time series kind of calculation. So basically, I'd like to initialize the DataFrame with columns A, B and timestamp rows, all 0 or all NaN.

I'd then add initial values and go over this data calculating the new row from the row before, say row[A][t] = row[A][t-1]+1 or so.

I'm currently using the code as below, but I feel it's kind of ugly and there must be a way to do this with a DataFrame directly, or just a better way in general. Note: I'm using Python 2.7.

import datetime as dt
import pandas as pd
import scipy as s

if __name__ == '__main__':
    base = dt.datetime.today().date()
    dates = [ base - dt.timedelta(days=x) for x in range(0,10) ]
    dates.sort()

    valdict = {}
    symbols = ['A','B', 'C']
    for symb in symbols:
        valdict[symb] = pd.Series( s.zeros( len(dates)), dates )

    for thedate in dates:
        if thedate > dates[0]:
            for symb in valdict:
                valdict[symb][thedate] = 1+valdict[symb][thedate - dt.timedelta(days=1)]

    print valdict
Jan 5 in Python by anonymous
• 8,880 points
112 views

1 answer to this question.

0 votes

Here's a couple of suggestions:

Use date_range for the index:

import datetime
import pandas as pd
import numpy as np

todays_date = datetime.datetime.now().date()
index = pd.date_range(todays_date-datetime.timedelta(10), periods=10, freq='D')

columns = ['A','B', 'C']

Note: we could create an empty DataFrame (with NaNs) simply by writing:

df_ = pd.DataFrame(index=index, columns=columns)
df_ = df_.fillna(0) # with 0s rather than NaNs

To do these type of calculations for the data, use a numpy array:

data = np.array([np.arange(10)]*3).T

Hence we can create the DataFrame:

In [10]: df = pd.DataFrame(data, index=index, columns=columns)

In [11]: df
Out[11]: 
            A  B  C
2012-11-29  0  0  0
2012-11-30  1  1  1
2012-12-01  2  2  2
2012-12-02  3  3  3
2012-12-03  4  4  4
2012-12-04  5  5  5
2012-12-05  6  6  6
2012-12-06  7  7  7
2012-12-07  8  8  8
2012-12-08  9  9  9
answered Jan 5 by Gitika
• 65,870 points

Related Questions In Python

0 votes
1 answer

Creating an empty list in Python

Here is how you can test which ...READ MORE

answered Aug 17, 2018 in Python by Priyaj
• 58,120 points
440 views
0 votes
2 answers

How to create empty pandas dataframe?

To create a simple empty DataFrame, use ...READ MORE

answered Aug 10, 2020 in Python by Nehal
• 140 points

edited Aug 10, 2020 by Nehal 9,629 views
0 votes
1 answer

How to create empty pandas dataframe only with column names?

You can do it like this: df=pd.DataFrame(columns=["Name","Old","Ne ...READ MORE

answered Apr 5, 2019 in Python by Hari
6,146 views
0 votes
1 answer

How to find an element in Pandas Dataframe?

You can do it like this: Suppose df is ...READ MORE

answered Apr 8, 2019 in Python by Tina
20,319 views
0 votes
1 answer

How to rename columns in pandas (Python)?

You can use the rename function in ...READ MORE

answered Apr 30, 2018 in Data Analytics by DeepCoder786
• 1,720 points

edited Jun 8, 2020 by MD 651 views
0 votes
1 answer

What is the Difference in Size and Count in pandas (python)?

The major difference is "size" includes NaN values, ...READ MORE

answered Apr 30, 2018 in Data Analytics by DeepCoder786
• 1,720 points

edited Jun 8, 2020 by Gitika 1,458 views
0 votes
2 answers

Replacing a row in pandas data.frame

key error. I love python READ MORE

answered Feb 18, 2019 in Data Analytics by anonymous
8,745 views
0 votes
1 answer

Converting a pandas data-frame to a dictionary

Emp_dict=Employee.to_dict('records') You can directly use the 'to_dict()' function ...READ MORE

answered May 23, 2018 in Data Analytics by Bharani
• 4,620 points
3,504 views
+2 votes
4 answers

How can I replace values with 'none' in a dataframe using pandas

Actually in later versions of pandas this ...READ MORE

answered Aug 13, 2018 in Python by bug_seeker
• 15,550 points
38,776 views
0 votes
3 answers

Python Pandas: ValueError: DataFrame constructor not properly called!

The syntax is not right. The right ...READ MORE

answered Mar 28, 2019 in Python by Ritu
53,604 views