Pandas DataFrames in a loop, df.to_csv()

0 votes

I am trying to write a df to a csv from a loop, each line represents a df, but I am finding some difficulties once the headers are not equal for all dfs, some of them have values for all dates and others no.

I am writing the df using a function similar to this one:

def write_csv():
    for name, df in data.items():
        df.to_csv(meal+'mydf.csv', mode='a')

and it creates a csv for each meal (lunch an dinner) each df is similar to this:

Name    Meal    22-03-18    23-03-18    25-03-18        
Peter   Lunch   12          10          9

or:

Name    Meal    22-03-18    23-03-18    25-03-18        
Peter   Dinner  12          10          9

I was trying to use pandas concatenate, but I am not finding a way to implement this in the function. My goal is to have the headers with all the dates (as the example of desired output), independent if the DataFrame appended to the csv have or not values in all dates.

Actual output:
Name    Meal    22-03-18    23-03-18    25-03-18        
Peter   Lunch   12          10          9       
Mathew  Lunch   12          11          11         10     9
Ruth    Lunch   9           9           8          9    
Anna    Lunch   10          12          11         13     10


output with headers:
Name    Meal    22-03-18    23-03-18    25-03-18           
Peter   Lunch   12          10          9       
Name    Meal    21-03-18    22-03-18    23-03-18    24-03-18    25-03-18
Mathew  Lunch   12          11          11          10          9
Name    Meal    21-03-18    22-03-18    24-03-18    25-03-18    
Ruth    Lunch   9           9           8           9   
Name    Meal    21-03-18    22-03-18    23-03-18    24-03-18    25-03-18
Anna    Lunch   10          12          11          13          10



Output desired:
Name    Meal    21-03-18    22-03-18    23-03-18    24-03-18    25-03-18
Peter   Lunch   12          10          9   
Mathew  Lunch               12          11          11           10
Ruth    Lunch   9           9           8           9
Anna    Lunch   10          12          11          13           10

Sep 25, 2018 in Python by bug_seeker
• 15,350 points
865 views

1 answer to this question.

0 votes

can you try something like this? not sure if is exactly what you want, but it will concatenate dataframes without fully overlapping columns

def write_csv():
    df2 = pd.DataFrame()
    for name, df in data.items():
        df2 = df2.append(df)
    df2.to_csv('mydf.csv')
answered Sep 25, 2018 by Priyaj
• 56,900 points

Related Questions In Python

+2 votes
2 answers

How can I replace values with 'none' in a dataframe using pandas

Actually in later versions of pandas this ...READ MORE

answered Aug 13, 2018 in Python by bug_seeker
• 15,350 points
6,634 views
0 votes
1 answer

How to convert a Pandas GroupBy object to DataFrame in Python

g1 here is a DataFrame. It has a hierarchical index, ...READ MORE

answered Nov 12, 2018 in Python by Nymeria
• 3,520 points
6,825 views
0 votes
1 answer

How to use Pandas HDF5 as a Database in Python?

HDF5 works fine for concurrent read only ...READ MORE

answered Nov 30, 2018 in Python by Nymeria
• 3,520 points

edited Dec 10, 2018 by Nymeria 103 views
0 votes
1 answer

How to rename columns in pandas (Python)?

It is easy by just adding ".columns" ...READ MORE

answered Apr 30, 2018 in Data Analytics by DeepCoder786
• 1,720 points
108 views
0 votes
1 answer

What is the Difference in Size and Count in pandas (python)?

The major difference is size includes NaN ...READ MORE

answered Apr 30, 2018 in Data Analytics by DeepCoder786
• 1,720 points
731 views
0 votes
2 answers
0 votes
1 answer

Converting a pandas data-frame to a dictionary

Emp_dict=Employee.to_dict('records') You can directly use the 'to_dict()' function ...READ MORE

answered May 23, 2018 in Data Analytics by Bharani
• 4,550 points
1,440 views
0 votes
1 answer

How do I copy a file in python?

Use the shutil module. copyfile(src, dst) Copy the contents ...READ MORE

answered Jul 31, 2018 in Python by Priyaj
• 56,900 points
37 views
0 votes
1 answer

Is there a label/goto in Python?

No, Python does not support labels and ...READ MORE

answered Aug 1, 2018 in Python by Priyaj
• 56,900 points
120 views