How to convert pyspark Dataframe to pandas Dataframe

Question

Hi Guys,

I am trying to create one Machine Learning model using pyspark. I want to convert my pyspark dataframe to pandas dataframe for some operation. How can I do that?

Answer 1

Hi@akhtar,

To convert pyspark dataframe into pandas dataframe, you have to use this below given command.

$ pandas_df = spark_df.select("*").toPandas()

Hope this will help you.

To know more about Pyspark, it's recommended that you join Pyspark course online.

Thanks.