Question

How to index one csv file with no header after converting the csv to a dataframe i need to name the columns in order to normalize in minmaxScaler

Answer 1

Hi@Manas,

You can read your dataset from CSV file to Dataframe and set header value to false. So it will create a data frame with the index value.

df = spark.read.format("csv").option("header", "false").load("csvfile.csv")

After that, you can replace the index value with column name.

val df2 = df.withColumnRenamed(0,"DateOfBirth")
           .withColumnRenamed(1,"salary")
df2.printSchema()