How to get Spark dataset metadata

Question

I want to convert a dataset into some other object. I need the metadata of the dataset before converting it. So is there any function in spark that can help?

Thanks in advance.

Answer 1

There are a bunch of functions that can help you here.\

For the schema of dataset ds, you can use ds.schema

you have ds.schema.size, ds.schema.fields, ds.schema.fieldNames

Also, you can see the details with ds.printSchema()

Hope this helps