Is Spark Sql provides indexing to improve processing speed

Question

Hi,

Is Spark Sql provides indexing concept? I tried but I am getting error. It shows not found in the org module.

Can somebody help me?

Thank You

Answer 1

Hi@akhtar,

There is no concept of indexing in Spark Sql. That is why you are getting error. The fundamental reason why indexing over external data sources is not in the Spark scope is that Spark is not a data management system but a batch data processing engine. Spark Sql provides the interface to run queries.

Hope this will help

Thank You