How to read mp4 video file stored at HDFS using pyspark

Question

Is there any way to read the video file (mp4) using spark? I need to read the file and extract frame by frame data.

Answer 1

Hi@Amey,

You can enable WebHDFS to do this task. Follow the below given steps.

1. Enable WebHDFS in HDFS configuration file. (hdfs-site.xml)
   Set dfs.webhdfs.enabled as true.

2. Restart HDFS daemons.

3. We can now access HDFS with the WebHDFS API.

Now you can browse your video by using curl command.

$ curl -i "http://<HOST>:<PORT>/webhdfs/v1/<PATH>

As we know video file is combination of arrays. So read your video file in a variable.

Hope this helps!

To know more about Pyspark, it's recommended that you join Pyspark Certification today.

Thanks.