Can anyone help me with the right path to get a Hadoop file system so that I can read and write a certain file in HDFS? I tried a few methods as follows:
final Configuration conf = new Configuration();
final FileSystem fs = FileSystem.get(conf);
The Class configuration documentation says the core-site.xml properties automatically get loaded when an object is created and the required file is in the classpath and there would be no need to set it again.
The HDFS-site.xml is not added, yet it seems to work fine.
Is it a recommended option to just configure Core-site.xml and skip HDFS-site.xml or is it mandatory to define both? please let me know.