How to create files in hdfs

Question

Every time I want to use hdfs, I have to create a file in local system and then copy it into hdfs. Is there a way I can directly create files in hdfs?

Answer 1

You can create a file directly in hdfs using put:

hadoop fs -put - /path/to/file/filename

Or you can use HdfsCLI python module:

# Loading a file in memory.
with client.read('features') as reader:
  features = reader.read()

# Directly deserializing a JSON object.
with client.read('model.json', encoding='utf-8') as reader:
  from json import load
  model = load(reader)

Link to HdfsCLI: https://hdfscli.readthedocs.io/en/latest/quickstart.html#command-line-interface