How to create files in hdfs?

0 votes
Every time I want to use hdfs, I have to create a file in local system and then copy it into hdfs. Is there a way I can directly create files in hdfs?
Jan 25 in Big Data Hadoop by Sanam
84 views

1 answer to this question.

0 votes

You can create a file directly in hdfs using put:

hadoop fs -put - /path/to/file/filename

Or you can use HdfsCLI python module:

# Loading a file in memory.
with client.read('features') as reader:
  features = reader.read()

# Directly deserializing a JSON object.
with client.read('model.json', encoding='utf-8') as reader:
  from json import load
  model = load(reader)

Link to HdfsCLI: https://hdfscli.readthedocs.io/en/latest/quickstart.html#command-line-interface

answered Jan 25 by Omkar
• 67,290 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is the standard way to create files in your hdfs file-system?

Well, it's so easy. Just enter the below ...READ MORE

answered Sep 22, 2018 in Big Data Hadoop by Frankie
• 9,810 points
87 views
0 votes
1 answer

How to create new directory in hdfs using java code?

Try this: { Configuration config ...READ MORE

answered Dec 6, 2018 in Big Data Hadoop by Omkar
• 67,290 points
46 views
0 votes
1 answer

How to copy files in hdfs with wildcards?

You can use the hdfs command: hdfs fs ...READ MORE

answered Dec 11, 2018 in Big Data Hadoop by Omkar
• 67,290 points
110 views
0 votes
1 answer

How to create a Hive table from sequence file stored in HDFS?

There are two SerDe for SequenceFile as ...READ MORE

answered Dec 17, 2018 in Big Data Hadoop by Omkar
• 67,290 points
376 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,510 points
2,392 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,510 points
244 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
12,188 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
896 views
0 votes
1 answer

How to read HDFS and local files with the same code in Java?

You can try something like this: ​ ...READ MORE

answered Nov 22, 2018 in Big Data Hadoop by Omkar
• 67,290 points
339 views
0 votes
1 answer

How to concatenate hdfs files and store in output file?

You can use a combination of cat and put command. Something ...READ MORE

answered Dec 5, 2018 in Big Data Hadoop by Omkar
• 67,290 points
356 views