How will you submit job as particular user?

0 votes
While running mapreduce job, the jobs are run as the default user. What if I want to run the job as a different user? How can I do that?
Jan 30 in Big Data Hadoop by Amulya
42 views

1 answer to this question.

0 votes

You need to create a new user before submitting the Job. Please refer to the below details:

I. Create a New User

Ubuntu: $ sudo adduser –ingroup

Redhat: $ useradd -g

All: $ passwd

II. Change permission of hadoop temp directory
First, change the permission of a directory in HDFS where
hadoop stores its temporary data. You will find it in parameter hadoop.tmp.dir in core-site.xml file. Then run this command:

$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory for new user

Now we have to give write permissions to hadoop.tmp.dir. Run this command:

$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS

$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory

The new user will not have permission to run mapreduce jobs. For that, you will have to change ownership:

$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ sunewuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path
answered Jan 30 by Omkar
• 67,660 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How will the Fair Scheduler handle more than one Job?

In your case when job B gets ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,710 points
49 views
0 votes
1 answer

How will you choose various file formats for storing and processing data using Apache Hadoop ?

The decision to choose a particular file ...READ MORE

answered Sep 27, 2018 in Big Data Hadoop by zombie
• 3,690 points
123 views
0 votes
1 answer
0 votes
1 answer

How will you backup an HBase cluster?

Hey, In 2 ways we can perform HBase ...READ MORE

answered May 22 in Big Data Hadoop by Gitika
• 25,340 points
74 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,304 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
391 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,294 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,187 views
0 votes
1 answer

How to save Spark dataframe as dynamic partitioned table in Hive?

Hey, you can try something like this: df.write.partitionBy('year', ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 67,660 points
1,836 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Omkar
• 67,660 points
39 views