How will you submit job as particular user

0 votes
While running mapreduce job, the jobs are run as the default user. What if I want to run the job as a different user? How can I do that?
Jan 31, 2019 in Big Data Hadoop by Amulya
1,233 views

1 answer to this question.

0 votes

You need to create a new user before submitting the Job. Please refer to the below details:

I. Create a New User

Ubuntu: $ sudo adduser –ingroup

Redhat: $ useradd -g

All: $ passwd

II. Change permission of hadoop temp directory
First, change the permission of a directory in HDFS where
hadoop stores its temporary data. You will find it in parameter hadoop.tmp.dir in core-site.xml file. Then run this command:

$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory for new user

Now we have to give write permissions to hadoop.tmp.dir. Run this command:

$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS

$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory

The new user will not have permission to run mapreduce jobs. For that, you will have to change ownership:

$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ sunewuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path
answered Jan 31, 2019 by Omkar
• 69,220 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How will the Fair Scheduler handle more than one Job?

In your case when job B gets ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,072 views
0 votes
1 answer

How will you choose various file formats for storing and processing data using Apache Hadoop ?

The decision to choose a particular file ...READ MORE

answered Sep 27, 2018 in Big Data Hadoop by zombie
• 3,790 points
1,757 views
0 votes
1 answer

How will you decide whether you need to use the Capacity Scheduler or the Fair Scheduler?

Fair Scheduling is the process in which ...READ MORE

answered Dec 20, 2018 in Big Data Hadoop by Frankie
• 9,830 points
2,940 views
0 votes
1 answer

How will you backup an HBase cluster?

Hey, In 2 ways we can perform HBase ...READ MORE

answered May 22, 2019 in Big Data Hadoop by Gitika
• 65,770 points
1,532 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
11,067 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,569 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
109,017 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,350 points
4,633 views
0 votes
1 answer

How to save Spark dataframe as dynamic partitioned table in Hive?

Hey, you can try something like this: df.write.partitionBy('year', ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 69,220 points
8,542 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,220 points
993 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP