How will you submit job as particular user

0 votes
While running mapreduce job, the jobs are run as the default user. What if I want to run the job as a different user? How can I do that?
Jan 30, 2019 in Big Data Hadoop by Amulya
233 views

1 answer to this question.

0 votes

You need to create a new user before submitting the Job. Please refer to the below details:

I. Create a New User

Ubuntu: $ sudo adduser –ingroup

Redhat: $ useradd -g

All: $ passwd

II. Change permission of hadoop temp directory
First, change the permission of a directory in HDFS where
hadoop stores its temporary data. You will find it in parameter hadoop.tmp.dir in core-site.xml file. Then run this command:

$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory for new user

Now we have to give write permissions to hadoop.tmp.dir. Run this command:

$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS

$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory

The new user will not have permission to run mapreduce jobs. For that, you will have to change ownership:

$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ sunewuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path
answered Jan 30, 2019 by Omkar
• 69,110 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How will the Fair Scheduler handle more than one Job?

In your case when job B gets ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
176 views
0 votes
1 answer

How will you choose various file formats for storing and processing data using Apache Hadoop ?

The decision to choose a particular file ...READ MORE

answered Sep 27, 2018 in Big Data Hadoop by zombie
• 3,790 points
414 views
0 votes
1 answer

How will you decide whether you need to use the Capacity Scheduler or the Fair Scheduler?

Fair Scheduling is the process in which ...READ MORE

answered Dec 20, 2018 in Big Data Hadoop by Frankie
• 9,810 points
1,111 views
0 votes
1 answer

How will you backup an HBase cluster?

Hey, In 2 ways we can perform HBase ...READ MORE

answered May 22, 2019 in Big Data Hadoop by Gitika
• 65,870 points
365 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
7,115 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,141 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
51,698 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,689 views
0 votes
1 answer

How to save Spark dataframe as dynamic partitioned table in Hive?

Hey, you can try something like this: df.write.partitionBy('year', ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 69,110 points
6,253 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Omkar
• 69,110 points
253 views