How will you submit job as particular user?

0 votes
While running mapreduce job, the jobs are run as the default user. What if I want to run the job as a different user? How can I do that?
Jan 30 in Big Data Hadoop by Amulya
24 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

You need to create a new user before submitting the Job. Please refer to the below details:

I. Create a New User

Ubuntu: $ sudo adduser –ingroup

Redhat: $ useradd -g

All: $ passwd

II. Change permission of hadoop temp directory
First, change the permission of a directory in HDFS where
hadoop stores its temporary data. You will find it in parameter hadoop.tmp.dir in core-site.xml file. Then run this command:

$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory for new user

Now we have to give write permissions to hadoop.tmp.dir. Run this command:

$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS

$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory

The new user will not have permission to run mapreduce jobs. For that, you will have to change ownership:

$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ sunewuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path
answered Jan 30 by Omkar
• 65,820 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How will the Fair Scheduler handle more than one Job?

In your case when job B gets ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 9,030 points
27 views
0 votes
1 answer

How will you choose various file formats for storing and processing data using Apache Hadoop ?

The decision to choose a particular file ...READ MORE

answered Sep 27, 2018 in Big Data Hadoop by zombie
• 3,690 points
50 views
0 votes
1 answer
0 votes
1 answer

How hadoop mapreduce job is submitted to worker nodes?

Alright, I think you are basically looking ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
1,186 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
1,643 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
130 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
7,966 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
555 views
0 votes
1 answer

How to save Spark dataframe as dynamic partitioned table in Hive?

Hey, you can try something like this: df.write.partitionBy('year', ...READ MORE

answered Nov 6, 2018 in Big Data Hadoop by Omkar
• 65,820 points
312 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Omkar
• 65,820 points
17 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.