What are the roles of a Big Data Engineer

0 votes
Nov 22, 2018 in Career Counselling by Kapoor
• 1,070 points
881 views

5 answers to this question.

0 votes

These are some typical roles and responsibilities of a Big Data Engineer:

  • Selecting and integrating Big Data tools and frameworks.
  • Implementing ETL process
  • Monitoring performance and advising any necessary infrastructure changes
  • Defining data retention policies
  • Management of Hadoop cluster
  • Building stream-processing systems
  •  Integration of data from multiple data sources
answered Nov 22, 2018 by Abdul
0 votes
  • Statistical analysis and modeling
  • Database architectures
  • Hadoop-based technologies (e.g. MapReduce, Hive and Pig)
  • SQL-based technologies (e.g. PostgreSQL and MySQL)
  • NoSQL technologies (e.g. Cassandra and MongoDB)
  • Data modeling tools (e.g. ERWin, Enterprise Architect and Visio)
  • Python, C/C++ Java, Perl
  • MatLab, SAS, R
  • Data warehousing solutions
  • Predictive modeling, NLP and text analysis
  • Machine learning
  • Data mining
  • UNIX, Linux, Solaris and MS Windows
To know more about Big Data, It's recommended to join Big Data Architect today.
answered Nov 23, 2018 by meghanaa
• 140 points
Hello @meghanaa madam. Machine Learning is a separate domain, right? Then why is it included in Data Engineer roles?
0 votes

Big Data Engineer is mainly responsible for these things:

  • Development of Hadoop
  • processing data using hive and pig
  • manage production systems such as Hadoop, Kafka, Cassandra etc. 
answered Jan 17, 2019 by Layla
0 votes
A Big Data Engineer is responsible for the Databases and Data Processing Systems of the Organization. Designing, implementing and maintaining the Database is mainly the task of the Big Data Engineer.
answered Jan 22, 2019 by Lohit
0 votes
Adding to the above answers, a Data Engineer will also have to decide which architecture is more appropriate and also find a solution for unstructured data problems.
answered Jan 22, 2019 by Kiran

Related Questions In Career Counselling

0 votes
3 answers

What are the roles of a data scientist

These are the high-level roles: Data mining and ...READ MORE

answered Feb 11, 2019 in Career Counselling by Mariam
9,063 views
0 votes
3 answers

What are the roles and responsibilities of a Network engineer?

Being a network engineer you should have multiple ...READ MORE

answered Apr 2, 2019 in Career Counselling by reshma
917 views
0 votes
2 answers

What are the roles of security engineer in any company?

Some of the main roles of Security ...READ MORE

answered Apr 2, 2019 in Career Counselling by Monisha
570 views
–1 vote
5 answers

What are the roles and responsibilities of a full stack developer?

Key responsibilities of a full stack web ...READ MORE

answered Nov 22, 2021 in Career Counselling by Varun SAX
9,193 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,558 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,185 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,209 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
5 answers

What are the roles and responsibilities of a Data Scientist?

Following are one of the most important ...READ MORE

answered Feb 22, 2019 in Career Counselling by Barbara
2,440 views
0 votes
2 answers

What are the skills required to be a Data Engineer?

A Big Data Engineer will mainly work ...READ MORE

answered Jan 22, 2019 in Career Counselling by Sanam
1,012 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP