Scala: CSV file to Save data into HBase

0 votes
I want to Create Schema in Hbase. Develop code to connect to Hbase, Read Data and Write Data. How to do this?
Jul 25 in Apache Spark by Lohit
67 views

1 answer to this question.

0 votes

Check the reference code mentioned below:

def main(args: Array[String]): Unit = {


    val spark = SparkSession.builder().appName("sparkToHive").enableHiveSupport().getOrCreate()

    import spark.implicits._


    val config = HBaseConfiguration.create()

    config.set("hbase.zookeeper.quorum", "ip's")

    config.set("hbase.zookeeper.property.clientPort","2181")

    config.set(TableInputFormat.INPUT_TABLE, "tableName")


    val newAPIJobConfiguration1 = Job.getInstance(config)

    newAPIJobConfiguration1.getConfiguration().set(TableOutputFormat.OUTPUT_TABLE, "tableName")

    newAPIJobConfiguration1.setOutputFormatClass(classOf[TableOutputFormat[ImmutableBytesWritable]])


    val df: DataFrame = Seq(("foo", "1", "foo1"), ("bar", "2", "bar1")).toDF("key", "value1", "value2")


    val hbasePuts= df.rdd.map((row: Row) => {

      val put = new Put(Bytes.toBytes(row.getString(0)))

      put.addColumn(Bytes.toBytes("cf1"), Bytes.toBytes("value1"), Bytes.toBytes(row.getString(1)))

      put.addColumn(Bytes.toBytes("cf2"), Bytes.toBytes("value2"), Bytes.toBytes(row.getString(2)))

      (new ImmutableBytesWritable(), put)

    })


    hbasePuts.saveAsNewAPIHadoopDataset(newAPIJobConfiguration1.getConfiguration())

    }
answered Jul 25 by Hari

Related Questions In Apache Spark

0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

answered Jul 22 in Apache Spark by Vishwa
37 views
0 votes
1 answer

Scala: Convert text file data into ORC format using data frame

Converting text file to Orc: Using Spark, the ...READ MORE

answered Aug 1 in Apache Spark by Esha
131 views
0 votes
1 answer

where can i get spark-terasort.jar and not .scala file, to do spark terasort in windows.

Hi! I found 2 links on github where ...READ MORE

answered Feb 13 in Apache Spark by Omkar
• 67,600 points
118 views
0 votes
1 answer

Load CSV file data inot MySQL

If you are trying to load file ...READ MORE

answered Jul 25 in Apache Spark by Sita
26 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
3,029 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
339 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
14,957 views
0 votes
1 answer

How to create RDD from an external file source in scala?

Hi, To create an RDD from external file ...READ MORE

answered Jul 3 in Apache Spark by Gitika
• 25,340 points
56 views
0 votes
1 answer