In this article

What is HBase?

HBase is a low-latency NoSQL database that allows online transactional processing of big data. HBase is offered as a managed cluster integrated into the Azure environment. The clusters are configured to store data directly in Azure Blob storage, which provides low latency and increased elasticity in performance/cost choices. This enables customers to build interactive websites that work with large datasets, to build services that store sensor and telemetry data from millions of end points, and to analyze this data with Hadoop jobs. For more information on HBase and the scenarios it can be used for, see HDInsight HBase overview.

NOTE:

HBase (version 0.98.0) is only available for use with HDInsight 3.1 clusters on HDInsight (based on Apache Hadoop and YARN 2.4.0). For version information, see What's new in the Hadoop cluster versions provided by HDInsight?

Prerequisites

Before you begin this tutorial, you must have the following:

Provision an HBase cluster on the Azure portal

This section describes how to provision an HBase cluster using the Azure Management portal.

NOTE:

The steps in this article create an HDInsight cluster using basic configuration settings. For information on other cluster configuration settings, such as using Azure Virtual Network or a metastore for Hive and Oozie, see Provision an HDInsight cluster.

To provision an HDInsight cluster in the Azure Management portal

  1. Sign in to the Azure Management Portal.
  2. Click NEW on the lower left, and then click DATA SERVICESHDINSIGHTHBASE.
  3. Enter CLUSTER NAMECLUSTER SIZE, CLUSTER USER PASSWORD, and STORAGE ACCOUNT.

  4. Click on the check icon on the lower left to create the HBase cluster.

Create an HBase sample table from the HBase shell

This section describes how to enable and use the Remote Desktop Protocol (RDP) to access the HBase shell and then use it to create an HBase sample table, add rows, and then list the rows in the table.

It assumes you have completed the procedure outlined in the first section, and so have already successfully created an HBase cluster.

To enable the RDP connection to the HBase cluster

  1. From the Management portal, click HDINSIGHT from the left to view the list of the existing clusters.
  2. Click the HBase cluster where you want to open HBase Shell.
  3. Click CONFIGURATION from the top.
  4. Click ENABLE REMOTE from the bottom.
  5. Enter the RDP user name and password. The user name must be different from the cluster user name you used when provisioning the cluster. TheEXPIRES ON data can be up to seven days from today.
  6. Click the check on the lower right to enable remote desktop.
  7. After the RPD is enabled, click CONNECT from the bottom of the CONFIGURATION tab, and follow the instructions.

To open the HBase Shell

  1. Within your RDP session, click on the Hadoop Command Line shortcut located on the desktop.

  2. Change the folder to the HBase home directory:

    cd %HBASE_HOME%\bin
  3. Open the HBase shell:

    hbase shell

To create a sample table, add data and retrieve the data

  1. Create a sample table:

    create 'sampletable', 'cf1'
  2. Add a row to the sample table:

    put 'sampletable', 'row1', 'cf1:col1', 'value1'
  3. List the rows in the sample table:

    scan 'sampletable'

Check cluster status in the HBase WebUI

HBase also ships with a WebUI that helps monitoring your cluster, for example by providing request statistics or information about regions. On the HBase cluster you can find the WebUI under the address of the zookeepernode.

http://zookeepernode:60010/master-status

In a HighAvailability (HA) cluster, you will find a link to the current active HBase master node hosting the WebUI.

Bulk load a sample table

  1. Create samplefile1.txt containing the following data, and upload to Azure Blob Storage to /tmp/samplefile1.txt:

    row1    c1  c2
    row2 c1 c2
    row3 c1 c2
    row4 c1 c2
    row5 c1 c2
    row6 c1 c2
    row7 c1 c2
    row8 c1 c2
    row9 c1 c2
    row10 c1 c2
  2. Change the folder to the HBase home directory:

    cd %HBASE_HOME%\bin
  3. Execute ImportTsv:

    hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns="HBASE_ROW_KEY,a:b,a:c" -Dimporttsv.bulk.output=/tmpOutput sampletable2 /tmp/samplefile1.txt
  4. Load the output from prior command into HBase:

    hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles /tmpOutput sampletable2

Use Hive to query an HBase table

Now you have an HBase cluster provisioned and have created an HBase table, you can query it using Hive. This section creates a Hive table that maps to the HBase table and uses it to queries the data in your HBase table.

To open cluster dashboard

  1. Sign in to the Azure Management Portal.
  2. Click HDINSIGHT from the left pane. You shall see a list of clusters created including the one you just created in the last section.
  3. Click the cluster name where you want to run the Hive job.
  4. Click QUERY CONSOLE from the bottom of the page to open cluster dashboard. It opens a Web page on a different browser tab.
  5. Enter the Hadoop User account username and password. The default username is admin, the password is what you entered during the provision process. A new browser tab is opened.
  6. Click Hive Editor from the top. The Hive Editor looks like :

To run Hive queries

  1. Enter the HiveQL script below into Hive Editor and click SUBMIT to create an Hive Table mapping to the HBase table. Make sure that you have created the sampletable table referenced here in HBase using the HBase Shell before executing this statement.

    CREATE EXTERNAL TABLE hbasesampletable(rowkey STRING, col1 STRING, col2 STRING)
    STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,cf1:col1,cf1:col2')
    TBLPROPERTIES ('hbase.table.name' = 'sampletable');

    Wait until the Status is updated to Completed.

  2. Enter the HiveQL script below into Hive Editor, and then click SUBMIT button. The Hive query queries the data in the HBase table:

    SELECT count(*) FROM hbasesampletable;
  3. To retrieve the results of the Hive query, click on the View Details link in the Job Session window when the job finishes executing. The Job Output shall be 1 because you only put one record into the HBase table.

To browse the output file

  1. From Query Console, click File Browser from the top.
  2. Click the Azure Storage account used as the default file system for the HBase cluster.
  3. Click the HBase cluster name. The default Azure storage account container uses the cluster name.
  4. Click user.
  5. Click admin. This is the Hadoop user name.
  6. Click the job name with the Last Modified time matching the time when the SELECT Hive query ran.
  7. Click stdout. Save the file and open the file with Notepad. The output shall be 1.

Use HBase REST Client Library for .NET C# APIs to create an HBase table and retrieve data from the table

The Microsoft HBase REST Client Library for .NET project must be downloaded from GitHub and the project built to use the HBase .NET SDK. The following procedure includes the instructions for this task.

  1. Create a new C# Visual Studio Windows Desktop Console application.
  2. Open NuGet Package Manager Console by click the TOOLS menu, NuGet Package ManagerPackage Manager Console.
  3. Run the following NuGet command in the console:

    Install-Package Microsoft.HBase.Client

  4. Add the following using statements on the top of the file:

    using Microsoft.HBase.Client;
    using org.apache.hadoop.hbase.rest.protobuf.generated;
  5. Replace the Main function with the following:

    static void Main(string[] args)
    {
    string clusterURL = "https://<yourHBaseClusterName>.azurehdinsight.net";
    string hadoopUsername= "<yourHadoopUsername>";
    string hadoopUserPassword = "<yourHadoopUserPassword>"; string hbaseTableName = "sampleHbaseTable"; // Create a new instance of an HBase client.
    ClusterCredentials creds = new ClusterCredentials(new Uri(clusterURL), hadoopUsername, hadoopUserPassword);
    HBaseClient hbaseClient = new HBaseClient(creds); // Retrieve the cluster version
    var version = hbaseClient.GetVersion();
    Console.WriteLine("The HBase cluster version is " + version); // Create a new HBase table.
    TableSchema testTableSchema = new TableSchema();
    testTableSchema.name = hbaseTableName;
    testTableSchema.columns.Add(new ColumnSchema() { name = "d" });
    testTableSchema.columns.Add(new ColumnSchema() { name = "f" });
    hbaseClient.CreateTable(testTableSchema); // Insert data into the HBase table.
    string testKey = "content";
    string testValue = "the force is strong in this column";
    CellSet cellSet = new CellSet();
    CellSet.Row cellSetRow = new CellSet.Row { key = Encoding.UTF8.GetBytes(testKey) };
    cellSet.rows.Add(cellSetRow); Cell value = new Cell { column = Encoding.UTF8.GetBytes("d:starwars"), data = Encoding.UTF8.GetBytes(testValue) };
    cellSetRow.values.Add(value);
    hbaseClient.StoreCells(hbaseTableName, cellSet); // Retrieve a cell by its key.
    cellSet = hbaseClient.GetCells(hbaseTableName, testKey);
    Console.WriteLine("The data with the key '" + testKey + "' is: " + Encoding.UTF8.GetString(cellSet.rows[0].values[0].data));
    // with the previous insert, it should yield: "the force is strong in this column" //Scan over rows in a table. Assume the table has integer keys and you want data between keys 25 and 35.
    Scanner scanSettings = new Scanner()
    {
    batch = 10,
    startRow = BitConverter.GetBytes(25),
    endRow = BitConverter.GetBytes(35)
    }; ScannerInformation scannerInfo = hbaseClient.CreateScanner(hbaseTableName, scanSettings);
    CellSet next = null;
    Console.WriteLine("Scan results"); while ((next = hbaseClient.ScannerGetNext(scannerInfo)) != null)
    {
    foreach (CellSet.Row row in next.rows)
    {
    Console.WriteLine(row.key + " : " + Encoding.UTF8.GetString(row.values[0].data));
    }
    } Console.WriteLine("Press ENTER to continue ...");
    Console.ReadLine();
    }
  6. Set the first three variables in the Main function.

  7. Press F5 to run the application.

What's Next?

In this tutorial, you have learned how to provision an HBase cluster, how to create tables, and and view the data in those tables from the HBase shell. You also learned how use Hive to query the data in HBase tables and how to use the HBase C# APIs to create an HBase table and retrieve data from the table.

To learn more, see:

微软Azure云平台Hbase 的使用的更多相关文章

  1. 微软Azure云主机及blob存储的网络性能测试

    http://www.cnblogs.com/sennly/p/4137024.html 微软Azure云主机及blob存储的网络性能测试 1. 测试目的 本次测试的目的在于对微软Azure的云主机. ...

  2. 【Microsoft Azure 的1024种玩法】三.基于Azure云平台构建Discuz论坛

    [简介] Discuz!是一套通用社区论坛软件系统,用户在不需要任何编程的基础上,通过简单的设置和安装,在互联网上搭建起具备完善功能.很强负载能力和可高度定制的论坛服务. [前期文章] [操作步骤] ...

  3. 【Microsoft Azure 的1024种玩法】二.基于Azure云平台的安全攻防靶场系统构建

    简介 本篇文章将基于在Microsoft Azure云平台上使用Pikachu去构建安全攻防靶场,Pikachu使用世界上最好的语言PHP进行开发,数据库使用的是mysql,因此运行Pikachu需要 ...

  4. 多云时代,海外微软Azure云与国内阿里云专线打通性能测试

    本文地址:http://www.cnblogs.com/taosha/p/6528730.html 在云计算的大时代,大型客户都有业务全球拓展的需求,考虑到成本,时间因素,一般都是选择云计算,现在云计 ...

  5. Azure 云平台用 SQOOP 将 SQL server 2012 数据表导入 HIVE / HBASE

    My name is Farooq and I am with HDinsight support team here at Microsoft. In this blog I will try to ...

  6. 微软Azure云主机测试报告

    http://www.cnblogs.com/sennly/p/4135658.html 1. 测试目的 本次测试的目的在于对微软云主机做性能测试,评估其是否能够满足我们业务的需求. 2. 测试项目 ...

  7. 微软Azure开始支持Docker技术

    前一段时间还在与微软的技术人员讨论媒体转换服务的效率问题,如果应用 Docker将会有质的提高,没想到国外的Azure已经开始支持了,相信国内Azure支持也不远了.微软正在努力确保Azure成为开发 ...

  8. 用手机应用追踪城市噪声污染——微软Azure助力解决城市问题

    噪声无孔不入的城市地带(图片来自于网络) 2014年4月19日发行的<经济学人>杂志预言,到2030年,中国人口的70%(约10亿人)会在城市中居住.中国城镇化的高速发展一方面大大提高了 ...

  9. 微软云平台媒体服务实践系列 1- 使用静态封装为iOS, Android 设备实现点播(VoD)方案

    微软的云平台媒体服务为流媒体服务提供了多种选择,在使用流媒体服务为企业做流媒体方案时,首先需要确认要流媒体接收目标,如针对广大iOS, Android移动设备,由于它们都支持HLS 格式的流媒体,基于 ...

随机推荐

  1. Duilib自定义控件响应指定命令(转载)

    转载:http://blog.csdn.net/panxianzhan/article/details/50772893 duilib在UIManager.h里的EVENTTYPE_UI枚举里定义了很 ...

  2. 不同分辨率的LCM进行兼容

    1.读取不同LCM ID的时序不同,如何处理? 2.开机后android会自动resize图片资源的大小,但开机logo无法自行resize,如何操作?保证识别到不同分辨率的LCM后开机logo可以正 ...

  3. .Net鼠标随动窗口

    就像QQ宠物或者迅雷悬浮窗口一样,鼠标点下去窗体跟着鼠标动 主要是两个时间的加载 MouseDown和MouseMove事件 MouseDown事件: private int _StartX ;//鼠 ...

  4. ERROR 1044 (42000) ERROR 1142 (42000): SELECT command denied to user ''@'localhost' for table 'user'

    ERROR: Access denied for user 'root'@'localhost' (using password: NO)    发现:    mysql -u root@localh ...

  5. Metasploit基础命令

    msf > show exploits 列Metasploip的所有可用的渗透测试框架.在MSF终端中可以针对渗透测试中发现的安全漏洞来实施相应的渗透攻击. msf > show auxi ...

  6. JS学习笔记(二) 数据类型

    参考资料: 1. http://www.w3school.com.cn/js/js_datatypes.asp 2. http://blog.sina.com.cn/s/blog_85c1dc1001 ...

  7. C#形参,实参,值传递参数,引用传递参数,输出参数,参数数组的学习

    1)形参 形参顾名思义就是形式上的参数,不是实际的参数,它代替实际传入方法的值.在方法体代码中代表了值本身参与运算.形参定义于参数中,它不同于方法体内局部变量,因为是一个变量,在它的作用域内不允许存在 ...

  8. MVC 3 数据验证 Model Validation 详解

    在MVC 3中 数据验证,已经应用的非常普遍,我们在web form时代需要在View端通过js来验证每个需要验证的控件值,并且这种验证的可用性很低.但是来到了MVC 新时代,我们可以通过MVC提供的 ...

  9. 数据库分库分表(sharding)

    地址: http://blog.csdn.net/column/details/sharding.html

  10. java使用方法总结

    第一章 字符串 . 获取字符串的长度: length() . 判断字符串的前缀或后缀与已知字符串是否相同 前缀 startsWith(String s) .后缀 endsWith(String s) ...