Skip to content

Latest commit

 

History

History
117 lines (95 loc) · 2.99 KB

hbase.md

File metadata and controls

117 lines (95 loc) · 2.99 KB
Raw

HBase

  • distributed, column-oriented persistent multidimensional sorted map
  • storing column-family into memory/disc
  • disc = hdfs or filesystem
  • column family has 'veracity' - version of the record based on timestamp
  • Value = Table + RowKey + Family + Column + Timestamp
data is sparse - a lot of column has null values
fast retrieving data by 'key of the row' + 'column name'
contains from: (HBase HMaster) *---> (HBase Region Server)

SQL for Hbase - Phoenix SQL

Why HBase

hbase-why.png

HBase Architecture

hbase-architecture.jpg

HBase ACID

hbase-acid.png

logical view

hbase-record-logical-view.png

phisical view

hbase-record-phisical-view.png

logical to phisical view

hbase-logical-to-phisical-view.png

Table characteristics

hbase-table-characteristics.png

Column-Family

hbase-column-family.png

manage HBase

start/stop hbase

$HBASE_HOME/bin/start-hbase.sh
$HBASE_HOME/bin/stop-hbase.sh

interactive shell

cheat sheet

$HBASE_HOME/bin/hbase shell

path to jars from hbase classpath

hbase classpath

commands

  • list of the tables
list
  • create table
create table 'mytable1'
  • description of the table
descibe 'my_space:mytable1'
  • count records
count 'my_space:mytable1'
  • delete table
drop table 'mytable1'
disable table 'mytable1'
  • iterate through a table, iterate with range
scan 'my_space:mytable1'
scan 'my_space:mytable1', {STARTROW=>"00223cfd-8b50-979d29164e72:1220", STOPROW=>"00223cfd-8b50-979d29164e72:1520"}
  • save results into a file
echo " scan 'my_space:mytable1', {STARTROW=>"00223cfd-8b50-979d29164e72:1220", STOPROW=>"00223cfd-8b50-979d29164e72:1520"} " | hbase shell > out.txt
  • insert data
put 'mytable1', 'row0015', 'cf:MyColumnFamily2', 'my value 01'
  • read data
get 'mytable1', 'row0015'

Java

java app

java \
    -cp /opt/cloudera/parcels/SPARK2/lib/spark2/jars/*:`hbase classpath`:{{ deploy_dir }}/lib/ingest-pipeline-orchestrator-jar-with-dependencies.jar \
    -Djava.security.auth.login.config={{ deploy_dir }}/res/deploy.jaas \
    com.bmw.ad.ingest.pipeline.orchestrator.admin.TruncateSessionEntriesHelper \
    --hbase-zookeeper {{ hbase_zookeeper }} \
    --ingest-tracking-table-name {{ ingest_tracking_table }} \
    --file-meta-table-name {{ file_meta_table }} \
    --component-state-table-name {{ component_state_table }} \
    --session-id $1