[Sqoop1] Interact MySQL with HDFS/Hive/HBase

install sqoop1 on mac

brew install sqoop

#if you have set env profiles, uncomment profiles in conf/sqoop-env.sh

1.MySQL -> HDFS

1.1.import table

sqoop import --connect jdbc:mysql://localhost/test --direct --username root --P --table t1 --warehouse-dir /mysql/test --fields-terminated-by ','

1.2.import schema

sqoop import-all-tables --connect jdbc:mysql://localhost/test --direct --username root -P --warehouse-dir /mysql/test --fields-terminated-by ','

2.MySQL -> Hive

2.1.import definition

sqoop create-hive-table --connect jdbc:mysql://localhost/test --table t1 --username root --P --hive-database test

2.2.import table

sqoop import --connect jdbc:mysql://localhost/test --username root --P --table t1 --hive-import --hive-database test --hive-table t1 --fields-terminated-by ','

2.3.import schema

sqoop import-all-tables --connect jdbc:mysql://localhost/test --username root --P --hive-import --hive-database test --fields-terminated-by ','

3.MySQL -> HBase

3.1.definition

sqoop import --connect jdbc:mysql://localhost/test --username root --P --table t1

3.2.import table, need create table in hbase first

sqoop import --connect jdbc:mysql://localhost/test --username root --P --table t1 --hbase-bulkload --hbase-table test.t1 --column-family basic --fields-terminated-by ','

3.3.import table without creating table in hbase, but pay attention to hbase/sqoop version

sqoop import --connect jdbc:mysql://localhost/test --username root --P --table t1 --hbase-bulkload --hbase-create-table --hbase-table test.t1 --column-family basic --fields-terminated-by ','

4.HDFS/Hive/HBase -> MySQL

sqoop export --connect jdbc:mysql://localhost/test --username root --P --table t1 --export-dir /user/hive/warehouse/test.db/t1 --fields-terminated-by ','