(一)hive导入数据
101 浏览量
2021-01-20
12:18:09
上传
评论
收藏 237KB PDF 举报
(一一)hive导入数据导入数据
创建数据库创建数据库+运行运行sql文件文件
1.create database XXX；
2.sql文件路径下运行hive -f XXX.sql
注：若出现错误line1:0 不识别，检查sql文件和编码格式（应该为utf-8 无bom）
UE修改文件格式
用用datax导入导入hive表表
元数据存于mysql
1.配置文件：mysql2hive.json
{
"job": {
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
//mysql选取的列
"column": ["id","username"],
"connection": [
{
//mysql连接参数
"jdbcUrl": ["jdbc:mysql://192.168.43.20:3306/test"],
"table": ["target_user"] }
],
"password": "111111",
"username": "root",
"where": ""
}
},
"writer": {
"name": "hdfswriter",
"parameter": {
"column": [
{"name":"id","type":"string"}
{"name":"username","type":"string"}
],
"compress": "gzip",
//修改hdfs地址
"defaultFS": "hdfs://192.168.43.20:8020",
"fieldDelimiter": ",",
"fileName": "target_user",
"fileType": "text",
//修改hdfs路径，就是hive表路径
"path": "/user/hive/warehouse/mysql2hive",
"writeMode": "append"
}
}
}
],
"setting": {
"speed": {
"channel": "1"
}
评论0
最新资源