无论才能、知识多么卓著,如果缺乏热情,则无异纸上画饼充饥,无补于事。

使用logstash-input-jdbc如何将数据导入至同一索引下

Logstash | 作者 fengzxia | 发布于2018年03月21日 | 阅读数:4800

小白一个,今天在学习使用logstash-input-jdbc同步mysql数据时,我想要将两个查询的数据分别放置在同一index下的不同type中,写法如下:
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/blog?zeroDateTimeBehavior=convertToNull&useSSL=true"
jdbc_user => "root"
jdbc_password => "root"
jdbc_driver_library => "D:\dev\maven\repository\mysql\mysql-connector-java\5.1.38\mysql-connector-java-5.1.38.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
statement => "select * from article;"
schedule => "* * * * *"
type => "article"
}
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/blog?zeroDateTimeBehavior=convertToNull&useSSL=true"
jdbc_user => "root"
jdbc_password => "root"
jdbc_driver_library => "D:\dev\maven\repository\mysql\mysql-connector-java\5.1.38\mysql-connector-java-5.1.38.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
statement => "select * from icon;"
schedule => "* * * * *"
type => "icon"
}
}
output {
stdout {
codec => json_lines
}
if[type] == "article"{
elasticsearch {
hosts => "localhost:9200"
index => "contacts4"
document_type => "article"
document_id => "%{id}"
}
}
if[type] == "icon"{
elasticsearch {
hosts => "localhost:9200"
index => "contacts4"
document_type => "icon"
document_id => "%{id}"
}
}
}
它会给出警告,信息如下
][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"4", :_index=>"contacts4", :_type=>"article", :_routing=>nil}, #<LogStash::Event:0x611194af>], :response=>{"index"=>{"_index"=>"contacts4", "_type"=>"article", "_id"=>"4", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [contacts4] as the final mapping would have more than 1 type: [icon, article]"}}}}
[2018-03-21T16:05:52,840][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"8", :_index=>"contacts4", :_type=>"article", :_routing=>nil}, #<LogStash::Event:0x57f48d93>], :response=>{"index"=>{"_index"=>"contacts4", "_type"=>"article", "_id"=>"8", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [contacts4] as the final mapping would have more than 1 type: [icon, article]"}}}}
[2018-03-21T16:05:52,841][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"13", :_index=>"contacts4", :_type=>"article", :_routing=>nil}, #<LogStash::Event:0x282b7e44>], :response=>{"index"=>{"_index"=>"contacts4", "_type"=>"article", "_id"=>"13", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [contacts4] as the final mapping would have more than 1 type: [icon, article]"}}}}
不能把这两个同时插入至同一index中的不同type,当我把他们分别插入至两个不同的索引时却是可以的
已邀请:

laoyang360 - 《一本书讲透Elasticsearch》作者,Elastic认证工程师 [死磕Elasitcsearch]知识星球地址:http://t.cn/RmwM3N9;微信公众号:铭毅天下; 博客:https://elastic.blog.csdn.net

赞同来自: fengzxia

确认下,如果是es新版本,es6.x的新版本不支持一个索引多个type了

要回复问题请先登录注册