之前同事建过一个索引,现在新增了一个聚合的需求,我就加了几个字段。
现在的问题是,之前的数据没有这些字段,我现在的想法是,从之前的索引中,一条一条的把数据取出来,然后根据数据的某个字段从Oracle中查询到对应的值用于新索引的字段赋值。
我查了下可以用Scroll来做,不过好像很慢,不知道大家有什么好办法。
附整个索引的mapping。
PUT /pahfpt_v1
{
"settings": {
"index.number_of_shards" : 5,
"number_of_replicas" : 1
},
"mappings": {
"_default_": {
"_all": {
"enabled": false
}
},
"pt_oper_log": {
"properties": {
"fun_name": {
"type": "string",
"index": "not_analyzed"
},
"id": {
"type": "long"
},
"mod_id": {
"type": "long"
},
"mod_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_data_ids": {
"type": "string",
"index": "not_analyzed"
},
"oper_desc": {
"type": "string",
"index": "not_analyzed",
"doc_values": false
},
"oper_id": {
"type": "long"
},
"oper_ip": {
"type": "string",
"index": "not_analyzed"
},
"oper_real_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_time": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
},
"oper_type": {
"type": "long"
},
"oper_user_name": {
"type": "string",
"index": "not_analyzed"
},
"sub_sys_id": {
"type": "long",
"index": "not_analyzed"
},
"sub_sys_name": {
"type": "string",
"index": "not_analyzed"
},
"police_department": {
"type": "string",
"index": "not_analyzed"
},
"police_station": {
"type": "string",
"index": "not_analyzed"
},
"police_office": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_ip": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_org": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
新增的字段有police_department、police_station、police_office、oper_camera_name、oper_camera_org等。
其中 police_department、police_station、police_office是根据之前的字段oper_id(用户的id,PT_USER_INFO这个表中有这个用户的org_id,然后查询到这个用户所在的公安局、分局、派出所;oper_camera_name、oper_camera_org是根据oper_data_ids去分别查询PT_CAMERA_INFO表和PT_ORG_INFO表得到的),也就是说现在我要在索引旧索引的时候,还要去查询数据库,为新增的字段赋值,这个就比较慢了,不知道大家有什么好的办法?请指点,谢谢!
现在的问题是,之前的数据没有这些字段,我现在的想法是,从之前的索引中,一条一条的把数据取出来,然后根据数据的某个字段从Oracle中查询到对应的值用于新索引的字段赋值。
我查了下可以用Scroll来做,不过好像很慢,不知道大家有什么好办法。
附整个索引的mapping。
PUT /pahfpt_v1
{
"settings": {
"index.number_of_shards" : 5,
"number_of_replicas" : 1
},
"mappings": {
"_default_": {
"_all": {
"enabled": false
}
},
"pt_oper_log": {
"properties": {
"fun_name": {
"type": "string",
"index": "not_analyzed"
},
"id": {
"type": "long"
},
"mod_id": {
"type": "long"
},
"mod_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_data_ids": {
"type": "string",
"index": "not_analyzed"
},
"oper_desc": {
"type": "string",
"index": "not_analyzed",
"doc_values": false
},
"oper_id": {
"type": "long"
},
"oper_ip": {
"type": "string",
"index": "not_analyzed"
},
"oper_real_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_time": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
},
"oper_type": {
"type": "long"
},
"oper_user_name": {
"type": "string",
"index": "not_analyzed"
},
"sub_sys_id": {
"type": "long",
"index": "not_analyzed"
},
"sub_sys_name": {
"type": "string",
"index": "not_analyzed"
},
"police_department": {
"type": "string",
"index": "not_analyzed"
},
"police_station": {
"type": "string",
"index": "not_analyzed"
},
"police_office": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_name": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_ip": {
"type": "string",
"index": "not_analyzed"
},
"oper_camera_org": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
新增的字段有police_department、police_station、police_office、oper_camera_name、oper_camera_org等。
其中 police_department、police_station、police_office是根据之前的字段oper_id(用户的id,PT_USER_INFO这个表中有这个用户的org_id,然后查询到这个用户所在的公安局、分局、派出所;oper_camera_name、oper_camera_org是根据oper_data_ids去分别查询PT_CAMERA_INFO表和PT_ORG_INFO表得到的),也就是说现在我要在索引旧索引的时候,还要去查询数据库,为新增的字段赋值,这个就比较慢了,不知道大家有什么好的办法?请指点,谢谢!
2 个回复
Jea - 一只猿
赞同来自:
那么你不必重建, 因为没有设置dynamic:false你可以直接增加字段并存入, 如果这么干的话, 建议先存入一条数据
like this
Jea - 一只猿
赞同来自: