1.了解 Kibana 最新的针对时序型数据的 Time Series Visual Builder，看这两篇博客就够了：

http://t.cn/R9lbflp 和 http://t.cn/R9lbq3W

2.基于 Kibana 的开源的分析方案，实时掌握加密货币(如：比特币)的最新行情：

http://t.cn/R9lGuZi

3.Github 工程师团队分享的如何实现项目标签的智能推荐：

http://t.cn/R9lqS2A

 

编辑：Medcl

归档：https://elasticsearch.cn/article/217

订阅：https://tinyletter.com/elastic-daily
  收起阅读 »

1.大规模Elasticsearch集群管理心得 http://t.cn/RSrUeoY

来自携程wood的干货分享，对于大规模100+ node的管理感兴趣的同学快来看看吧！

2.Elasticsearch前沿：ES 5.x改进详解与ES6展望 http://t.cn/R90xiRF

Elasticsearch 6.0都快来了，你不会还不知道 5.x 的特性吧？！快来看看 medcl 的分享吧

3.Elastic Stack 官方实例 http://t.cn/RyCRVU4

Elastic Stack的产品不会用？！来看看官方的例子吧，从es 到 X-Pack 应有尽有，快来学习吧！




编辑：rockybean

归档：https://elasticsearch.cn/article/216

订阅：https://tinyletter.com/elastic-daily 收起阅读 »

1. 基于Elasticsearch构建千亿流量日志搜索平台实战 http://t.cn/R9hVYoW

来自七牛云的大数据高级工程师使用ES的经验分享




2. Autocomplete Using Elasticsearch http://t.cn/R9Sjr07

怎么实现谷歌、百度搜索的自动补全，这里利用es提供了一种可能的方法




3. 谈谈ES的Recovery http://t.cn/R9SsrK2

关于ES集群的快速恢复，社区里的这篇文章值得一看。




编辑：bsll

归档：https://elasticsearch.cn/article/215

订阅：https://tinyletter.com/elastic-daily 收起阅读 »

installDir=$1

curl -L -O https://artifacts.elastic.co/d ... ar.gz



mkdir $installDir

echo "安装的位置$installDir"

tar -xvf elasticsearch-5.2.2.tar.gz -C $installDir



mv $installDir/elasticsearch-5.2.2 $installDir/elasticsearch

echo "添加es用户 esuser 组 esuser"

groupadd esuser

useradd esuser -g esuser -p esuser



echo "输入es用户密码"

passwd esuser



cd $installDir

chown -R esuser:esuser $installDir/elasticsearch

###sh +x $installDir/elasticsearch/bin/elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name



function install_mvn(){

wget http://repos.fedorapeople.org/ ... .repo -O /etc/yum.repos.d/epel-apache-maven.repo

yum -y install apache-maven



}

function getik(){

ver="v5.2.2"

yum -y install git

git clone https://github.com/medcl/elasticsearch-analysis-ik

cd elasticsearch-analysis-ik

git checkout tags/$ver

mvn clean

mvn compile

mvn package

cd -



}

install_mvn

version=5.2.2

getik

mkdir $installDir/elasticsearch/plugins/ik

unzip elasticsearch-analysis-ik/target/releases/elasticsearch-analysis-ik-$version.zip -d $installDir/elasticsearch/plugins/ik

1：需要传入安装路径这个参数
2：默认是安装5.2.2，可以在脚本中修改
注意：
因为不能上传.sh结尾的，我把后缀名改为.doc 使用的时候再改回.sh即可
 
有不完善的地方，还望大家指正(:
 

installDir=$1

curl -L -O https://artifacts.elastic.co/d ... ar.gz



mkdir $installDir

echo "安装的位置$installDir"

tar -xvf elasticsearch-5.2.2.tar.gz -C $installDir



mv $installDir/elasticsearch-5.2.2 $installDir/elasticsearch

echo "添加es用户 esuser 组 esuser"

groupadd esuser

useradd esuser -g esuser -p esuser



echo "输入es用户密码"

passwd esuser



cd $installDir

chown -R esuser:esuser $installDir/elasticsearch

###sh +x $installDir/elasticsearch/bin/elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name



function install_mvn(){

wget http://repos.fedorapeople.org/ ... .repo -O /etc/yum.repos.d/epel-apache-maven.repo

yum -y install apache-maven



}

function getik(){

ver="v5.2.2"

yum -y install git

git clone https://github.com/medcl/elasticsearch-analysis-ik

cd elasticsearch-analysis-ik

git checkout tags/$ver

mvn clean

mvn compile

mvn package

cd -



}

install_mvn

version=5.2.2

getik

mkdir $installDir/elasticsearch/plugins/ik

unzip elasticsearch-analysis-ik/target/releases/elasticsearch-analysis-ik-$version.zip -d $installDir/elasticsearch/plugins/ik

收起阅读 »

1. 23 Useful Elasticsearch Example Queries  http://t.cn/R9Sjxf5

ES作为全文检索引擎，最常用的查询都在这里了




2. All About Analyzers

Part 1: http://t.cn/R9SFtYd

Part 2: http://t.cn/R9SFxPZ

如何利用ES提供的tokenizer filter以及正则等组件定制自己的analyzer，这篇文章给出了方向。




3. Elasticsearch Security: Authentication, Encryption, and Backup http://t.cn/R9SQJlK

安全问题永远不会过时，用这篇文章检查下你的ES集群是否有安全隐患




编辑：bsll

归档：https://elasticsearch.cn/article/213

订阅：https://tinyletter.com/elastic-daily 收起阅读 »

岗位职责（工作内容）

• 负责数据分析产品全栈式开发；
• 指导开发人员工作，提升团队整体的技术能力；
• 学习研究业界前沿技术，并迅速转化为项目生产力。

任职资格、技能和经验

• 全栈开发能力，熟悉JS/CSS/Html，Node.js等；
• 前端开发能力优秀者优先；
• 具有数据分析类产品设计或研发经验技术优先；
• 注重代码质量，能高效编写优雅的代码；
• 思路清晰，善于思考，能独立分析和解决问题，责任心强，具备良好的团队合作精神和承受压力的能力；
• 具备广泛的技术视野和很强的技术前瞻性。

 
工作地点北京、杭州，薪资待遇优厚，欢迎各位大神！
联系方式：wending.ywd@alibaba-inc.com 收起阅读 »

测试集合：模拟一亿条

0," nnrIuS","raet","lnsr","inu ","saia",83.405273,73.302012,3991,24,"N"," usA","airport","rra i"

1,"omlritp","aaVe","y Mu","AaVV","NMc ",15.459643,-20.826241,2627,54,"a","eemo","airport","MaArp"

2,"kyaneMr","iasm","raAA"," tnt","inls",16.606066,38.663728,2761,53,"o","arIi","airport","uiron"

BKD Tree 
https://www.elastic.co/blog/lucene-points-6.0
Block k-d trees are a simple yet powerful data structure. At index time, they are built by recursively partitioning the full space of N-dimensional points to be indexed into smaller and smaller rectangular cells, splitting equally along the widest ranging dimension at each step of the recursion. However, unlike an ordinary k-d tree, a block k-d tree stops recursing once there are fewer than a pre-specified (1024 in our case, by default) number of points in the cell.

At that point, all points within that cell are written into one leaf block on disk and the starting file-pointer for that block is saved into an in-heap binary tree structure. In the 1D case, this is simply a full sort of all values, divided into adjacent leaf blocks. There are k-d tree variants that can support removing values, and rebalancing, but Lucene does not need these operations because of its write-once per-segment design.
 
At search time, the same recursion takes place, testing at each level whether the requested query shape intersects the left or right sub-tree of each dimensional split, and recursing if so. In the 1D case, the query shape is simply a numeric range whereas in the 2D and 3D cases, it is a geo-spatial shape (circle, ring, rectangle, polygon, cube, etc.).

测试集合：模拟一亿条

0," nnrIuS","raet","lnsr","inu ","saia",83.405273,73.302012,3991,24,"N"," usA","airport","rra i"

1,"omlritp","aaVe","y Mu","AaVV","NMc ",15.459643,-20.826241,2627,54,"a","eemo","airport","MaArp"

2,"kyaneMr","iasm","raAA"," tnt","inls",16.606066,38.663728,2761,53,"o","arIi","airport","uiron"

1. General Multidimensional Space Points
   Search for points with exact given values. 
  Search for points which has one of the value from a given set of values. 
Search for points within a given range. 
Get the number of points which has exact point.
Get the number of points within a given range. (Ranges are multidimensional ranges. In 3D, they are boxes.)
Divide points into range-buckets and get the count in each buckets. (Range bucket is a range which has a label in it)
 
2. Locations on the planet surface. (Latitude, Longitude)
  Find closest set of airports to a given town.  
  Find the set of airports within a given radius from a particular town.
  Find the set of airports inside a country. (Country can be given as a polygon) 
  Find the set of airports within a given range of Latitudes and Longitudes. It is a Latitude, Longitude box query. (For a examples: Airports closer to the equatorial) 
  Find the set of airports closer to a given path. (Path can be something like a road. Find the airports which are less than 50km away from a given highway)
  Count the airports in each country by giving country maps as polygons.
 
search  result:
 
Loading Data is finished ----------------------------------------------------------------------
建索引花费时间：982ms
LatLon - Box Query Example------------------------------------------------------------------------------
search_LatLon_Box 花费时间：69ms

LatLon - K Nearest------------------------------------------------------------------------------
search_LatLon_Nearest 花费时间：108ms

DoublePoint 1D Point Exact------------------------------------------------------------------------------
search_Double_1D_Exact 花费时间：10ms

DoublePoint 1D - Range------------------------------------------------------------------------------
search_Double_1D_range 花费时间：8ms

DoublePoint 1D - Range Buckets -----------------------------------------------------------------------------
search_Double_1D_range_bucket 花费时间：58ms

DoublePoint multi dimensional - Range------------------------------------------------------------------------------
search_Double_MiltiDimensional_Range 花费时间：1ms
 
 
  收起阅读 »

1. X-Pack Alternatives http://t.cn/RaFzzv1

如果你看重了 elastic 付费套件 X-Pack 中的某个功能但又囊中羞涩，不妨来看看社区的其他选择方案。当然，还是推荐你去买 X-Pack ，官方出品，质有保障！

2.Elasticsearch as a Graph Database  http://t.cn/R9Xgj2X


听说过图数据库吧？你知道 es 也可以在这个领域发挥能力吗？快来看看吧！请自备梯子哦！

3.Scaling Elasticsearch  http://t.cn/R9Xev3r

听说你的es集群频繁GC，压力巨大，要扩容了？来看看这篇文章，科学扩容有保障！请自备梯子哦！


招聘：

阿里云近期会推出ES云产品，正在组建ES专家小组，工作地点北京、杭州，薪资待遇优厚。详情请看如下链接：https://elasticsearch.cn/article/209





编辑：rockybean

归档：https://elasticsearch.cn/article/210

订阅：https://tinyletter.com/elastic-daily 收起阅读 »

岗位描述：
1. 负责阿里云上搜索云产品的设计和研发工作，确保项目质量和进度 
2. 能深入理解产品和业务，推动技术不断升级，解决客户和平台问题。
岗位要求：
1. 编程基本功扎实，熟悉常用数据结构和算法，擅长Java编程语言，熟悉JVM机制，熟悉shell、python等脚本语言； 
2. 学习能力较强，有较好的逻辑思维能力，较强的抽象、概括和总结能力，有较好的沟通交流能力，善于主动思考，对技术有强烈激情； 
3. 熟悉ElasticSearch/Lucene开源系统
4. 熟悉分布式系统，例如hadoop、spark、flink，有云计算相关开发经验者优先 
5. 具有敏捷开发经验者优先，具有完整产品生命周期开发者优先
 
阿里云近期会推出ES云产品，正在组建ES专家小组，工作地点北京、杭州，薪资待遇优厚，简历请发送至ruijie.guo@alibaba-inc.com 收起阅读 »

日常使用中的一些经验，给使用ES的筒子们一些建议，如有错误，请多多包含..
 
































  收起阅读 »

1. 安全播报：超过5000个kibana实例裸奔在互联网 http://t.cn/R9JLxE9
你的kibana也在裸奔吗？戳这里
 
 2. string类型已死，字符串永生 http://t.cn/R9xxGwq
还在疑惑ES5为什么移除了string类型？这里有你想要的答案。

3. 机器学习与日志分析 http://t.cn/R9xxJtU
不要被潮流淘汰：人工分析日志是徒劳的，机器学习是日志分析的趋势，玩转日志分析和机器学习。

4. 另类玩法：用Elasticsearch和Grafana分析你的GitHub项目 http://t.cn/R9xXkZE
想快速直观炫酷的了解自己的github project，这篇文章教你新姿势。

编辑：金桥

归档：https://elasticsearch.cn/article/207
订阅：https://tinyletter.com/elastic-daily 收起阅读 »

消息来自：https://medium.com/%40SergiuSe ... 4af48
因为这个网址不存在，所以搬过来大家一起看看，请自查自家服务器是不是快乐的在裸奔，嘿，要管管了啊。
试试：
https://www.zoomeye.org/search ... Dhost 
https://www.shodan.io/search?query=kibana​ 
 

Over 5,000 Kibana instances exposed on the internet

I’m not a big fan of writing articles so I’ll keep it short… I was using Shodan.io recently for research purposes and while searching for different devices I came across 5,591 Kibana instances exposed over the internet. A significant number of those instances didn’t use any authentication mechanisms and several had +100 million log events recorded.

The query syntax that I used was the following: kibana port:”5601".


 
Risk: Kibana is deployed alone or together with Elasticsearch and Logstash (the ELK Stack) for log management purposes and it gained notoriety in the last couple of years as an open source alternative to more expensive commercial solutions. Log management solutions usually contain sensitive info and should not be exposed over the internet… (people who are familiar with information security know what I’m talking about).

Solution: For all the entities affected please refer to the following link and enable authentication on your Kibana implementations: https://www.elastic.co/guide/e ... .html


 
去年的大规模勒索事件，大家应该还记得吧，什么，ES你也裸奔着，你。。。
  收起阅读 »

1. ELK 与 Raspberry Pi 的另类极客玩法  http://t.cn/R9MLk7E
只要你有一个树莓派， 就可以轻松打造一个跑在“云”上的便携式 ELK 集群。

2. Elasticsearch 安全 Search Guard 落地实践  http://t.cn/R9ZeqNp
Search Guard 是一款 Elasticsearch 比较通用且方便的认证插件，这篇文章主要讲解了如何快速接入 Search Guard 插件。

3. Docker Logging with the ELK Stack
Part 1  http://t.cn/R9MUnJS 
Part 2  http://t.cn/R9M4UNz
关于 Docker 日志的采集应该是 ELK 应用最广泛的一面，具体的一些细节可以参考上述文档。该文档共有两部分。

编辑：江水

归档：https://elasticsearch.cn/article/205 
订阅：https://tinyletter.com/elastic-daily 收起阅读 »

1. Elasticsearch 6.0 将严格校验 Content-Type http://t.cn/R9VmPqx
 
大家知道 Elasticsearch 是 HTTP+Restfu 风格的，在 5.x 及以前的版本，Content-Type 一直是松散不校验的，从而存在跨站脚本攻击的可能，所以从 6.0 开始，所有带请求体的 HTTP 请求都需要带上正确的 Content-type 才能正常执行，同时也意味着你的客户端是不是需要更新或者升级了，另外 6.0 马上就要发布了哦。

2. Elasticsearch: How to avoid index throttling, deep dive in segments merging http://t.cn/R9V1sZH

Segment 的合并会严重影响 Elasticsearch 的性能，但你知道 Elasticsearch 什么时候会进行合并么？这篇文章从源码层面比较详细的介绍了 Elasticsearch 内部的 Segment 合并策略，感兴趣的同学可以仔细读一下。

3. Making your search not suck with Elasticsearch http://t.cn/R9Vg1SO

系列文章，主要介绍文本分析原理以及如何优化 Elasticsearch 的相关性评分，完善搜索结果。

4.极客邦科技发布站内搜索（InfoQ） http://t.cn/R9Vru3R

Powered by Elasticsearch！看起来不错哦，虽然目前功能还比较简单。
如果您有基于 Elasticsearch 实现的酷站，也欢迎投稿哦。
 
今天是八一建军节，解放军同志们辛苦了！


编辑：Medcl
归档：https://elasticsearch.cn/article/203​
订阅：https://tinyletter.com/elastic-daily 收起阅读 »

1. Logstash Persistent Queue http://t.cn/R9ctBCQ
Logstash 5.x 新加入了持久化队列功能，想要了解的同学不妨看看官网的这篇介绍哦！

2. 在Elasticsearch中应用机器学习排序LTR http://t.cn/RX2AVnS
相信不少同学在开发中遇到过修改排序结果的需求，常见的操作是使用function_score 来自定义排序分值，但要做到个性化搜索的话，往往离不开数据挖掘、机器学习的算法，那么如何整合这些算法到Elasticsearch中呢？该文提供了一个思路，推荐阅读，开阔视野！

3.用ElasticSearch搭建自己的搜索和分析引擎  http://t.cn/R9can85
来看下腾讯WeTest团队是如何调研和测试 Elasticsearch的，文章中提到的论坛统计分析功能是一个常见的需求，推荐大家阅读并实践！

4.X-Pack Machine Learning Online Training http://t.cn/R9c6vnt
价值 $400 的 elastic 机器学习在线教程免费啦！免费啦！免费啦！还不赶紧去注册！

编辑：rockybean
归档：https://elasticsearch.cn/article/202
订阅：https://tinyletter.com/elastic-daily 收起阅读 »