1. (自备梯子)推荐三个数据库相关(MySQL,MongoDB,Elasticsearch)的课程。
http://t.cn/RjPvlq1
2. 将 ELASTICSEARCH 写入速度优化到极限
http://t.cn/RWs8yvS
3. 零点之战！探访阿里巴巴8大技术专家，提前揭秘2017双11关键技术。
http://t.cn/RjPPzGc
4. 只等你来 | Elastic Meetup 广州交流会
https://elasticsearch.cn/article/364

编辑：至尊宝
归档：https://elasticsearch.cn/article/371
订阅：https://tinyletter.com/elastic-daily

[原文链接](https://segmentfault.com/a/1190000011966008)

距离上一篇 [esrally 教程](https://segmentfault.com/a/1190000011174694)过去快2个月了，这期间不停有同学来询问使用中遇到的问题，尤其由于其测试数据存储在国外 aws 上，导致下载极慢。为了让大家快速上手使用 esrally，我 build 了一个可用的 docker 镜像，然后将 13GB 的测试数据拉取到国内的存储上，通过百度网盘的方式分享给大家。大家只要按照下面简单的几步操作就可以顺畅地使用 esrally 来进行相关测试了。

操作步骤

废话不多说，先上菜！

1. 拉取镜像
   `<br /> docker pull rockybean/esrally<br />
   
2. 下载数据文件 链接:http://pan.baidu.com/s/1eSrjZgA 密码:aagl
   
3. 进入下载后的文件夹 rally_track,执行如下命令开始测试
   <br /> docker run -it -v $(PWD):/root/track rockybean/esrally esrally race --track-path=/root/track/logging --offline --pipeline=benchmark-only --target-hosts=192.168.1.105:9200<br />
   
   打完收工！
   
   

   几点说明
   

   
   

   数据文件介绍
   

   esrally 自带的测试数据即为 rally_track 文件夹中的内容，主要包括:
   
   

   • Geonames(geonames): for evaluating the performance of structured data.
     
   • Geopoint(geopoint): for evaluating the performance of geo queries.
     
   • Percolator(percolator): for evaluating the performance of percolation queries.
     
   • PMC(pmc): for evaluating the performance of full text search.
     
   • NYC taxis(nyc_taxis): for evaluating the performance for highly structured data.
     
   • Nested(nested): for evaluating the performance for nested documents.
     
   • Logging(logging): for evaluating the performance of (Web) server logs.
     
   • noaa(noaa): for evaluating the performance of range fields.
     
     可以根据自己的需要下载对应的测试数据，不必下载全部，保证对应文件夹下载完全即可。
     
     
     

     命令解释
     

     
     

     docker 相关
     

     docker run -it rockybean/esrally esrally 为执行的 esrally 命令，-v $(PWD):/root/track是将 rally_docker 文件夹映射到 docker 容器中，$(PWD)是获取当前目录的意思，所以在此前要 cd 到 rally_docker 目录，当然你写全路径也是没有问题的。
     
     esrally 的 docker 镜像比较简单，可以参看 [ github 项目介绍][1]。
     
     

     esrally 相关
     

     该镜像是通过自定义 track 的方式来加载数据，所以命令行中用到 --track=/root/track/logging 的命令行参数。注意这里的 /root/track 即上面我们绑定到容器的目录，更换 logging 为其他的数据集名称即可加载其他的测试数据。
     
     该容器只支持测试第三方 es 集群，即 --pipeline=benchmark-only 模式。这应该也是最常见的压测需求了。
     
     
     愉快地去玩耍吧！
     
     
     
     
     [1]: https://github.com/rockybean/esrally-docker

elasticsearch的dsl开发工具sense被google下架了，kibana console是很好的替代品。但是，我们的es集群前些日子因为应付安全检查改为https+basic auth方式（详细配置过程见本人博文：http://blog.csdn.net/jiashiwen ... 14374），kibana需要进行若干配置才能工作。另外老系统中还有elasticsearch2.3.3遗留，需要kibana4.5.1+sense。




一、elasticsearch5.5.2+kibana5.5.2

1.下载与elasticsearch版本号一致的kibana安装包，笔者目前开发环境5.5.2，对应kibana版本也为5.5.2（最新的5.6版本会报不兼容错误，不能运行）。




2.配置config/kibana.yml文件，主要配置项如下 
# The URL of the Elasticsearch instance to use for all your queries.
#elasticsearch.url: "http://localhost:9200"
elasticsearch.url: "https://192.168.1.1:9281/"
​
​
# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
​
​
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
elasticsearch.ssl.certificate: /home/develop/kibana-5.6.3-linux-x86_64/config/crts/eshttp.crt
elasticsearch.ssl.key: /home/develop/kibana-5.6.3-linux-x86_64/config/crts/eshttp.key
​
​
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full
elasticsearch.ssl.verificationMode: none各项配置看文件内说明，写的很清楚，这里就不翻译了，其中最重要的是这两样elasticsearch.ssl.certificate和elasticsearch.ssl.key，一定要与服务端保持一致。由于证书是自己生成的，校验项elasticsearch.ssl.verificationMode的值需要改为none。




启动kibana后，通过http://localhose:5601访问即可

1. sense为什么不能用了，看看ES官方怎么说？
   http://t.cn/RlB3B62
   
   
2. 使用allocation API快速定位分片分配问题
   http://t.cn/RlrzTsD
   
   
3. ES6.0有关防止硬盘被填满的改进
   http://t.cn/RlrU3Nr
   
   
4. 喜大普奔，ES社区支持Markdown编辑器了
   https://elasticsearch.cn/article/366
   
   
5. Elastic 收购网站搜索 SaaS 服务领导者 Swiftype
   http://t.cn/Rl3a4P2
   
   
6. 只等你来 | Elastic Meetup 广州交流会
   https://elasticsearch.cn/article/364
   
   
   • 编辑：bsll
     
   • 归档：https://elasticsearch.cn/article/368
     
   • 订阅：https://tinyletter.com/elastic-daily
     

[立即申请](https://www.elastic.co/about/c ... 923231)

Support Engineer - Mandarin Speaking

Location: Beijing, China

Department: Support

Responsibilities

• Ensuring customer issues are resolved within our committed service level agreements.
  
• Maintain strong relationships with our customers for the delivery of support.
  
• Have a mindset of continuous improvement, in terms of efficiency of support processes and customer satisfaction.
  
  

  Experience
  

• Demonstrable experience in of support in technology businesses
  
• Experience working across multi-cultural and geographically distributed teams
  
  

  Key Skills
  

  
  

• Strong verbal and written communication skills in both Mandarin and English.
  
• Customer orientated focus.
  
• Team player, ability to work in a fast pace environment with a positive and adaptable approach.
  
• Knowledge of databases or search technologies a plus.
  
• Demonstrated strong technical understanding of software products.
  
  

  Additional Information
  

  
  

• Competitive pay and benefits
  
• Stock options
  
• Catered lunches, snacks, and beverages in most offices
  
• An environment in which you can balance great work with a great life
  
• Passionate people building great products
  
• Employees with a wide variety of interests
  
• Distributed-first company with employees in over 30 countries, spread across 18 time zones, and speaking over 30 languages!
  
  [立即申请](https://www.elastic.co/about/c ... 923231)
  
  Elastic is an Equal Employment employer committed to the principles of equal employment opportunity and affirmative action for all applicants and employees. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status or any other basis protected by federal, state or local law, ordinance or regulation. Elastic also makes reasonable accommodations for disabled employees consistent with applicable law.
  
  

  About Elastic
  

  Elastic is the world's leading software provider for making structured and unstructured data usable in real time for search, logging, security, and analytics use cases. Founded in 2012 by the people behind the Elasticsearch, Kibana, Beats, and Logstash open source projects, Elastic's global community has more than 80,000 members across 45 countries, and since its initial release. Elastic's products have achieved more than 100 million cumulative downloads. Today thousands of organizations, including Cisco, eBay, Dell, Goldman Sachs, Groupon, HP, Microsoft, Netflix, The New York Times, Uber, Verizon, Yelp, and Wikipedia, use the Elastic Stack, X-Pack, and Elastic Cloud to power mission-critical systems that drive new revenue opportunities and massive cost savings. Elastic is backed by more than $104 million in funding from Benchmark Capital, Index Ventures, and NEA; has headquarters in Amsterdam, the Netherlands, and Mountain View, California; and has over 500 employees in more than 30 countries around the world.
  
  

  Our Philosophy
  

  
  We’re always on the search for amazing people, people who have deep passion for technology and are masters at their craft. We build highly sophisticated distributed systems and we don’t take our technology lightly. In Elasticsearch, you’ll have the opportunity to work in a vibrant young company next to some of the smartest and highly skilled technologists the industry has to offer. We’re looking for great team players, yet we also promote independence and ownership. We’re hackers… but of the good kind. The kind that innovates and creates cutting edge products that eventually translates to a lot of happy, smiling faces.
  
  

  LifeAtElastic
  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  

  

  
  
  
  [有兴趣加入我们么？点击申请](https://www.elastic.co/about/c ... 923231)
  

为了改善大家的创作体验，提高大家的写作和分享热情！?，经过两天的不懈奋斗，终于把 Markdown 编辑器搬上来了。
目前只支持文章的发布，可以通过切换编辑器来选择 Markdown 编辑模式。
希望不要再以编辑器作为理由发只有链接的文章了。
???????????

• 支持 Github 风格的 Markdown 格式
  
• 支持本站附件功能
  
• 支持 emoj 符号
  
• 支持自动的页面导航
  
• 以前的文章可再次编辑，切换 Markdown 模式然后修改保存
  
  

  如何使用？
  

  1. 点击【发起】，选择文章
     
  2. 切换绿色按钮，将编辑器切换到 Markdown，然后在文本框内输入 Markdown 格式的内容即可。
     
     
     在线 Markdown 脚本编辑预览工具：https://elasticsearch.cn/stati ... down/
     
     ----
     
     以下为样式测试参考，忽略其意义。
     
     
     

     
     
     ----------- 常用格式-----------------
     
     ```
     
     

     标题1
     

     标题2
     

     标题3
     

     标题4
     

     标题5
     

     标题6
     

     超大标题 //等于号写于文字下方
     ===
     标题 //同超大标题
     ---
     
     短代码
     注：长代码块，用三个: `
     
     This is the first level of quoting.
     
     

     This is nested blockquote.
     
     Back to the first level.
     
     
     

• Red
  
• Green
  
• Blue
  
  
• Red
  
• Green
  
• Blue
  
  
• Red
  
• Green
  
• Blue
  
  
  1. 这是第一个
     
  2. 这是第二个
     
  3. 这是第三个
     
     
• • *
    *
    ***
    
• • -
    ---
    
    [markdown-syntax](http://daringfireball.net/projects/markdown/syntax)
    
    [id]: http://example.com/ "Optional Title Here"
    This is [an example][id] reference-style link.
    
    内容
    内容
    内容
    内容
    
    
    
    
    
    <http://elastic.co/>;
    
    info@elastic.o
    
    
    四个空格
    一个tab
    
    
    ```
    ----------- 样式预览-----------------
    
    

    标题1
    

    标题2
    

    标题3
    

    标题4
    

    标题5
    

    标题6
    

    
    超大标题 //等于号写于文字下方
    ===
    标题 //同超大标题
    ---
    
    短代码
    
    
    This is the first level of quoting.
    
    

    This is nested blockquote.
    
    Back to the first level.
    
    
    

• Red
  
• Green
  
• Blue
  
  
• Red
  
• Green
  
• Blue
  
  
• Red
  
• Green
  
• Blue
  
  
  1. 这是第一个
     
  2. 这是第二个
     
  3. 这是第三个
     
     
• • *
    *
    ***
    
• • -
    ---
    
    [markdown-syntax](http://daringfireball.net/projects/markdown/syntax)
    
    [id]: http://example.com/ "Optional Title Here"
    This is [an example][id] reference-style link.
    
    内容
    内容
    内容
    内容
    
    
    
    
    
    <http://elastic.co/>;
    
    info@elastic.o
    
    
    四个空格
    一个tab
    
    
    ---------
    
    
    https://github.com/infinitbyte/gopa 的 README 内容
    
    ---------
    <img width="200" alt="What a Spider!" src="" rel="nofollow" target="_blank">https://raw.githubusercontent. ... gt%3B
    
    GOPA, A Spider Written in Go.
    
    [](https://travis-ci.org/infinitbyte/gopa)
    [](https://goreportcard.com/repor ... e/gopa)
    [](https://coveralls.io/github/in ... master)
    [](https://gitter.im/infinitbyte/ ... Dbadge)
    
    
    

    Goal
    

    
    

• Light weight, low footprint, memory requirement should < 100MB
  
• Easy to deploy, no runtime or dependency required
  
• Easy to use, no programming or scripts ability needed, out of box features
  
  
  

  Screenshoot
  

  
  <img width="800" alt="What a Spider! GOPA Spider!" src="" rel="nofollow" target="_blank">https://raw.githubusercontent. ... gt%3B
  
  
  ---
  
  
  

• How to use
  
• Setup
  
• Download Pre Built Package
  
• Compile The Package Manually
  
• Optional Config
  
• Start
  
• Stop
  
• Configuration
  
• UI
  
• API
  
• Contributing
  
• License
  
  
  
  

  How to use
  

  
  

  Setup
  

  
  First of all, get it, two opinions: download the pre-built package or compile it yourself.
  
  

  Download Pre Built Package
  

  
  Go to [Release](https://github.com/infinitbyte/gopa/releases) or [Snapshot](https://github.com/infinitbyte ... leases) page, download the right package for your platform.
  
  Note: Darwin is for Mac
  
  

  Compile The Package Manually
  

  
  

• Mac/Linux: Run make build to build the Gopa.
  
  
• Windows: Checkout this wiki page - [How to build GOPA on windows](https://github.com/infinitbyte ... indows).
  
  So far, we have:
  
  gopa, the main program, a single binary.
  
  config/, elasticsearch related scripts etc.
  
  gopa.yml, main configuration for gopa.
  
  
  
  

  Optional Config
  

  
  By default, Gopa works well except indexing, if you want to use elasticsearch as indexing, follow these steps:
  
  

• Create a index in elasticsearch with script config/gopa-index-mapping.sh
  
  
  

  Example

  
  

  curl -XPUT "http://localhost:9200/gopa-index" -H 'Content-Type: application/json' -d'
  
  {
  
  "mappings": {
  
  "doc": {
  
  "properties": {
  
  "host": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "snapshot": {
  
  "properties": {
  
  "bold": {
  
  "type": "text"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "content_type": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "file": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "h1": {
  
  "type": "text"
  
  },
  
  "h2": {
  
  "type": "text"
  
  },
  
  "h3": {
  
  "type": "text"
  
  },
  
  "h4": {
  
  "type": "text"
  
  },
  
  "hash": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "id": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "images": {
  
  "properties": {
  
  "external": {
  
  "properties": {
  
  "label": {
  
  "type": "text"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  }
  
  }
  
  },
  
  "internal": {
  
  "properties": {
  
  "label": {
  
  "type": "text"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  }
  
  }
  
  }
  
  }
  
  },
  
  "italic": {
  
  "type": "text"
  
  },
  
  "links": {
  
  "properties": {
  
  "external": {
  
  "properties": {
  
  "label": {
  
  "type": "text"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  }
  
  }
  
  },
  
  "internal": {
  
  "properties": {
  
  "label": {
  
  "type": "text"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  }
  
  }
  
  }
  
  }
  
  },
  
  "path": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "sim_hash": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "lang": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "size": {
  
  "type": "long"
  
  },
  
  "text": {
  
  "type": "text"
  
  },
  
  "title": {
  
  "type": "text",
  
  "fields": {
  
  "keyword": {
  
  "type": "keyword"
  
  }
  
  }
  
  },
  
  "version": {
  
  "type": "long"
  
  }
  
  }
  
  },
  
  "task": {
  
  "properties": {
  
  "breadth": {
  
  "type": "long"
  
  },
  
  "created": {
  
  "type": "date"
  
  },
  
  "depth": {
  
  "type": "long"
  
  },
  
  "id": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "original_url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "reference_url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "schema": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  },
  
  "status": {
  
  "type": "integer"
  
  },
  
  "updated": {
  
  "type": "date"
  
  },
  
  "url": {
  
  "type": "keyword",
  
  "ignore_above": 256
  
  }
  
  }
  
  }
  
  }
  
  }
  
  }
  
  }'

  
  
  
  
  _Note: Elasticsearch version should > v5.0_
  
  
• Enable index module in gopa.yml, update the elasticsearch's setting:
  ```
  
• module: index
  enabled: true
  ui:
  enabled: true
  elasticsearch:
  endpoint: http://dev:9200
  index_prefix: gopa-
  username: elastic
  password: changeme
  ```
  
  
  
  

  Start
  

  
  Gopa doesn't require any dependencies, simply run ./gopa to start the program.
  
  Gopa can be run as daemon(Note: Only available on Linux and Mac):
  

  
  

  Example

  
  

  
  
  ➜  gopa git:(master) ✗ ./bin/gopa --daemon
  
  ________ ________ __________  _____
  
  /  _____/ \_____  \\______   \/  _  \
  
  /   \  ___  /   |   \|     ___/  /_\  \
  
  \    \_\  \/    |    \    |  /    |    \
  
  \______  /\_______  /____|  \____|__  /
  
  \/         \/                \/
  
  [gopa] 0.10.0_SNAPSHOT
  
  ///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
  
  
  
  [10-21 16:01:09] [INF] [instance.go:23] workspace: data/gopa/nodes/0
  
  [gopa] started.

  
  
  
  
  Also run `./gopa -h` to get the full list of command line options.
  
  
  

  Example

  
  

  
  
  ➜  gopa git:(master) ✗ ./bin/gopa -h
  
  ________ ________ __________  _____
  
  /  _____/ \_____  \\______   \/  _  \
  
  /   \  ___  /   |   \|     ___/  /_\  \
  
  \    \_\  \/    |    \    |  /    |    \
  
  \______  /\_______  /____|  \____|__  /
  
  \/         \/                \/
  
  [gopa] 0.10.0_SNAPSHOT
  
  ///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
  
  
  
  Usage of ./bin/gopa:
  
  -config string
  
  the location of config file (default "gopa.yml")
  
  -cpuprofile string
  
  write cpu profile to this file
  
  -daemon
  
  run in background as daemon
  
  -debug
  
  run in debug mode, wi
  
  -log string
  
  the log level,options:trace,debug,info,warn,error (default "info")
  
  -log_path string
  
  the log path (default "log")
  
  -memprofile string
  
  write memory profile to this file
  
  -pidfile string
  
  pidfile path (only for daemon)
  
  -pprof string
  
  enable and setup pprof/expvar service, eg: localhost:6060 , the endpoint will be: http://localhost:6060/debug/pprof/ and http://localhost:6060/debug/vars</pre>
  
  

  
  
  
  ### Stop
  
  It's safety to press `ctrl+c` stop the current running Gopa, Gopa will handle the rest,saving the checkpoint,
  you may restore the job later,the world is still in your hand.
  
  If you are running `Gopa` as daemon, you may stop it like this:
  
  ```
  kill -QUIT `pgrep gopa`
  ```
  
  ## Configuration
  
  ## UI
  
  
• Search Console <a href="http://127.0.0.1:9001/" rel="nofollow" target="_blank">http://127.0.0.1:9001/</a>
  
• Admin Console <a href="http://127.0.0.1:9001/admin/" rel="nofollow" target="_blank">http://127.0.0.1:9001/admin/</a>
  
  

  API
  

  
  

• TBD
  
  
  

  Contributing
  

  
  You are sincerely and warmly welcomed to play with this project,
  from UI style to core features,
  or just a piece of document,
  welcome! let's make it better.
  
  
  License
  =======
  Released under the [Apache License, Version 2.0](https://github.com/infinitbyte ... ICENSE) .
  
  
  Also XSS Test
  
  alert('XSS test');