ELK，萌萌哒

sense不能用了改用kibana吧

Elasticsearch • JiaShiwen 发表了文章 • 0 个评论 • 5334 次浏览 • 2017-11-11 12:34 • 来自相关话题

elasticsearch的dsl开发工具sense被google下架了，kibana console是很好的替代品。但是，我们的es集群前些日子因为应付安全检查改为https+basic auth方式（详细配置过程见本人博文：http://blog.csdn.net/jiashiwen ... 14374），kibana需要进行若干配置才能工作。另外老系统中还有elasticsearch2.3.3遗留，需要kibana4.5.1+sense。




一、elasticsearch5.5.2+kibana5.5.2

1.下载与elasticsearch版本号一致的kibana安装包，笔者目前开发环境5.5.2，对应kibana版本也为5.5.2（最新的5.6版本会报不兼容错误，不能运行）。




2.配置config/kibana.yml文件，主要配置项如下 
# The URL of the Elasticsearch instance to use for all your queries.
#elasticsearch.url: "http://localhost:9200"
elasticsearch.url: "https://192.168.1.1:9281/"
​
​
# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "user"
#elasticsearch.password: "pass"
elasticsearch.username: "admin"
elasticsearch.password: "admin"
​
​
# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files validate that your Elasticsearch backend uses the same key files.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key
elasticsearch.ssl.certificate: /home/develop/kibana-5.6.3-linux-x86_64/config/crts/eshttp.crt
elasticsearch.ssl.key: /home/develop/kibana-5.6.3-linux-x86_64/config/crts/eshttp.key
​
​
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full
elasticsearch.ssl.verificationMode: none各项配置看文件内说明，写的很清楚，这里就不翻译了，其中最重要的是这两样elasticsearch.ssl.certificate和elasticsearch.ssl.key，一定要与服务端保持一致。由于证书是自己生成的，校验项elasticsearch.ssl.verificationMode的值需要改为none。




启动kibana后，通过http://localhose:5601访问即可

社区日报 第97期 (2017-11-11)

社区日报 • bsll 发表了文章 • 0 个评论 • 1915 次浏览 • 2017-11-11 09:37 • 来自相关话题

  1. sense为什么不能用了，看看ES官方怎么说？
    http://t.cn/RlB3B62

  2. 使用allocation API快速定位分片分配问题
    http://t.cn/RlrzTsD

  3. ES6.0有关防止硬盘被填满的改进
    http://t.cn/RlrU3Nr

  4. 喜大普奔，ES社区支持Markdown编辑器了
    https://elasticsearch.cn/article/366

  5. Elastic 收购网站搜索 SaaS 服务领导者 Swiftype
    http://t.cn/Rl3a4P2

  6. 只等你来 | Elastic Meetup 广州交流会
    https://elasticsearch.cn/article/364

Elastic 招聘技术支持工程师，坐标北京

求职招聘 • medcl 发表了文章 • 6 个评论 • 5421 次浏览 • 2017-11-10 23:16 • 来自相关话题

DG5LrOuXUAEgq4Q.jpg




[立即申请](https://www.elastic.co/about/c ... 923231)

Support Engineer - Mandarin Speaking


Location: Beijing, China


Department: Support


Responsibilities

  • Ensuring customer issues are resolved within our committed service level agreements.
  • Maintain strong relationships with our customers for the delivery of support.
  • Have a mindset of continuous improvement, in terms of efficiency of support processes and customer satisfaction.

    Experience

  • Demonstrable experience in of support in technology businesses
  • Experience working across multi-cultural and geographically distributed teams

    Key Skills


  • Strong verbal and written communication skills in both Mandarin and English.
  • Customer orientated focus.
  • Team player, ability to work in a fast pace environment with a positive and adaptable approach.
  • Knowledge of databases or search technologies a plus.
  • Demonstrated strong technical understanding of software products.

    Additional Information


  • Competitive pay and benefits
  • Stock options
  • Catered lunches, snacks, and beverages in most offices
  • An environment in which you can balance great work with a great life
  • Passionate people building great products
  • Employees with a wide variety of interests
  • Distributed-first company with employees in over 30 countries, spread across 18 time zones, and speaking over 30 languages!

    [立即申请](https://www.elastic.co/about/c ... 923231)

    Elastic is an Equal Employment employer committed to the principles of equal employment opportunity and affirmative action for all applicants and employees. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status or any other basis protected by federal, state or local law, ordinance or regulation. Elastic also makes reasonable accommodations for disabled employees consistent with applicable law.

    About Elastic

    Elastic is the world's leading software provider for making structured and unstructured data usable in real time for search, logging, security, and analytics use cases. Founded in 2012 by the people behind the Elasticsearch, Kibana, Beats, and Logstash open source projects, Elastic's global community has more than 80,000 members across 45 countries, and since its initial release. Elastic's products have achieved more than 100 million cumulative downloads. Today thousands of organizations, including Cisco, eBay, Dell, Goldman Sachs, Groupon, HP, Microsoft, Netflix, The New York Times, Uber, Verizon, Yelp, and Wikipedia, use the Elastic Stack, X-Pack, and Elastic Cloud to power mission-critical systems that drive new revenue opportunities and massive cost savings. Elastic is backed by more than $104 million in funding from Benchmark Capital, Index Ventures, and NEA; has headquarters in Amsterdam, the Netherlands, and Mountain View, California; and has over 500 employees in more than 30 countries around the world.

    Our Philosophy


    We’re always on the search for amazing people, people who have deep passion for technology and are masters at their craft. We build highly sophisticated distributed systems and we don’t take our technology lightly. In Elasticsearch, you’ll have the opportunity to work in a vibrant young company next to some of the smartest and highly skilled technologists the industry has to offer. We’re looking for great team players, yet we also promote independence and ownership. We’re hackers… but of the good kind. The kind that innovates and creates cutting edge products that eventually translates to a lot of happy, smiling faces.

    LifeAtElastic



    DEFMWTGV0AAOxBT.jpg



    DFml2iQXcAAI4Dk.jpg



    DFR1IpGVoAAyYfH.jpg



    DHciIhpUMAAwk7W.jpg



    DI-WzDWXoAAx8Z0.jpg



    DKSTN1SVAAAoVgd.jpg



    DJtbgW6UEAAmKu_.jpg



    DIpqETTXoAAktBP.jpg



    DKv1Gu1UMAAwJal.jpg




    [有兴趣加入我们么？点击申请](https://www.elastic.co/about/c ... 923231)

elasticsearch5.6比5.2版本区别

回复

Elasticsearch • es_shengbin 发起了问题 • 1 人关注 • 0 个回复 • 3367 次浏览 • 2017-11-10 18:20 • 来自相关话题

es memory lock的问题

回复

Elasticsearch • benhairui 发起了问题 • 2 人关注 • 0 个回复 • 4730 次浏览 • 2017-11-10 17:58 • 来自相关话题

社区支持 Markdown 编辑器

默认分类 • medcl 发表了文章 • 4 个评论 • 4511 次浏览 • 2017-11-10 17:36 • 来自相关话题

为了改善大家的创作体验，提高大家的写作和分享热情！?，经过两天的不懈奋斗，终于把 Markdown 编辑器搬上来了。
目前只支持文章的发布，可以通过切换编辑器来选择 Markdown 编辑模式。
希望不要再以编辑器作为理由发只有链接的文章了。
???????????


  • 支持 Github 风格的 Markdown 格式
  • 支持本站附件功能
  • 支持 emoj 符号
  • 支持自动的页面导航
  • 以前的文章可再次编辑，切换 Markdown 模式然后修改保存

    如何使用？

    1. 点击【发起】，选择文章
    2. 切换绿色按钮，将编辑器切换到 Markdown，然后在文本框内输入 Markdown 格式的内容即可。


      在线 Markdown 脚本编辑预览工具：https://elasticsearch.cn/stati ... down/

      ----

      以下为样式测试参考，忽略其意义。

      1510307393004.jpg



      ----------- 常用格式-----------------

      ```

      标题1

      标题2

      标题3

      标题4

      标题5
      标题6

      超大标题 //等于号写于文字下方
      ===
      标题 //同超大标题
      ---

      短代码
      注：长代码块，用三个: `

      This is the first level of quoting.

      This is nested blockquote.

      Back to the first level.


  • Red
  • Green
  • Blue

  • Red
  • Green
  • Blue

  • Red
  • Green
  • Blue

    1. 这是第一个
    2. 这是第二个
    3. 这是第三个

    • *
      *
      ***
  • Red
  • Green
  • Blue

  • Red
  • Green
  • Blue

  • Red
  • Green
  • Blue

    1. 这是第一个
    2. 这是第二个
    3. 这是第三个

    • *
      *
      ***
  • Light weight, low footprint, memory requirement should < 100MB
  • Easy to deploy, no runtime or dependency required
  • Easy to use, no programming or scripts ability needed, out of box features


    Screenshoot


    <img width="800" alt="What a Spider! GOPA Spider!" src="" rel="nofollow" target="_blank">https://raw.githubusercontent. ... gt%3B


    ---


  • How to use
  • Setup
  • Download Pre Built Package
  • Compile The Package Manually
  • Optional Config
  • Start
  • Stop
  • Configuration
  • UI
  • API
  • Contributing
  • License



    How to use


    Setup


    First of all, get it, two opinions: download the pre-built package or compile it yourself.

    Download Pre Built Package


    Go to [Release](https://github.com/infinitbyte/gopa/releases) or [Snapshot](https://github.com/infinitbyte ... leases) page, download the right package for your platform.

    Note: Darwin is for Mac

    Compile The Package Manually


  • Mac/Linux: Run make build to build the Gopa.

  • Windows: Checkout this wiki page - [How to build GOPA on windows](https://github.com/infinitbyte ... indows).

    So far, we have:

    gopa, the main program, a single binary.

    config/, elasticsearch related scripts etc.

    gopa.yml, main configuration for gopa.



    Optional Config


    By default, Gopa works well except indexing, if you want to use elasticsearch as indexing, follow these steps:

  • Create a index in elasticsearch with script config/gopa-index-mapping.sh


    Example
    curl -XPUT "http://localhost:9200/gopa-index" -H 'Content-Type: application/json' -d'
    
{
    
"mappings": {
    
"doc": {
    
"properties": {
    
"host": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"snapshot": {
    
"properties": {
    
"bold": {
    
"type": "text"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"content_type": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"file": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"h1": {
    
"type": "text"
    
},
    
"h2": {
    
"type": "text"
    
},
    
"h3": {
    
"type": "text"
    
},
    
"h4": {
    
"type": "text"
    
},
    
"hash": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"id": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"images": {
    
"properties": {
    
"external": {
    
"properties": {
    
"label": {
    
"type": "text"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
}
    
}
    
},
    
"internal": {
    
"properties": {
    
"label": {
    
"type": "text"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
}
    
}
    
}
    
}
    
},
    
"italic": {
    
"type": "text"
    
},
    
"links": {
    
"properties": {
    
"external": {
    
"properties": {
    
"label": {
    
"type": "text"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
}
    
}
    
},
    
"internal": {
    
"properties": {
    
"label": {
    
"type": "text"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
}
    
}
    
}
    
}
    
},
    
"path": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"sim_hash": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"lang": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"size": {
    
"type": "long"
    
},
    
"text": {
    
"type": "text"
    
},
    
"title": {
    
"type": "text",
    
"fields": {
    
"keyword": {
    
"type": "keyword"
    
}
    
}
    
},
    
"version": {
    
"type": "long"
    
}
    
}
    
},
    
"task": {
    
"properties": {
    
"breadth": {
    
"type": "long"
    
},
    
"created": {
    
"type": "date"
    
},
    
"depth": {
    
"type": "long"
    
},
    
"id": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"original_url": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"reference_url": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"schema": {
    
"type": "keyword",
    
"ignore_above": 256
    
},
    
"status": {
    
"type": "integer"
    
},
    
"updated": {
    
"type": "date"
    
},
    
"url": {
    
"type": "keyword",
    
"ignore_above": 256
    
}
    
}
    
}
    
}
    
}
    
}
    
}'



    _Note: Elasticsearch version should > v5.0_

  • Enable index module in gopa.yml, update the elasticsearch's setting:
    ```
  • module: index
    enabled: true
    ui:
    enabled: true
    elasticsearch:
    endpoint: http://dev:9200
    index_prefix: gopa-
    username: elastic
    password: changeme
    ```



    Start


    Gopa doesn't require any dependencies, simply run ./gopa to start the program.

    Gopa can be run as daemon(Note: Only available on Linux and Mac):


    Example


    
➜  gopa git:(master) ✗ ./bin/gopa --daemon
    
________ ________ __________  _____
    
/  _____/ \_____  \\______   \/  _  \
    
/   \  ___  /   |   \|     ___/  /_\  \
    
\    \_\  \/    |    \    |  /    |    \
    
\______  /\_______  /____|  \____|__  /
    
\/         \/                \/
    
[gopa] 0.10.0_SNAPSHOT
    
///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
    

    
[10-21 16:01:09] [INF] [instance.go:23] workspace: data/gopa/nodes/0
    
[gopa] started.



    Also run `./gopa -h` to get the full list of command line options.


    Example

    
➜  gopa git:(master) ✗ ./bin/gopa -h
    
________ ________ __________  _____
    
/  _____/ \_____  \\______   \/  _  \
    
/   \  ___  /   |   \|     ___/  /_\  \
    
\    \_\  \/    |    \    |  /    |    \
    
\______  /\_______  /____|  \____|__  /
    
\/         \/                \/
    
[gopa] 0.10.0_SNAPSHOT
    
///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
    

    
Usage of ./bin/gopa:
    
-config string
    
the location of config file (default "gopa.yml")
    
-cpuprofile string
    
write cpu profile to this file
    
-daemon
    
run in background as daemon
    
-debug
    
run in debug mode, wi
    
-log string
    
the log level,options:trace,debug,info,warn,error (default "info")
    
-log_path string
    
the log path (default "log")
    
-memprofile string
    
write memory profile to this file
    
-pidfile string
    
pidfile path (only for daemon)
    
-pprof string
    
enable and setup pprof/expvar service, eg: localhost:6060 , the endpoint will be: http://localhost:6060/debug/pprof/ and http://localhost:6060/debug/vars</pre>




    ### Stop

    It's safety to press `ctrl+c` stop the current running Gopa, Gopa will handle the rest,saving the checkpoint,
    you may restore the job later,the world is still in your hand.

    If you are running `Gopa` as daemon, you may stop it like this:

    ```
    kill -QUIT `pgrep gopa`
    ```

    ## Configuration

    ## UI

  • Search Console <a href="http://127.0.0.1:9001/" rel="nofollow" target="_blank">http://127.0.0.1:9001/</a>
  • Admin Console <a href="http://127.0.0.1:9001/admin/" rel="nofollow" target="_blank">http://127.0.0.1:9001/admin/</a>

    API


  • TBD


    Contributing


    You are sincerely and warmly welcomed to play with this project,
    from UI style to core features,
    or just a piece of document,
    welcome! let's make it better.


    License
    =======
    Released under the [Apache License, Version 2.0](https://github.com/infinitbyte ... ICENSE) .


    Also XSS Test

    alert('XSS test');

Elasticsearch自动写副本分片失败

回复

Elasticsearch • ximing 发起了问题 • 1 人关注 • 0 个回复 • 2920 次浏览 • 2017-11-10 16:58 • 来自相关话题

logstash output email 日志抛错

Logstash • luosm 回复了问题 • 4 人关注 • 3 个回复 • 11325 次浏览 • 2021-02-05 10:08 • 来自相关话题

java API 聚合

Elasticsearch • redhat 回复了问题 • 2 人关注 • 1 个回复 • 1979 次浏览 • 2017-11-10 18:44 • 来自相关话题

date_index_name processor 如何设置动态的 prefix

Elasticsearch • medcl 回复了问题 • 2 人关注 • 2 个回复 • 2268 次浏览 • 2017-11-13 12:01 • 来自相关话题

elastic query内容包含"+"号，无法获得预期查询结果怎么办？

Elasticsearch • hufuman 回复了问题 • 2 人关注 • 1 个回复 • 2554 次浏览 • 2017-11-11 06:51 • 来自相关话题

Elasticsearch多汇聚查询返回数据格式

Elasticsearch • zhangg7723 回复了问题 • 2 人关注 • 1 个回复 • 2925 次浏览 • 2017-11-10 14:05 • 来自相关话题

存在自有 mapping 会忽略动态模板中的 mapping 么？

Elasticsearch • rochy 回复了问题 • 5 人关注 • 3 个回复 • 3012 次浏览 • 2018-11-06 17:44 • 来自相关话题

再次打扰，向你请教一下es集群节点的如果出现硬盘只读的问题

Elasticsearch • zhangg7723 回复了问题 • 3 人关注 • 2 个回复 • 2704 次浏览 • 2017-11-10 14:09 • 来自相关话题

社区日报 第96期 (2017-11-10)

社区日报 • laoyang360 发表了文章 • 0 个评论 • 1845 次浏览 • 2017-11-10 06:37 • 来自相关话题

1、源码解析 | elasticsearch 索引创建和更新
http://t.cn/RlHuOKx
2、业界良心 |  《Elasticsearch5.6.3 Java API 中文手册》
https://elasticsearch.cn/article/362
3、PPT | 基于 Mesos/Docker 的 Elasticsearch 容器化私有云
http://t.cn/RlHuTQR 
4、只等你来 | Elastic Meetup 广州交流会
https://elasticsearch.cn/article/364

编辑：laoyang360
归档：https://elasticsearch.cn/article/365
订阅：https://tinyletter.com/elastic-daily
 
Copyright © 2025· CC BY-NC-SA 3.0