请问大量小文件读会影响数据的写入么?
kennywu76 回复了问题 • 4 人关注 • 1 个回复 • 2713 次浏览 • 2018-01-18 11:27
elk如何支持多套环境?
Chenx1242 回复了问题 • 3 人关注 • 2 个回复 • 2110 次浏览 • 2017-12-29 14:42
elasticsearch安装问题
phoenix 回复了问题 • 2 人关注 • 2 个回复 • 2409 次浏览 • 2017-12-22 11:34
急!求助各位高手,我的kafka客户端发消息到kafka总是在报“incorrect broker.version.fallback”,是什么原因啊?????????
回复yhacmilan 回复了问题 • 1 人关注 • 1 个回复 • 4582 次浏览 • 2017-12-20 19:20
elastic stack 大家用什么做权限控制, 能否实现kibana多租户, 开源的最好
xinfanwang 回复了问题 • 7 人关注 • 2 个回复 • 8595 次浏览 • 2018-02-07 16:33
求问关于grok正则表达式怎么学习啊?
kuanglin 回复了问题 • 2 人关注 • 1 个回复 • 3647 次浏览 • 2017-11-16 15:13
社区支持 Markdown 编辑器
medcl 发表了文章 • 4 个评论 • 4242 次浏览 • 2017-11-10 17:36
为了改善大家的创作体验,提高大家的写作和分享热情!?,经过两天的不懈奋斗,终于把 Markdown 编辑器搬上来了。
目前只支持文章的发布,可以通过切换编辑器来选择 Markdown 编辑模式。
希望不要再以编辑器作为理由发只有链接的文章了。
???????????
- 支持 Github 风格的 Markdown 格式
- 支持本站附件功能
- 支持 emoj 符号
- 支持自动的页面导航
- 以前的文章可再次编辑,切换 Markdown 模式然后修改保存
如何使用?
- 点击【发起】,选择文章
- 切换绿色按钮,将编辑器切换到 Markdown,然后在文本框内输入 Markdown 格式的内容即可。
在线 Markdown 脚本编辑预览工具:https://elasticsearch.cn/stati ... down/
----
以下为样式测试参考,忽略其意义。
----------- 常用格式-----------------
```
标题1
标题2
标题3
标题4
标题5
标题6
超大标题 //等于号写于文字下方
===
标题 //同超大标题
---
短代码
注:长代码块,用三个: `
This is the first level of quoting.
This is nested blockquote.
Back to the first level.
- Red
- Green
- Blue
- Red
- Green
- Blue
- Red
- Green
- Blue
- 这是第一个
- 这是第二个
- 这是第三个
- *
*
***
- -
---
[markdown-syntax](http://daringfireball.net/projects/markdown/syntax)
[id]: http://example.com/ "Optional Title Here"
This is [an example][id] reference-style link.
内容
内容
内容
内容
![这是张外链图片](https://static-www.elastic.co/ ... %3D845)
<http://elastic.co/>
info@elastic.o
四个空格
一个tab
```
----------- 样式预览-----------------
标题1
标题2
标题3
标题4
标题5
标题6
超大标题 //等于号写于文字下方
===
标题 //同超大标题
---
短代码
This is the first level of quoting.
This is nested blockquote.
Back to the first level.
- Red
- Green
- Blue
- Red
- Green
- Blue
- Red
- Green
- Blue
- 这是第一个
- 这是第二个
- 这是第三个
- *
*
***
- -
---
[markdown-syntax](http://daringfireball.net/projects/markdown/syntax)
[id]: http://example.com/ "Optional Title Here"
This is [an example][id] reference-style link.
内容
内容
内容
内容
![这是张外链图片](https://static-www.elastic.co/ ... %3D845)
<http://elastic.co/>
info@elastic.o
四个空格
一个tab
---------
https://github.com/infinitbyte/gopa 的 README 内容
---------
<img width="200" alt="What a Spider!" src="" rel="nofollow" target="_blank">https://raw.githubusercontent. ... gt%3B
GOPA, A Spider Written in Go.
[![Travis](https://travis-ci.org/infinitb ... master)](https://travis-ci.org/infinitbyte/gopa)
[![Go Report Card](https://goreportcard.com/badge ... e/gopa)](https://goreportcard.com/repor ... e/gopa)
[![Coverage Status](https://coveralls.io/repos/git ... master)](https://coveralls.io/github/in ... master)
[![Join the chat at https://gitter.im/infinitbyte/gopa](https://badges.gitter.im/infinitbyte/gopa.svg)](https://gitter.im/infinitbyte/ ... Dbadge)
Goal
- Light weight, low footprint, memory requirement should < 100MB
- Easy to deploy, no runtime or dependency required
- Easy to use, no programming or scripts ability needed, out of box features
Screenshoot
<img width="800" alt="What a Spider! GOPA Spider!" src="" rel="nofollow" target="_blank">https://raw.githubusercontent. ... gt%3B
---
- How to use
- Setup
- Download Pre Built Package
- Compile The Package Manually
- Optional Config
- Start
- Stop
- Configuration
- UI
- API
- Contributing
- License
How to use
Setup
First of all, get it, two opinions: download the pre-built package or compile it yourself.
Download Pre Built Package
Go to [Release](https://github.com/infinitbyte/gopa/releases) or [Snapshot](https://github.com/infinitbyte ... leases) page, download the right package for your platform.
Note: Darwin is for Mac
Compile The Package Manually
- Mac/Linux: Run
make build
to build the Gopa.- Windows: Checkout this wiki page - [How to build GOPA on windows](https://github.com/infinitbyte ... indows).
So far, we have:
gopa
, the main program, a single binary.
config/
, elasticsearch related scripts etc.
gopa.yml
, main configuration for gopa.
Optional Config
By default, Gopa works well except indexing, if you want to use elasticsearch as indexing, follow these steps:
- Create a index in elasticsearch with script
config/gopa-index-mapping.sh
Example
curl -XPUT "http://localhost:9200/gopa-index" -H 'Content-Type: application/json' -d'
{
"mappings": {
"doc": {
"properties": {
"host": {
"type": "keyword",
"ignore_above": 256
},
"snapshot": {
"properties": {
"bold": {
"type": "text"
},
"url": {
"type": "keyword",
"ignore_above": 256
},
"content_type": {
"type": "keyword",
"ignore_above": 256
},
"file": {
"type": "keyword",
"ignore_above": 256
},
"h1": {
"type": "text"
},
"h2": {
"type": "text"
},
"h3": {
"type": "text"
},
"h4": {
"type": "text"
},
"hash": {
"type": "keyword",
"ignore_above": 256
},
"id": {
"type": "keyword",
"ignore_above": 256
},
"images": {
"properties": {
"external": {
"properties": {
"label": {
"type": "text"
},
"url": {
"type": "keyword",
"ignore_above": 256
}
}
},
"internal": {
"properties": {
"label": {
"type": "text"
},
"url": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"italic": {
"type": "text"
},
"links": {
"properties": {
"external": {
"properties": {
"label": {
"type": "text"
},
"url": {
"type": "keyword",
"ignore_above": 256
}
}
},
"internal": {
"properties": {
"label": {
"type": "text"
},
"url": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"path": {
"type": "keyword",
"ignore_above": 256
},
"sim_hash": {
"type": "keyword",
"ignore_above": 256
},
"lang": {
"type": "keyword",
"ignore_above": 256
},
"size": {
"type": "long"
},
"text": {
"type": "text"
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"version": {
"type": "long"
}
}
},
"task": {
"properties": {
"breadth": {
"type": "long"
},
"created": {
"type": "date"
},
"depth": {
"type": "long"
},
"id": {
"type": "keyword",
"ignore_above": 256
},
"original_url": {
"type": "keyword",
"ignore_above": 256
},
"reference_url": {
"type": "keyword",
"ignore_above": 256
},
"schema": {
"type": "keyword",
"ignore_above": 256
},
"status": {
"type": "integer"
},
"updated": {
"type": "date"
},
"url": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}'
_Note: Elasticsearch version should > v5.0_
- Enable index module in
gopa.yml
, update the elasticsearch's setting:
```- module: index
enabled: true
ui:
enabled: true
elasticsearch:
endpoint: http://dev:9200
index_prefix: gopa-
username: elastic
password: changeme
```
Start
Gopa doesn't require any dependencies, simply run./gopa
to start the program.
Gopa can be run as daemon(Note: Only available on Linux and Mac):
Example
➜ gopa git:(master) ✗ ./bin/gopa --daemon
________ ________ __________ _____
/ _____/ \_____ \\______ \/ _ \
/ \ ___ / | \| ___/ /_\ \
\ \_\ \/ | \ | / | \
\______ /\_______ /____| \____|__ /
\/ \/ \/
[gopa] 0.10.0_SNAPSHOT
///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
[10-21 16:01:09] [INF] [instance.go:23] workspace: data/gopa/nodes/0
[gopa] started.
Also run `./gopa -h` to get the full list of command line options.
Example
➜ gopa git:(master) ✗ ./bin/gopa -h
________ ________ __________ _____
/ _____/ \_____ \\______ \/ _ \
/ \ ___ / | \| ___/ /_\ \
\ \_\ \/ | \ | / | \
\______ /\_______ /____| \____|__ /
\/ \/ \/
[gopa] 0.10.0_SNAPSHOT
///last commit: 99616a2, Fri Oct 20 14:04:54 2017 +0200, medcl, update version to 0.10.0 ///
Usage of ./bin/gopa:
-config string
the location of config file (default "gopa.yml")
-cpuprofile string
write cpu profile to this file
-daemon
run in background as daemon
-debug
run in debug mode, wi
-log string
the log level,options:trace,debug,info,warn,error (default "info")
-log_path string
the log path (default "log")
-memprofile string
write memory profile to this file
-pidfile string
pidfile path (only for daemon)
-pprof string
enable and setup pprof/expvar service, eg: localhost:6060 , the endpoint will be: http://localhost:6060/debug/pprof/ and http://localhost:6060/debug/vars</pre>
### Stop
It's safety to press `ctrl+c` stop the current running Gopa, Gopa will handle the rest,saving the checkpoint,
you may restore the job later,the world is still in your hand.
If you are running `Gopa` as daemon, you may stop it like this:
```
kill -QUIT `pgrep gopa`
```
## Configuration
## UI
- Search Console
<a href="http://127.0.0.1:9001/" rel="nofollow" target="_blank">http://127.0.0.1:9001/</a>
- Admin Console
<a href="http://127.0.0.1:9001/admin/" rel="nofollow" target="_blank">http://127.0.0.1:9001/admin/</a>
API
- TBD
Contributing
You are sincerely and warmly welcomed to play with this project,
from UI style to core features,
or just a piece of document,
welcome! let's make it better.
License
=======
Released under the [Apache License, Version 2.0](https://github.com/infinitbyte ... ICENSE) .
Also XSS Test
alert('XSS test');
logstash+elasticsearch,时区问题
xjj001 回复了问题 • 8 人关注 • 2 个回复 • 15287 次浏览 • 2018-06-08 12:20
elastalert的rule文件写中文无法运行
jhondom 回复了问题 • 2 人关注 • 2 个回复 • 3133 次浏览 • 2019-04-10 08:36
es5.6安装x-pack 修改密码不成功,报No handler found for uri
rockybean 回复了问题 • 2 人关注 • 4 个回复 • 3895 次浏览 • 2017-09-26 15:42