Skip to content

Commit

Permalink
update elk
Browse files Browse the repository at this point in the history
  • Loading branch information
abregman committed Feb 16, 2020
1 parent 00c5724 commit e2e2898
Showing 1 changed file with 37 additions and 2 deletions.
39 changes: 37 additions & 2 deletions resources/elk.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,18 @@ Name | Comments
[Grok Patterns](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns) | Logstash Patterns Code
[RE used by Grok](https://github.com/kkos/oniguruma/blob/master/doc/RE) |

## Tools
## Logstash Tools

Name | Comments
:------|:------:
[GrokDebug](https://grokdebug.herokuapp.com) | Test Grok Patterns

## Kibana Tools

Name | Comments
:------|:------:
[Rubban](https://github.com/sherifabdlnaby/rubban) | "Kibana Automatic Index Pattern Discovery and Other Curating Tasks"

https://grokdebug.herokuapp.com

## Logstash Cheat Sheet

Expand All @@ -41,10 +50,36 @@ input {
tcpdump -Xni eth0 port 5140
```

* Load Parquet files

```
Use https://github.com/Parquet/parquet-compatibility/blob/master/parquet-compat/src/test/java/parquet/compat/test/ConvertUtils.java
Specifically https://github.com/Parquet/parquet-compatibility/blob/master/parquet-compat/src/test/java/parquet/compat/test/ConvertUtils.java#L111
Once it's convreted to CSV, you process it with
input {
file {
path => "/path/to/your/parquet/as/csv/file"
}
}
filter {
csv {
columns => ["col1", "col2"]
}
}
```

## Elsaticsearch Cheat Sheet

* Test elasticsearch

```
curl localhost:9200
```

* List indexes

```
curl 'localhost:9200/_cat/indices?v'
```

0 comments on commit e2e2898

Please sign in to comment.