elasticsearch-RESTful-API-常用操作.md

什么是 ElasticSearch

ElasticSearch 是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java语言开发的，并作为Apache许可条款下的开放源码发布，是一种流行的企业级搜索引擎。

HTTP RESTful API 常用操作

• 查询和过滤上下文

$ curl -X GET "localhost:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "title":   "Search"        }}, 
        { "match": { "content": "Elasticsearch" }}  
      ],
      "filter": [ 
        { "term":  { "status": "published" }}, 
        { "range": { "publish_date": { "gte": "2019-01-01" }}} 
      ]
    }
  }
}
'

• 查询 ES 中所有索引模板名称

$ curl localhost:9200/_template  | jq keys

• 查询一个索引模板详细信息

$ curl localhost:9200/_template/logstash  | jq

• 查询 ES 集群健康状态

$  curl -s -XGET 'http://localhost:9200/_cluster/health?pretty'

• 查询 ES 集群设置

curl -s -XGET 'http://localhost:9200/_cluster/settings' | jq

• 下架 ES 集群中一个节点

$ curl -X PUT "http://localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
  "transient" : {
    "cluster.routing.allocation.exclude._name" : "node-3"
  }
}'

# 除了_name 之外, 还可以用_ip、_host进行匹配

• 设置 discovery.zen.minimum_master_nodes

# 法定个数就是 ( master 候选节点个数 / 2) + 1 ，默认为 1
$ curl -X PUT "http://localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
    "persistent" : {
        "discovery.zen.minimum_master_nodes" : 2
    }
}'

• 增加一个default_template模板，设置副本为0 (默认副本为1)，不推介这样做，集群没有备份数据

$ curl -XPUT "localhost:9200/_template/default_template" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*"],
  "settings": {
    "index": {
      "number_of_replicas": 0
    }
  }
}'

• 把现有的 ES 集群中 index 副本去掉，不推介这样做，集群没有备份数据

$ curl -X PUT "localhost:9200/_all/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "number_of_replicas" : 0
    }
}'

• 添加自定义nginx索引模板

$ curl -XPUT "http://localhost:9200/_template/nginx_template" -H 'Content-Type: application/json' -d'
{
  "template" : "*nginx*",
  "version" : 60001,
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
      "dynamic_templates" : [ {
        "message_field" : {
          "path_match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text",
            "norms" : false
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "text", "norms" : false,
            "fields" : {
              "keyword" : { "type": "keyword", "ignore_above": 256 }
            }
          }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date"},
        "@version": { "type": "keyword"},
        "geoip"  : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "half_float" },
            "longitude" : { "type" : "half_float" }
          }
        }
      }
    }
  }
}
'

elasticdump 导出数据

• 导出kubernetes pod name名为test 并且 log 字段中匹配access数据

$ elasticdump \
  --input=http://localhost:9200/logstash-2019.01.06 \
  --output=/tmp/test-2019-01-06-query.json \
  --limit=10000 \
  --searchBody '{
  "query": {
    "bool": {
      "must": [
        {
          "match": { "kubernetes.pod_name": "test" },
          "match": { "log": "*access*" }}
      ],
      "filter": {
        "range": {
          "@timestamp": {
            "gte": "2019-01-06T00:00:00.000+00:00",
            "lt":  "2019-01-06T10:00:00.000+00:00"
          }
        }
      }
    }
  }
}'