Original link: https://chegva.com/5750.html
1. What is a monitor?
watcher is a plug-in for elasticsearch that can provide alerts and notifications based on changes in data, by using Watcher to monitor changes or exceptions in data and perform necessary response operations.
Since version 5.0, watcher has become a part of x-pack, which means that after x-pack is installed, watcher is already installed.
In order to use the feature of Watcher in ES, a certificate containing this feature must be obtained.
Features of each version of ES: https://www.elastic.co/subscriptions
Typical usage scenarios:
-
Monitor social media as another way to detect failures in user-facing automated systems such as ATMs or ticketing systems. Notify service technicians when the number of tweets and posts in an area exceeds critical thresholds.
-
Monitor your infrastructure and track disk usage over time. Please open a help desk ticket when any server may run out of available space within the next few days.
-
Track network activity to detect malicious activity and proactively change firewall configurations to deny malicious users.
-
Monitor Elasticsearch and notify system administrators immediately if a node leaves the cluster or if query throughput exceeds expected ranges.
-
Track application response times and open a help desk ticket if page load time exceeds SLA by more than 5 minutes. If the SLA is exceeded by one hour, call the duty manager.
With Watcher, you can monitor the data indicators in the index, the disk usage of the cluster server, network activity, the status of the ES cluster, the corresponding time of the program and other attributes, and take corresponding actions according to the monitored situation.
These scenarios are characterized by the fact that relevant data or data changes can be identified through regular Elasticsearch queries, and the query results can be checked against conditions.
Perform one or more actions if the condition is true—send an email, notify a third-party system, or store query results.
2. How does the monitor work?
1. Schedule scheduling plan
A scheduler for running queries and checking conditions.
2. Query query
The query to run as input to the condition. The monitor supports the full Elasticsearch query language, including aggregations.
3. Condition conditions
Conditions that determine whether to execute an action. You can use simple conditions (always true), or use scripts for more complex scenarios.
4. Actions
One or more actions, such as sending an email, pushing data to a 3rd party system via a webhook, or indexing query results.
Watcher execution process description:
Load input data as payload in the Watcher execution context. This makes the data available for all subsequent steps in the execution process. This step is controlled by the input of Watcher.
Evaluate the monitor Watcher Condition to determine whether to continue processing the Watcher. If the condition is satisfied (calculation result is true), the processing proceeds to the next step. If it is not satisfied (evaluates to false), stop executing Watcher.
Apply transformations to the monitoring payload (if required).
When the addition is satisfied and the Watcher is not restricted, the Watcher action is executed.
3. How to create a monitor Watcher
1. Trigger
Determines when to check Watcher. There must be a trigger in Watcher.
2. Input
Load data into Watcher payload. If no input is specified, an empty load is loaded.
3. Condition
Controls whether to execute monitoring actions. If no condition is specified, the condition defaults to always.
4. Transform
Process monitoring payload data to make it ready for monitoring operations Actions.
Transformations can be defined at the monitoring level or as action-specific transformations. Digital conversion is an optional configuration.
5. Actions
Specifies what happens when the monitoring conditions are met.
Example usage:
{
"trigger": {
"schedule": {
"interval": "60s" #Execute once a minute }
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": [
"chegva.com_log*" #Query index name ],
"rest_total_hits_as_int": true,
"body": {
"query": {
"bool": {
"minimum_should_match": 1,
"must_not": {
"term": {
"agentinfo.agent_id": "agent001" #Exclude an item }
},
"should": [
{
"match_phrase": {
"error.code": "AGENT_NODE"
}
},
{
"match_phrase": {
"error.code": "AGENT_COLD_START_TIMEOUT"
}
},
...... #error.code satisfies the condition as or
{
"match_phrase": {
"error.code": "AGENT_LOST_CONNECT"
}
}
],
"filter": {
"range": {
"@timestamp": {
"from": "||-1m", #Monitor data within one minute "to": ""
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0 #The amount of data generated is greater than 0
}
}
},
"actions": {
"test_issue": {
"webhook": {
"scheme": "http",
"host": "110.112.119.x", #nginx server for alarm configuration, to proxy alarm messages to alarm group robots "port": 8888,
"method": "post",
"params": {},
"headers": {},
"body": """{"msgtype": "text", "text": { "content": "agentnum:\n agent_version:\n ... "}}"""
}
}
}
}
refer to:
-
ES Detailed Explanation – Getting Started: Basic Use of Query and Aggregation
-
ElasticSearch aggregation query Restful syntax and JavaApi detailed explanation (based on ES7.6)
This article is transferred from: https://chegva.com/5750.html
This site is only for collection, and the copyright belongs to the original author.