[role="xpack"]
[[monitoring-with-metricbeat]]
=== Collect {ls} monitoring data with {metricbeat}
[subs="attributes"]
++++
<titleabbrev>{metricbeat} collection</titleabbrev>
++++

You can use {metricbeat} to collect data about {ls} and ship it to the
monitoring cluster. The benefit of Metricbeat collection is that the monitoring
agent remains active even if the {ls} instance does not. 

//NOTE: The tagged regions are re-used in the Stack Overview.

To collect and ship monitoring data:

. <<disable-default,Disable default collection of monitoring metrics>>
. <<define-cluster__uuid,Specify the target `cluster_uuid` (optional)>>
. <<configure-metricbeat,Install and configure {metricbeat} to collect monitoring data>>

[float]
[[disable-default]]
==== Disable default collection of {ls} monitoring metrics

--
// tag::disable-ls-collection[]
The `monitoring` setting is in the {ls} configuration file (logstash.yml), but is
commented out: 

[source,yaml]
----------------------------------
monitoring.enabled: false
----------------------------------

Remove the `#` at the beginning of the line to enable the setting.
// end::disable-ls-collection[]

--

[float]
[[define-cluster__uuid]]
==== Define `cluster_uuid` (Optional)
To bind the metrics of {ls} to a specific cluster, optionally define the `monitoring.cluster_uuid`
in the configuration file (logstash.yml):

[source,yaml]
----------------------------------
monitoring.cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
----------------------------------

[float]
[[configure-metricbeat]]
==== Install and configure {metricbeat}

. {metricbeat-ref}/metricbeat-installation-configuration.html[Install {metricbeat}] on the
same server as {ls}. 

. Enable the `logstash-xpack` module in {metricbeat}. +
+
--
// tag::enable-ls-module[]
To enable the default configuration in the {metricbeat} `modules.d` directory, 
run: 

*deb, rpm, or brew:* +

["source","sh",subs="attributes"]
----
metricbeat modules enable logstash-xpack
----

*linux or mac:*

["source","sh",subs="attributes"]
----
./metricbeat modules enable logstash-xpack
----

*win:*

["source","sh",subs="attributes"]
----
PS > .{backslash}metricbeat.exe modules enable logstash-xpack
----

For more information, see 
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and 
{metricbeat-ref}/metricbeat-module-beat.html[beat module]. 
// end::enable-beat-module[]
--

. Configure the `logstash-xpack` module in {metricbeat}. +
+
--
// tag::configure-beat-module[]
The `modules.d/logstash-xpack.yml` file contains these settings:

[source,yaml]
----------------------------------
  - module: logstash
    metricsets:
      - node
      - node_stats
    period: 10s
    hosts: ["localhost:9600"]
    #username: "user"
    #password: "secret"
    xpack.enabled: true
----------------------------------
 
Set the `hosts`, `username`, and `password` to authenticate with {ls}.
For other module settings, it's recommended that you accept the
defaults.

By default, the module collects {ls} monitoring data from
`localhost:9600`. 

To monitor multiple {ls} instances, specify a list of hosts, for example:
[source,yaml]
----------------------------------
hosts: ["http://localhost:9601","http://localhost:9602","http://localhost:9603"]
----------------------------------

// end::configure-ls-module[]

// tag::remote-monitoring-user[]
*Elastic security.* If the Elastic {security-features} are enabled, provide a user 
ID and password so that {metricbeat} can collect metrics successfully: 

.. Create a user on the production cluster that has the 
`remote_monitoring_collector` {ref}/built-in-roles.html[built-in role]. 

.. Add the `username` and `password` settings to the module configuration 
file (`logstash-xpack.yml`).
// end::remote-monitoring-user[]
--

. Optional: Disable the system module in the {metricbeat}.
+
--
// tag::disable-system-module[]
By default, the {metricbeat-ref}/metricbeat-module-system.html[system module] is
enabled. The information it collects, however, is not shown on the
*Stack Monitoring* page in {kib}. Unless you want to use that information for
other purposes, run the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules disable system
----------------------------------------------------------------------
// end::disable-system-module[] 
--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster 
(referred to as the _monitoring cluster_) to store the data. Using a separate 
monitoring cluster prevents production cluster outages from impacting your 
ability to access your monitoring data. It also prevents monitoring activities 
from impacting the performance of your production cluster.

For example, specify the {es} output information in the {metricbeat} 
configuration file (`metricbeat.yml`):

[source,yaml]
----------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
  
  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes 
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
cluster that stores the monitoring data must have at least one ingest node.

If the {es} {security-features} are enabled on the monitoring cluster, you 
must provide a valid user ID and password so that {metricbeat} can send metrics 
successfully: 

.. Create a user on the monitoring cluster that has the 
`remote_monitoring_agent` {ref}/built-in-roles.html[built-in role]. 
Alternatively, use the `remote_monitoring_user` 
{ref}/built-in-users.html[built-in user]. 
+
TIP: If you're using index lifecycle management, the remote monitoring user
requires additional privileges to create and read indices. For more
information, see `<<feature-roles>>`.

.. Add the `username` and `password` settings to the {es} output information in 
the {metricbeat} configuration file.

For more information about these configuration options, see 
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] to begin
collecting monitoring data. 

. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}]. 

Your monitoring setup is complete.
