Elasticsearch provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Learn more about Elasticsearch in our product documentation.
You can configure the Elasticsearch environment in which Mattermost is deployed in System Console > Environment > Elasticsearch. You can also edit the config.json
file as described in the following tables. Changes to configuration settings in this section require a server restart before taking effect.
Enable Elasticsearch indexing
Available in legacy Enterprise Edition E10/E20
Configure Mattermost to index new posts automatically.
|
|
|
Notes:
|
Server connection address
Available in legacy Enterprise Edition E10/E20
The address of the Elasticsearch server. |
|
CA path
Optional path to the Custom Certificate Authority certificates for the Elasticsearch server. |
|
Note: Available from Mattermost v7.8. Can be used in conjunction with basic auth credentials or to replace them. Leave this setting blank to use the default Certificate Authority certificates for the operating system. |
Client certificate path
Optional client certificate for the connection to the Elasticsearch server in the PEM format. |
|
Note: Available from Mattermost v7.8. Can be used in conjunction with basic auth credentials or to replace them. |
Client certificate key path
Optional key for the client certificate in the PEM format. |
|
Note: Available from Mattermost v7.8. Can be used in conjunction with basic auth credentials or to replace them. |
Skip TLS verification
Available in legacy Enterprise Edition E10/E20
The certificate step for TLS connections can be skipped.
|
|
Server username
Available in legacy Enterprise Edition E10/E20
(Optional) The username to authenticate to the Elasticsearch server. String input. |
|
Server password
Available in legacy Enterprise Edition E10/E20
(Optional) The password to authenticate to the Elasticsearch server. String input. |
|
Enable cluster sniffing
Available in legacy Enterprise Edition E10/E20
Configure Mattermost to automatically find and connect to all data nodes in a cluster.
|
|
Select the Test Connection button in the System Console to validate the connection between Mattermost and the Elasticsearch server. |
Bulk indexing
Available in legacy Enterprise Edition E10/E20
Configure Mattermost to start a bulk index of all existing posts in the database. |
|
Select the Index Now button in the System Console to start a bulk index of all posts. If the indexing process is canceled, the index and search results will be incomplete. |
Indexes to skip while purging
Specify index names to ignore while purging indexes. Separate multiple index names with commas. Use an asterisk (*) to match a sequence of index name characters. |
|
Purge indexes
Available in legacy Enterprise Edition E10/E20
Purge the entire Elasticsearch index. Typically only used if the index has corrupted and search isn’t behaving as expected. |
|
Select the Purge Indexes button in the System Console to purge the index. After purging the index, create a new index by selecting the Index Now button. |
Enable Elasticsearch for search queries
Available in legacy Enterprise Edition E10/E20
Configure Mattermost to use Elasticsearch for all search queries using the latest index
|
|
Enable Elasticsearch for autocomplete queries
Available in legacy Enterprise Edition E10/E20
Configure Mattermost to use Elasticsearch for all autocompletion queries on users and channels using the latest index.
|
|
Note: Autocompletion results may be incomplete until a bulk index of the existing users and channels database is finished. |
Post index replicas
Available in legacy Enterprise Edition E10/E20
The number of replicas to use for each post index. Numerical input. Default is 1. |
|
Important note: If this setting is changed, the changed configuration only applies to newly-created indexes. To apply the change to existing indexes, purge and rebuild the index after changing this setting. |
Post index shards
Available in legacy Enterprise Edition E10/E20
The number of shards to use for each post index. Numerical input. Default is 1. |
|
Important note: If this setting is changed, the changed configuration only applies to newly-created indexes. To apply the change to existing indexes, purge and rebuild the index after changing this setting. |
Channel index replicas
Available in legacy Enterprise Edition E10/E20
The number of replicas to use for each channel index. Numerical input. Default is 1. |
|
Channel index shards
Available in legacy Enterprise Edition E10/E20
The number of shards to use for each channel index. Numerical input. Default is 1. |
|
User index replicas
Available in legacy Enterprise Edition E10/E20
The number of replicas to use for each user index. Numerical input. Default is 1. |
|
User index shards
Available in legacy Enterprise Edition E10/E20
The number of shards to use for each user index. Numerical input. Default is 1. |
|
Aggregate search indexes
Available in legacy Enterprise Edition E10/E20
Elasticsearch indexes older than the age specified by this setting, in days, will be aggregated during the daily scheduled job. Numerical input. Default is 365 days. |
|
Note: If you’re using data retention and Elasticsearch, configure this with a value greater than your data retention policy. |
Post aggregator start time
Available in legacy Enterprise Edition E10/E20
The start time of the daily scheduled aggregator job. Must be a 24-hour time stamp in the form Default is 03:00 (3 AM) |
|
Index prefix
Available in legacy Enterprise Edition E10/E20
The prefix added to the Elasticsearch index name. |
|
Note: When this setting is used, all Elasticsearch indexes created by Mattermost are given this prefix. You can set different prefixes so that multiple Mattermost deployments can share an Elasticsearch cluster without the index names colliding. |
Live indexing batch size
Available in legacy Enterprise Edition E10/E20
The number of new posts needed before those posts are added to the Elasticsearch index. Once added to the Index, the post becomes searchable. On servers with more than 1 post per second, we suggest setting this value to the average number of posts over a 20 second period of time. Numerical input. Default is 1. Every post is indexed synchronously as they are created. |
|
Note: It may be necessary to increase this value to avoid hitting the rate limit or resource limit of your Elasticsearch cluster on installs handling more than 1 post per second. What exactly happens when I increase this value? The primary impact is that a post will be indexed into Elasticsearch after the threshold of posts is met which then makes the posts searchable within Mattermost. So, if you set this based on our recommendations for larger servers, and you make a post, you cannot find it via search for ~ 10-20 seconds, on average. Realistically, no users should see or feel this impact due to the limited amount of users who are actively searching for a post this quickly. You can set this value to a lower average or higher average as well, depending on your Elasticsearch server specifications. During busy periods, this delay will be faster as more traffic is happening, causing more posts and a quicker time to hit the index number. During slow times, expect the reverse. |
How to find the right number for your server
You must understand how many posts your server makes every minute. Run the query below to calculate your server’s average posts per minute.
Note that this query can be heavy, so we recommend that you run it during non-peak hours. Additionally, you can adjust the
WHERE
clause to see the posts per minute over a different time period. Right now31536000000
represents the number of milliseconds in a year.SELECT AVG(postsPerMinute) as averagePostsPerMinute FROM ( SELECT count(*) as postsPerMinute, date_trunc('minute', to_timestamp(createat/1000)) FROM posts WHERE createAt > ( (extract(epoch from now()) * 1000 ) - 31536000000) GROUP BY date_trunc('minute', to_timestamp(createat/1000)) ) as ppm;
Decide the acceptable index window for your environment, and divide your average posts per minute by that. We suggest 10-20 seconds. Assuming you have
600
posts per minute on average, and you want to index every 20 seconds (60 seconds / 20 seconds = 3`
) you would calculate600 / 3
to come to the number200
. After 200 posts, Mattermost will index the posts into Elasticsearch. So, on average, there would be a 20-second delay in searchability.Edit the
config.json
or run mmctl to modify theLiveIndexingBatchSize
settingIn the ``config.json``
{ "ElasticsearchSettings": { "LiveIndexingBatchSize": 200 } }
Via mmctl
mmctl config set ElasticsearchSettings.LiveIndexingBatchSize 200
Via an environment variable
MM_ELASTICSEARCHSETTINGS_LIVEINDEXINGBATCHSIZE = 200
Restart the Mattermost server.
Bulk indexing time window
Available in legacy Enterprise Edition E10/E20
The maximum time window, in seconds, for a batch of posts being indexed by the Bulk Indexer. This setting serves as a performance optimization for installs with over ~10 million posts in the database. Numerical input in seconds. Default is 3600 seconds (1 hour). Approximate this value based on the average number of seconds for 2,000 posts to be added to the database on a typical day in production. |
|
Note: Setting this value too low will cause bulk indexing jobs to run slowly. |
Request timeout
Available in legacy Enterprise Edition E10/E20
The timeout, in seconds, for Elasticsearch calls. Numerical input in seconds. Default is 30 seconds. |
|
Trace
Available in legacy Enterprise Edition E10/E20
Options for printing Elasticsearch trace errors.
|
|