Skip to main content

AWS OpenSearch server setup

AWS OpenSearch Service allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an OpenSearch server. The post index is stored on the OpenSearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated.

Deploying AWS OpenSearch includes the following two steps: setting up AWS OpenSearch, and configuring Mattermost.

Set up AWS OpenSearch

From Mattermost v9.11, beta support is available for AWS OpenSearch v1.x and v2.x. This document covers both on‑premises and AWS OpenSearch setup, including manual steps and Terraform examples.

We highly recommend that you set up an AWS OpenSearch server on a separate machine from the Mattermost server.

  1. To install on-premise OpenSearch, provision a dedicated server (e.g. Ubuntu 22.04 LTS).

  2. Install Java (OpenSearch requires Java 11+):

    sudo apt update
    sudo apt install -y openjdk-11-jdk
    java -version
  3. Download & extract OpenSearch 2.x:

    wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.9.0/opensearch-2.9.0-linux-x64.tar.gz
    tar -xzf opensearch-2.9.0-linux-x64.tar.gz
    sudo mv opensearch-2.9.0 /usr/share/opensearch
  4. Create a dedicated user & set permissions:

    sudo useradd --no-create-home --shell /bin/false opensearch
    sudo chown -R opensearch:opensearch /usr/share/opensearch
  5. Configure systemd:

    [Unit]
    Description=OpenSearch
    Wants=network-online.target
    After=network-online.target

    [Service]
    Type=notify
    User=opensearch
    Group=opensearch
    ExecStart=/usr/share/opensearch/bin/opensearch
    Restart=on-failure
    LimitNOFILE=65536
    LimitNPROC=4096

    [Install]
    WantedBy=multi-user.target
  6. Edit opensearch.yml to include the following:

    cluster.name: mattermost-cluster
    node.name: node-1
    path.data: /var/lib/opensearch
    path.logs: /var/log/opensearch
    network.host: 0.0.0.0
    discovery.seed_hosts: ["<other-node-ip>"]
    cluster.initial_master_nodes: ["node-1", "node-2"]
  7. Enable & start OpenSearch:

    sudo systemctl daemon-reload
    sudo systemctl enable opensearch
    sudo systemctl start opensearch
    sudo systemctl status opensearch
  8. Install the icu-analyzer plugin to the /usr/share/opensearch/plugins directory by running the following command:

    sudo /usr/share/opensearch/bin/opensearch-plugin install analysis-icu

    Restart OpenSearch on each node to load the newly installed plugin before verifying it; _cat/plugins only reports active plugins, so a restart is required first. For production clusters, restart nodes one at a time (rolling restart) rather than all at once. Confirm the plugin is installed and active on every node before continuing. Replace the host, admin password, and path to your cluster's CA certificate:

    curl --silent --show-error --fail-with-body --cacert <path-to-root-ca.pem> --user 'admin:<password>' 'https://<your-opensearch-host>:9200/_cat/plugins?v&h=name,component,version&s=name,component'

    Running this command should show analysis-icu once per node in the cluster. If the analysis-icu line is missing for any node, the plugin is not installed there.

(Optional) CJK language analyzer plugins: To improve search for Korean, Japanese, or Chinese content, install one or more of the following language-specific analyzer plugins: analysis-nori (Korean), analysis-kuromoji (Japanese), and analysis-smartcn (Chinese).

sudo /usr/share/opensearch/bin/opensearch-plugin install analysis-nori
sudo /usr/share/opensearch/bin/opensearch-plugin install analysis-kuromoji
sudo /usr/share/opensearch/bin/opensearch-plugin install analysis-smartcn

After installing the CJK plugins, restart OpenSearch to load them:

sudo systemctl restart opensearch

Then enable the EnableCJKAnalyzers configuration setting. See Enabling Chinese, Japanese, and Korean Search for additional CJK search configuration options.

Terraform (Docker) Example

provider "docker" {
host = "unix:///var/run/docker.sock"
}

resource "docker_image" "opensearch" {
name = "opensearchproject/opensearch:2.9.0"
}

resource "docker_container" "opensearch" {
name = "opensearch"
image = docker_image.opensearch.latest

ports {
internal = 9200
external = 9200
}
ports {
internal = 9600
external = 9600
}

env = [
"cluster.name=mattermost-cluster",
"network.host=0.0.0.0",
"discovery.type=single-node", # remove for multi-node
]

restart = "unless-stopped"
}

resource "null_resource" "install_icu_plugin" {
depends_on = [docker_container.opensearch]

provisioner "local-exec" {
command = "docker exec opensearch /usr/share/opensearch/bin/opensearch-plugin install analysis-icu && docker restart opensearch"
}
}

Configure Mattermost

Follow these steps to configure Mattermost to use your AWS OpenSearch server and to generate the post index:

  1. Go to System Console > Environment > Elasticsearch.
  2. Set Enable Elasticsearch Indexing to true to enable the other the settings on the page.
  3. Ensure Backend type is set to opensearch.
  4. Set the Server Connection Address to your Elasticsearch or OpenSearch cluster endpoint.
  5. Monitor cluster health: curl https://mattermost-os-xxxxx.us-east-1.es.amazonaws.com/_cluster/health
  1. (Optional) Enter Server Username used to access the enterprise search server.
  2. (Optional) Enter Server Password associated with the username.
  3. Set Enable Cluster Sniffing (Optional). Sniffing finds and connects to all data nodes in your cluster automatically.
  1. Optional CA and client certificate configuration settings are available for use with basic authentication credentials or to replace them. See the Enterprise search configuration settings documentation for details.
  2. Select Test Connection and then select Save. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with Elasticsearch or AWS OpenSearch.

Build the post index of existing messages

Select Index Now. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set Enable Elasticsearch for search queries to false so that database search is available during the indexing process.

Enable enterprise search

Ensure bulk indexing is complete before enabling enterprise search, otherwise search results will be incomplete.

Set Enable Elasticsearch for search queries to true, and setting Enable Elasticsearch for autocomplete to true. Save your configuration updates and restart the Mattermost server.

Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch or AWS OpenSearch server.

Enterprise search limitations

  1. Elasticsearch and AWS OpenSearch use a standard selection of "stop words" to keep search results relevant. Results for the following words will not be returned: "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", and "with".
  2. Searching stop words in quotes returns more results than just the searched terms (ticket).
  3. By default, search results are limited to a user's team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch or AWS OpenSearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch or AWS OpenSearch request applying those filters. From Mattermost v11.6, admins can allow searching public channels without membership so that users can find messages in public channels they haven't joined, scoped to teams they belong to.
  4. User search and autocomplete served by Elasticsearch or AWS OpenSearch match on username, nickname, and first and last name only. Email addresses aren't indexed, so no user can be found by email address, including system admins. User searches served from the database continue to match on email address. For channel and team member searches, system admins always match on email address, and all other users also require Show email address to be enabled. The System Console > User Management > Users list always matches on email address, independently of that setting.