Confluence DC semantic search uses OpenSearch's ML (Machine Learning) Commons plugin to generate and store vector embeddings for Confluence content. When a user searches, a neural (vector) query is executed against those embeddings, returning results ranked by semantic similarity rather than keyword overlap.

For this to work, the OpenSearch cluster must be able to run ML tasks i.e. it must be able to host and serve a text-embedding model. How this is configured depends on whether you manage your own OpenSearch cluster or use Amazon OpenSearch Service.

Before enabling semantic search for Confluence, confirm that the OpenSearch cluster has one of the following:

• Dedicated OpenSearch nodes with the ml node role.

• A deliberate, capacity-reviewed configuration that allows ML Commons tasks to run on data nodes.

• For Amazon OpenSearch Service production domains, a supported remote inference setup using ML connectors rather than disabling the production safeguard that keeps ML tasks off data nodes.

Step 1: Check whether ML is already enabled

セルフホスト型 OpenSearch

Run the following from OpenSearch Dashboards Dev Tools or any REST client:

GET /_cat/nodes?v&h=name,node.roles

Example output:

name                  node.roles
data-e5b89ad7         data,ingest,ml
os-node-02            data,ingest,cluster_manager

Look for ml  in the node.roles column. Also check the current cluster setting:

GET /_cluster/settings?include_defaults=true&filter_path=**.ml_commons

Example output:

{
    "persistent": {
        "plugins": {
            "ml_commons": {
                "only_run_on_ml_node": "true",
                "native_memory_threshold": "90"
            }
        }
    },
    ...
}

The output should indicate that at least one node has the ml role, and plugins.ml_commons.only_run_on_ml_node is true (default).

AWS Managed OpenSearch

Run the following from OpenSearch Dashboards Dev Tools or any REST client:

GET /_plugins/_ml/stats

Example output:

{
    "ml_model_index_status": "non-existent",
    "ml_config_index_status": "green",
    "ml_connector_count": 0,
    ...
}

 Also check whether remote inference is enabled:

GET /_cluster/settings?include_defaults=true&filter_path=**.plugins.ml_commons.only_run_on_ml_node,**.plugins.ml_commons.remote_inference.enabled,**.plugins.ml_commons.task_dispatcher.eligible_node_role.*

Example output:

{
    "persistent": {
        "plugins": {
            "ml_commons": {
                "only_run_on_ml_node": "true"
            }
        }
    },
    "defaults": {
        "plugins": {
            "ml_commons": {
                  ...
                },
                "remote_inference": {
                    "enabled": "true"
                }
            }
        }
    }
}

Do not set plugins.ml_commons.only_run_on_ml_node to false on production AWS domains. AWS explicitly recommends production domains use connectors instead.

If ML is already enabled, skip to Step 3: Configure your model. Otherwise, follow the section below that matches your deployment.

Step 2: Enable ML on your OpenSearch cluster

Depending on whether your OpenSearch deployment is self-managed (e.g. on-premise, bare metal, or on Cloud VMs), or you’re using an AWS managed OpenSearch service, you will need to follow two different approaches to enabling ML.

セルフホスト型 OpenSearch

1. Add one or more OpenSearch nodes with the ml role. For production, use at least two ML nodes if semantic search availability is important.
   Example opensearch.yml for a dedicated ML node: 

   node.name: os-ml-1
   # Assign the ml role only — this node will not store index shards
   node.roles: [ ml ]
   cluster.name: <your-cluster-name>
   discovery.seed_hosts: [ <your-seed-host-1>, ... <your-seed-host-n> ]

   A node configured with only the ml role will not store any index shards. It dedicates all its memory and CPU to ML inference. If you also want it to store data (not recommended for production), add data to node.roles.

2. Start the mew node and confirm is has joined the cluster:

   GET /_cat/nodes?v&h=name,node.roles

   The new node should appear with ml in its roles column:

   name            node.roles
   os-ml-1         ml

3. Ensure cluster settings are correct:

   GET /_cluster/settings?include_defaults=true&filter_path=**.ml_commons

   Should return output where only_run_on_ml_node is true and native_memory_threshold has a reasonable, non-zero value e.g 90. If either of these are incorrect, run the following:

   PUT /_cluster/settings
   {
         "persistent": {
           "plugins.ml_commons.only_run_on_ml_node": true,
           "plugins.ml_commons.native_memory_threshold": 90
       }
   }

4. Confirm ML is ready:

   GET /_plugins/_ml/stats

   Should return an HTTP 200 response with output similar to:

   {
       "ml_model_index_status": "non-existent",
       "ml_config_index_status": "green",
       "ml_connector_count": 0,
       ...
   }

   You can refer to the OpenSearch documentation for a complete list of explanations and instructions. 

AWS Managed OpenSearch

Amazon OpenSearch Service is a fully managed offering. You do not have access to opensearch.yml and cannot directly assign the ml node role to nodes via configuration files. There are two approaches available; Amazon Bedrock (no model deployment required) or Amazon SageMaker (requires model deployment). Both approaches are well documented on the AWS developer guide site - https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html. We recommend that you follow the guidance there, as this will be updated by AWS accrodingly.

Step 3: Configure your model

Once your cluster has ML capability enabled (step above), you must register and deploy a local sentence-transformer model. OpenSearch loads the model into memory on the ML-eligible nodes and uses it to generate embeddings during indexing and search.

Once again, depending on your deployment, you will be following a different set of instructions. For AWS Managed OpenSearch, refer the AWS documentation, which too will differ on the chosen remote inference tooling (AWS Bedrock vs AWS SageMaker).

For self-hosted OpenSearch clusters, perform the steps below:

1. (Optional) Register a model group - Model groups let you organise and control access to related models. Registration is optional (ML Commons creates a default group) but is recommended for clarity: 

   POST /_plugins/_ml/model_groups/_register
   {
     "name": "confluence-semantic-search",
     "description": "Sentence-transformer models for Confluence DC Semantic Search"
   }

   Note the returned model_group_id which will be used in the next step.

2. Register a pre-trained model - OpenSearch provides a curated set of pre-trained sentence-transformer models from HuggingFace. These are downloaded directly from the OpenSearch model repository, no external file hosting is required.

   huggingface/sentence-transformers/all-MiniLM-L6-v2 (384 dimensions, ~80 MB) produces good quality English-language embeddings and is fast enough for real-time search inference but we recommend finding the right model for your needs. If you require multi-language support then refer to the OpenSearch list of pre-trained models.

   Register the model using the TORCH_SCRIPT format (recommended):

   POST /_plugins/_ml/models/_register
   {
     "name": "huggingface/sentence-transformers/all-MiniLM-L6-v2",
     "version": "1.0.1",
     "model_group_id": "<model_group_id>",
     "model_format": "TORCH_SCRIPT"
   }

   Both TORCH_SCRIPT and ONNX formats are supported. TORCH_SCRIPT is the default and works on all supported platforms. ONNX can be faster on some hardware configurations. Check the pre-trained models page for further details.

   The register call is asynchronous and returns a task_id. Poll until the task completes:

   GET /_plugins/_ml/tasks/<task_id>
   
   // Expected response when complete:
   {
     "model_id": "cleMb4kBJ1eYAeTMFFg4",
     "task_type": "REGISTER_MODEL",
     "function_name": "TEXT_EMBEDDING",
     "state": "COMPLETED",
     ...
   }

   Note the model_id which will be used in the next step.

3. Deploy the model - Deploying the model reads the registered model chunks from the OpenSearch model index and loads the model into memory on the ML-eligible nodes.

   POST /_plugins/_ml/models/<model_id>/_deploy

   The register call is asynchronous and returns a task_id. Poll until the task completes:

   GET /_plugins/_ml/tasks/<task_id>
   
   // Expected response when complete:
   {
     "task_type": "DEPLOY_MODEL",
     "state": "COMPLETED",
     ...
   }

   Deployment can take up to 120 seconds depending on model size and hardware. If the task enters a FAILED state, check that the ML nodes have sufficient native memory. The most common cause is the native memory circuit breaker triggering.

   As per the OpenSearch documentation:

   “If a cluster or node is restarted, then you need to redeploy the model. To learn how to set up automatic redeployment, see Enable auto redeploy.”

4. Test the model - Verify the model generates embeddings correctly before configuring Confluence DC:

   POST /_plugins/_ml/models/<model_id>/_predict
   {
     "text_docs": ["Confluence semantic search test"],
     "return_number": true,
     "target_response": ["sentence_embedding"]
   }

   A successful response will contain a sentence_embedding array with 384 float values (for all-MiniLM-L6-v2):

   {
     "inference_results": [
       {
         "output": [
           {
             "name": "sentence_embedding",
             "data_type": "FLOAT32",
             "shape": [384],
             "data": [-0.023315024, 0.08975691, 0.078479774, ...]
           }
         ]
       }
     ]
   }

   Once confirmed working, record the model_id — this is what you will supply to Confluence DC when enabling Semantic Search.

Step 4: Configure Confluence for semantic search

1. Configure confluence.cfg.xml:
   Add the following property to confluence.cfg.xml on each Confluence DC node.

   <property name="opensearch.vector.model.id"><model_id></property>

   Replace <model_id> with the model ID recorded from Step 3.2 (e.g. cleMb4kBJ1eYAeTMFFg4). Once the changes are in place, restart the nodes.

   This will not affect your existing search and indexing, until after step 4.2 (reindex) has been actioned.

2. Reindex
   Navigate to Administration → Content Indexing and perform a full reindex of the Confluence site. Note that this will not require downtime as OpenSearch currently provides a Blue/Green reindexing mechanism which avoids downtime.

   Reindexing with an ML inference pipeline configured is considerably more resource-intensive than a standard reindex, because every document must pass through model inference to regenerate its embeddings before being written making the operation bound by inference latency rather than I/O, with slower throughput, higher CPU and memory, and additional storage from the generated vectors.

   Once the reindex completes, you can confirm that semantic search is active by opening the quick search UI (press / or click the search bar). You should see a Semantic search toggle on the top of the left-hand panel:
   

Optional - Rollback

To disable Semantic Search and revert to keyword-only search:

1. Remove (or comment out) the opensearch.vector.model.id property from confluence.cfg.xml on every node.

2. Perform a rolling restart of all Confluence DC nodes.

3. Trigger a full reindex. This will re-index content without the embedding pipeline, effectively clearing the semantic search configuration from active use.

Tuning search result quality

The quality of semantic search results depends heavily on the embedding model you choose and the shape of your content. Confluence provides configuration properties that let you adjust result quality. The right values depend on your model's score range and your content. Use the guidance below as a starting point, and iterate based on what you observe in your environment.

Understanding the problem: weak matches and recency boost

Semantic search can sometimes return results that appear unrelated to the search query. This typically happens because:

• Weak semantic matches are still returned. The neural query returns the closest vector matches it can find, even when none of them are particularly similar to the query. These "weak matches" have low relevance scores, but they are still included in the result set.

• Recency scoring amplifies weak matches. By default, Confluence boosts search results based on how recently pages were modified. A weakly matching page that was edited recently can end up ranked above a strongly matching page that hasn't been touched in a while — producing results that feel irrelevant.

There are two complementary approaches to address this, and you may choose to apply one or both.

Approach 1: Filter out weak matches

Use the k, min_score, and max_distance properties for title and contentBody fields to control which results are included in the first place. See Recognized properties below for the full list of available properties and their defaults.

• min_score — Set a minimum similarity score threshold. Results below this threshold are excluded entirely. This is generally the most intuitive option when your model produces scores in a predictable range.

• max_distance — Set a maximum vector distance threshold. Results beyond this distance are excluded. Useful when working with distance-based space types such as l2.

• k — Limit the number of nearest-neighbour candidates returned by the neural query. A lower value reduces the chance of weak matches being included.

Approach 2: Disable recency scoring

Set search.scoring.recency to false in confluence.cfg.xml. This disables the time-decay boost applied to search results, so that results are ranked purely by their relevance score rather than how recently they were edited.

This is a good option when:

• You want strong semantic matches to always appear at the top, regardless of when the page was last modified.

• You have not yet identified the right min_score / max_distance threshold for your model and want to improve result ordering while you experiment.

Disabling recency scoring does not remove weak matches from results, it only changes their ranking. Combining this with min_score or max_distance filtering gives you the most control.

認識済みのプロパティ

These properties can be set in confluence.cfg.xml on each Confluence node. For example:

<property name="search.scoring.recency">false</property>
<property name="search.semantic.contentBody.neural.min_score">0.5</property>