Shared file system: migrate to remote OpenSearch

前提条件

If you don’t have the bundled search password, reset the credentials. This will allow you to perform REST API calls on the OpenSearch endpoints.

Step 1: Take a snapshot of the bundled search

1. To define a path.repo for bundled search, add the following to the $BITBUCKET_HOME/shared/search/config/opensearch.yml file. This can be any path as long as atlbitbucket has access to it, and the path has enough disk space.
   

   path.repo: ['${BITBUCKET_HOME}/shared/search/repositories']

2. To load the modified opensearch.yml, restart the bundled search by restarting Bitbucket.
   

   cd $BITBUCKET_INST/bin
   ./stop-bitbucket.sh
   ./start-bitbucket.sh

3. Create the snapshot repository.
   1. Check what you have defined as path.repo in the cluster settings. This must not be empty since this is where the snapshots will be stored.
      

      curl -u bitbucket -X GET "http://localhost:7992/_cluster/settings?include_defaults=true&filter_path=*.path.repo"

   2. Create the snapshot repository at ${BITBUCKET_HOME}/shared/search/repositories/my-fs-repository.
      

      curl -u bitbucket -X PUT "http://localhost:7992/_snapshot/my-fs-repository" -H 'Content-Type: application/json' -d'
      {
        "type": "fs",
        "settings": {
          "location": "my-fs-repository"
        }
      }
      '

      
      
   3. Check if the snapshot repository was created.
      

      curl -u bitbucket -X GET http://localhost:7992/_snapshot/my-fs-repository

4. Take a snapshot.
   1. We will only include the indices with a "bitbucket-" prefix.
      

      curl -u bitbucket -X PUT "http://localhost:7992/_snapshot/my-fs-repository/bundled-search-snapshot-20250728" -H 'Content-Type: application/json' -d'{
        "indices": "bitbucket-*",
        "ignore_unavailable": true,
        "include_global_state": true,
        "metadata": {
          "taken_by": "user",
          "taken_because": "To move to remote OpenSearch"
        }
      }'

   2. Check if the snapshot was created. Run this to monitor the progress of the snapshot creation until the state turns from IN_PROGRESS to SUCCESS.
      

      curl -u bitbucket -X GET http://localhost:7992/_snapshot/my-fs-repository/_all

Step 2: Install and configure a remote OpenSearch server

Complete steps 1 to 3 from the Install and configure a remote OpenSearch server page, and then return to this instruction.

Step 3: Restore the snapshot on the remote search server

1. On the remote OpenSearch server, define the path.repo by passing it when launching the container. Alternatively, modify the opensearch.yml and restart OpenSearch.
   

   docker run -d -p 7992:9200 -p 9600:9600 \
   --env discovery.type=single-node \
   --env "OPENSEARCH_INITIAL_ADMIN_PASSWORD=RemoteOpenSearch#123" \
   --env "OPENSEARCH_JAVA_OPTS"="-Xms2000m -Xmx2000m" \
   --env "path.repo=/usr/share/opensearch/data/repositories" \
   --env "plugins.security.ssl.http.enabled=false" \
   --name bbs-opensearch \
   --pull missing \
   -v /mnt/opensearch-data:/usr/share/opensearch/data \
   opensearchproject/opensearch:2.19.1

2. Copy the snapshot repository files from the Bitbucket host to the path.repo of the remote search server.
   

   cp -R $BITBUCKET_HOME/shared/search/repositories/* /mnt/opensearch-data/repositories/

3. Register the copied repository.
   1. We specify location as my-fs-repository since the file system would have /mnt/opensearch-data/repositories/my-fs-repository.
      

      curl -u admin -X PUT "$REMOTE_OPENSEARCH_URL/_snapshot/my-fs-repository" -H 'Content-Type: application/json' -d'{
        "type": "fs",
        "settings": {
          "location": "my-fs-repository"
        }
      }'

      
      
   2. Check if the snapshot repository was created.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-fs-repository

      
      
   3. Check if the snapshot is present. It should return the bundled-search-snapshot-<creation-date>.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-fs-repository/bundled-search-snapshot-20250728
      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-fs-repository/_all

      
      
4. Delete any bitbucket-* indexes on the remote OpenSearch.
   1. Check to confirm no bitbucket-* indexes are present.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_cat/indices?v

      
      
   2. If there are bitbucket-* indices, run the DELETE call below.
      

      Make sure this call targets the remote OpenSearch, and not the bundled search, because this will wipe out the Bitbucket indexes.

      
      

      curl -u admin -X DELETE "$REMOTE_OPENSEARCH_URL/bitbucket-*"

      
      
5. Restore the snapshot.
   1. Restore the snapshot that only contains bitbucket-* indexes.
      

      curl -u admin -X POST "$REMOTE_OPENSEARCH_URL/_snapshot/my-fs-repository/bundled-search-snapshot-20250728/_restore" -H 'Content-Type: application/json' -d'{
          "indices": "bitbucket-*",
          "include_global_state": false
      }'

      
      
   2. Check recovery progress. It will return {} if the restore has completed.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_recovery?active_only=true

      
      
   3. Check the restored indexes if their sizes are as expected.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_cat/indices?v

Step 4: Connect Bitbucket to remote OpenSearch

1. Restart Bitbucket, but this time, don’t start the bundled search.
   

   cd $BITBUCKET_INST/bin
   ./stop-bitbucket.sh
   ./start-bitbucket.sh --no-search

2. Go to server settings:

   • for Bitbucket 9.5 or earlier, $BITBUCKET_BASE_URL/admin/server-settings 
   • for Bitbucket 9.6 or later, $BITBUCKET_BASE_URL/plugins/servlet/search-server 
3. Update the search server URL with the $REMOTE_OPENSEARCH_URL.
4. Update the search server credentials.
5. Run a test connection to verify it’s working, and then save the changes.

Bitbucket will run an incremental sync once it detects that the search server URL has changed  (similar to running a POST /rest/indexing/latest/sync).

Amazon S3: migrate to remote OpenSearch

Follow these steps if you don’t have access to the file system of the remote OpenSearch server. These steps describe how to use an Amazon S3 bucket as a snapshot repository.

Step 1: Take a snapshot of the bundled search

1. Create an Amazon S3 bucket.
2. Retrieve the AWS Access and Secret keys that have access to the S3 bucket as described in Update access keys.
3. Install the repository-s3 plugin. This will create the  $BITBUCKET_INST/opensearch/plugins/repository-s3 path that contains the repository-s3-<opensearch-version>.jar and other dependencies of this plugin.
   

   export BITBUCKET_HOME=/Users/aescultura/bitbucket-home/atlassian-bitbucket-9.6.5
   cd $BITBUCKET_INST/opensearch/bin
   sudo -E ./opensearch-plugin install repository-s3

4. Add the AWS access and secret keys to the OpenSearch keystore.
   

   export BITBUCKET_HOME=/Users/aescultura/bitbucket-home/atlassian-bitbucket-9.6.5
   cd $BITBUCKET_INST/opensearch/bin
   sudo -E ./opensearch-keystore add s3.client.default.access_key
   # Prompt:
   # The opensearch keystore does not exist. Do you want to create it? [y/N]y
   # Enter value for s3.client.default.access_key:
   sudo -E ./opensearch-keystore add s3.client.default.secret_key
   # Prompt:
   # Enter value for s3.client.default.secret_key:

   
   
5. Copy the updated keystore in $BITBUCKET_INST/opensearch/config/opensearch.keystore to $BITBUCKET_HOME/shared/search/config/opensearch.keystore.
   

   cp $BITBUCKET_INST/opensearch/config/opensearch.keystore $BITBUCKET_HOME/shared/search/config/opensearch.keystore

   
   
6. Restart Bitbucket so that the bundled search is aware of the changes to the S3 type snapshot and to the keystore.
   

   ./stop-bitbucket.sh
   ./start-bitbucket.sh

   
   
7. Register the snapshot repository using the OpenSearch REST API.
   1. This command creates the snapshot repository with arn:aws:s3:::bb-opensearch-snapshot. In this example, the S3 bucket was created in the ap-southeast-1 region, so we need to provide the correct region in the json body.
      

      curl -u bitbucket -X PUT "http://localhost:7992/_snapshot/my-s3-repository" -H 'Content-Type: application/json' -d'
      {
        "type": "s3",
        "settings": {
          "bucket": "bb-opensearch-snapshot",
          "base_path": "bundled-search-snapshot",
          "region": "ap-southeast-1"
        }
      }
      '

      
      
   2. Check if the snapshot repository was created.
      

      curl -u bitbucket -X GET http://localhost:7992/_snapshot/my-s3-repository

      
      
8. Take a snapshot.
   1. We will only include the indices with a "bitbucket-" prefix.
      

      curl -u bitbucket -X PUT "http://localhost:7992/_snapshot/my-s3-repository/bundled-search-snapshot-20250828" -H 'Content-Type: application/json' -d'{
        "indices": "bitbucket-*",
        "ignore_unavailable": true,
        "include_global_state": true,
        "metadata": {
          "taken_by": "user",
          "taken_because": "To move to remote OpenSearch"
        }
      }'

      
      
   2. Check if the snapshot was created. Run this to monitor the progress of the snapshot creation until the state turns from IN_PROGRESS to SUCCESS.
      

      curl -u bitbucket -X GET http://localhost:7992/_snapshot/my-s3-repository/_all

      
      
   3. To see if the snapshot was created, check the S3 bucket in the AWS Console.

Step 2: Install and configure a remote OpenSearch server

Complete steps 1 to 3 from the Install and configure a remote OpenSearch server page, and then return to this instruction.

Step 3: Restore the snapshot on the remote search server

1. Get the OpenSearch version of the bundled search from the version.number in the response. We will use this in the next step.
   

   curl -u bitbucket -X GET "http://localhost:7992"

2. Create a Dockerfile with the contents below. This will create an OpenSearch image that has the repository-s3 plugin installed. Note that the remote OpenSearch version must match the bundled search version to restore snapshots.
   

   FROM opensearchproject/opensearch:2.19.1
   RUN /usr/share/opensearch/bin/opensearch-plugin install --batch repository-s3
   RUN /usr/share/opensearch/bin/opensearch-keystore create

3. Create a Docker image using the Dockerfile.
   

   cd /path/to/Dockerfile

   Use the following command to create an image named opensearch-219-with-custom-plugin.

   docker build -t opensearch-219-with-custom-plugin .

4. Launch the container using the custom Docker image. Refer to Install and configure a remote OpenSearch server if you need to configure anything else for your OpenSearch server.
   

   docker run -d -p 7992:9200 -p 9600:9600 \
   --env discovery.type=single-node \
   --env "OPENSEARCH_INITIAL_ADMIN_PASSWORD=RemoteOpenSearch#123" \
   --env "OPENSEARCH_JAVA_OPTS"="-Xms2000m -Xmx2000m" \
   --env "path.repo=/usr/share/opensearch/data/repositories" \
   --env "plugins.security.ssl.http.enabled=false" \
   --env AWS_ACCESS_KEY_ID=<your-s3-access-key> \
   --env AWS_SECRET_ACCESS_KEY=<your-s3-secret-key> \
   --name bbs-opensearch \
   --pull missing \
   opensearch-219-with-custom-plugin

   
   
5. Update the keystore with the AWS Access and Secret keys.
   

   docker exec bbs-opensearch bash -c \
     'echo $AWS_ACCESS_KEY_ID | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.access_key'
   docker exec bbs-opensearch bash -c \
     'echo $AWS_SECRET_ACCESS_KEY | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.secret_key'

   
   
6. Restart the container to reflect the keystore changes.
   

   docker restart bbs-opensearch

   
   
7. Create the snapshot repository.
   1. Create the snapshot repository, pointing it to the same S3 bucket that the bundled search used.
      

      curl -u admin -X PUT "$REMOTE_OPENSEARCH_URL/_snapshot/my-s3-repository" -H 'Content-Type: application/json' -d'
      {
        "type": "s3",
        "settings": {
          "bucket": "bb-opensearch-snapshot",
          "base_path": "bundled-search-snapshot",
          "region": "ap-southeast-1"
        }
      }
      '

      
      
   2. Check if the snapshot repository was created.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-s3-repository

      
      
   3. Check if the snapshot is present. It should return the bundled-search-snapshot-<creation-date>.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-s3-repository/bundled-search-snapshot-20250828
      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_snapshot/my-s3-repository/_all

      
      
8. Delete any bitbucket-* indexes on the remote OpenSearch.
   1. Check to confirm no bitbucket-* indexes are present.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_cat/indices?v

      
      
   2. If there are bitbucket-* indices, run the DELETE call below.
      

      Make sure this call targets the remote OpenSearch and not the bundled search, as this will wipe out the Bitbucket indexes.

      curl -u admin -X DELETE "$REMOTE_OPENSEARCH_URL/bitbucket-*"

      
      
9. Restore the snapshot.
   1. Restore the snapshot that only contains bitbucket-* indexes.
      

      curl -u admin -X POST "$REMOTE_OPENSEARCH_URL/_snapshot/my-s3-repository/bundled-search-snapshot-20250828/_restore" -H 'Content-Type: application/json' -d'{
          "indices": "bitbucket-*",
          "include_global_state": false
      }'

      
      
   2. Check recovery progress. It will return {} if the restore has completed.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_recovery?active_only=true

      
      
   3. Check the restored indexes if their sizes are as expected.
      

      curl -u admin -X GET $REMOTE_OPENSEARCH_URL/_cat/indices?v

Step 4: Connect Bitbucket to remote OpenSearch

1. Restart Bitbucket, but this time, don’t start the bundled search.
   

   cd $BITBUCKET_INST/bin
   ./stop-bitbucket.sh
   ./start-bitbucket.sh --no-search

2. Go to server settings:

   • for Bitbucket 9.5 or earlier, $BITBUCKET_BASE_URL/admin/server-settings
   • for Bitbucket 9.6 or later, $BITBUCKET_BASE_URL/plugins/servlet/search-server
3. Update the Search server URL with the $REMOTE_OPENSEARCH_URL.
4. Update the search server credentials.
5. Run a test connection to verify it’s working, and then save the changes.

Bitbucket will run an incremental sync once it detects that the search server URL has changed  (similar to doing a POST /rest/indexing/latest/sync).