要約

Elasticsearch fails to index content from Bitbucket.

環境

• Bitbucket Data Center 7.7.1 with one node only
• Bundled Elasticsearch 7.5

診断

This error is found on the search logs:

[2020-01-06T11:05:13,248][WARN ][o.e.m.j.JvmGcMonitorService] [bitbucket_bundled][gc][6577647] overhead, spent [1.5s] collecting in the last [2.3s]
[2020-01-06T11:05:17,287][WARN ][o.e.m.j.JvmGcMonitorService] [bitbucket_bundled][gc][6577649] overhead, spent [2.7s] collecting in the last [3s]
[2020-01-06T11:06:27,699][WARN ][o.e.m.j.JvmGcMonitorService] [bitbucket_bundled][gc][young][6577718][7632] duration [2.2s], collections [1]/[2.3s], total [2.2s]/[5.7m], memory [2.7gb]->[1.8gb]/[3.8gb], all_pools {[young] [969.4mb]->[1.5mb]/[1gb]}{[survivor] [89mb]->[136.4mb]/[136.5mb]}{[old] [1.7gb]->[1.7gb]/[2.6gb]}

原因

This issue is caused because Elasticsearch heap memory is not properly sized to the size of the content available to index. The messages above show that the JVM is running continuous garbage collection cycles cleaning objects from memory, due to the small heap size allocated. During those GC cycles, the JVM will pause making Elasticsearch not responsive to index content.

ソリューション

Increase JVM heap memory available for Elasticsearch.

1. For the bundled version of Elasticsearch edit the <BITBUCKET_HOME>/search/config/jvm.option and change the following properties from:

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

変更後:

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms2g
-Xmx2g

2. Restart Bitbucket.

 We recommend increasing the heap in 1GB increments until the overhead messages are not shown anymore on the search logs.