"journal state store is corrupt" エラーによりインデックスが動作しない
問題
検索に最近のページが含まれず、最近の更新マクロが操作しない。
atlassian-confluence.log
に次のメッセージが表示される。
WARN [http-bio-8090-exec-3] [confluence.impl.journal.DefaultJournalManager] enqueue Newly enqueued entry in journal
[main_index] has an ID [X] that should have been higher than the journal state store's most-recent-id [Y]. it is
likely that this node's journal state store is corrupt.
原因
journal は Confluence での変更を追跡するために使用されます。これは、各ノードがこのログを約 5 秒間隔でチェックし、自身のインデックスを適宜更新する、Data Center インストレーションで特に便利です。
This issue occurs when there is an entry in the journal with an ID higher than what the journal is expecting. This can be due to a corruption of the Confluence index itself. Or, it can be an issue with your database setup.
Alternatively, if you are using an Oracle RAC database, you may be encountering this bug: - CONF-35764Getting issue details... STATUS
ソリューション 1
- Confluence を終了します (Data Center の場合はすべてのノードを終了)。
<CONF_HOME_Dir>/
にあるjournal
ディレクトリを移動するかリネームします。- コンテンツ インデックスをゼロから再構築します (すべてのノード)。
- Confluence を再起動します。
- Rebuild the index from the Administration menu
Resolution 2: Change the next value for "seq_journal_entry_id" in PostgreSQL
In some cases, especially when migrating from one database to another the next sequence number can be lower than the last ID that is used for the index in the <confluence-home>/journal/main_index file. In that case, we should update the next sequence number to a value that is greater than the number in <confluence-home>/journal/main_index file.
Check the last value for 'seq_journal_entry_id' by using the below SQL query:
SELECT last_value FROM pg_catalog.pg_sequences WHERE sequencename='seq_journal_entry_id';
If the above value is lower than the value in <confluence-home>/journal/main_index file, follow the below steps:
- Use the below query to ALTER the related view. Try to give a value (NEW_VALUE) that is higher than the value <confluence-home>/journal/main_index file. Preferably the value in main_index file +100.
ALTER SEQUENCE seq_journal_entry_id RESTART WITH <NEW_VALUE>;
- Rebuild the index from the Administration menu.
ソリューション 3: Oracle RAC
Follow the workaround from - CONF-35764Getting issue details... STATUS
- SEQ_JOURNAL_ENTRY_ID シーケンスを並べ替えるように変更します:
ALTER SEQUENCE SEQ_JOURNAL_ENTRY_ID ORDER;
- インデックスを再構築します。
Resolution 4: Postgres Sequence Cache
Postgres does not sequentially create IDs by default, so a sequence needs to be set up in the DB. Postgres should create IDs 1 at a time to make sure they are sequential. If your DB was somehow changed to create more than 1 at a time, it can start to create these non-sequentially.
To Confirm:
Enable DEBUG logging in Confluence on the following packages:
com.atlassian.bonnie.search.extractor com.atlassian.confluence.search.lucene com.atlassian.confluence.internal.index.AbstractBatchIndexer com.atlassian.confluence.impl.journal
- Make a few changes in confluence, such as editing several pages.
Check the logs. You should see each change logged with a journal entry ID:
2019-07-29 07:40:46,967 DEBUG [http-nio-127.0.0.1-8090-exec-5] [confluence.impl.journal.DefaultJournalManager] enqueue Enqueued JournalEntry: JournalEntry{id=5936925, journalId='JournalIdentifier{journalName=edge_index}', creationDate=Mon Jul 29 07:40:46 MDT 2019, type='ADD_DOCUMENT', message='{"edgeId":"162070548","userKey":"1c0c8294582c611c015847a7c66e0032","targetId":162070548,"date":1564407548478,"typeKey":"page.create"}'} 2019-07-29 07:40:51,108 DEBUG [http-nio-127.0.0.1-8090-exec-8] [confluence.impl.journal.DefaultJournalManager] enqueue Enqueued JournalEntry: JournalEntry{id=5936863, journalId='JournalIdentifier{journalName=main_index}', creationDate=Mon Jul 29 07:40:51 MDT 2019, type='ADD_CHANGE_DOCUMENT', message='com.atlassian.confluence.pages.Page-162070546'}
- In the logs above, you can see the first entry was given an ID of 5936925, and the second, 5936863. This is not only non-sequential, but a big jump.
Check the sequence in Postgres by exporting the schema/generating a DDL. Or run this command:
Postgres versions below 10SELECT cache_value FROM seq_journal_entry_id;
Postgres 10 and aboveSELECT cache_size FROM pg_catalog.pg_sequences WHERE sequencename='seq_journal_entry_id';
The sequence should look like this:
CREATE SEQUENCE public.seq_journal_entry_id START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;
- If the CACHE value is anything other than 1, Postgres may generate IDs in the wrong order.
To Fix:
- Confluence をシャットダウンします。
Run the following query to update the Postgres Sequence:
ALTER SEQUENCE public.seq_journal_entry_id CACHE 1;
- Re-enable the DEBUG logging mentioned above
- Verify that journal IDs are now created in sequential order.