問題

Something went wrong. Contact administrator pop-up message in Insight can be seen when making one of the following actions:

• Filtering on or searching for objects in Insight 

• When triggering an Insight Reindex or when attempting to edit the configuration of an Object-referencing attribute
• When attempting to delete an Insight Schema from the UI.

The following appears in the atlassian-jira.log:

2021-05-06 14:27:55,871 insight-InsightThreadGroup-worker-thread-6 ERROR admin 863x2041x1 1gmljw3 127.0.0.1 /rest/insight/1.0/index/reindex/start [i.r.i.index.model.ObjectIndexImpl] Index not completed successfully. objectIds: ([1, 2, 5, 6, 85])
com.google.common.cache.CacheLoader$InvalidCacheLoadException: loadAll failed to return a value for 3
    at com.google.common.cache.LocalCache.getAll(LocalCache.java:4025)
    at com.google.common.cache.LocalCache$LocalLoadingCache.getAll(LocalCache.java:4972)
    at io.riada.insight.index.model.InsightIndexBase.loadAll(InsightIndexBase.java:81)
    at io.riada.insight.index.model.ObjectIndexImpl.reindexObjects(ObjectIndexImpl.java:318)
    at com.riadalabs.jira.plugins.insight.services.core.index.ReindexServiceImpl$ReindexObjectsJob.executeTask(ReindexServiceImpl.java:543)
    at com.riadalabs.jira.plugins.insight.services.core.index.ReindexServiceImpl$ReindexObjectsJob.executeTask(ReindexServiceImpl.java:520)
    at com.riadalabs.jira.plugins.insight.services.core.multithreadservice.InsightServiceJob.call(InsightServiceJob.java:42)
    at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

• 「objectIds」リストには、Insight により該当する操作でインデックス再作成が試みられたすべてのオブジェクト ID が表示されます (リストは長くなる可能性があります)。
• 「return a value for」とは、Insight で上記の問題が最初に発生したオブジェクト ID の値のことです。

診断

環境

• This problem has been detected on Insight major versions > 8.4.x

• It causes the Insight Index not completed successfully due to some corrupted objects due to some kind of race condition. This happens after operations as
  • Deletion of objects
  • Deletion/change of attributes
  • Deletion of ObjectTypes
  • Deletion of ObjectShemas

Diagnostic Steps

To diagnose this problem:

1. Review InvalidCacheLoadException: loadAll failed to return a value for xxx KB and run the SQL queries suggested there. If none rows are returned, please proceed with the following steps shared in this article.
2. You might look for the above-mentioned errors in the Jira logs and run the below query on the Jira database:

3. A row should be returned and if you have null values for all the right tables indicates that object got corrupted and needs to be deleted from your database.

   FOR POSTGRES:

   SELECT
     OAV."TEXT_VALUE",
     O."NAME",
     O."OBJECT_TYPE_ID",
     OTA."NAME",
     OA."ID",
     OA."OBJECT_TYPE_ATTRIBUTE_ID",
     OA."OBJECT_ID",
     OAV."ID",
     OAV."OBJECT_ATTRIBUTE_ID"FROM "AO_8542F1_IFJ_OBJ" O
     LEFT OUTER JOIN "AO_8542F1_IFJ_OBJ_ATTR" OA ON O."ID" = OA."OBJECT_ID"
     LEFT OUTER JOIN "AO_8542F1_IFJ_OBJ_TYPE_ATTR" OTA ON OTA."ID" = OA."OBJECT_TYPE_ATTRIBUTE_ID"
     LEFT OUTER JOIN "AO_8542F1_IFJ_OBJ_ATTR_VAL" OAV ON OA."ID" = OAV."OBJECT_ATTRIBUTE_ID"
   WHERE O."ID" = ?;

   ? needs to be replaced by the the Object ID you found after the following message "com.google.common.cache.CacheLoader$InvalidCacheLoadException: loadAll failed to return a value for X "

   You may have more than one exception so you will need to perform the same steps for each of the values. 

4. Example of corrupted object

    TEXT_VALUE |    NAME     | OBJECT_TYPE_ID | NAME | ID | OBJECT_TYPE_ATTRIBUTE_ID | OBJECT_ID | ID | OBJECT_ATTRIBUTE_ID 
   ------------+-------------+----------------+------+----+--------------------------+-----------+----+------------
   
         null  | webex        |  97 |  null    | null   |           null               |     null      |  null  |            
   (1 row)

5. Before we move to delete those objects, run the following query. It should return four standard Object Type attributes, just making sure that the problem is not on the object type level.

   SELECT * FROM "AO_8542F1_IFJ_OBJ_TYPE_ATTR" WHERE "OBJECT_TYPE_ID" = ? AND ("NAME" IN ('Key', 'Created', 'Updated') OR "LABEL"='t');

   ? needs to be replaced by the Object Type ID from the first query output

   1. Query should return exactly 4 attributes (Key, Created, Updated and the current Label), and all additional Attributes, if configured. 

      1. If none attributes are retuned, there is an issue with the Object Type and needs to be removed by following Cannot Delete Object Types on Insight Asset Management due to database constraint violation exception
      2. If the 4 attributes are returned, proceed with the Resolution steps below:

原因

Some manual database changes is required for Insight 

ソリューション

1. Jira を停止します。
2. Execute the SQL queries below in Jira database.

3. Check all inbound references pointing to the corrupt object (if any), as these may have been the initial reason for the object not deleting:

   SELECT * FROM "AO_8542F1_IFJ_OBJ_ATTR_VAL" WHERE "REFERENCED_OBJECT_ID"= ?

4. Delete these references (attribute values) if any.

   DELETE FROM "AO_8542F1_IFJ_OBJ_ATTR_VAL" WHERE "REFERENCED_OBJECT_ID"=?

5. Once the inbound references to the corrupt objects are removed, you should be able to DELETE the corrupt object correctly, using:

   DELETE FROM "AO_8542F1_IFJ_OBJ" WHERE "ID" = ?;

   ? needs to be replaced by the Object ID you found after the following message "com.google.common.cache.CacheLoader$InvalidCacheLoadException: loadAll failed to return a value for X "

   You may have more than one exception so you will need to perform the same steps for each of the values. 

6. Clean the plugin and tomcat caches by removing the contents of these directories:
   1. JIRA_HOME/plugins/.osgi-plugins/
   2. JIRA_HOME/plugins/.bundled-plugins/
   3. JIRA_INSTALL/work/
7. If your Insight is configured to "Restore Insight index from file" - please remove this file, so Insight will re-index upon start (the file should be located in the path  /var/atlassian/application-data/jira/caches/insight_indexes) OR $JIRA_HOME/caches/insight_indexes
8. Jira を起動します。