How To Estimate FishEye Repository Index

お困りですか?

アトラシアン コミュニティをご利用ください。

コミュニティに質問

プラットフォームについて: Server および Data Center のみ。この記事は、Server および Data Center プラットフォームのアトラシアン製品にのみ適用されます。

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Fisheye および Crucible は除く

目的

The purpose of this article is to provide you with steps to estimate the length of time it will take to index your FishEye repository.

This page is based on getting estimates using Git.


Step 1 - Identify Repository Size

The most important factors that determine slurp time are the total number of commits (reachable both from remote heads and tags).

Run the following commands to retrieve the corresponding values:

  • total commits reachable
  • number of paths changed in every commit
  • number of tags
  • number of paths for the most recent tag

$ git rev-list --all --count
1000000
$ git show-ref --heads --no-abbrev | wc -l
       0
$ git show-ref -d --tags | wc -l
    6000
$ git ls-tree -r origin/<recent_tag> | wc -l
   50000

Step 2 - Identify the Number of Commits Fisheye Can Process Per Day

(info) This information is only available if DEBUG logging is enabled on Fisheye/Crucible. 

To enable debug logging

  •  Go to Administration > Global settings > Server
  •  Under Debug logging, click on Turn debug logging ON

Retrieve the FishEye logs and execute the following command:

grep "<- Processing" atlassian-fisheye-YYYY-MM-DD.log | awk 'BEGIN {total = 0}{total += $15}END{printf("%d commits processed in %dms total (%.1fms avg)\n", NR, total, total / NR);}'
1000000 commits processed in 75525706ms total (42.9ms avg)

Of which, we look for the unique commits that were processed:

grep "<- Processing" atlassian-fisheye-YYYY-MM-DD.log | awk '{print $13}' | sort | uniq | wc -l
   50000

Step 3 - Calculate Estimate

From the above, we now know that we have 1 million commits within our repository. Of those, only 50 thousand were able to be processed in a single day.  This is 5% of the total number of commits we need to be indexed. This means that it would take approximately 20 days to index this repository. 

Bonus - Check (Git) Index Progress

Fisheye doesn't provide any indication of a Git repository indexing progress via its UI, but one can hit a REST endpoint that returns a number of commits processed so far:
<FECRU_URL>/rest-service-fecru/admin/repositories-v1/<REPOSITORY_KEY> 
E.g.:

{
"name": "processor-sdk-linux-local",
"displayName": "processor-sdk-linux-local",
"state": "RUNNING",
"enabled": true,
"indexingStatus": {
"linesOfContentIndexingInProgress": false,
"indexingStateCounts": {
"INFILLED": 0,
"METADATA_INDEXED": 0,
"COMPLETE": 0,
"SCANNED": 0,
"UNKNOWN": 33958,
"CONTENT_INDEXED": 0
},
"initialScanningComplete": false,
"fullRepositorySlurpDone": false,
"incrementalIndexingInProgress": false,
"message": "Processing commit dc335d9735220b3a9ece5ec2d95864b1e8ff06a0",
"error": false,
"fullIndexingInProgress": true,
"crossRepositoryRescanInProgress": false
}
}


The number we're interested in is indexingStatus.indexingStateCounts.UNKNOWN and can be compared to a total number of commits in the repo (git rev-list --all --count) to get a rough estimate of indexing progress.


Last modified on Mar 5, 2024

この内容はお役に立ちましたか?

はい
いいえ
この記事についてのフィードバックを送信する
Powered by Confluence and Scroll Viewport.