Troubleshooting SVN Indexing Related Issues in Fisheye/Crucible


アトラシアン コミュニティをご利用ください。


This page lists the troubleshooting tips required to investigate some common SVN issues reported while indexing a repository in Fisheye/Crucible.

Recommendations for SVN integration

Best Practices

SVN Client
Latency of SVN communication
  • Fisheye requires detailed information for every commit in configured SVN repository. The faster it can talk to SVN, the faster indexing will proceed.
  • The http/s protocol has the slowest performance during the initial scan. The svn protocol (svn://) is faster and the file protocol (file:///) is the fastest.
  • Therefore if you find your initial scan takes an extended amount of time (more than a day or two), you should consider switching over from the http/s protocol to the svn or file protocol to define the location of your SVN repository. (Use svnsync to mirror the repository onto the fisheye server, so that you can access it with the file protocol.). In order for SVN protocol to work you need to have set up an svnserve based server.
SVN Branch/Tag Structure
  • Ensure the SVN branch and tag structure has been selected correctly in Fisheye repository configuration. If Fisheye does not know which files are tags and branches, it will treat all files as trunk files. This can significantly increase the effective size of your repository and the initial scan time as well as impact runtime performance. This guide has a number of examples to help setup correct SVN branch and tag mapping in Fisheye.

Fisheye Index Structure
  • Fisheye is a caching application. It caches all the repository information. Therefore, big repositories do take time to be indexed.

    This index/repository cache is stored in the var directory defined in the Fisheye configuration, under a subdirectory named var/cache/name of the repository. For example, the indexes for the repository FE are under var/cache/FE, with one subdirectory for each index:

    • Metadata index is stored in var/cache/FE/idx0
    • Content index is stored in var/cache/FE/idx1
    • DiffText index is stored in var/cache/FE/idx2
Limit Indexing Scope
  • You can limit the repository content that Fisheye will index by either:
    • Setting a start revision to start indexing from a particular revision.
    • Exclude certain file types/paths that are not required to be indexed.
    • Turn OFF the store-diff info property. Some of the benefits are:
      • Speed up indexing; Fisheye won't have to retrieve and index all diffs.
      • Reduce disk footprint. The diff index files can become quite large, so disabling diffs can lead to significant savings.
      • Reduce disk I/O, particularly write operations. If disk I/O is a performance bottleneck, disabling diff indexing could help.
      • Reduce the load on your SVN server during indexing, but you'll only notice the difference if you've got a lot of commit activity going on.
Indexing Thread Configuration

       Fisheye uses one thread each for incremental and initial indexing for each repository. 3 is a recommended thread count. This means a maximum of 3 repositories will be processed concurrently for full re-indexing, and a maximum of 3 repositories will be processed concurrently for incremental indexing (the periodic indexing that occurs to keep Fisheye up to date with repository contents). Other repositories will be queued. Having more number of threads configured will add extra overhead on server when indexing and negatively impact performance.

Amount of heap allocated to the Fisheye instance
  • The more indexing threads you allocate, the more heap (memory) that Fisheye will consume. Without sufficient memory, the Fisheye instance will spend a lot of time garbage collecting.
Number of processor cores available to Fisheye
  • The more processors available to Fisheye, the faster indexing will go. Please refer to the hardware requirements for more details.
Block size for SVN indexing
  • This setting determines the number of changesets that are processed at a time. A larger number reduces the number of calls to SVN, but consumes more memory. A lower number increases the number of calls to SVN, but reduces memory consumption.


Debugging Slow Indexing

Command Line Tool
  • Monitor the size of your <FISHEYE_INST>/var/cache/name of the repository folder on disk as this should be similar to your SVN repository on the server. If there is a huge size mismatch, that is usually an indication of incorrect SVN branch and tag mapping in Fisheye. If there is a huge mismatch, you may use a command line tool shipped with the latest version of Fisheye, that can be downloaded from here. Please note that this does not require you to upgrade to use it, as this is command line tool.  You can simply run it against your current version.  To run the tool:
    •   Download the latest Fisheye version
    •   Unpack it to some folder (e.g. /tmp/fecru-2.10.1)
    •   Go into that folder cd /tmp/fecru-2.10.1
    •   Point FISHEYE_INST environment variable to the folder where your current instance home is, e.g. export FISHEYE_INST=/var/fisheye
    •   Run svn symbolic rule analyser on your repository, with a command ./bin/ svnrules <Repo Name> | tee <Repo Name>-svn-rules.log

This command may take a while as it would run svn log -v against the specified repository, but it would attempt to analyse all the copy operations in your repository and would advice where it is most likely that there is a missing svn symbolic rule definition.  Once this has run see the end of the generated <Repo Name>-svn-rules.log file, it would contain summary of the 10 most common folders.
You don't need to stop your current instance to perform this check.  The generated output would help to determine what the missing svn symbolic rules are.  But please bear in mind the tool may report false positives, as sometimes it is genuine activity in the repository to copy some files or folders within the trunk. 

Debug Logs
  • With debug logging enabled, look for log entries beginning with [IncrementalPinger<Thread No.> <Repository Name>]. and [InitialPinger<Thread No.> <Repository Name>] . These entries correspond to indexing related tasks currently running in the application and the Thread No. corresponds to which thread is being used to complete the task on that repository. Using the timestamp on the log, you can then investigate the last time that Fisheye ran an incremental/initial index on a repository.
  • Debug logs will also report the exact commands being run on the repository at the time of initial and incremental indexing, for example:
2013-04-08 14:05:45,971 DEBUG [IncrementalPinger2 git_tutorial] LoggingProcessMonitor-onBeforeStart - Starting process: git fetch /Users/ganand/Git Tutorial/git_tutorial/work/hello/ +refs/*:refs/* 
2013-04-08 14:05:46,006 DEBUG [IncrementalPinger2 git_tutorial] LoggingProcessMonitor-onAfterFinished - Finished process: git fetch /Users/ganand/Git Tutorial/git_tutorial/work/hello/ +refs/*:refs/*  took 33ms

In the above example, the "Starting process" and "Finished process" are the messages used to track the start and end of a command and how long did it take to complete - "took 33ms" in this case. This can be helpful in identifying some long running commands or if they do not complete at all when there are no corresponding "Finished process" log entries. You may then run the same commands via the command line directly on your server and compare the results.

  • Logs will also lodge the time taken to index a particular repository, for example:
2013-04-08 14:06:05,833 INFO  [InitialPinger3 git] fisheye.console BaseRepositoryScanner-ping - [git] starting initial scan of repository
2013-04-08 14:06:11,817 INFO  [InitialPinger3 git] fisheye.console BaseRepositoryScanner-ping - [git] finished initial scan of repository in 5s
  • With debug logging enabled, you can also look for the exact command that Fisheye is failing at while indexing, for example:
2013-04-08 14:28:40,272 DEBUG [InitialPinger3 svn] SvnThrottledClient-start - Executing ls -r HEAD https://URL/public/@HEAD
2013-04-08 14:28:41,834 WARN  [InitialPinger3 svn] SvnRepositoryTester-getServerRootURL - Unable to get Repository Root URL for svn:https://URL/public/
2013-04-08 14:28:41,834 DEBUG [InitialPinger3 svn] SvnRepositoryTester-pingAndValidateAccess - Validating access to https://URL/public/
2013-04-08 14:28:41,834 DEBUG [InitialPinger3 svn] SvnThrottledClient-start - Executing propget fisheye.access -r HEAD https://URL/public/@HEAD
2013-04-08 14:28:43,124 DEBUG [InitialPinger3 svn] RepositoryStatus-setMessage - Status change [svn]: Could not access https://URL/public/
2013-04-08 14:28:43,124 ERROR [InitialPinger3 svn] SvnRepositoryTester-pingAndValidateAccess - Exception getting FishEye access control
com.cenqua.fisheye.rep.RepositoryClientException: org.tigris.subversion.javahl.ClientException: svn: E175011: Repository moved permanently to '/public'; please relocate
svn: E175002: PROPFIND request failed on '/public'

In the above example, the command is "svn ls -r HEAD https://URL/public/@HEAD". You can run it via the command line to check the result returned from your SVN server.

Further Resources
  • The Fisheye Knowledge base articles for SVN integration are very useful for identifying the underlying cause.
  • You may also use the bundled Hercules Log Scanner to scan your logs for known issues and get recommendations for resolution accordingly.
最終更新日 2018 年 11 月 2 日


Powered by Confluence and Scroll Viewport.