Troubleshooting SVN indexing issues in FishEye and Crucible
This page lists the troubleshooting tips required to investigate some common SVN issues reported while indexing a repository in Fisheye and Crucible.
Debugging slow indexing
Command line tool
- Monitor the size of your
<FISHEYE_INST>/var/cache/repository_namefolder on disk as this should be similar to your SVN repository on the server. If there is a huge size mismatch, that is usually an indication of incorrect SVN branch and tag mapping in Fisheye.
- If you observe a large size mismatch, you can use the 'svnrules' command line tool to identify missing svn symbolic rules. The tool is shipped with the latest version of Fisheye. For more information, see Verifying SVN symbolic rules.
This command may take a while as it runs svn log -v against the specified repository. It attempts to analyze all the copy operations in your repository and reports where it is most likely that there are missing svn symbolic rule definitions. Once the tool has finished, check the end of the generated <Repo Name>-svn-rules.log file – it contains a summary of the 10 most common folders for which symbolic rules may be missing.
You don't need to stop your current Fisheye instance to perform this check. Please note that the tool may report false positives, as sometimes it is genuine activity in the repository to copy some files or folders within the trunk.
Analyzing debug logs
- With debug logging enabled, look for log entries beginning with
. These entries correspond to indexing related tasks currently running in the application and the Thread No. corresponds to which thread is being used to complete the task on that repository. Using the timestamp on the log, you can then investigate the last time that Fisheye ran an incremental/initial index on a repository. and [
InitialPinger<Thread No.><Repository Name>]
- Debug logs will also report the exact commands being run on the repository at the time of initial and incremental indexing, for example:
2013-04-08 14:05:45,971 DEBUG [IncrementalPinger2 git_tutorial] fisheye.app LoggingProcessMonitor-onBeforeStart - Starting process: git fetch /Users/ganand/Git Tutorial/git_tutorial/work/hello/ +refs/*:refs/* . . . 2013-04-08 14:05:46,006 DEBUG [IncrementalPinger2 git_tutorial] fisheye.app LoggingProcessMonitor-onAfterFinished - Finished process: git fetch /Users/ganand/Git Tutorial/git_tutorial/work/hello/ +refs/*:refs/* took 33ms
In the above example, the "Starting process" and "Finished process" are the messages used to track the start and end of a command and how long did it take to complete - "took 33ms" in this case. This can be helpful in identifying some long running commands or if they do not complete at all when there are no corresponding "Finished process" log entries. You may then run the same commands via the command line directly on your server and compare the results.
- Logs will also lodge the time taken to index a particular repository, for example:
2013-04-08 14:06:05,833 INFO [InitialPinger3 git] fisheye.console BaseRepositoryScanner-ping - [git] starting initial scan of repository . . . 2013-04-08 14:06:11,817 INFO [InitialPinger3 git] fisheye.console BaseRepositoryScanner-ping - [git] finished initial scan of repository in 5s
- With debug logging enabled, you can also look for the exact command that Fisheye is failing at while indexing, for example:
2013-04-08 14:28:40,272 DEBUG [InitialPinger3 svn] fisheye.app SvnThrottledClient-start - Executing ls -r HEAD https://URL/public/@HEAD 2013-04-08 14:28:41,834 WARN [InitialPinger3 svn] fisheye.app SvnRepositoryTester-getServerRootURL - Unable to get Repository Root URL for svn:https://URL/public/ 2013-04-08 14:28:41,834 DEBUG [InitialPinger3 svn] fisheye.app SvnRepositoryTester-pingAndValidateAccess - Validating access to https://URL/public/ 2013-04-08 14:28:41,834 DEBUG [InitialPinger3 svn] fisheye.app SvnThrottledClient-start - Executing propget fisheye.access -r HEAD https://URL/public/@HEAD 2013-04-08 14:28:43,124 DEBUG [InitialPinger3 svn] fisheye.app RepositoryStatus-setMessage - Status change [svn]: Could not access https://URL/public/ 2013-04-08 14:28:43,124 ERROR [InitialPinger3 svn] fisheye.app SvnRepositoryTester-pingAndValidateAccess - Exception getting FishEye access control com.cenqua.fisheye.rep.RepositoryClientException: org.tigris.subversion.javahl.ClientException: svn: E175011: Repository moved permanently to '/public'; please relocate svn: E175002: PROPFIND request failed on '/public'
In the above example, the command is "svn ls -r HEAD". You can run it via the command line to check the result returned from your SVN server.