Improve Fisheye scan performance
This page contains information about improving the performance of Fisheye repository scans.
Background information
When you add a repository, Fisheye needs to perform a once-off scan through the repository to build up its initial index and cache. This scan can take some time. Until this scan is complete, you may find that some data is not displayed. As a guide, Fisheye should be able to process about 100KB-200KB per second on an averaged-size PC. If Fisheye is accessing the repository over the network (e.g. over a NFS mount), then you should expect the initial scan to take longer.
General improvements
You can increase the speed of your scans using the following options:
- If your repository is non-local, set up a local repository mirror on the Fisheye server. This will provide a major speed boost for anyone scanning a repository across a network.
- Exclude unused file types, unused directories and specific large files from Fisheye.
Improve update performance during initial scan
One option is break large repositories into multiple smaller repositories. While this technique will not improve the overall initial scan time, it allows for all fully scanned repositories to be updated while the initial scan is still being performed on those remaining.
In Fisheye 1.3.4 and later, the initial and incremental scans happen in separate, single threads. So, splitting the repositories will allow incremental scans to run concurrently alongside the initial scans. You may also wish to split projects into separate repositories, since permissions in Fisheye are applied on a per-repository basis.
Improving initial scan performance for an SVN repository
svnsync is not an atomic protocol. It generally performs the commit as the svn sync user and then updates the revprops to match the original commit, under some circumstances, this can cause the wrong user to be attributed as the author of a changeset. More information available at Wrong User Reported When Indexing SVN Mirror Repository.
The http/s protocol has the slowest performance during the initial scan. The svn protocol (svn://) is faster and the file protocol (file:///) is the fastest.
Therefore if you find your initial scan takes an extended amount of time (more than a day or two), you should consider switching over from the http/s protocol to the svn or file protocol to define the location of your SVN repository. (Use svnsync to mirror the repository onto the fisheye server, so that you can access it with the file protocol.)
E.g. Switch from
https://example.com/svn/project/
to
svn://example.com/svn/project/
or
file:///home/user/some/location/svn/project
In order for SVN protocol to work you need to have set up an svnserve based server.
More information on how to troubleshoot SVN indexing related issues can be found here.
Performance support
If you have implemented at least one of the suggestions above but are still experiencing slow performance, ask us for help:
- First read the Tuning Fisheye performance document.
- Turn on debug logging using the command line debug flag.
- Allow Fisheye to continue its initial scan overnight.
- Create a new support request in the Fisheye project, including your server environment and log files with the problem description.