要約

After creating a new repository with Git LFS enabled, when attempting to push the repository for the first time, the push takes a long time. If the push completes, the repository in the Bitbucket Server user interface will still appear as empty and contain only the .gitattributes file. The Git LFS files are not available on the server.

Subsequent pushes

It has also been observed that subsequent pushes return with the Everything up-to-date message. When the Git client debug log is enabled, the following can be found in the log:

trace git-lfs: tq: running as batched queue, batch size of 100
trace git-lfs: tq: sending batch of size 0
Everything up-to-date

環境

• Bitbucket Server/DC 7+

診断

Diagnostic Steps

To confirm that this is what is happening, we create a brand new repository and time the expected push time.

• Create a new repository in Bitbucket Server
• Update the push URL for the local repository by following the instructions in the user interface
• Push the repository
• When checking the logs, the initial Git push completes in a short time:

# start of the push
17:04:53.890919 run-command.c:369       trace: run_command: 'ssh' '-p' '7999' 'git@<base_url>' 'git-receive-pack '\''/<project_slug>/<repo_name>.git'\'''
..
# end of the push
17:04:54.164698 pkt-line.c:80           packet:         push< 0000

• The push of Git LFS changes instead takes much longer. Note the 50 seconds "gap" in the logs:

# push of one ref (<ref_name1> in this case)
17:05:43.285345 trace git-lfs: pre-push: refs/heads/<ref_name1> <ref_hash_a> refs/heads/<ref_name1> <ref_hash_b>
17:05:43.285677 trace git-lfs: run_command: git rev-list --objects <ref_hash_a> --not --remotes=origin --
17:05:43.287041 trace git-lfs: run_command: git cat-file --batch


# start of push of the next ref (<ref_name2> in this case)
# 50 seconds later
17:06:37.575044 trace git-lfs: pre-push: refs/heads/<ref_name2> <ref_hash_c> refs/heads/<ref_name2> <ref_hash_d>

The total time required for a Git LFS push can be estimated by multiplying the following:

Push time for a single ref * (number of branches + number of tags)

For big repositories (with a significant history), this can take days.

原因

The push of the repository consists of two steps:

• the push of the Git repository itself
• the transfer of the Git LFS files

While the push of the Git repository completes quickly, the transfer of the Git LFS files requires a longer time.

ソリューション

This approach will break down the push of the repository in two phases:

• Push the Git repository (without the LFS objects)
• Push the Git LFS objects only

Preparation steps

• Create a new repository (as a clone of the source - not yet converted - one)
• Perform the clone as usual by running git clone --mirror
• Convert the repository as usual (for example by using bfg)

Push the Git repository (without the LFS objects)

• Do not initialize Git LFS on the client. In other words, do not run the git lfs install in the repository. By skipping this step, we will not create the hook to upload the Git LFS files once the push completes, we will perform this step manually later.
• Push the repository by running git push --force (this should complete quickly)

At this stage, Bitbucket Server will:

• be aware of the new refs
• have the updated .gitattributes file
• not have the Git LFS files yet

When accessing one of the Git LFS converted files, the following will be displayed:

• the details of the Git LFS file, if Git LFS is not enabled for the repository
• an error ("File not found") if Git LFS is enabled for the repository
  
  

This is what we will need to do to push the Git LFS files:

Identify the Git LFS objects

In the cloned repository, run the following command:

find .git/lfs/objects  -type f

This will return the Git LFS objects id to be pushed. For example:

./lfs/objects/00/78/007887412de71307d189f8cf38131958f7c14e204ade0ac643aa1416ed2b908f
./lfs/objects/a2/f1/a2f1e465fb27e154b76c150be4ee8155e39452e26f0517e2eca84a8455502e63
./lfs/objects/a5/c7/a5c72c2609067bce941c84df309a17733777d72b0ff17400fe267c5b03d525d1
./lfs/objects/fd/5d/fd5dc2782cc30c70b70e19850a6547ee12dd4542553744306441a60d14b7caf7

Push the Git LFS objects
注意:

• it is not required to git lfs install at this stage because, by calling Git LFS commands directly, we are bypassing the hooks. However, it should be installed now (or at any time later)
• LFS has to be enabled on the Server for this repository before running the next step ("Repository admin settings" -> "Large File Storage (LFS)" -> "Allow LFS")

For each of the id returned by the command above, run the following command:

git lfs push origin --object-id <hash>

Following the example above, this is what needs to be run:

git lfs push origin --object-id 007887412de71307d189f8cf38131958f7c14e204ade0ac643aa1416ed2b908f
git lfs push origin --object-id a2f1e465fb27e154b76c150be4ee8155e39452e26f0517e2eca84a8455502e63
git lfs push origin --object-id a5c72c2609067bce941c84df309a17733777d72b0ff17400fe267c5b03d525d1
git lfs push origin --object-id fd5dc2782cc30c70b70e19850a6547ee12dd4542553744306441a60d14b7caf7

The image (a Git LFS object) will now be visible in the user interface.

Post push checks

To confirm that all files have been transferred, it is possible to count the Git LFS files on the client and on the server and make sure that the number matches.