Plans with git submodule fail in Bamboo Server/ DC with "Repository not found" or "Authentication failed" messages

お困りですか?

アトラシアン コミュニティをご利用ください。

コミュニティに質問

プラットフォームについて: Server および Data Center のみ。この記事は、Server および Data Center プラットフォームのアトラシアン製品にのみ適用されます。

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Fisheye および Crucible は除く

要約

Plan build fails if a plan checks out a Bitbucket Server or Cloud repository that contains git submodules (if submodules are enabled in advanced repository settings in Bamboo).

診断

You will find authentication errors in plan build logs. If the submodule has been declared using an SSH Git URL.

Repository not found
The requested repository does not exist, or you do not have permission to access it.
fatal: Could not read from remote repository.

If the Git submodule has been added with HTTPS URL:

fatal: Authentication failed for 'https://bitbucket-server.com/scm/vpe/adat.git/'
fatal: clone of 'https://bitbucket-server.com/scm/scm/vpe/adat.git' 
into submodule path '/opt/data/bamboo/xml-data/build-dir/MTP-MT-JOB1/submodule' failed

Or if it shows that the task was canceled in the build log:

Task was canceled - 'Checkout gen-sw Repository(1)' of type com.atlassian.bamboo.plugins.vcs:task.vcs.checkout

原因

Bamboo tries to checkout submodules but fails because it does not have username/password (for https URLs) or SSH key (for SSH Git URLs). In case of submodules  the clone happens outside of Bamboo SSH proxy.

One of the other causes may be the ~/.git-credentials file has corrupted entries for username or password, or the file itself is missing from the Bamboo server/agent.

ソリューション

There are three solutions to this problem . The first solution implies changes to .gitmodules file and adding SSH key to repository settings. The second one is all about configuring SSH keys (or passwordless access for https repositories) on all agent nodes.

The third one is to update the ~/.git-credentials file to refresh the username or password or create a new one if it is not present.

Option 1: Use Relative Paths

  1. To force Bamboo to use SSH proxy, use relative paths instead of HTTPS or SSH Git URLs.

    The following .gitmodules

    [submodule "module1"]
     path = submodule1
     url = ssh://git@bitbucket-server.com:7999/project1/repo1.git
    [submodule "submodule2"]
     path = submodule2
     url = ssh://git@bitbucket-server.com:7999/project1/repo2.git

    Should become:

    [submodule "module1"]
     path = submodule1
     url = ../repo1.git
    [submodule "submodule2"]
     path = submodule2
     url = ../repo2.git

    In this particular example the relative path is ../repo1 and ../repo2, thus we assume that the main repo is in the same Bitbucket project. If Bitbucket projects are different, a relative path should look like this: ../../projectName/repo.git

  2. Having replaced URLs with relative paths, a public SSH key needs to be added to submodule repositories in Bitbucket.
    1. Copy SSH key from the advanced settings of the main repository in Bamboo:
    2. Save the key as Access Token in Bitbucket server repository settings:
    3. For Bitbucket Cloud you may skip Step 2 if you have already uploaded private key to Bamboo and added a public key to your profile when adding the main repository to a plan.

Option #2: Use Absolute URLs

If for some reason changes to .gitmodules are not possible and either https or ssh URLS are used, agent environments need to be configured to let Git clone the repositories without password prompts. Depending on the URL and the OS instructions vary.

Configuration needs to be performed on all remote agent machines (and Bamboo server machine as well if local agents are used).

Linux: HTTPS

echo "https://admin:password@git.mybitbucket.com" > ~/.git-credentials

The downside of this approach is that credentials are store in a plan text file which isn't secure. Also, this command needs to be run on all new agent machines (unless the job that uses Git repo with submodules has dedicated agents)

(info) The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.

Linux: SSH

Either generate or copy id_rsa to ~/.ssh. Make sure the .ssh directory permissions are 700 (drwx------) and the private key (id_rsa) is 600 (-rw-------). Do not forget to add a public key as an access token to Bitbucket repository settings or user profile.

Again, the downside here is that a private key is stored on agent machines, and it needs to be copied to any new agent machines (unless dedicated agents are used).

Windows: HTTPS

There are two options here:

(info) The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.

Windows: SSH

Follow instructions for Linux users. File and directory permission issues are possible. You may need to manually clone the repository to troubleshoot such issues.

Option #3: Update/create  the .git-credentials file 


  • As the user which is running Bamboo ,please perform a git clone of the main repository(one which calls the submodules) in the Bamboo remote agent.
git clone http://bitbucket:7990/scm/bam/submodule_demo.git
  • Navigate to the repository
cd submodule_demo
  • Perform an update of the submodule by running the below command 
git submodule update --init --recursive

This prompts for password, please provide the username and password for the Bitbucket user which has access to those submodule repositories.

The Bitbucket repository username and password is now updated in the .git-credentials file if present. If not its creates a new .git-credentials file under the home directory in the Bamboo remote agent.

(info) The user(configured above) needs to added into the Repository Permissions in Bitbucket repository for the external repository.



最終更新日: 2024 年 2 月 20 日

この内容はお役に立ちましたか?

はい
いいえ
この記事についてのフィードバックを送信する
Powered by Confluence and Scroll Viewport.