Bamboo Data Center ephemeral agent refuses to start with error message "pods is forbidden"
プラットフォームについて: Data Center のみ - この記事は、Data Center プラットフォームのアトラシアン製品にのみ適用されます。
この KB は Data Center バージョンの製品用に作成されています。Data Center 固有ではない機能の Data Center KB は、製品のサーバー バージョンでも動作する可能性はありますが、テストは行われていません。サーバー*製品のサポートは 2024 年 2 月 15 日に終了しました。サーバー製品を利用している場合は、アトラシアンのサーバー製品のサポート終了のお知らせページにて移行オプションをご確認ください。
*Fisheye および Crucible は除く
本記事で説明している手順は、現時点でのものとなります。そのため、一部のお客様で特定の状況下で動作したという報告がありますが、正式にサポートされているわけではなく、お客様の特定のシナリオで動作することを保証するものではありません。
本番環境での実施の前に一通り非本番環境で検証し、成功しなかった場合にはサポートされている代替案にフォール バックしてください。
要約
Bamboo job failed to start on an ephemeral agent with below error message on cluster communication log.
Error from server (Forbidden): pods is forbidden環境
- This issue and the solution has been tested on Bamboo Data Center 9.4.1 and 9.3.2
- This happened on an Openshift Kubernetes cluster 4.12
診断
The cluster communication logs reveal an "Unauthorized" message when trying to create the agent pod. To access these logs, navigate to the Bamboo administration page. From there, go to the Ephemeral agents section and select Pods. On the Manage pods and ephemeral agents page, you can find the cluster communication log.
Jan 16, 2024, 2:03:26 PM Launching Ephemeral Agent pod eph-ephe-job1-6-flfhpdkq for EPHEMERAL - EPHE - Default Job #6 (EPH-EPHE-JOB1-6)
error: You must be logged in to the server (error when creating "/opt/atlassian/atlassian-bamboo-9.4.1/temp/pod1931317460300020553.yaml": Unauthorized)
Jan 16, 2024, 2:03:54 PM Launching Ephemeral Agent pod eph-ephe-job1-6-qgblqhra for EPHEMERAL - EPHE - Default Job #6 (EPH-EPHE-JOB1-6) [2. attempt of 3]
error: You must be logged in to the server (error when creating "/opt/atlassian/atlassian-bamboo-9.4.1/temp/pod14230423914922192937.yaml": Unauthorized)
Jan 16, 2024, 2:04:24 PM Launching Ephemeral Agent pod eph-ephe-job1-6-njxxvsbf for EPHEMERAL - EPHE - Default Job #6 (EPH-EPHE-JOB1-6) [3. attempt of 3]
error: You must be logged in to the server (error when creating "/opt/atlassian/atlassian-bamboo-9.4.1/temp/pod11931554224856638448.yaml": Unauthorized)
Jan 16, 2024, 2:04:24 PM The Ephemeral Agent's pod eph-ephe-job1-6-njxxvsbf for EPHEMERAL - EPHE - Default Job #6 (EPH-EPHE-JOB1-6) failed to start - dropping launch request
Jan 16, 2024, 2:15:42 PM Launching Ephemeral Agent pod eph-ephe-job1-7-alphnzop for EPHEMERAL - EPHE - Default Job #7 (EPH-EPHE-JOB1-7)The below error message is also seen on the <bamboo-home>/logs/atlassian-bamboo.log file. The service account "system:serviceaccount:default:bamboo" is forbidden from listing pods.
2024-01-16 14:19:06,156 WARN [http-nio-8085-exec-3] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:19:16,293 WARN [http-nio-8085-exec-24] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:19:26,432 WARN [http-nio-8085-exec-11] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:19:36,592 WARN [http-nio-8085-exec-12] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:19:46,729 WARN [http-nio-8085-exec-8] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:19:56,902 WARN [http-nio-8085-exec-14] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
2024-01-16 14:20:07,043 WARN [http-nio-8085-exec-4] [KubernetesExecutor] Cannot perform KubernetesExecutor#getPods: Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:bamboo" cannot list resource "pods" in API group "" in the namespace "default"
原因
The service account used here has not been provided with required privilege to create pods in the default namespace.
Please also note that namespace could have been modified in the Ephemeral agent template by adding namespace: bamboo-ephemeral to the metadata section.
metadata:
name: '{{NAME}}'
namespace: bamboo-ephemeral
labels:
'{{RESOURCE_LABEL}}': <value>
From <bamboo-home>/logs/atlassian-bamboo.log file, we can see this user, "system:serviceaccount:default:bamboo". This is a service account named bamboo in the default namespace.
Use the command below to find out the role assigned to the service account. In this case, there was no output which explains why the service account had no permission to create a pod in the default namespace.
oc get rolebindings -o json | jq -r '
.items[] |
select(
.subjects // [] | .[] |
[.kind,.namespace,.name] == ["ServiceAccount","default","bamboo"]
) |
.metadata.name'
ソリューション
Assign the edit role to the service account in question.
oc project default #Switch to the default namespace
oc get sa bamboo #Confirm the presence of the service account in the current namespace
oc adm policy add-role-to-user edit -z bamboo #assign the edit role to service account
Now you can check the role bindings assigned to the user using the below
oc get rolebindings -o json | jq -r '
.items[] |
select(
.subjects // [] | .[] |
[.kind,.namespace,.name] == ["ServiceAccount","default","bamboo"]
) |
.metadata.name'
edit
Now try running the build again using the ephemeral agent and it should work fine.