Bamboo fails to lodge spot instance request since the provided credentials do not have permission to create the service-linked role
問題
When Bamboo lodges a spot instance request, the request is instantly abandoned and the below error is logged to <bamboo-home>/atlassian-bamboo.log or in the UI at Bamboo Administration >> (Elastic Bamboo) >> Instances:
2018-05-16 15:08:34,601 ERROR [elastic-pool-3-thread-4] [RemoteEC2InstanceImpl] EC2 instance order for image ami-9345bbf1 failed.
com.amazonaws.services.ec2.model.AmazonEC2Exception: The provided credentials do not have permission to create the service-linked role for EC2 Spot Instances. (Service: AmazonEC2; Status Code: 403; Error Code: AuthFailure.ServiceLinkedRoleCreationNotPermitted; Request ID: 5df32feb-eff8-413b-90cc-fddfa769fedf)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:15651)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:15627)
at com.amazonaws.services.ec2.AmazonEC2Client.executeRequestSpotInstances(AmazonEC2Client.java:14443)
at com.amazonaws.services.ec2.AmazonEC2Client.requestSpotInstances(AmazonEC2Client.java:14419)
...
at com.atlassian.aws.ec2.awssdk.launch.AwsSpotInstanceLauncher.call(AwsSpotInstanceLauncher.java:76)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.launchInstance(RemoteEC2InstanceImpl.java:363)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.backgroundStart(RemoteEC2InstanceImpl.java:346)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.access$100(RemoteEC2InstanceImpl.java:39)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$1.run(RemoteEC2InstanceImpl.java:95)
at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$CatchingRunnableDecorator.run(RemoteEC2InstanceImpl.java:79)
...The request is instantly abandoned and the below log entries are present in the instance log in the Bamboo UI at Bamboo Administration >> (Elastic Bamboo) >> Instances:
May 16, 2018 2:32:30 PM Requested that new elastic instance be created for configuration: Ubuntu stock image / ami-000000
May 16, 2018 2:32:30 PM Detected that a pending instance request for image [Ubuntu stock image], ami: ami-000000 has been abandoned.原因
This occurs because with AWS EC2, when lodging a Spot Request – there is a service-linked role that needs to be created (or exist already) in IAM called AWSServiceRoleForEC2Spot. If the role doesn't exist, AWS will attempt to create it automatically:
If the IAM user configured for Elastic Bamboo use does not have the iam:CreateServiceLinkedRole permission, this action will fail with a permission error.
ソリューション
Grant the IAM user configured for Elastic Bamboo use, the below IAM permission:
iam:CreateServiceLinkedRole
The next time Bamboo makes a spot request, the IAM role AWSServiceRoleForEC2Spot will be created. After role creation, that permission can be from the IAM policy and it will continue to work since the role now exists.
For more information on the API that this permission grants access to, please see the Amazon documentation below: