Database Corruption - Can't load default bamboo plan page
症状
We see this type of errors in the Bamboo logs when accessing the dashboard on http://<host> :<port>/<context>/allPlans.action
java.lang.RuntimeException: org.springframework.orm.hibernate.HibernateObjectRetrievalFailureException: No row with the given identifier exists: 26444020, of class: com.atlassian.bamboo.plan.AbstractPlan; nested exception is net.sf.hibernate.UnresolvableObjectException: No row with the given identifier exists: 26444020, of class: com.atlassian.bamboo.plan.AbstractPlan
at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:120)
at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:88)
at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:191)
at com.atlassian.bamboo.plan.cache.ImmutablePlanManagerImpl.getPlanByKey(ImmutablePlanManagerImpl.java:129)
at sun.reflect.GeneratedMethodAccessor259.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy68.getPlanByKey(Unknown Source)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$1$1.call(ImmutablePlanCacheServiceImpl.java:97)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$1$1.call(ImmutablePlanCacheServiceImpl.java:89)
at com.atlassian.bamboo.plan.cache.CacheLoadContextSupport.callRethrowingExceptionsAsRuntimeExceptions(CacheLoadContextSupport.java:145)
at com.atlassian.bamboo.plan.cache.CacheLoadContextSupport.access$200(CacheLoadContextSupport.java:21)
at com.atlassian.bamboo.plan.cache.CacheLoadContextSupport$4$1.doInHibernate(CacheLoadContextSupport.java:122)
at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:370)
at org.springframework.orm.hibernate.HibernateTemplate.execute(HibernateTemplate.java:337)
at com.atlassian.bamboo.plan.cache.CacheLoadContextSupport$4.doInTransaction(CacheLoadContextSupport.java:116)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
at com.atlassian.bamboo.plan.cache.CacheLoadContextSupport.load(CacheLoadContextSupport.java:109)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$1.load(ImmutablePlanCacheServiceImpl.java:107)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$1.load(ImmutablePlanCacheServiceImpl.java:76)
at com.google.common.cache.CustomConcurrentHashMap$ComputingValueReference.compute(CustomConcurrentHashMap.java:3426)
at com.google.common.cache.CustomConcurrentHashMap$Segment.compute(CustomConcurrentHashMap.java:2322)
at com.google.common.cache.CustomConcurrentHashMap$Segment.getOrCompute(CustomConcurrentHashMap.java:2291)
at com.google.common.cache.CustomConcurrentHashMap.getOrCompute(CustomConcurrentHashMap.java:3802)
at com.google.common.cache.ComputingCache.get(ComputingCache.java:46)
at com.atlassian.bamboo.collections.AlwaysInvalidatingCacheDecorator.get(AlwaysInvalidatingCacheDecorator.java:66)
at com.atlassian.bamboo.collections.AlwaysInvalidatingCacheDecorator.getUnchecked(AlwaysInvalidatingCacheDecorator.java:100)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.internalGetPlanByKey(ImmutablePlanCacheServiceImpl.java:439)
at com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.getImmutablePlanByKey(ImmutablePlanCacheServiceImpl.java:210)
at com.atlassian.bamboo.caching.DashboardCachingManagerImpl$1.apply(DashboardCachingManagerImpl.java:164)
at com.atlassian.bamboo.caching.DashboardCachingManagerImpl$1.apply(DashboardCachingManagerImpl.java:160)
at com.google.common.collect.Lists$TransformingRandomAccessList.get(Lists.java:451)
at java.util.AbstractList$Itr.next(AbstractList.java:345)
at com.google.common.collect.Iterators$8.next(Iterators.java:781)
at com.google.common.collect.Iterators$7.computeNext(Iterators.java:644)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
at com.atlassian.bamboo.plan.cache.PlansState.<init>(PlansState.java:47)
at com.atlassian.bamboo.caching.DashboardCachingManagerImpl.initCaches(DashboardCachingManagerImpl.java:168)
at com.atlassian.bamboo.caching.DashboardCachingManagerImpl.getAllTopLevelPlans(DashboardCachingManagerImpl.java:50)
at sun.reflect.GeneratedMethodAccessor825.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at com.atlassian.bamboo.security.acegi.intercept.aopalliance.AuthorityOverrideMethodSecurityInterceptor.invoke(AuthorityOverrideMethodSecurityInterceptor.java:29)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy112.getAllTopLevelPlans(Unknown Source)
at com.atlassian.bamboo.webwork.StarterAction.getPlans(StarterAction.java:238)
at com.atlassian.bamboo.webwork.StarterAction.doStart(StarterAction.java:139)
at sun.reflect.GeneratedMethodAccessor1207.invoke(Unknown Source)原因
There could be several root causes to this kind of errors. One cause could be that the database is not using the correct storage engine. In MYSQL for example, the storage engine should be InnoDB which supports row-level locking to accommodate simultaneous write access by multiple sessions. MyISAM can introduce subtle corruptions since it supports table-level locking. If you are using MYSQL, follow the guide on this article to identify whether you are using MyISAM: Convert from MySQL MyISAM to InnoDB Storage Engine. Another possibility is that the server went down during the expiry of builds/branch that left some data uncleaned properly
ソリューション
The resolution of this type of database corruptions can be running SQL queries to delete the records from the database that refer to the fields that don't exist. This can be done by following the steps below:
- Run Bamboo, and see on which field it is failing (usually Bamboo shows the table names, column names and the names of the constraints involved)
- It might be a good idea to run a search on the whole DB dump file to see if the specified identifier exists, and in which tables.
- Shut down Bamboo,
- After making sure that you have a DB backup, delete the rows mentioned in the previous step.
- Start Bamboo, and if there are any other errors, go to step 1.
Each specific DB corruption case might be different - involving different tables. This SQL queries might be useful to clean the DB (make sure to use the correct ID mentioned in the Bamboo log in case you have exactly the same type of problem). This will return the rows that are affected:
select * from build where master_ID=26444020;
RECOMMENDED: You can also make Bamboo perform this deletion by performing the below steps. This is the safer alternative to deleting build rows manually from the database:
Run the below SQL to flag it for deletion:
update build set marked_for_deletion =1 where master_ID=26444020;- Restart Bamboo so the deletion service picks up the update and performs the deletion.
最終更新日 2017 年 7 月 7 日
Powered by Confluence and Scroll Viewport.