Exceeds maximum permitted number of rows per writeset
問題
次のうちの 1 つ以上が観察される。
- Cannot run manual builds
- Change detection process fails
The following appears in the atlassian-bamboo.log
file:
org.springframework.orm.hibernate4.HibernateJdbcException: JDBC exception on Hibernate data access: SQLException for SQL [insert into COMMIT_FILES (COMMIT_ID, COMMIT_FILE_NAME, COMMIT_FILE_REIVISION) values (?, ?, ?)]; SQL state [HY000]; error code [1364]; could not execute batch; nested exception is org.hibernate.exception.GenericJDBCException: could not execute batch
at org.springframework.orm.hibernate4.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:170)
at org.springframework.orm.hibernate4.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:730)
at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:592)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at sun.reflect.GeneratedMethodAccessor151.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy31.commit(Unknown Source)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:484)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy70.updateResultSummaryWithBuildChanges(Unknown Source)
at com.atlassian.bamboo.chains.ChainExecutionManagerImpl$1.getChainState(ChainExecutionManagerImpl.java:256)
at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.tryStartChainState(ChainExecutionManagerImpl.java:337)
at com.atlassian.bamboo.chains.ChainExecutionManagerImpl.delayedStart(ChainExecutionManagerImpl.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy151.delayedStart(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy151.delayedStart(Unknown Source)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl$1$1$1.lambda$call$0(PlanExecutionManagerImpl.java:413)
at com.atlassian.bamboo.util.CacheAwareness$3.call(CacheAwareness.java:159)
at com.atlassian.bamboo.util.CacheAwareness$3.call(CacheAwareness.java:155)
at com.atlassian.bamboo.util.CacheAwareness.withValuesOlderThanTimestampReloaded(CacheAwareness.java:188)
at com.atlassian.bamboo.util.CacheAwareness.withValuesOlderThanTimestampReloaded(CacheAwareness.java:154)
at com.atlassian.bamboo.util.CacheAwareness.withValuesOlderThanTimestampReloaded(CacheAwareness.java:219)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl$1$1$1.call(PlanExecutionManagerImpl.java:412)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl$1$1$1.call(PlanExecutionManagerImpl.java:403)
at io.atlassian.util.concurrent.ManagedLocks$ManagedLockImpl.withLock(ManagedLocks.java:293)
at com.atlassian.bamboo.plan.PlanExecutionLockServiceImpl.lock(PlanExecutionLockServiceImpl.java:85)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl.doWithProcessLock(PlanExecutionManagerImpl.java:783)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl.access$400(PlanExecutionManagerImpl.java:130)
at com.atlassian.bamboo.plan.PlanExecutionManagerImpl$1$1.run(PlanExecutionManagerImpl.java:397)
at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:51)
at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31)
at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20)
at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.hibernate.exception.GenericJDBCException: could not execute batch
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:136)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.addToBatch(BatchingBatch.java:98)
at org.hibernate.persister.collection.AbstractCollectionPersister.recreate(AbstractCollectionPersister.java:1311)
at org.hibernate.action.internal.CollectionRecreateAction.execute(CollectionRecreateAction.java:67)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:425)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:177)
at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:584)
... 53 more
Caused by: java.sql.BatchUpdateException: Field 'LIST_POSITION' doesn't have a default value
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2055)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1467)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:127)
... 65 more
Caused by: java.sql.SQLException: Field 'LIST_POSITION' doesn't have a default value
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2825)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2459)
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2007)
... 68 more
診断
環境
- Bamboo connected to the MariaDB flavour of the MySQL database.
Diagnostic Steps
- The problem started happening after a recent upgrade to Bamboo 5.12 onwards.
原因
From Bamboo 5.12 onwards, the column LIST_POSITION was removed from the COMMIT_FILES table in Bamboo. However, in this case, the column is yet to be removed from your database. Now, when Bamboo issues an insert to that table, it no longer specifies any value for the LIST_POSITION column, and because a "default value" has not been defined, your database is rejecting the insert as invalid.
This is a result from a failed upgrade task:
2017-03-11 08:27:27,359 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [AbstractUpgradeManager] --------------------------------------------------
2017-03-11 08:27:27,359 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [AbstractUpgradeManager] 51205 : Drop obsolete column in COMMIT_FILES table
2017-03-11 08:27:27,359 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [AbstractUpgradeManager] --------------------------------------------------
2017-03-11 08:27:27,359 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [UpgradeTask51205DropObsoleteCommitFilesColumn] Dropping primary key on COMMIT_FILES
2017-03-11 08:27:29,053 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [UpgradeTask51205DropObsoleteCommitFilesColumn] The primary key constraint on COMMIT_FILES has NOT been dropped.
2017-03-11 08:27:29,076 WARN [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [UpgradeTask51205DropObsoleteCommitFilesColumn] The primary key constraint on COMMIT_FILES has NOT been dropped.
java.sql.SQLException: wsrep_max_ws_rows exceeded
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2819)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2768)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:894)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:732)
at com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:75)
at com.atlassian.bamboo.utils.db.MySqlDbmsBean.dropPrimaryKey(MySqlDbmsBean.java:117)
at com.atlassian.bamboo.upgrade.tasks.v5_12.UpgradeTask51205DropObsoleteCommitFilesColumn.doUpgrade(UpgradeTask51205DropObsoleteCommitFilesColumn.java:32)
at org.hibernate.jdbc.WorkExecutor.executeWork(WorkExecutor.java:54)
at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2076)
at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2073)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.coordinateWork(JdbcCoordinatorImpl.java:313)
at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2095)
at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2080)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate$1$1.doInHibernate(BambooConnectionTemplate.java:41)
at org.springframework.orm.hibernate4.HibernateTemplate.doExecute(HibernateTemplate.java:341)
at org.springframework.orm.hibernate4.HibernateTemplate.executeWithNativeSession(HibernateTemplate.java:309)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate$1.doInTransactionWithoutResult(BambooConnectionTemplate.java:35)
at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate.execute(BambooConnectionTemplate.java:30)
at com.atlassian.bamboo.upgrade.tasks.AbstractInHibernateTransactionUpgradeTask.doUpgrade(AbstractInHibernateTransactionUpgradeTask.java:35)
at com.atlassian.bamboo.upgrade.AbstractUpgradeManager.runUpgradeTask(AbstractUpgradeManager.java:206)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:114)
at com.atlassian.bamboo.upgrade.UpgradeLauncher$1.call(UpgradeLauncher.java:117)
at com.atlassian.bamboo.upgrade.UpgradeLauncher$1.call(UpgradeLauncher.java:113)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:51)
at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31)
at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20)
at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52)
at java.lang.Thread.run(Thread.java:745)
2017-03-11 08:27:29,076 WARN [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [UpgradeTask51205DropObsoleteCommitFilesColumn]
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '2753009' for key 'PRIMARY'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1041)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2819)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2768)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:894)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:732)
at com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:75)
at com.atlassian.bamboo.utils.db.AbstractDbmsBean.dropColumn(AbstractDbmsBean.java:82)
at com.atlassian.bamboo.upgrade.tasks.v5_12.UpgradeTask51205DropObsoleteCommitFilesColumn.doUpgrade(UpgradeTask51205DropObsoleteCommitFilesColumn.java:43)
at org.hibernate.jdbc.WorkExecutor.executeWork(WorkExecutor.java:54)
at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2076)
at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2073)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.coordinateWork(JdbcCoordinatorImpl.java:313)
at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2095)
at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2080)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate$1$1.doInHibernate(BambooConnectionTemplate.java:41)
at org.springframework.orm.hibernate4.HibernateTemplate.doExecute(HibernateTemplate.java:341)
at org.springframework.orm.hibernate4.HibernateTemplate.executeWithNativeSession(HibernateTemplate.java:309)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate$1.doInTransactionWithoutResult(BambooConnectionTemplate.java:35)
at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:34)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at com.atlassian.bamboo.persistence.BambooConnectionTemplate.execute(BambooConnectionTemplate.java:30)
at com.atlassian.bamboo.upgrade.tasks.AbstractInHibernateTransactionUpgradeTask.doUpgrade(AbstractInHibernateTransactionUpgradeTask.java:35)
at com.atlassian.bamboo.upgrade.AbstractUpgradeManager.runUpgradeTask(AbstractUpgradeManager.java:206)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:114)
at com.atlassian.bamboo.upgrade.UpgradeLauncher$1.call(UpgradeLauncher.java:117)
at com.atlassian.bamboo.upgrade.UpgradeLauncher$1.call(UpgradeLauncher.java:113)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:51)
at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31)
at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20)
at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52)
at java.lang.Thread.run(Thread.java:745)
2017-03-11 08:27:29,078 INFO [15-UpgradeTaskBackgroundThread:pool-30-thread-1] [AbstractUpgradeManager] Completed task 51205 successfully.
The upgrade task 51205 removes the LIST_POSITION column from the COMMIT_FILES table. Although the upgrade task reports as completed successfully, the operation failed with a wsrep_max_ws_rows exceeded message. Older versions of MariaDB used to have this variable set to 131072. Newer versions, however, have this variable set to 0, which essentially allows writesets to be any size.
Check the current variable value using the following steps:
- Log in to MySQL as a root user.
次のコマンドを実行します。
SHOW VARIABLES LIKE 'wsrep_max_ws_rows';
ソリューション
- Set the maximum permitted number of rows per writeset (wsrep_max_ws_rows) to 0 and restart the MySQL server.
- Re-do the Bamboo upgrade.