[Other doc versions]
[Doc downloads (PDF, HTML, XML)]
Stash DIY Backup was introduced in Stash 2.12 as an alternative strategy to using the Stash Backup Client. It allows you to:
pg_dump
: バックエンドのデータベースが PostgreSQL の場合、またはsqlcmd
: バック エンドのデータベースが MS SQL Server の場合、差分バックアップ用の適切なコマンドとともに使用。
rsync
(利用可能であれば)The the key to reducing Stash downtime to a minimum is the use of optimal, vendor-specific database and file system backup tools. Such tools are generally able to take snapshots (though sometimes in a vendor-specific format) in much less time than the generic, vendor-neutral format used by the Stash Backup Client.
Stash DIY Backup does require you to write some code in a language of your choice to perform the required backup steps, using the REST API available for Stash 2.12.
DIY Backup supports Windows and Linux platforms, and Stash versions 2.12 and higher. DIY Backup supports both Stash Server and Stash Data Center instances equally - any DIY Backup solution that works on one should work on the other without modification.
For information about other backup strategies for Stash, see Data recovery and backups. That page also discusses the tight coupling between the Stash file system on disk and the database that Stash uses.
Please note that the examples on this page are provided as guidance for developing a DIY Backup solution. As such, the third-party tools described are for example only – you will need to choose the tools that are appropriate to your own specific installation of Stash.
選択したサードパーティ ツールについては、ベンダーのドキュメントをご確認ください。アトラシアンではこのようなツールのサポートを提供できません。
On this page:
動作確認済みのサンプル リクエストを Bitbucket からダウンロードできます。
このページでは次のような内容について説明します。
bash
シェル スクリプトを使用した、PostgreSQL データベースおよびローカル ファイルシステム用の完全な DIY Backup ソリューションについて説明します。You can use this solution directly if your Stash instance has the same or similar configuration, or use this as a starting point to develop your own DIY Backup solution tailored to your hardware configuration.
When you use DIY Backup instead of the Stash Backup Client, you have complete control over the backup steps, and can implement any custom processes you like in the language of your choice. For example, you can use your database's incremental or fast snapshot tools and/or your file server's specific tools as part of a DIY Backup.
The DIY Backup works in a similar way to the Stash Backup Client and does the following:
rsync
を実行する。pg_dump
を使用して、データベースの完全に一貫したバックアップを作成する。rsync
を使用して、ファイルシステムの完全に一貫したバックアップを作成する。A user will get an error message if they try to access the Stash web interface, or use the Stash hosting services, when Stash is in maintenance mode.
As an indication of the unavailability time that can be expected, in Atlassian's internal use we have seen downtimes for Stash of 7–8 minutes with repositories totalling 6 GB in size when using the Stash Backup Client. For comparison, using Stash DIY Backup for the same repositories typically results in a downtime of less than a minute.
DIY Backup backs up all the same data as the Stash Backup Client:
このセクションでは、次のツールを使用する、完全な DIY Backup ソリューションを紹介します。
bash
-スクリプト用jq
- an open source command line JSON processor for parsing the REST responses from Stashpg_dump
(または sqlcmd
) - PostgreSQL データベースのバックアップ用rsync
- ファイルシステムのバックアップ用tar
- バックアップ アーカイブの作成用このアプローチに小規模な変更を加えることで、以下で DIY Backups を実行するために使用できます。
Download the scripts from Bitbucket.
The scripts below are for example only and are not kept in sync with the scripts found in the Bitbucket repository.
These scripts implement one particular solution for performing a DIY Backup:
#!/bin/bash ########################################################## # Configure the settings in this section to suit your Stash installation. SCRIPT_DIR=$(dirname $0) # Contains all variables used by the other scripts. source ${SCRIPT_DIR}/stash.diy-backup.vars.sh # Contains util functions (bail, info, print). source ${SCRIPT_DIR}/stash.diy-backup.utils.sh # Contains functions that perform lock/unlock and backup of a Stash instance. source ${SCRIPT_DIR}/stash.diy-backup.common.sh # The following scripts contain functions which are dependant on the configuration of this Stash instance. # Generally every each of them exports certain functions, which can be implemented in different ways. # Exports the following functions: # stash_prepare_db - for making a backup of the DB if differential backups a possible. Can be empty. # stash_backup_db - for making a backup of the Stash DB. source ${SCRIPT_DIR}/stash.diy-backup.postgresql.sh # Exports the following functions: # stash_prepare_home - for preparing the filesystem for the backup. # stash_backup_home - for making the actual filesystem backup. source ${SCRIPT_DIR}/stash.diy-backup.rsync.sh # Exports the following functions: # stash_backup_archive - for archiving the backup folder and puting the archive in archive folder. source ${SCRIPT_DIR}/stash.diy-backup.tar.sh ########################################################## # This section does NOT need any configuration. # The actual back up process. It has the following steps: # Prepare the database and the filesystem for taking a backup. stash_prepare_db stash_prepare_home # Locking the Stash instance, starting an external backup and waiting for instance readiness. stash_lock stash_backup_start stash_backup_wait # Backing up the database and reporting 50% progress. stash_backup_db stash_backup_progress 50 # Backing up the filesystem and reporting 100% progress. stash_backup_home stash_backup_progress 100 # Unlocking the Stash instance. stash_unlock # Making an archive for this backup. stash_backup_archive ##########################################################
The stash.diy-backup.sh
script above references the following scripts:
Once you have downloaded the Bash scripts, you need to customize two files:
stash.diy-backup.vars.sh
stash.diy-backup.sh
for your setup.
For example, if your Stash server is called stash.example.com,
uses port 7990, and has its home directory in /stash-home
, here's how you might configure stash.diy-backup.vars.sh
:
#!/bin/bash
# Used by the scripts for verbose logging. If not true only errors will be shown.
STASH_VERBOSE_BACKUP=TRUE
# The base url used to access this Stash instance.
STASH_URL= http://stash.example.com:7990
# The username and password for the account used to make backups (and has permission for this).
STASH_BACKUP_USER= admin
STASH_BACKUP_PASS= admin
# The name of the database used by this instance.
STASH_DB= stash
# The path to Stash home folder (with trailing /).
STASH_HOME= /stash-home
# The path to the working folder for the backup.
STASH_BACKUP_ROOT= /stash-backup
STASH_BACKUP_DB=${STASH_BACKUP_ROOT}/stash-db/
STASH_BACKUP_HOME=${STASH_BACKUP_ROOT}/stash-home/
# The path to where the backup archives are stored.
STASH_BACKUP_ARCHIVE_ROOT= /stash-backup-archives
The supplied stash.diy-backup.vars.sh
is written to use PostgreSQL, rsync, and tar by default. But if you want to use different tools, you can also customize the top section of this file to use different tools as follows:
# The following scripts contain functions which are dependant on the configuration of this Stash instance.
# Generally, each of them exports certain functions, which can be implemented in different ways.
# Exports the following functions:
# stash_prepare_db - for making a backup of the DB if differential backups are possible. Can be empty.
# stash_backup_db - for making a backup of the Stash DB.
source ${SCRIPT_DIR}/ stash.diy-backup.postgresql.sh
# Exports the following functions:
# stash_prepare_home - for preparing the filesystem for the backup.
# stash_backup_home - for making the actual filesystem backup.
source ${SCRIPT_DIR}/ stash.diy-backup.rsync.sh
# Exports the following functions:
# stash_backup_archive - for archiving the backup folder and moving the archive to the archive folder.
source ${SCRIPT_DIR}/ stash.diy-backup.tar.sh
DIY Backup が機能するには、2 つのディレクトリを作成する必要があります。
${STASH_BACKUP_ROOT}
is a working directory (/stash-backup
in our example) where copies of Stash's home directory and database dump are built during the DIY Backup process. ${STASH_BACKUP_ARCHIVE_ROOT}
is the directory (/stash-backup-archives
in our example) where the final backup archives are saved. Bash スクリプトが以下の条件を満たしている場合、このスクリプトをあらゆるホストで実行できます。
${STASH_BACKUP_ROOT}
and ${STASH_BACKUP_ARCHIVE_ROOT}
directories,${STASH_HOME}
ディレクトリへの読み取りアクセス、curl
commands on the Stash server. This is true regardless of whether you have an instance of Stash Server or Stash Data Center. It doesn't matter whether the filesystem access is direct or over NFS, or whether the network access is direct to a Stash node or to a load balancer / reverse proxy.
stash.diy-backup.vars.sh
が正しく設定できたら、ターミナル ウィンドウでバックアップを実行してください。
$ ./stash.diy-backup.sh
The first time you run the backup, rsync
will do most of the work since the /stash-backup
working directory is initially empty. This is normal. Fortunately, this script performs one rsync
before locking Stash, followed by a second rsync
while Stash is locked. This minimizes downtime.
2 回目以降のバックアップ実行時、/stash-backup
にはすでにファイルが存在するため、バックアップ プロセスはより短時間で完了します。表示される出力は次のようになるはずです。
When restoring Stash, you must run the
stash.diy-restore.sh
script on the machine that Stash should be restored to. In order to ensure accidental restores do not delete existing data, you should never restore into an existing home directory.
The new database should be configured following the instructions in Connecting Stash to an external database and its sub-page that corresponds to your database type.
${STASH_BACKUP_ARCHIVE_ROOT}
ディレクトリにある利用可能なバックアップを確認するには、次のように入力してください。
$ ./stash.diy-restore.sh
次のように表示されるはずです。
バックアップを復元するには、stash.diy-restore.sh
を、ファイル名を引数として実行してください。
$ ./stash.diy-restore.sh stash-20140320-092743-063.tar.gz
出力は次のように表示されます。
必要に応じて、実行中のバックアップ操作をキャンセルできます。
バックアップのキャンセル方法
This section is optional and provides background information about how you might use the Stash REST APIs if you need to rewrite the DIY Backup scripts described above in your preferred language or to customize them heavily.
ここでは、Bash での curl
コマンドを示していますが、任意の言語を使用できます。
次の手順が含まれます。
Before you lock Stash you can perform any preparation you like. It makes sense to perform as much processing as possible before you lock Stash, to minimize downtime later. For example, you could perform an rsync
:
rsync -avh --delete --delete-excluded --exclude=/caches/ --exclude=/data/db.* --exclude=/export/ --exclude=/log/ --exclude=/plugins/.*/ --exclude=/tmp --exclude=/.lock ${STASH_HOME} ${STASH_BACKUP_HOME}
The next step in making a backup of a Stash instance is to lock the instance for maintenance. This can be done using a POST request to the /mvc/maintenance/lock
REST point (where STASH_URL
points to the Stash instance, STASH_BACKUP_USER
is a Stash user with backup permissions, and STASH_BACKUP_PASS
is this user's password).
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X POST \ -H "Content-type: application/json" \ "${STASH_URL}/mvc/maintenance/lock"
{ "unlockToken":"0476adeb6cde3a41aa0cc19fb394779191f5d306", "owner": { "displayName":"admin", "name":"admin" } }
If successful, the Stash instance will respond with a 202 and will return a response JSON similar to the one above. The unlockToken
should be used in all subsequent requests where $STASH_LOCK_TOKEN
is required. This token can also be used to manually unlock this Stash instance.
Next, all connections to both the database and the filesystem must be drained and latched. When backing up the database, we have two options:
Both options make a POST
request to /mvc/admin/backups
but if we require a custom database backup (the second option) we need to add the ?external=true
parameter (requires Stash 2.12+):
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X POST \ -H "X-Atlassian-Maintenance-Token: ${STASH_LOCK_TOKEN}" \ -H "Accept: application/json" \ -H "Content-type: application/json" \ "${STASH_URL}/mvc/admin/backups?external=true"
{ "id":"d2e15c3c2da282b0990e8efb30b4bffbcbf09e04", "progress": { "message":"Closing connections to the current database", "percentage":5 }, "state":"RUNNING", "type":"BACKUP", "cancelToken":"d2e15c3c2da282b0990e8efb30b4bffbcbf09e04" }
If successful the Stash instance will respond with 202 and a response JSON similar to the one above will be returned. The cancelToken
can be used to manually cancel the back up process.
If you use Stash to back up your database (i.e., you POST
to /mvc/admin/backups
without the ?external=true
parameter), Stash will save the complete database backup (in a vendor independent format) to a file name of the form
${STASH_HOME}/export/backup-admin-20140325-223629-897Z.zip
Your backup process must include this file in the final archive.
Part of the back up process, even if Stash is doing a database backup, includes draining and latching the connections to the database and the filesystem. Before continuing with the back up we have to wait for the Stash instance to report that this has been done. To get details on the current status we make a GET
request to the /mvc/maintenance
REST point.
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X GET \ -H "X-Atlassian-Maintenance-Token: ${STASH_LOCK_TOKEN}" \ -H "Accept: application/json" \ -H "Content-type: application/json" \ "${STASH_URL}/mvc/maintenance"
{ "task":{ "id":"0bb6b2ed52a6a12322e515e88c5d515d6b6fa95e", "progress":{ "message":"Backing up Stash home", "percentage":10 }, "state":"RUNNING", "type":"BACKUP" }, "db-state":"DRAINED", "scm-state":"DRAINED" }
This causes the Stash instance to report its current state. We have to wait for both db-state
and scm-state
to have a status of DRAINED
before continuing with the back up.
At this point we are ready to create the actual backup of the Stash filesystem. For example, you could use rsync again:
rsync -avh --delete --delete-excluded --exclude=/caches/ --exclude=/data/db.* --exclude=/export/ --exclude=/log/ --exclude=/plugins/.*/ --exclude=/tmp --exclude=/.lock ${STASH_HOME} ${STASH_BACKUP_HOME}
ここに示す rsync オプションは例に過ぎませんが、バックアップ プロセスに必要なファイルのみを含め、それ以外を除外する方法を示しています。詳細な説明については、rsync
またはお好きなツールのドキュメントを参照してください。
データベース バックアップを作成する際には、ベンダー固有のバックアップ ツールを使用できます。たとえば、PostgreSQL を使用している場合は pg_dump
を使用できます。
pg_dump -Fd ${STASH_DB} -j 5 --no-synchronized-snapshots -f ${STASH_BACKUP_DB}
While performing these operations, good practice is to update the Stash instance with progress on the backup so that it's visible in the UI. This can be done by issuing a POST
request to /mvc/admin/backups/progress/client
with the token and the percentage completed as parameters:
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X POST \ -H "Accept: application/json" \ -H "Content-type: application/json" "${STASH_URL}/mvc/admin/backups/progress/client?token=${STASH_LOCK_TOKEN}&percentage=${STASH_BACKUP_PERCENTAGE}"
The Stash instance will respond to this request with an empty 202 if everything is OK.
The amount of progress that Stash displays to users is calculated differently, depending on whether you used the external=true parameter
with the /mvc/admin/backups
REST endpoint described in the section above.
external=true
, Stash backs up the database and divides the 100 per cent progress into 50% user DIY Backup, and 50% Stash (preparation + database).external=true
, your code must handle backing up of both the filesystem and the database, and Stash divides the 100 per cent progress into 90% user DIY Backup, and 10% Stash preparation.Once we've finished the backup process we must report to the Stash instance that progress has reached 100 percent. This is done using a similar request to the progress request. We issue a POST
request to /mvc/admin/backups/progress/client
with the token and 100 as the percentage:
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X POST \ -H "Accept: application/json" \ -H "Content-type: application/json" "${STASH_URL}/mvc/admin/backups/progress/client?token=${STASH_LOCK_TOKEN}&percentage=100"
The Stash will respond with an empty 202 if everything is OK. The back up process is considered completed once the percentage is 100. This will unlatch the database and the filesystem for this Stash instance.
The final step we need to do in the back up process is to unlock the Stash instance. This is done with a DELETE
request to the /mvc/maintenance/lock
REST point:
curl -s \ -u ${STASH_BACKUP_USER}:${STASH_BACKUP_PASS} \ -X DELETE \ -H "Accept: application/json" \ -H "Content-type: application/json" \ "${STASH_URL}/mvc/maintenance/lock?token=${STASH_LOCK_TOKEN}"
The Stash instance will respond to this request with an empty 202 if everything is OK, and will unlock access.