About Bitbucket Mesh
Bitbucket Mesh is a distributed, replicated, and horizontally scalable Git repository storage system, which increases performance and improves the resilience of Bitbucket.
Since clustering was introduced in Bitbucket Data Center, Git repositories have been hosted on a shared file system (specifically as NFS based filesystem), which often becomes a performance bottleneck. NFS introduces additional latency, which isn’t significant for large operations, like streaming a pack file, but can be extremely punishing for small operations, like accessing refs or loose objects. Bitbucket Mesh utilizes local disks for fast read/write access and moves Git processing closer to the storage for reduced I/O latency. As a result, it greatly improves the performance of Bitbucket.
Bitbucket Data Center is designed for high availability. However, not all the elements are truly HA. The shared file system can become a single point of failure. Bitbucket Mesh offers a solution to this problem. When repositories are migrated to Mesh, they are replicated to multiple Mesh nodes. That ensures the loss of any single node has no impact on the availability of the repositories it hosted because each still has replicas available on other nodes. When Mesh nodes are brought back online, they automatically repair their replicas and are returned to service.
Bitbucket Mesh architecture allows horizontal scaling of Git storage and processing. Repositories are replicated to multiple Mesh nodes, and each replica is capable of actively serving both read and write traffic. Each replica adds capacity and does so more efficiently than adding Bitbucket Data Center cluster nodes because each has independent storage in addition to its own CPUs and RAM.
What does Mesh look like?
Mesh is most easily understood through two diagrams.
The image below represents a two-node cluster deployed with NFS-based Git repository storage. This is, in principle, how Bitbucket Data Center is deployed on all versions before 8.0.
Bitbucket DC 7.x with 3 cluster nodes and NFS storage
Now observe the architecture of a Bitbucket Data Center 8.0 instance with Bitbucket Mesh. Mesh replaces NFS as storage for Git repositories, bringing Git processing closer to the storage. Distribution and replication of data also improves availability and prevents the NFS Server from being a single point of failure.
Bitbucket DC 8.x with 3 cluster nodes and 3 Mesh nodes
Check if this version of Mesh suits your needs
As we’re continuing to build improvements to Mesh, you may find that this version of Mesh is not yet ready for you.
We recommend that you take some time to check out the below list of things we’re working on and vote or watch the items that are most important to you:
- Large, active repositories (repos with more than 10,000 refs and/or greater than 4GB in size) are not recommended yet. - BSERV-13300Getting issue details... STATUS
- Repositories with more than 10 forks that have been enabled for ref synchronization may be slow and are not recommended yet. - BSERV-13268Getting issue details... STATUS
- Rebalancing Mesh partitions is not yet supported. Adding Mesh nodes after the initial set up doesn’t increase your ability to scale the load of repositories hosted on the previously created Mesh nodes. - BSERV-13271Getting issue details... STATUS
- Removal of Mesh nodes using the UI or REST API is not yet supported. If you need to remove a Mesh node, contact support. - BSERV-13272Getting issue details... STATUS
Mesh doesn’t support self-healing immediately after disaster recovery yet. Repositories will repair after the first write is committed to each repository. Inconsistencies between nodes will exist until this occurs. Automatic self-healing is on our roadmap. - BSERV-13312Getting issue details... STATUS
- Controlling the replication factor of Mesh is not yet supported, and this may limit the ability to scale out for very busy repositories. - BSERV-13301Getting issue details... STATUS
- Migration of LFS objects to Mesh is not yet supported. - BSERV-13269Getting issue details... STATUS
- Mesh node deployment into multiple availability zones (Multi-AZ Mesh deployments) in AWS is not yet supported. - BSERV-13270Getting issue details... STATUS
Meanwhile, we encourage you to test and explore Mesh and let us know your feedback.
Ready to get started setting up Mesh?
Got other questions on Mesh? Check out these FAQs