Implement backup infrastructure

Description

The current infrastructure is based around rsnapshot and file copies.

A new solution based on Attic and its snapshot-enabled repository is to be implemented.

Additionally, the QCOW2 format used for virtual disks in the KVM cluster also offers snapshots. Those should be investigated and possibly integrated into the backup infrastructure as a simpler form of backups at a lower-level.

Environment

None

Attachments

1

Activity

Show:

Giovanni Tirloni September 4, 2015 at 1:36 PM

We've had a few weeks of backups using Attic and everything seems to be working as expected.

Giovanni Tirloni August 18, 2015 at 5:59 PM

The backup code has been running and seems stable. It's triggered by a backup.sh script in the ops repository (ops/ansible/bin/backup.sh) that initiates database dumps first and then initiates attic to perform the remote backup.

Giovanni Tirloni August 3, 2015 at 9:46 PM

Attic has been implemented throughout all the new infrastructure (KVM-based) with an Ansible role and supporting scripts (see: http://github.com/inclusive-design/ops/ansible/roles/attic and the scripts in the bin directory). The original Attic role was enhanced to support local and remote repositories as well as pruning, among other changes.

The server tor1-prd-bkp01 was deployed with a 1TB area for backups (running on tor1-prd-srv06, away from the other VMs). Attic is configured to use remote repositories over SSH, one per server (see /backup/attic). Each server has a passwordless SSH key that is used to connect to the central backup server as the 'attic' user (in the future this could be changed to one user per server but I felt it was too much complexity for now).

Next step is to request Yong/Paul for an area in their CommVault backup system to store our backups off-site.

Fixed

Details

Assignee

Reporter

Priority

Created July 29, 2015 at 2:13 PM
Updated September 4, 2015 at 1:36 PM
Resolved September 4, 2015 at 1:36 PM