Today, I want to talk a little backup backups. Specifically, how to backup your Openstack environment. But not only how to backup the contents of your Openstack environment, but how to backup Openstack itself.
The thing to keep in mind here is that OpenStack is based around a modular architecture in which a number of different components can be combined together to offer cloud services on standardized hardware. These modules are freely available under the Apache license.
Backing up OpenStack
Backup solutions are typically developed with either operating systems or applications in mind. OpenStack is neither. OpenStack is merely a collection of components that can be combined to provide various types of cloud services. As such, OpenStack administrators must consider what needs to be backed up and how to perform the backup.
OpenStack backups should focus on backing up configuration files and databases for Openstack itself. The configuration files can be backed up at the file level since Openstack is just software running on a Linux machine.
The /etc/nova /var/lib/nova folders should be backed up on both the cloud controller and the compute nodes. However, you must exclude the /var/lib/nova/instances folder on any compute nodes. This folder contains live KVM instances. Restoring a backup that was made of a live KVM instance will typically result in an unbootable image.
One of the most important folders to include in your backup is /etc/swift/. This folder contains the ring files, ring builder files, and swift configuration files. If the contents of this folder are lost, the cluster data will become inaccessible. As such, it is a good idea to copy the contents of this folder to each storage node so that multiple backups exist within your storage cluster.
Some other folders that contain configuration data and should be included in your backups include:
/etc/keystone
/var/log/keystone
/etc/cinder
/var/log/cinder
/etc/glance
/var/log/glance
/var/lib/glance
/var/lib.glance/images
/var/log/keystone
/etc/cinder
/var/log/cinder
/etc/glance
/var/log/glance
/var/lib/glance
/var/lib.glance/images
In addition to the folders listed above, there are also several databases that need to be backed up. Typically the databases will reside on the cloud controller, which doubles as a MySQL Server. This server hosts databases related to the Keystone, Cinder, Nova, and Glance components of OpenStack.
You can back up these databases by using the mysqldump command. The command requires you to specify the names of the databases that you want to back up as well as an output file. For example, if you wanted to back up the keystone database to a file named KeystoneBackup, you could do so with the following command:
# mysqldump –opt keystone > KeystoneBackup.sql
As a shortcut, you can substitute the all-databases parameter in place of the database name. For instance, if you wanted to back up all of the databases to a file named MyCloud, you could use the following command:
# mysqldump –opt –all-databases >MyCloud.sql
What's missing?
Backing up configuration files and databases will allow you to protect your OpenStack configuration, but there are some things that are not protected by this type of backup. This method does not protect individual objects within object storage. Similarly, block storage data is also left unprotected. According to the OpenStack documentation, these types of data are left for users to back up on their own.
You can use any compatible backup application. The OpenStack documentation basically says that it is up to the users to back up data residing on the virtual machines that they create. As such, the backup application would have to be compatible with the virtual machines. It should be noted at this point that users of the public cloud have the same problem. Public cloud providers also leave it up to the user to backup their virtual machines and the data that this virtual machines use.
Of course, this raises the question of how you can better protect an OpenStack cloud. One thing to keep in mind is that like any cloud environment, OpenStack makes use of server virtualization. In fact, OpenStack is designed to work with a number of different hypervisors. You can see the full hypervisor support matrix at:https://wiki.openstack.org/wiki/HypervisorSupportMatrix
One way that you can better protect your OpenStack environment is to adopt a backup application that is specifically designed for the hypervisor that you are using. You will still need to protect the OpenStack configuration files and databases, but you can use the backup software to protect the individual virtual machines and their contents
Another thing that you can do is to adopt a backup application that is OpenStack aware. However, this is more easily said than done. As previously mentioned, OpenStack is a collection of modular components that can be used to construct a private cloud. As such, none of the major backup products come preconfigured to back up OpenStack clouds.
Backup vendor Druva recently made headlines when they announced that their inSync software now supports OpenStack based scale-out storage. The software is designed to access OpenStack storage using the SWIFT OpenStack storage access protocol. It will also have the ability to back up file and object storage, as well as mobile endpoints (laptops, smart phones, etc).
Similarly, Zmanda supports the OpenStack framework with its Amanda enterprise backup software. The software is designed to create backups from the remote server layer.
Both Druva and Zmanda back up specific OpenStack resources, as opposed to the entire OpenStack infrastructure. It should be possible to also use traditional backup apps like NetBackup for Linux to back up the required components. However, NetBackup is not OpenStack aware. It would, therefore, be the backup admin's responsibility to manually configure a backup job that includes all of the required config data and databases.
The key to adequately protecting your OpenStack environment is to determine what it is that needs to be protected and then build a backup solution to meet those needs. While there are commercial products that can back up certain OpenStack resources, those products may not offer the level of protection that you require. You may have to combine commercial backup products with script-based backup techniques.
