On 09/03/14 11:52 AM, Nico Kadel-Garcia wrote: > On Sun, Mar 9, 2014 at 11:28 AM, Digimer <lists at alteeve.ca> wrote: >> Would you mind elaborating on this? If a snapshot is a point-in-time >> image of a VM (or even normal FS), why would DB backups be at risk >> (assuming things like fsync are used)? >> >> I'm asking in general terms... no idea if this is something AWS specific. >> >> digimer > > It's a general issue. If a system snapshot is used to correctly > preserve both the disk image, and the "state" of the VM including > memory, well and good. The state is recoverable. There's always a risk > that interrupted network transactions left things in an unexpectedly > inconsistent state that the VM is not equipped to handle: I'm thinking > particularly of "wget" or other download transactions where the > download software was not intelligent enough to verify the download > before proceeding. I've been through this a lot lately with "chef" > software. It's compounded by network based filesystem transactions, > such as interactions with NFS or CIFS filesystems, which cannot be > synchronized with the OS snapshot. > > But simply relying on the disk image from such an AWS snapshot, > without recovering the full system state, is a potential adventure. > I've not myself had opportunity to play with this kind of restoration, > so I'm uncertain whether AWS allows access to the plain disk image, or > automatically would bring the full VM state with it for re-activation > of the snapshot. If you're just getting at the disk images, using > "fsync" before the snapshots is helpful, but any atomic transaction > that is in progress at the time of the disk image snapshot is not > verifiable in the atomicity of that transaction. This particularly > includes precisely the sort of "page mapped" data, sitting in RAM, > that the "fsync" command helps write to disk. > > And snapshots cheduled from outside controllers, such as automatic > snapshots, cannot be reliably synced with system specific "fsync" > database suspension commands without a great deal of integration > between the outside system, and the local host, that VM's are not > supposed to normally need. I went through great deal of this some > years back, shutting down databases, running "LVM" to get a disk > snapshot, then running "rsnapshot" against the *snapshot* to avoid > getting an inconsistent state of the database into the backup system. > > And there are some *funky* databases out there. Ask sometime about the > "Use hardlinked RCS files for source control of multiple project > branches" sometime, if you'd like to wince a lot. This is very useful, thank you kindly for sharing. I suppose I always considered the "it's like recovering for the server losing power" as "usually works" and equating that to "good enough" backup. So I suppose, at best, using snapshot images as a backup ... backup method would be valid... I could see the benefit of recovering the VM, and then if anything wasn't right, using it as the target for restoring data from the proper backup. Thanks again! -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education?