Site Snapshot Feature

[johnnyoffline]

In the admin, there should be the ability to take an entire site-wide snapshot with the click of a button.

"Take Site Snapshot Now."

It freezes a copy of the database in it’s current incarnation, slots, slot history, etc. everything.
Not sure how to handle actual files.

Then if a user makes a huge amount of changes to their site, they could take another snapeshot and roll back to a previous site snapshot if needed.

Snapshots should be browse-able in a logged-out only view. They don’t need to log-in to a snapshot, because the point of it is to view content differences, not manage them.

Browsing a snapshot should absolutely should have a url structure that looks like:
http://snapshot11-06-2010.clientsdomain.com/

These subdomains would only be viewable to an admin logged-in.

Browsing to them not logged-in prompts a login.

A global bar sites at the top of the snapshot that says:

Browsing Snapshot 11 January 2010:
– Restore Site from this Snapshot
– Delete this Snapshot
– Download Snapshot to File
– Import / Load Snapshot from File
– Cancel / Leave

[boutell]

Managing snapshots is fairly easy. But browsing snapshots in a read-only view as you've suggested is superhard. We'll want to break this up a bit.

This feature will probably be MySQL-specific for sanity. Hasn't hurt WordPress? any.

Migrations will need to be triggered when a snapshot is loaded to make it compatible with the current code on the site. A snapshot taken from a site running newer code should be rejected with a message to upgrade the site first. We need our own simple but effective MySQL-based migration mechanism (not as scary as it sounds) since it's clear that Doctrine migrations are not ready to solve this kind of problem for plugins.

Snapshots will be BIG. Big big big. Think of all the media. This feature will be slow and possibly run into PHP execution time limit issues, we'll have to keep an eye on that.

[boutell]

This is partially implemented (the apostrophe:demo-fixtures task and the corresponding task that shrinkwraps). It is indeed quite slow.

[lsmith]

The snapshot should probably be done via CLI script, that potentially locks the current site from editing while its running. Now the question is why even bother with the DB backup part? Seems to me like this is best left to an admin, which would also make the feature less RDBMS dependent.

Now this would make the feature simply a solution to be able to easily have a read-only version to which a running installation can easily switch running inside a separate DB?

[tboutell]

We don't mind being RDBMS dependent. It hasn't hurt WordPress? acceptance one bit.

mysqldump gives you about 90% of what people want from this feature...

[tboutell]

We've had the same discussion about the possibility of using this to set up read only instances of sites, yes.

[lsmith]

Given how things are shaping up with MySQL (aka not so good), I do think its a good idea to keep options open (mainly also supporting SQLite - great for testing - and PostgreSQL). But this could be left to others as long as you just make it possible to hook in other RDBMS for those who care.

FYI: We are mostly using MySQL these days still of course. However with the NoSQL options, we see more and more that where MYSQL used to shine being a "relaxed" RDBMS, we are not considering using NoSQL for that and a "serious" RDBMS like PostgreSQL for our relational needs.

That being said, as you pointed out doing it all within the context of a single web request is probably not feasible anyway, so I propose that there should just be a way to temporarily lock all editing, call a script (this one could be configurable to support other RDBMS and OS) to make a backup, add another script to load the backup and then be able to "permanently" mark at DB snapshot as read only.

[tboutell]

Re: the status of MySQL, it's an open source project. When people didn't like the direction the new maintainers of XFree86 were taking, they forked it in a real hurry and created x.org. If there is a real threat to the future of MySQL something similar will undoubtedly happen. In the meantime it has tremendous momentum and the concern is mostly theoretical.

That said, our code is not *intentionally* MySQL-specific. We just have no business case whatsoever for spending time on the issue. We will accept patches that address compatibility *without* breaking MySQL or wrecking performance. It just doesn't make sense for me to spend work hours fighting with SQLite or PostgresQL or Oracle.

[lsmith]

I wasnt really focused about Oracle MySQL, but more that its been a long while (imho since 4.1) since there has been a good new stable release. Anyway, I am happy as long as you remain open to patches that cover other RDBMS and I do acknowledge that with a feature like this its very hard to properly support different RDBMS, so I do think its wise to be cautious what to official support. But with an approach like I outlined above its easy for people to add support for a new RDBMS .. or even a peculiar new or old version of MySQL or one of the emerging variants of MySQL.

[tboutell]

We are talking about supporting XML import and export, mainly because it would provide a great way for third parties to write exporters from many CMSes while we would maintain just one importer (and a compatible exporter).