The following is our internal process for deploying new Apostrophe client sites. We share it with you to better document useful tools like apostrophe:deploy and project:sync-content. Please note that it is not mandatory at all to use those tools with Apostrophe. But we like and recommend them and also feel this document would benefit from third-party feedback.
- The team at P'unk Avenue
Our process is to develop sites on Macs, using svn for version control and local copies of Apache and PHP to interact with the site before it is deployed. Completed code is then deployed via rsync to a staging server for approval by the client and then deployed from there to the actual production server. In some cases a staging server is not available, in which case we deploy directly to production, although our preference is naturally to use a staging server.
Note that we do not use svn for final deployment. Instead we use svn to keep the code up to date on development Macs and use various Symfony tasks (based on rsync, mysqldump and related commands) to deploy code and content as needed.
To deploy the site to production, it is necessary to sync the code first from a development Mac to the staging server and then on to the production server. This assures that each change has been properly reviewed. It also means that "hotfixes" are never made directly to a server and such changes will be overwritten by future deployments that do follow the process. Contributing directly at the code level requires svn access to the project.
The config/databases.yml file must have database settings for the production server as well as for the staging server. Examine this file to see how the staging settings are set up (it is a human-readable configuration file; take care to preserve the indentation as it is significant). If you prefer, for security reasons, not to keep this file in svn where developers can see it, we can take it off the list of files that are synced to to the production server, and you can manually maintain a special production copy there. This is common on sites where P'unk Avenue does not have direct access to the production server.
PLEASE NOTE: do not make direct changes to the database without consulting with the rest of the developers involved. In particular the various tables that make up the Apostrophe content management model should be manipulated through Apostrophe's Doctrine-based model layer and never directly with SQL statements. Otherwise the page tree and versioning system can be easily damaged. There are rare exceptions but it is best to discuss them with P'unk Avenue first.
The config/properties.ini file must contain ssh credentials (but no password) for the production server. Examine this file to see how the staging settings are set up. Since this file does not contain the password there is no security risk in adding the production settings to this file.
When syncing code to the production server for the first time, you will receive some errors when running the apostrophe:deploy task. This is normal and due to the fact that the web/index.php file has not been set up on that server yet. This file is not synced because it is specific to each server. (Note to experienced Symfony developers: this is a departure from the default "frontend_dev.php" approach that we prefer because it simplifies debugging and deployment. index.php acts as the sole "switch" that determines which settings should be used on a particular host.)
After the first code sync, copy web/index.php from staging to production manually, and edit the file so that it enables the prod environment rather than the staging environment:
ProjectConfiguration::getApplicationConfiguration('frontend', 'staging', true);
To sync code from one server to another, we use the apostrophe:deploy Symfony task. The following command will push the current code from the staging server or a development Mac to the production server. Note that this does not affect the content on the site, only the code. This is the way to deploy fixes to the code, or to initially deploy the site to production:
./symfony apostrophe:deploy production prod
You will be prompted several times for the ssh password. This command can take a long time and sometimes runs quietly. It carries out several Symfony commands:
[Locally] project:permissions project:deploy apostrophe:standby apostrophe:fix-remote-permissions [Remotely] apostrophe:standby [Locally] project:deploy apostrophe:live [Remotely] apostrophe:bump-asset-generation cc doctrine:migrate apostrophe:migrate cc apostrophe:live
Also, the project:deploy task is given specific rsync arguments that ensure rsync is not fooled by overlapping timestamps if two different developers have deployed recently. This is important to avoid surprising results especially with the APC cache.
Note that the apostrophe:fix-remote-permissions task uses a password set in config/properties.ini to request that the remote website adjust permissions on files that must be writable by both command line tasks and the webserver. The relevant section of properties.ini is:
Permissions and deployment are a source of great frustration in Symfony development: files created by Apache are usually not writable by cron jobs and vice versa. If everything is in the database you're OK, but if you need to manage files you have a problem on your hands. The best way to address this issue is to run command line tasks and Apache as the same user, which does not introduce any great new security risk, since if PHP is compromised it is still possible to call system() even when the Apache user has no shell. However, if you are not comfortable with this, apostrophe:fix-remote-permissions is a useful workaround. it invokes the aSync/fixPermissions action, which carries out the same steps as the project:permissions task, but does so as Apache.
To deploy new code from a development Mac to the staging server (which we recommend doing first before deploying anything to the production server), use this command:
./symfony apostrophe:deploy staging staging
You can then run:
./symfony apostrophe:deploy production prod
Directly on the staging server to deploy the final step to production after the client has approved the changes. If P'unk Avenue does not have direct access to the production server then this command is carried out by the client system administrator.
Our default rsync options are carefully chosen, hard-won and really effective. They replace what needs to be replaced, remove what needs to be removed and update timestamps so that APC does the right thing in every case.
But if you need to add additional options or replace our options entirely, you can. A common case is telling apostrophe:deploy (and therefore project:deploy) not to try to set permissions on a server where we don't have the access rights to do so.
You can override all of the options or just add some. It's usually a good idea to add some instead, because you stay in sync with our best practices on everything else. Note that many rsync options can be cancelled with a corresponding --no option. Here's what it looks like in properties.ini:
[servershortname] host=whatever user=whatever port=whatever ; These are the defaults you get if you don't set this option rsync_options="-azvCcI --no-t --force --delete --progress" ; On this server we can't set permissions, so don't try extra_rsync_options=--no-p
apostrophe:deploy uses the new apostrophe:standby task to put sites on hold so that race conditions do not occur during the deployment process. For sites that begin with our latest sandbox, you do not have to do anything to take advantage of this, but you can choose to edit web/apostrophe-standby/index.html as you see fit to change the standby message. You can add asset files in web/apostrophe-standby. Files in any other folder will NOT be visible on the server during the deployment. This is intentional.
For sites created before apostrophe:standby was introduced, the task has no effect until you upgrade your .htaccess file. All lines mentioning apostrophe-standby are commented out by apostrophe:live and uncommented by apostrophe:standby. See the sandbox .htaccess file for the lines you need. Take note that you need to add a RewriteCond line to EACH of your RewriteRules.
You can also copy the web/standby folder from the sandbox as a starting point for your own standby page in an existing project.
Pushing code to the production server is normal. But pushing content to another server should be done with great care and caution. Always think about what machine you are syncing from and what machine you are syncing to. There is NO way to undo this operation, unless proper backups of the database and the web/uploads folder are being made. Measure twice, cut once.
The following command, typed on the staging server, will sync content FROM the staging server TO the production server. This should only be done ONCE when the production server is first set up, and then once more after content has been frozen on staging in anticipation of launch:
ALMOST CERTAINLY A BAD IDEA (except the very first time production is set up):
./symfony project:sync-content frontend staging to prod@production
You will be prompted several times for the ssh password. This command can take a long time and sometimes runs quietly.
Note that the project:sync-content task copies both the MySQL database and the web/uploads and data/a_writable folders, as well as any other data folders specified in app.yml (usually these are the only ones).
The following command, typed on the staging server, will sync content FROM the production server back down TO the staging server. This command is quite useful for making sure the staging server’s content is a realistic test of what will happen when new code changes are eventually pushed to production. Content "lives" in production (after launch), so it makes sense to periodically refresh the content on staging with the current content of production:
./symfony project:sync-content frontend staging from prod@production
Note the use of “from” rather than “to” above. Always proofread this command carefully.
The following command, typed on a development Mac, will sync content FROM the staging server back down TO the development Mac for realistic testing:
./symfony project:sync-content frontend dev from staging@staging
Make sure you clear the cache with good old:
After syncing content. Code deployments do this automatically on the remote server.
"I don't see the fix!" "Did you clear your browser cache?" "Um..."
If you have had this conversation with your client then you understand why we added a neat feature called asset generations.
When Apostrophe generates a minified CSS or JS file, it will automatically prepend a unique "asset generation" string to it, provided that app_a_asset_generation is set. This string stays the same until the next deployment, when apostrophe:deploy remotely triggers the apostrophe:bump-asset-generation task, updating asset_generation.yml with a new random string.
If you cloned your project from our sandbox recently you don't have to do anything special to take advantage of this feature. If your project is older but your apostrophePlugin is up to date with the 1.5 stable branch in svn, you can add this to app.yml to take advantage of asset generations:
all: a: other_setting: something <?php if (file_exists(dirname(__FILE__) . '/asset_generation.yml')) require dirname(__FILE__) . '/asset_generation.yml' ?> other_setting: something
Note the blank lines before and after the <?php block. These help avoid parsing errors if the whitespace around this directive and in the asset generation file is not exactly perfect.
It is possible to clear the editing history so that old versions of text cannot be discovered via the history button. In production this is usually a bad idea, however during the transition from in-house development to the creation of content by clients it can prevent professional embarrassments:
“Why ‘production prod’ and not just production?” It’s possible for Symfony sites to have several environments on one server although we don’t recommend that practice or use it on our client projects.
“Why frontend?” Symfony projects can contain several sub-applications. Our projects typically contain only one "application" because we believe in progressively enhancing the user's experience to include admin features rather than creating a typically less user-friendly "back end" application that is often neglected in the design process.
"I made a change to the code on the staging or production server and someone deployed and now the change is gone. How do I make changes that stick?" You need svn access to the project so you can participate in version control and avoid conflicts with other developers on the project. Code changes are typically made on a development laptop and then committed with 'svn commit,' never by hotfixing files on servers.
Complete developer documentation for Apostrophe, our content management system, is available in the Apostrophe manual. For client technical staff interested in contributing at a designer or developer level, that is the right place to start reading.