Running the Sites Backend – Process Progression

Over the years, the how and why of managing the websites’ backend – the files and databases has seen a gradual progression to more complex methods.  I think in many ways it showcases the common route smaller e-commerce / online businesses progress in their processes so I figured I’d write it up.

Single Site – Everything Live

In the beginning, we had single site with everything live on the site.  So any changes we made was automatically on the website as we had to test changes ‘live’.  Bugs, code fixes, new content – it all went up live.  This meant that we had to be careful when we started deploying code and keep all the back-up files on our computer in case something went wrong and we couldn’t figure out the code fix quickly.  On the other hand, it also meant that there was only one site to ever worry about and everyone who worked on it had access to the same files (mostly – see below for potential problems).

This works fine if you don’t mind deploying code and fixes late at night or when you know there are few customers around.  It’s fine if you don’t have a lot of customers or don’t have a lot of big changes to do; but can be a mess if you have a ton of customers at any time of the day or worst; are trying to install a large upgrade / expansion / module.

Oh, the other major issue – all your developers (if you work with more than one) have access to all your ‘real’ databases including customer information. A potential privacy problem.

Staging and Production Sites

A staging site is a ‘fake’ website that (in theory) is exactly the same as your production site.  The staging site can be populated with ‘fake’ database information; reducing privacy problems while allowing you to continue to test code changes.  In addition; because the site is not live your devleopers can put up a partial fix, test it out and then come back to it at a later date (or leave you to do a test).   Timing becomes less of an issue because the staging site can be broken without affecting front-end sales.

Once a change is considered production ready, you can then download the changed files and send the files over to the live site.  This is generally a manual process and one that you have to do yourself.  Part of the reason you implemented this entire process is for privacy reasons.  It makes no sense to ask your developer to do the fix.

Of course, as any IT person will tell you – just because it worked on the staging site doesn’t mean it will work on production.  Sometimes that means a bit of scrambling; but it’s a lot less likely to be a problem.  We’ve been doing this process for the last 3 years or so; bumbling our way through multiple sites, trying to remember to take backups as necessary and keeping multiple versions of the files on our home computer.

Developer, Staging, Production & Version Control

We’ve recently grown-up and moved to a more complex system with 3 sites and version control.

The Developer sites reside on the developer’s server where they test code.  It’s the working version of the site where all the changes are tested in multiple versions till a ‘good’ fix is ready.

Then the ‘good’ fix is sent to Staging, where it’s deployed.  Here, I do the test to ensure there’s no bugs that the developer has missed.  If there are, I send the bug report to the developer who goes back to working on the Developer site before uploading the fix.  If there isn’t, we deploy to Production.

It’s very similar to the above method; except for the addition of Version Control.  We use Springloops for our version control system and can’t be happier.

Version control systems do a few important things for us:

a) it automatically keeps a repository of all files – old and new.  It keeps dates and keeps information about changes so we can ‘roll back’ to an older version with just a click of a button.  No more hunting for files and hoping we had backed it up properly; its all done.

b) deployment of code can be set up to be automatic to Staging servers and Manual with Production servers, while keeping deployment simple.  Quite literally; a click of a button again – so no more worrying if we had missed a file.  The exact same set of files get sent to both; so it removes ‘human error’ from the equation.

c) it allows multiple developers to work on the site at the same time.  Even if a pair of developers download the same file and make different changes, the software will show and indicate any conflicts.  This way, no one developer’s work is ‘over-written’ by accident.

d) it restricts access even further.  With Springloops, we provide access to the repository but not to the actual FTP site. It also lets us invite multiple developers and kick them out easily while keeping track of all the files they’ve touched.

Truthfully, I cannot be happier that we found this solution.  It allows us to get more changes done with new modules and to roll out changes easier.  It’s something I’d recommend to anyone with a website that they have numerous changes on.