A long and twisted road

2024 Edit Update: This article was written in 2007, as part of the rollout of our second generation public website and client management system. The ASP based system it talks about replacing was v1 of our site… the incarnation you’re currently reading is… v6 or maybe v7 I think? I could probably do a full article on both the evolution of web programming over the years using purenrg.com as a case study, or even a full retrospective in “what the hell went wrong?” with this particular rebuild. Titling this “long and Twisted Road” was more apt than I realized at the time. I was about 3 months into development on the new Drupal based site whent his was written… Thought it would all be wrapped up by February at the latest. It went live in June.

Technological progress is like an axe in the hands of a pathological criminal.

— Albert Einstein

The current (well, the “old system” if your reading this) Pure Energy Systems website / management system was originally conceived many many moons years ago, during the earliest days of planning what would become the current incarnation Pure Energy Systems. At the time coding began on the actual system we had a rough outline of what we wanted the system to do, and what functionality it was going to need; the list looked something like this:

• Take orders, notify when an order comes in
• Allow clients to submit a trouble ticket, notify when a new ticket comes in.
• Give us a way to keep track of client information such as billing dates, plans, account notes.

That was it, the entire scope of what we started with summed up nicely in three little lines. Of course before the system even went live we realized some additional things would be needed (basic Payment integration, allowing for the system to generate links to Paypal or other payment services depending on the method the client chose on the order screen, and also the plan in question), but I don’t think anybody really grasped the scale of what would be eventually needed, or how many things we left out of the list entirely.

Over the years since a lot of what most people would consider “basic” features were added into the system, things that should have probably been there from the start, but were simply missed. Features like a way for clients to change their “contact email address” on an account, without having to email support specifically asking for it to be changed. A way for us to pull a list of email addresses for every client on a specific server (So we could mass-email everyone on a specific server when an upgrade or downtime was coming up), a quick report to let us see which accounts were past due in their billings, you know, little things like that. 🙂

After a couple of years of this, and the chorus of discord internally with the system was beginning to grow astronomically. It wasn’t just a matter of little features being missing anymore, there were simply features/improvements we wanted to implement, that our management system was holding us back from being able to offer. Things such as moving to a more cost effective domain registration system for our clients, or supporting multiple hosting accounts under one “account” and therefore one “bill” where just not possible within the framework of the old site, at least, not without a major amount of development time and resources being committed to the project.

The decision was made that we would have to either:

1. Invest quite a bit of time/effort into adding a slew of new features into the current system.
2. Procure one of the many “out of the box” billing/client management software packages for hosting providers that is now available on the open market.
3. Re-create our own in-house system from scratch.

Investing time/effort into our old system was the first idea, however there was one hang up; The old system was written entirely around Microsoft products… IIS / ASPi (VBScript) / SQLServer. We used to work with IIS/ASP back before our public launch, we had a number of web-projects that all ran on ASP based sites; our own needs for affordable hosting for said ASP sites is actually what prompted our journey into this business in the first place. But Windows / IIS hosting has never been a service we have marketed to the public at large, we’ve always felt that open-source alternatives such as Apache/Linux made for a more cost-effective hosting platform for clients, and that’s the area we’ve focused ourselves on. Over the years since then those original projects have either moved on (to other owners, other fates, or redeveloped in PHPi), and the only “Microsoft-tech” based site we’ve run for quite sometime was our very own website / ordering / management system. Add into this the fact that “nobody codes in ASP anymore, everything is ASP.net now!”, and just how unfit the old system was for our real needs, we realized we were looking at a pretty much a complete re-write of the old site, not just implementing some additional features into the system. So the idea of investing more time/effort into the ASP-based site and thus re-committing ourselves to an ASP solution for years to come didn’t make a whole lot of sense versus moving everything over to a PHP-based solution if possible.

The idea of recreating everything from scratch as a PHP site wasn’t bad either, but there were already a half-dozen reputable hosting provider management software packages on the market, most all of them in PHP, most all of them loved by many, hated by many others. We decided to wander down the “Out of the box” road and see where it took us. We looked at a number of packages from different developers, we tried out many trial versions, and ultimately decided on the product we were going to use. It wouldn’t fit our “Desires” list 100%, but it was closer then anything else out there, and it definitely met everything on our “Must Have!” list. So we had a plan. We started a redesign of our site to incorporate the new package, and just before placing the actual order for said software, we start hearing rumors of a brand new major release to the software is coming “In a couple of months”. After confirming with the developer that they were working on a major new release (which was to be “lightyears beyond the version out there now” and would “Support everything you desire” , but came with one snafu “It’s not exactly a clean upgrade from the current version”… We decided to wait for this new release before progressing any further. Quite simply we didn’t want to go through all the work of building a site around the current version, get clients familiar with it, and then 4 months later be doing everything over again.

However, “a couple of months” stretched into six months, then almost a year, and the new release never materialized. Then when it did, we discovered why it took so long, it wasn’t just an upgrade, it was a whole new system. They had re-written everything from scratch, and this new version, while it did offer many new features we had waited for, was MISSING many features that had been standard in the previous version. Long story short, they’re now something like 17 releases into this new version, and it’s still buggy and missing some of the features that were in the original product we looked at and fell in love with so long ago.

We could have, I suppose, went with their “old” version and used that, but there’s just something about building a site (and a business) around what is effectively already an “orphaned” project at this point that didn’t appeal to us. I know they still occasionally patch the old version, but it’s not going to get any real upgrades or features added at this point. So a few months ago when the topic of revamping our own website came back up once again, we once again started looking at the various packages on the market to see if there was anything that would fill our needs.

Some are missing key features we absolutely need. Some are incredible looking packages (HSphere!) but are more then just account management, they encompass everything (client control panel, etc) and we don’t want to move from cPanel at this point in time. And a couple came very, very close to what we needed, and most of what we wanted.

But then we run into the big one “Integration”. We’re huge fans of integration these days. After years of:

• Every hosting account has a cPanel username/password
• Every hosting account has a support site username/password for the www site.
• Every client with a domain registration with us has a username/password for the domain system

We want to simplify the whole username/password thing as much as possible, for everyone. We want:

• One username, One Password, covers as much as possible.

Yes, due to the way cPanel works, if you have multiple hosting accounts with us you will always have a separate username/password for each, within cPanel. But you should only have to remember one to log into our actual site, open tickets, everything. If you have 5 hosting accounts and 10 domains registered with us, we want you to be able to log in with ONE username/password to manage ALL of it right here on our site. If we decide to one day add forums to our site, that should use the same credentials. If we decide to add anything to the site, it should use the same credentials. This is a big one for us, it’s something of a holy-grail we’ve been wanting to achieve (or at the very least, pave the road towards) for sometime.

We don’t think that’s too much to shoot for, some developers apparently do. In all fairness, most of the products we looked at DO handle this for hosting accounts / domain registration, it’s when you start adding in things like FAQ programs, Forums, Trouble Ticket Systems, etc that they start to run into issues with multiple usernames/passwords. Or even worse, the billing software developer tries to cram everything (ticket system, faq, forms, etc) into their billing software, but only provide a marginal solution for those features (ie: their system has a FAQ system, but it’s horrible to use and featureless).

In short, we ultimately decided that the only way we were going to be truly happy with our new system and website was going to be to completely build it ourselves.

Since we also wanted a content management system to make for easier updates / modifications to the site itself, we decided on the Drupal based system you see here now. There is a bit of a learning curve (from raw php development to coding within the Drupal APIs/Functions) that we dealing with, but this is not entirely our first experience with Drupal (it is however our first run at something on this large of a scale). Using Drupal as the core framework gives us the content management system we seek for the site itself, while allowing us to create our own modules to handle our actual business related needs. All integrated under one platform, and more importantly, one username / password for both clients and staff 🙂

Will it take longer then simply dropping in a “off the shelf” product? Sure. But it’s the only way we see to get the flexibility we want going forward. And hopefully, the end result will make this long, twisted journey worth while.