PowerBlogs.Com Development

The Redundancy Plan

Ok, here's the plan. It's ambitious but achievable, and worth it. (And it shouldn't drive prices up, either.)

Servers will work in pairs, though the implementation will allow for trios, etc. They will both serve web pages as well as take new posts, comments, etc. The servers will automatically sync with each other in real-time on a peer-to-peer basis, so they'll be capable of operating independently if one of them goes down or they get separated, and they'll automatically sync up as soon as they can talk to each other again. (Each server will have a sync daemon which stores all updates with a revision number; when a server reconnects it will just ask for all udpates from the last revision it knew about — this will ensure smooth resyncing.)

Reports will be generated by an off-site server, taking the load off of the web servers to ensure that report generation will never bog them down. (I'll have to get a feel for how quickly I'll be able to process the reports, but it might be reasonable to get the processing up to every 2 hours, which would be really nice.)

All of the servers will be DNS slaves (listed in the master record) so that there won't be any single point of failure. DNS will hit the two servers in round-robin fashion.

As I said, it's ambitious, but it will result in a system with no single point of failure that should be able to incorporate new servers quickly and seamlessly. Getting this done is going to be top priority here at Powerblogs. The transition to the two redundant servers should be completed within two weeks.

I'll keep you updated.

downtime

One of the Powerblogs servers is unable to reach the network. We're currently working with our upstream provider to get the issue fixed (they're not sure yet, but it looks to be either the physical infrastructure or the switch). This is our new, more powerful server.

Obviously no one's happy about this. We were lulled into a false sense of security by our previous good experiences with them.

Obviously, we can't let things continue this way. We're working now on a redundancy plan to make our servers fully robust against upstream provider issues. I'll try to get the details out tonight once we're reasonably settled on what the plan and new design is going to be.

Update: we're back up. Apparently it was a network cable (now replaced) that had us down for so long.

I'll be updating with the plans for a redundant server as they firm up.

Update Service Pings

It is now possible to disable sending pings to the update services. If you're not interested in the update services, you now don't need to waste time or bandwidth on them. (The default is still to send them.)

Fixed a few problems with comment accounts

Comments accounts were supposed to be case insensitive, but there was one part of the code (mostly affecting reseting passwords) which wasn't; consequently people who entered their accounts with mixed case weren't able to reset their passwords. This has now been fixed.

Also, for some reason I was never able to figure out, once in a great while someone would have problems with cookies and would get told, when trying to post with their comment account, that the display name was already in use. I still don't get what was going on, but I've just disabled the relevant section when a person is using a comment account (because the display name is forced to be what's specified for the comment account), and this seems to have fixed things.