The product was based on a caching web server with a filter that would keep the Internet safe for school kids.  It had a caching web filter, a walled garden email system, and various other useful tools and applications.  This was part of my adventures in Chichester.

Porn, pumpkins and Apple

The underlying tech was bought in, so as senior developer I was initially responsible for reskinning it to match a look that some fancy design house had produced (in table layout, back before CSS was something we could rely on).  Instead of icons being squares seen straight on, they were zoomed in to part of the design, then skewed as if they were printed on after dinner mints seen from above and to the side, casting a shadow.  As I had to design a LOT of incidental icons, this was a pain in the arse - first I designed the icon, then I had to cut most of it out, then I had to apply it to a template so it looked 3D, which sometimes involved extending the design round the sides of the block.

I then had to integrate and in some cases replace the back-end code. For instance the original software allowed people to type in a new IP address for the server and dropped that unchecked in to the network configuration.  If you didn't notice that you'd hit the comma key instead of full stop, tough, you had a box that was no longer on any network.  I took great pains to replace that with something that checked both on the back and and with JavaScript on the front end.

Have I ever mentioned I had to write a search engine style spider to hunt for porn, so that we could populate the blacklist? Yeah that was an interesting couple of days.  I tested it on some non-porn sites first, and got us blocked from a news site (Slashdot) because I forgot to turn it off overnight and it was creating too much traffic.  Doh.

I remember my boss went to one sales pitch where the school already had a filtering product, but that was only based on word filtering without a blacklist to back it up.  He brought up some photo-based absolute filth and made the sale.

We had the only product in its class that ticked every single box on the BECTA accreditation list, even though our competitors were from much bigger companies (although partly this was because we could license in a filter for streaming media from the people who made RealPlayer).

This lead to us creating lightly skinned versions for a couple of other companies - one was Apple(!), who passed accreditation and then we never heard from them again; and another company that had done something to annoy BECTA and didn't pass despite it being the same software that had already passed twice with slightly different icons.

One of our competitors eventually said they wanted to buy us, turned up at the offices for the royal tour, were given access to the demo site, then turned around and created their own version that even copied the error messages.  Oh well.

I need warp speed in three minutes or we're all dead

You know how in Star Trek, Scotty (or one of his successors) says that it'll take so many hours to fix something, and the captain says he's got minutes or everyone's going to die?  So here's an incident that shows there's some truth in that cliché.  Just don't expect warp drives. Or death.

We'd just finished a major milestone: the latest upgrade was done, tested and basically ready to push out to customers.  An XBox was purchased as a thank-you (no games and you had to play it in the office, but better than the usual nothing), and I was ready for a few days of decompressing and completely not thinking about this product for the first time in many weeks.

Then the company supplying the underlying technology decided that we couldn't use their toolbox.  We could use all the tools, and all the underlying platform, just not the way the links to the different back-end pages needed to maintain everything were displayed.  I think they were a bit annoyed at an unpaid bill or something.

So my boss estimates it'll take two weeks for me to strip it all out and start again. He goes to the board.
"It'll take Rich at least four weeks", he lied.
"You've got two," they inform him.
"I don't know if that'll be enough, but I'll try and make it work."

So I'm standing there playing Halo and he comes over and apologetically explains the situation.  Bloody hell.  OK.  And then I go back to playing Halo.  For two days.

I can see him looking nervously out of the goldfish bowl that is his office window, but he says nothing.  He knows me well by now and just trusts I'll do what needs to be done.  Eventually.  Maybe.  I'm kind of enjoying that he's nervous.

Then it hits me.  A few weeks earlier I'd written myself some Perl code to create web pages that faked navigating through a computer's filing system.  I needed it to allow people to download the applications I'd written, and I'd even used it on Acorn User's website so people could browse old cover disc software.  All I needed to do was create a directory structure that mirrored the navigation of the toolbox, and switch the download a file part to follow a URL instead.

I spent a day modifying the code that built a toolbox using the names and implied URL of the directories, adding a simple scripting language that could override them if necessary (trickier URLs, more precise tool names, change sort order).  I even wrote a Perl script to seek and destroy every use of the old inline toolbox code, which saved hours of tedium. I then spent a couple of days drawing custom icons for every single option, because why not?

It was easier to maintain - any staff member could copy or move a directory via Samba to change the nav - and it was easier for people to navigate than the inline floating nonsense it replaced. And code, icons and two days of Halo - still less than a week.

Hidden depths

Someone watched the Sandra Bullock film The Net, and apparently there was a plot device where clicking on a pi symbol hidden on some web pages gave you access to... oh, I dunno, it wasn't a great film.  So we added a pi symbol to the toolbox and linked it to a bunch of games - Flash or DHTML versions of Lemmings, Asteroids and the like.  I think it was so that if we were bored out on installs we'd have something to occupy us, although we made sure the school techs knew about it so they could get some too.  Anyway the techs tell the teachers, the teachers let the kids in, and pretty soon we have to get rid of it all because shipping knockoff games in a commercial product probably doesn't end well.

The next person that mentions Cachemonkey is fired

With the problems we'd had with our main supplier, some of the problems the software had in it, and the money we were paying them (or, at times, not), it seemed to us techies that we could easily replace it with something built from open source components - Apache as the web server, Squid as the proxy, filtering done by a Squid plug-in.  By this point I'd already replaced and improved upon much of the toolbox, so that's just cut and paste.  We researched it all, came up with a project name*, put together a proposal, and took it to the Big Boss.

Not interested in the slightest.

This seemed crazy, so we grumbled and made sure that whenever something went wrong with the bought-in solution he knew our feelings on the matter.  To the point where he actually stood in the middle of the tech room and said "the next person that mentions Cachemonkey is fired!", and that was that.

Of course that wasn't that.  We just waited until he went on holiday for a week, the boss gave me a spare PC base unit, and I built it anyway.

Not the full thing, not in a week, but Linux was installed and configured, and it appeared as though the product was running due to a from-scratch recreation of the layout in PHP (then a new language to me, which I wanted to get in to more). Many of the options didn't actually do anything, but there was enough to fake it.

 

Big surprise, we weren't fired.

Although Cachemonkey never replaced the main product, some of our larger customers had been asking about a management console so the new code got repurposed.  It would ping scan all of the boxes in a metropolitan area network, and any boxes that didn't respond got re-pinged for a few seconds until it became obvious that it wasn't going to respond and was flagged as down.  The box also served as an edge cache for updates, so that customers could choose when to apply patches and filter updates.

* Cachemonkey - because all of our project names were whimsical and animal-based: monkey this, weasel that...

That seems like a really bad idea

We eventually added in a number of learning environments (interactive pages that taught science and maths through games) - there were several competing products, including one that we owned, and I had to get them all integrated.

Then we sold reskinned versions of the main software to one of these competitors.  I started to build a version with just their learning environment when the big boss had the bright idea that no, we'd include all of the learning environments so schools could just pay to unlock any and all of them. So basically: install, but hide, competing products to the company we were selling this to.  I told him in no uncertain terms what a Bad Idea this was, but I was overruled.  Apparently he had decided that what they didn't know wouldn't hurt them.

Of course they took the server apart, found the competing software, and went absolutely apeshit.

So who did they demand go all the way to London to personally fix the problem?  Was it the big boss?  No, they wanted the senior developer - me.  Anyone would have been able to run a few delete commands but they wanted the head guy to make a point.

I make it to their offices after 10am, and basically after getting shouted at a bit I was shoved in to a room with a monitor, a keyboard, and 93 boxes of neatly packed computer base units, and instructions that I was there as long as it took to get any competing software off every one of those servers.  I needed to be out of London before rush hour, so the expectation was I'd be there for three days.

I suppose I'd better just get started then.  I unpack the first server, plug it in, wait for it to start up, log in, type about three commands, shut it down, put it back in its box.  Soooooo looooong!  This is back when hard drives were real hard drives, not nice fast SSDs.  It was going to take me ages! (Rough maths - 93 boxes at about 10 minutes each, that's roughly 15.5 hours - so yeah, three days, when you include travelling to and from London.)

So 3:30pm rolls around and I'm finished, I'm heading for my favourite coffee shop and then home.  I haven't left the room in over five hours - no drinks, no toilet breaks (those two may be connected), haven't made a sound all day.  I say something terse like "Done. See ya." and breeze past my erstwhile captors.
"Wait - you've finished? Already?" they said incredulously.
"Pick any five and try them," I advised/dared them.

It was nothing particularly clever - I got it done by basically building an assembly line and zoning out so that I didn't even have to think about it.  You didn't need a monitor to power up a computer, nor to shut it down - just to see the commands you're typing.  And you don't need power to take a computer out of its box or put it back in again.  I got up to about six computers at once - two in various states of unboxing, one starting up, one plugged in to the monitor to take commands, one powering down and one going back in the box.  For over five hours straight.  Thanks boss.