Webmaster under-pressure

Traffic Spikes bring additional running costs

For the past week the site is experiencing massive traffic spikes like never before. I'm still wondering what is it that I've done different this year compared to the previous one... I mean it's normal for the traffic to go up on FM's release day and for a few weeks more, but I certainly didn't see this outburst coming. The issues that followed up this situation altered my sleep pattern, in other words I just don't sleep.

Just to give you an idea of what happened, our all time record for users online was 813 when the last Genie Scout was released back in March, while the all time record for daily unique visitors was 19,9k. Both records were broken and now they extend to 1362 and 42,2k respectively. As a result our hard work server has been unable to cope with this kind of traffic even after utilizing load balancing and aggressive caching.

I have been in constant contact with my server admin every day. Switching to fast-cgi and adding 2GB more RAM seemed to do the trick yesterday, but unfortunately fast-cgi failed today and we had to switch back to suphp (OK, I'll stop the tech talk now). We are trying to find a solution to get rid of downtime, as it is annoying for everyone, and it's hurting our credibility for both the visitors and our Google ranking. Changing host at this point would be a bad idea, as we would need roughly 2 weeks of compiling and optimizations. So the rational option is to look for an internal upgrade and further optimization in regards to proper caching and load management.

At any case we are looking at increased running costs, but that is essential for the well being of our site and my personal health too; I really need to relax for a couple of days. In order to cover the extra costs, I've been speaking with a number of people and trying out new ways for advertising. I wish to keep that to a minimum though and ensure they won't affect your site experience and most importantly the page loading speed. I'm currently negotiating a new deal that could potentially set me free from being always on the look out.


A quick flashback

FM Scout the website launched in January 2005, but the actual date I purchased the domain was a bit earlier; 10 Nov 2004. That's right! Today is the atypical 6 years anniversary. The idea behind the site was quite simple, to redistribute FMScout. Back in the day that was the most popular program for FM... well, it's successor Genie Scout successfully continues this tradition. Until late 2007 the site was nothing more than a mere download link and a couple of crappy articles of mine, as well as dead forums on top of that. And then I decided to take action, expand, gather a team of moderators, well... do something!

I guess that went well, except for my lack of experience around web servers and handling of high traffic sites. I had to move the site to a dedicated server and pay $200 per month for the next 2 years. That money couldn't be covered from the ad revenue, so I had to pay from my pocket. But that was alright, because I was happy knowing so many people visit the site and it is helping them with their FM games. I had to stop paying for that server last year and downgraded to a much cheaper VPS in order to cover the costs from advertising. That helped me a lot, and for half the months I didn't have to pay from my pocket.

After 3 years I'm still feeling helpless when it comes to servers and costs. I mean for the past year we had some issues with login, but there had been no problems with the server as the traffic wasn't massive at all. To get an idea of the average daily traffic during off season, we're talking about 3-4k. So you realize that with the traffic being 10x times up now, well it was impossible to cope with. In fact we did an upgrade during August and added traffic filters around late September to prepare for October and mainly November's madness, but that wasn't enough.


A misinterpreted idea I want to change

When most people hear about FM Scout, they would think it's the site for FM editors and scout utilities. That sounds OK, but it's a bit far from reality. Yes, we do have the most in-depth downloads section for data editors and scout utilities, but that is just one of the things we do. I believe we have the most popular downloads in general, because everything is free for all without any waiting times. Our downloads feature all kinds of files, but I must admit we have been low on ready to use tactics and training schedules. Graphics covered mostly skins, but has been in no way massive overall.

The blogs didn't go as good as I would like them to either. Just over 200 blogs within 2 years, that's not good at all. Last year we introduced a forum section for manager stories, in an effort to make it easier for people to write their stories in the forums instead of the blog, in case they were feeling awkward there. That worked, but still our stories database was a bit poor. It seems our forums haven't been the most crowded around the scene either. To add to the negatives, we also had no column or people to write the occasional interesting article with tips and advise.

Er, so we're left specializing in data editors and scout utilities, right? Well, no. We would always publish all the official hints and tips of the game first. Our good players area was albeit popular as well. Forums had various insightful bits and pieces, and a few blogs were quite good actually. The bottom line is we have been trying to expand in all possible areas (except podcasts), but at some point we lost focus as to what is it that we do best. I wish to change the idea people have about this site.


Season expectations signal a new era

We don't produce many things, but we can list them all. I'm referring to skins, facepacks, logopacks, kitpacks and various other gfx, custom competitions, data updates, mods and tricks, tools, soundpacks, and of course data editors, scout utilities, tools... well, anything FM! In fact we do produce some of those, but it's not something we specialize the most.

We can have everything here from around the scene, either as a direct download or as direct link to another site offering something exclusively. In the meantime we will be uploading our own ready to use tactics, training schedules, search filters, player shortlists, and others. The idea is to have everything there is to download.

OK, got it... you should have the most reliable downloads around. What else? We want to focus on manager stories; stories you can write here. We shall provide the space and make it easy for you to share your stories with fellow managers. This year blogs will be helping forums with those stories, and the users granted storyteller status will be our regular authors for this section.

We want to focus on player lists, and we partly made it already. We were the first to produce lists for wonderkids and free players on the new game, and we shall build on that by revising and adding more lists. With your help from the comments, we shall shape the best player lists around.

We want to focus on proper help/faq/tutorials section, and this has already kicked off with some success. Combine that with more insightful talk in the forums (we shall work towards arranging a series of topics for you to take part in) the idea is to learn something new and enrich your FM knowledge, improve as a player regardless if you are novice or advanced.

On top of that, we are starting a brand new section called FM Network. We will try to be in constant contact will all the fansites around the scene, and promote all their quality articles/material. This way you will know what's happening outside FM Scout and directly visit the pages that seem the most interesting from our network 'picks'. We are are happy to share traffic with the rest, because this should be the true spirit of a good fansite.


Looking ahead

To say this in FM terms, our site admittedly is high on potential ability, we just need to work harder to improve our current ability. I believe we are on the right path this season. There are still plenty of improvements to implement, and somehow the server issue should be solved soon. I'm confident we can make it though. A dark skin for the site is among my 'to-do' list for this week, and in fact most improvements could be done by next Sunday.

I wish to thank everyone for reading, and for your support through the years. It's a common cliche, but you are the site - there's no meaning into it without you coming here. Apologies for the downtime and any other issues that might go with that, rest assured we are trying hard to resolve them all and concentrate on delivering the best FM site for you.

Any comments/questions are welcome, I will be replying to all of them.