Tuesday, 7 August 2007

Status Update - 7th August

200,000 records and still going strong, there are around another 150,000 'in the queue' over the next week I will start concentrating on getting the most out of this already huge amount of data.
Alliance107,797 (53.9%)
Horde92,213 (46.1%)
Level 70's108,645 (54.32%)
Table Size2.60 GB

And here is a breakdown by class:
I am collecting data from 107 european realms simultaneously. More realm-balance related info will be published soon too.

Daily traffic has tailed off since the WoWInsider post but is averaging about 350 page views a day (up from around 50 page views a day).


Anonymous said...

absolutely awesome work!
i love statistics ;-)
can't wait to see your server based population stats :D

Blake said...

That is an absurd amount of data. I wonder how many people are querying the databases like you and compiling stats? That has to put a huge load on their servers. But keep it up - amazing data.

Okoloth said...

I know of several other projects working with Armory data, most of them on the US realms. As far as possible I try to reduce my impact on the Armory servers. My spider software has rate-limiting settings to slow/speed its operation throughout the day; and is 'intelligent' enough not to spam requests for the same arena team, guild or character. The XML fetching part of my software is almost as sophisticated as a full web browser, supporting HTTP compression should the Armory servers choose to send it compressed.

Coincidently the Armory servers are actually very fast at delivering XML data, the main reason they are perceived to be ‘slow’ is that your own web browser is ‘doing all the work’ in the form of a XSLT transform to format the XML for each page.

At the moment I feel I have reached a critical mass of data, enough to do some really powerful new work. Most importantly I have sufficient data for levels 1 to 69 allowing me to do some comprehensive analysis instead of focusing on level 70’s.

Anonymous said...

I think one thing that would/could prove quite informative would be to also calculate the sum of the ilvl of items worn by people.

This would provide a glimpse into which specs within which classes are obtaining the highest ilvl gear - be it from arena or pve raiding.

It would also help demonstrate which classes were most involved in these activities.