On the topic of Gear Scoring

A few recent comments have pointed out that the reports as they stand at the moment include everyone at level 80 regardless of gear level - from players that 'just dinged' seconds before my spider saw them all the way through to those who have been raiding since the WotLK launch-week. Gear scoring is something I wanted to use to extract different groups of players in the past, but I've finally got around to implementing it.

All items in the game have an internal 'item level' attribute as well as its more widely known rarity value (uncommon, rare, epic etc.), generally the higher value the item the higher the 'item level'. There are even achievements[1,2] linked to the 'item level' of your gear. The simplest way to create a 'gear score' is just to add up the 'item level's, if you want to make it more fair/accurate include a factor/multiplier based on rarity - giving an epic item a higher score than a rare item at the same 'item level'.

I was curious as to how the fansites and addons calculate their scores, so here is a quick summary:

Be Imba! - Their score includes Gear, Gems and Enchants. I'm not particularly interested in Gems & Enchants; I'm looking for a way to discriminate on Gear-only.

Warcrafter - Similar to Be Imba! their score appears to include Gems and Enchants as well as Gear.

GearScore (addon) - A pretty interesting spin on gear scoring, it works by adding up at the stats (+sta, +agi, +spell power) provided by the gear regardless of the 'item level'.

FuBar_GearRating (addon) - An inactive project, but still works with the latest FuBar on patch 3.0.8. It multiplies the 'item level' with a few factors including the rarity, slot and class of the character (e.g. a Hunters ranged slot item is more important than an idol/relic/wand slot item).

The method used by FuBar_GearRating turned out to be my favourite approach. I am not interested in Gems or Enchants, they are more indicators of the gold the characters are willing to spend than the gear level they have access to.

I implemented the same algorithm used by FuBar_GearRating in my character database, a quick sanity check proved the score I came up with matched the addon's in-game results. Next, I ran it on 249,882 level 80 characters and I got the following results.


The raw data behind this graph is available as a simple CSV file here:
http://armory-musings.appspot.com/download/gearScore_20090209.csv

There are 64,676 characters (25.8% of the sample) with a score of 200 or more, only 3,463 of those have a score of over 215. The long tail of characters with a score of less than 100 appears to be those who are practically naked or wearing RP outfits and the like.

Overall I am quite happy with the results, I'll be using this new data to partition the reports I generate. Also a recent brainwave concluded that anyone with a Resilience of over 200 is probably spec'd/geared for PvP and should also be in a separate report.

Comments / questions / suggestions welcomed :)

Class & Realm reports, updated automagically every 24 hours:
http://armory-musings.appspot.com/reports

The return of the Realm balance reports...

My spider software is still hurtling through lists of guilds and whatnot looking specifically for level 80 characters. Currently there are 260,389 level 80 characters used to generate the reports but this is ticking up by the second. I still have 50,464 guilds in the 'pending' queue waiting to be checked. Eventually the 71-79 characters will be refreshed as I expect a large number will have levelled by the time I return to them.

In other news, I have released a lot more information on the reports page. Each realm now has a separate 'Realm Balance' report showing the balance of classes for both Alliance and Horde on that realm. There is also a complete list of Guilds known on that realm. Don't panic if your realm report is a little on the empty side, your realm and its guilds are likely still somewhere in the 50,000+ TODO queue.

Lastly, I have released some sample data from 3,000 level 80 characters in both CSV and XML formats on the Developers page for you to play with. Limits on the size of files in Google App Engine has made it tricky to release larger data-sets but I have something planned.

Refreshed automagically every 6 hours:
http://armory-musings.appspot.com/reports

Armory XML: permanentenchant attribute decoded

Something that really bothered me with the previous reports was the mysterious "permanentenchant" attribute on each 'slot' element in the Armory's XML. It didn't correlate to anything useful on the usual fan sites, as far as I could tell it wasn't an 'item', 'spell' or 'object' ID as far as WowHead was concerned.

This morning I wrote a really simple script that does the following:

• Collate a list of all the permanentenchant attribute values from my database.


• For each permanentenchant ID:
  
  - Locate a character (realm & name) with that enchant in the correct slot in my database
  
  - Fetch the item tooltip for that character's enchanted slot and parse the 'enchant' element

Now I had a list mapping the permanentenchant ID to the text displayed in the tooltip '2669' -> '+40 Spell Power'. Pretty neat start, but not quite what I wanted - I needed to figure out that '+40 Spell Power' on a Weapon actually means 'Enchant Weapon - Major Spellpower'.

• Next, I try a pretty dumb lookup on WowHead '+40 Spell Power' didn't work but others like 'Unholy Weapon' were correctly identified.


• For anything that's left, I compare the '+40 Spell Power' to the huge enchantment list on WoWWiki and then return to WowHead with the name of the enchant as seen on WowWiki.

One clever thing I did add is the distinction between enchantments that are spells (by Enchanters) and enchantments that are items (Spellthread).

Thirty seconds later the script has finished and my list now maps:
permanentenchant:'2669' -> tooltip:'+40 Spell Power' -> spell:'27975'

The result of all this is I now publish an XML file from my database with all the permanentenchant ID's mapped either to a 'spell' or 'item' ID suitable for WowHead. There are a few in the list where both 'spell' and 'item' are 0 - these haven't been figured out and will require manual attention to update.

http://armory-musings.appspot.com/developers/files/permanentenchant.xml

This XML File is published under a Creative Commons Attribution 3.0 License, you are free to copy, distribute and adapt the data provided credit is given to the source of the data (i.e. a link to my site).

New Dataz!
http://armory-musings.appspot.com/

New Website, New Data, Daily Updates

I'm pleased to announce the launch of my new Armory Musings... project website. Information is updated (at least) daily direct from my spider database. The format of the reports will be improved over the next few days but all the data is under a Creative Commons license meaning you can copy, distribute and adapt the data in my reports as long as you link back to the original source (my site).

http://armory-musings.appspot.com/

Armory updates, and how to get at the new data

On the 25th of November a major Armory update added a whole host of new information with the introduction of the Statistics tab. Unlike the other tabs on the Armory where all the information is present in the XML this new Statistics tab fetches more data asynchronously.

Firstly if you have a look at the character-statistics.xml you will see something like this:

http://eu.wowarmory.com/character-statistics.xml?r=(realm)&n=(name)

  <statistics>
    <summary>
      <statistic name="Total gold acquired" quantityCopper="43" quantityGold="828" quantitySilver="89"/>
      <statistic name="Gold looted" quantityCopper="94" quantityGold="61" quantitySilver="3"/>
      <statistic name="Total kills that grant experience or honor" quantity="932"/>
      <statistic name="Total Killing Blows" quantity="997"/>
      <statistic name="Creatures killed" quantity="1907"/>
    </summary>
    <rootCategories>
      <category id="130" name="Character">
        <category name="Wealth"/>
        <category name="Consumables"/>
        <category name="Reputation"/>
        <category name="Gear"/>
      </category>
      <category id="141" name="Combat"/>
      <category id="128" name="Kills">
        <category name="Creatures"/>
        <category name="Honorable Kills"/>
        <category name="Killing Blows"/>
      </category>
      <category id="122" name="Deaths">
        <category name="Arenas"/>
        <category name="Battlegrounds"/>
        <category name="Dungeons"/>
        <category name="World"/>
        <category name="Resurrection"/>
      </category>
      <category id="133" name="Quests"/>
      <category id="14807" name="Dungeons & Raids">
        <category name="Classic"/>
        <category name="The Burning Crusade"/>
        <category name="Wrath of the Lich King"/>
      </category>
      <category id="132" name="Skills">
        <category name="Secondary Skills"/>
        <category name="Professions"/>
      </category>
      <category id="134" name="Travel"/>
      <category id="131" name="Social"/>
      <category id="21" name="Player vs. Player">
        <category name="Arenas"/>
        <category name="Battlegrounds"/>
        <category name="World"/>
      </category>
    </rootCategories>
  </statistics>

The <summary> section will be different for everyone but the category id's appear to be fixed. When you click the Character category (id 130 as you can see above) an XML call fires off. Low and behold an XML document comes back looking like this:

http://eu.wowarmory.com/character-statistics.xml?r=(realm)&n=(name)&c=130

<category name="Character" requestUrl="/character-statistics.xml">
  <statistic name="Talent tree respecs" quantity="--"/>
  <statistic name="Badges of Justice acquired" quantity="--"/>
  <statistic name="Emblems of Heroism acquired" quantity="--"/>
  <statistic name="Emblems of Valor acquired" quantity="--"/>
  <category name="Wealth">
    <statistic name="Total gold acquired" quantityCopper="43" quantityGold="828" quantitySilver="89"/>
    <statistic name="Average gold earned per day" quantityCopper="96" quantityGold="46" quantitySilver="4"/>
    <statistic name="Gold looted" quantityCopper="94" quantityGold="61" quantitySilver="3"/>
    <statistic name="Gold from quest rewards" quantityGold="473" quantitySilver="54"/>
    <statistic name="Gold earned from auctions" quantity="--"/>
    <statistic name="Auctions posted" quantity="--"/>
    <statistic name="Auction purchases" quantity="5"/>
    <statistic name="Most expensive bid on auction" quantityGold="110"/>
    <statistic name="Most expensive auction sold" quantity="--"/>
    <statistic name="Gold from vendors" quantityCopper="49" quantityGold="294" quantitySilver="31"/>
    <statistic name="Gold spent on travel" quantityCopper="90" quantityGold="7" quantitySilver="87"/>
    <statistic name="Gold spent at barber shops" quantityCopper="50" quantityGold="7" quantitySilver="29"/>
    <statistic name="Gold spent on postage" quantityCopper="30" quantitySilver="27"/>
    <statistic name="Gold spent on talent tree respecs" quantity="--"/>
    <statistic name="Most gold ever owned" quantityCopper="14" quantityGold="601" quantitySilver="99"/>
    <statistic name="Items pickpocketed" quantity="108"/>
  </category>
  <category name="Consumables">
    <statistic name="Bandages used" quantity="--"/>
    <statistic name="Bandage used most" quantity="--"/>
    <statistic name="Different bandage types used" quantity="--"/>
    <statistic name="Health potions consumed" quantity="1"/>
    <statistic highest="Super Healing Potion" name="Health potion used most" quantity="1"/>
    <statistic name="Different health potions used" quantity="1"/>
    <statistic name="Mana potions consumed" quantity="22"/>
    <statistic highest="Super Mana Potion" name="Mana potion used most" quantity="22"/>
    <statistic name="Different mana potions used" quantity="1"/>
    <statistic name="Elixirs consumed" quantity="--"/>
    <statistic name="Elixir consumed most" quantity="--"/>
    <statistic name="Different elixirs used" quantity="--"/>
    <statistic name="Flasks consumed" quantity="--"/>
    <statistic name="Flask consumed most" quantity="--"/>
    <statistic name="Different flasks consumed" quantity="--"/>
    <statistic name="Beverages consumed" quantity="126"/>
    <statistic highest="Pungent Seal Whey" name="Beverage consumed most" quantity="98"/>
    <statistic name="Different beverages consumed" quantity="9"/>
    <statistic name="Food eaten" quantity="10"/>
    <statistic name="Different foods eaten" quantity="3"/>
    <statistic highest="Poached Bluefish" name="Food eaten most" quantity="8"/>
    <statistic name="Healthstones used" quantity="--"/>
  </category>
  <category name="Reputation">
    <statistic name="Most factions at Exalted" quantity="4"/>
    <statistic name="Most factions at Revered or higher" quantity="9"/>
    <statistic name="Most factions at Honored or higher" quantity="18"/>
    <statistic name="Most Northrend factions at Exalted" quantity="--"/>
    <statistic name="Most Outland factions at Exalted" quantity="3"/>
    <statistic name="Most Alliance factions at Exalted" quantity="1"/>
    <statistic name="Total factions encountered" quantity="39"/>
  </category>
  <category name="Gear">
    <statistic name="Equipped epic items in item slots" quantity="2"/>
    <statistic name="Epic items acquired" quantity="1"/>
    <statistic name="Epic items looted" quantity="--"/>
    <statistic name="Legendary items acquired" quantity="--"/>
    <statistic name="Mounts owned" quantity="3"/>
    <statistic name="Vanity pets owned" quantity="3"/>
    <statistic name="Extra bank slots purchased" quantity="7"/>
    <statistic name="Greed rolls made on loot" quantity="1"/>
    <statistic name="Need rolls made on loot" quantity="--"/>
  </category>
</category>

You can retrieve any of the Statistic sub-pages independently of the rest of the character information. I haven't spent a lot of time looking into it but the category id's appear to be fixed and everyone seems to have the same set of categories on display (for the moment).

Blessing of Salvation

It’s been almost a year since my last post… With the recent launch of WotLK I have decided to resurrect this project and flood it with new data.

In a few days I expect to have the first batch of new data covering 100,000’s of level 70-80 characters. I am collecting data on lower level characters too, but mainly to check on them later when they have levelled up. I am concentrating the information on the level 70-80 range.

All the data will be hosted on a new website and updated daily directly from my database. The link to this new site will be posted later this week.

All data I publish from now on will be under the Creative Commons Attribution 3.0 License meaning you can copy, distribute and adapt my data provided you link back to the original source (i.e. my site).

In the mean time if you have any questions or suggestions, you can email at the usual address: okoloth (at) gmail (dot) com

Linking to World of Warcraft items from Blogger (Revisited)

Some time ago I wrote a simple piece of HTML/Javascript to produce item tooltips driven by Allakhazam. I'm pleased to retire this script in favor of some new code from Wowhead.

The simple instructions are here and are summarised below:

1). Add this piece of HTML code anywhere on your page:
<script src="http://www.wowhead.com/widgets/power.js"></script>

2). Add a regular HTML link to an item in Wowhead, example:
<a class="q5" href="http://www.wowhead.com/?item=19019">Thunderfury, Blessed Blade of the Windseeker</a>