Quest success rate research

"The research has been moved from user Punk rawk boi's blog to this page. Information here is not yet altered. If you feel like picking up the research where this user left it, please do. Information below is added by other users and might be very helpfull. WikiPim"

'''New raw data as of 19/02/13: 154 adventurers' results, gathered with adv bonus between 65 and 80 (lvls 107-123). Attempting several types of analysis (see "questions to be answered"), but if others want to take a crack at analysis as well, the full dataset is provided below.'''

Conclusions So Far
 * Estimated probability of success is 50% for happy advs and 33% for neutral ones, assuming it's only modified by whether or not they leave happy. Even angry advs have a slight chance to come back successful.
 * Whether an adventurer likes an item or not is likely most likely random, but modified by the factors under consideration (adv skill, item given, etc) - similar to haggling being random but modified by the ST skill, initial offer relative to market price, etc.
 * How good a reward you get has been confirmed to correlate only to the item requested, not to the item given.

Early Analysis - Full
 * I was once told that if an adventurer leaves cursing (noted below by "reaction = -2"), they have a 0% chance of success. However, one adv did leave cursing and come back with a reward. (Healer, requests devilish rod, given vial of mana, 78 points in adv, brings back red charm + 2.6K)
 * I assume that all variables in question (adv points, item given...) only contribute to whether an adventurer leaves happy or not, and once they have left, what mood they were in is the only thing contributing to their success chance going forward. With this in mind, I tallied the success chances based on their reaction:


 * ​ ​Positive reaction: 34/64 (53.13%) successful
 * ​ 42.14% < x < 63.88%  at p=.9
 * 36.56% < x < 69.21%  at p=.99
 * Guess at the true probability: 50%
 * Neutral reaction: 28/74 (37.84%) successful
 * 28.40 < x < 48.03  at p=.9
 * 23.84 < x < 48.03  at p=.99
 * Guess at the true probability: 33%
 * Negative reaction has too few data points to say anything conclusive (1/8)


 * Some interesting apparent conflicts have arisen. A few examples:
 * Archers
 * A: Requests banker's mark (6.5K), receives composite bow (2.3K) - 64% lower price and mismatched item class. With 65 points in adv, leaves happy.
 * B: Requests yeti armor (2.4K), receives perfect leather (1.9K) - 20% lower price and matched item class. With 79 points in adv, leaves neutral.
 * Engineers
 * A: Requests warrior's ring (3.2K), receives skull crusher (1.4K) - 58% lower price and mismatched item class. With 75 points in adv, leaves happy
 * B: Requests bile jar (2.7K), receives vial of mana (1.5K) - 46% lower price and matched item class. With 79 points in adv, leaves neutral.
 * Soldiers
 * A: Requests talisman of fear (2.6K), gets scutum (1.6K) - 36% lower price and mismatched item class. With 65 points in adv, leaves happy.
 * B: Requests labrys (2.4K), gets pirate axe (2.1K) - 13% lower price and matched item class. With 68 points in adv, leaves neutral.
 * War wizards
 * A: Requests ice shard scroll (5.5K), gets soldier's sword (1.2K) - 79% lower price and mismatched item class. With 65 points in adv, leaves happy.
 * B: Requests noble tiara (2.3K), gets cleric's hat (1.1K) - 53% lower price and matched item class.  With 65 points in adv, leaves neutral.
 * Possible explanations
 * Whether an adventurer likes an item or not is still due to random chance, but this chance is modified by the factors under consideration (adv points, item suggested, etc). Due to other game mechanics being known to work this way (eg. haggling and the ST skill), this seems the most likely.
 * Adventurers are more willing to accept cross-class items when they are requesting rings or pendants.
 * Counterexample with war wizards
 * A: Requests ice shard scroll (5.5K), gets soldier's sword (1.2K) - 79% lower price, mismatched item class, not requesting a ring or pendant. With 65 points in adv, leaves happy.
 * B: Requests talisman of fear (2.6K), gets vial of mana (1.5K) - 43% difference, 77 points in adv, leaves neutral.
 * Could extend this theory to "rings, pendants, and scrolls / consumables", but that seems pushing it...
 * Classes have a "favourite item" which is more important than class matching. This suggests archer B would have left happy with a composite bow, for instance.
 * There's another variable in play that was not being considered - eg, what else was in stock in the shop at the time.
 * Reward values
 * Looked at ratio of value of rewards brought back by successful advs (N=62) to the value of the item requested  / given. Rares converted at a semi-arbitrary value of 1.5K each.
 * Ratio to requested: mean=2.84, stdev=0.96
 * Ratio to given: mean=8.59, stdev=5.2
 * The much higher standard deviation on the second ratio supports the fact that this does not effect rewards (confirming what was previously thought).

Further questions to be answered:
 * How do you best ensure an adventurer leaves happy? i.e. attempting to quantify the effects on leaving happy due to...
 * Matching item class - either subclass (eg. potion) or superclass (eg. consumable)
 * Price difference
 * Adventurer class
 * "favoured item" of a given adventurer class, if any (ie. are archers more likely to leave happy with a composite bow than a similarly priced item with a matching item class?)
 * Are adventurers more willing to accept cross-class items when requesting rings or pendants?

Full Data

Archers

Barbarians {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Bards

Berserkers {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Engineers

Evokers {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Guards

Healers {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Illusionists

Knights {| border="0" cellpadding="1" cellspacing="1" class="article-table article-table-selected" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Monks

Musketeers {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Ninjas

Paladins {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Singing Axes

Soldiers {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

Sorcerers

Thieves {| border="0" cellpadding="1" cellspacing="1" class="article-table" style="width: 500px;" ! scope="col"|


 * }
 * }
 * }

War Wizards

Data below has become outdated with the update of september 6.

As a mathematician, I have been wondering what sort of success rates I was getting from my random adventurers. Once I got my Adventurer skill up to 50, I decided to start recording some information on my quests. It should be noted, that I did not start this experiment until after I hit level 110. As a result, I did not receive items as quest rewards. I had heard that the item rewards declined after hitting 100, and after receiving several items after that point, I was further motivated to start recording what awards I was receiving. Being my first attempt at recording the information, there are some things that I left out that I now realize I should have included. The following is the raw data that I recorded. I will provide some observations after the tables.

Archer: Barbarian: Bard:

Berzerker: No adventures during time period

Engineer: Evoker:

Guard:

Healer:

Illusionist:

Knight: No adventures during time period

Monk:

Musketeer:

Ninja:

Paladin:

Shaman: No adventures during time period

Singing Axe: No adventures during time period

Soldier:

Sorceror:

Thief:

War Wizard: No adventures during time period

Totals:
 * - The mats rewards were converted to cash at a rate of $625 per unit.

mean of the rewards (26/total) = $91,563

expected value for quests = ($91,563 * .44) + ($0 * .56) = $40,288

variance for quests (E(X^2) - E(X)^2) = $2,065,758,917

standard deviation for quests (SQRT(Variance)) = $45,450.62

So, what do these numbers mean? To begin, we look at the items I was supplying for the quests. The market value for each items ranges from 1.4K to 1.9K. This means that with an investment of around 1.7K your expected return is $40,288. The ROI (Return on Investment) for some random quest should be about 22.7. This would be an impossibly high ROI if we were dealing with the stock market, but since this is S&P, we can only use it as a marker for what I was earning during the course of this experiment. If the rewards from quests goes up as the shopkeep's level goes up, then the ROI will increase since the investment is staying the same.

Now, the incredibly high variance for this experiment is something which should be expected, and will also go up as quest rewards increase. If we assume that the 44% quest success rate remains true, then there will always be a 56% chance to get $0. And since quest rewards increase, the huge difference between success and failure will have the effect of making the variance skyrocket.

As my first attempt at an experiment for measuring quest success rates, I have found several things which I could improve upon. For instance, in my next attempt, I have decided to include the shopkeep's level along with the creature the adventurer is targetting. There are also some redundant columns in my data. I essentially just copied them from the excel sheet I created, where the success and failure columns were used for counting. Here, they are just wasted space. In the future, I'll probably change the success/failure/results columns into Response/Reward columns. I'll predefine each response (so you'll be able to distinguish between killing target and forgetting and selling item) into shorthand and show the rewards if there are some. Also, any future experiment will need to last longer since there were several classes that were not represented. This will allow me to provide some statistics for the individual classes in addition to overall stats. As it stands now it seems like the Archer and Ninja are the best classes where as the Healer and Engineer are among the worst. Also, with sufficient quests, I may diversify the items I supply the adventurer to see if the item changes the success rate.

If you have any suggestions or comments, please feel free to comment and I'll take them under consideration,

- PRB