Okay, before it turns into a slinging match, how about we do a few more tests.
Here's what I've done so far. Out of 3000, 30 blocks of 100. When I first started doing it, I worked out the percentage for the first two boxes, I thought something was wrong when the answer was 83.5% success - while close to 85%, it's still not 85%. After adding all the numbers up for all the boxes in the spreadsheet, I ended up with a total of 85.1% success (or 447 failures out of 3000), where the highest failure rate was 18% and lowest was 11%; 82% success and 89% success respectively. I am almost certain that if I were to do it again (I am not going to), the overall success rate would differ. I used a pure class mage, male, 18 int, playing on insane, and Colour Spray was my scroll of choice, while using CLUAConsole:CreateItem("scrl70",100) for each block of 100, I didn't reload at any time during the test, all I did was erase the spell and continued the trial.
This is not "proof" that either side is right or wrong, this is just the results of my testing over 5 hours. Perhaps someone else can work out the probability of how accurate 85% is as a number to aim for, offsetting these numbers with the numbers previously mentioned in other posts.
There's a difference between mathematical probability and statistical probability.
Statistical probability suggests that the likelihood of a certain outcome diminishes or increases depending on the results of previous trials. In other words, if your first toss comes up heads, then your second toss is statistically more likely to come up tails, because the statistical probability of it coming up heads twice in a row is lower than the statistical probability of it coming up heads and tails in equal turns.
Mathematical probability, on the other hand, means that with each toss of the coin there is a 50/50 chance of either outcome, regardless of how many times the coin is tossed. In other words, you might toss the coin once, and it would come up heads, and the second time you toss the coin there is the same probability of it coming up heads. Which is to say 50%, each time.
What we're dealing with is an expectation of statistical probability, which suggests that for every hundred trials there should be a direct correlation to the percent chance of success. In other words, for an 85% chance of success, there should be 85 successful results in 100 attempts. That's not what's happening.
What's happening is 100 individual attempts, each calculated on their own, using their own set of randomly generated numbers. Which means that each attempt has an 85% chance of success. This likelihood of success doesn't increase or diminish depending on the number of trials. You might have fifteen failures followed by eighty-five successes. You might have no successes at all. You might not have any failures. When it comes to randomly generated numbers, the key word is "random". The game's engine does not create results that take into account all previous results.
Anyone who has played tabletop D&D understands this. You could have a night where you roll nothing but 1's and 5's. That same night, the person to your left might roll a series of natural 20's. It doesn't mean the dice are rigged; it means you had a string of bad luck.
And sometimes you might have several strings of bad luck. Or you might find yourself rolling really well during combat but rolling terribly in social situations (such as writing spells, for instance).
@Aosaw You said everything I tried to convey, but much more eloquent and clear than I ever could.
Now, see, that's a lot of trials. And it does make it seem as though the percentage is working correctly. But unless we look at the code and see exactly how it's being calculated, we'll just be tossing out theories. It's not empirical to say "I did this a thousand times and this is what happened", because the behavior you're describing is only your experience, and someone else who recreates that same experiment (or a more thorough one) may very well come up with different results.
@Awong124 That's what I mean by anecdotal. Not that your trials never happened, but that they don't necessarily prove anything except that you happened to end up with those results.
@Aosaw, since it is highly unlikely that any of us will see the code (and thus not know the formula for working out the success rate), here is as a basic random number generator, it should look something like this (as a basic example) although the one in BG:EE will differ:
srand(time(0));
int i;
int ranum;
float y = 0;
float n = 0;
for (i = 1; i <= 100; i++){
ranum = 1 + rand() % 100;
if (ranum > 85){
n++;
}
if (ranum <= 85 && ranum > 0){
y++;
}
}
After doing the tests earlier, I spent a couple minutes putting together a random number generator (which gave similar results to that of the game). Generate number, if number is lower than X (85, in this case, though there's obviously an array in Overhauls case that gets the values from the INTMOD.2da file), then the spell has been learned successfully. If not, then spell has not been learned.
My total results range from 82% to 92% success, which means that the 85% chance to learn is still on a spell-by-spell basis (and only really visible in large numbers of tests), as you said before. Each spell is the 'roll' of a 'dice' (rand();), and in the long run, regardless of sample size, you're not likely to actually get 85% since that's only the number to work out individual spells.
Of course, not knowing what their code looks like, one can only guess. It would probably be worth Beamdog's time to take a quick look and make sure that it is actually correctly rolling the values, just in case. For me, they seem to be fine.
As an example: if I want to learn two spells, the first goes into the spell book first time, the second fails. According to those statistics, that's a 50/50 and nowhere near the 85% first thought. If both spells fail, then the probability looks to be about 0%, as in the current rate is 100% failure. The same applies in a much larger sample, if I want to learn 100 spells and the rand(); function keeps throwing back values above 85, I fail to learn the spell. I could learn 50 spells out of the hundred, and the result would still be entirely valid.
Well, you did the work, and your results show that the success rate is reasonably close to 85%. So I suppose there's nothing wrong. My sample size just wasn't large enough.
@Aosaw It's not empirical to say "I did this a thousand times and this is what happened", because the behavior you're describing is only your experience, and someone else who recreates that same experiment (or a more thorough one) may very well come up with different results.
That's what I mean by anecdotal. Not that your trials never happened, but that they don't necessarily prove anything except that you happened to end up with those results.
But the thing is, that's not the meaning of the terms empirical and anecdotal evidence. Collecting data is empirical evidence by definition. Anecdotal evidence is something like a story of somebody's experience (which I don't really see how that can be considered evidence at all). Empirical evidence isn't necessarily going to prove anything, it can suggest something to be true to a certain extent. Empirical evidence is always going to be gathered by somebody or something, so it's always going to be somebody's experience. Whether or not you trust the integrity of the empirical evidence is another matter.
Ya know with 85% probability.. I almost always learn a spell
That kind of pokes a hole in your "Mechanics are faulty" theory.
I also don't think some of you understand statistics. It is possible to flip 100 heads in a row. Is it probable? No, but the possibility remains..
"omg this coin flipped heads more than half the time. it most be broken."
That, @Aosaw, is a perfect example of anecdotal evidence. And I've already addressed enough times the concept of statistics and probability in this thread, so I'll just leave it alone.
@Aosaw You said everything I tried to convey, but much more eloquent and clear than I ever could.
Except that I pointed out that most of that post is incorrect. Sorry Aosaw, but I do think most of the points in that post were incorrect, not just a matter of differing opinions. You're welcome to show me otherwise though.
As an example: if I want to learn two spells, the first goes into the spell book first time, the second fails. According to those statistics, that's a 50/50 and nowhere near the 85% first thought. If both spells fail, then the probability looks to be about 0%, as in the current rate is 100% failure. The same applies in a much larger sample, if I want to learn 100 spells and the rand(); function keeps throwing back values above 85, I fail to learn the spell. I could learn 50 spells out of the hundred, and the result would still be entirely valid.
That's the role of the dice, so to speak.
This point I disagree with you. I don't disagree with your results, but I disagree with you on the concept of statistics. The examples in the first couple of sentences are meaningless, because if you just look at two or three trials it's obviously not going to mean anything. The same doesn't apply to a much larger sample, that is a major part of statistics, sample size is extremely important. Sure, if you get 50 successes out of a hundred with 85% chance of success it could be valid, but can you do it over and over again? You already showed in your own test that it's not likely, and certainly not repeatable.
Ah, maybe I was misunderstanding the term "empirical"; I understood it to mean "indisputable", as in "proves the point", which clearly wasn't what you meant.
I maintain that the best way to find an answer to the question would be to look at the code. But only the developers can do that, so...
This point I disagree with you. I don't disagree with your results, but I disagree with you on the concept of statistics. The examples in the first couple of sentences are meaningless, because if you just look at two or three trials it's obviously not going to mean anything. The same doesn't apply to a much larger sample, that is a major part of statistics, sample size is extremely important. Sure, if you get 50 successes out of a hundred with 85% chance of success it could be valid, but can you do it over and over again? You already showed in your own test that it's not likely, and certainly not repeatable.
I'll just take a bit right off the end as well as a bit out of the middle. This string of Ys and Ns are the result of the random output of 1000 tests: YYYYYYYYYYNNYYNYYYYNYYYYYYYYYNNYYNNYYYYNYYYYNYYNNYYYNYNYYYYYNYYYYYYYYYYNYNNNYYYYYYYYNYNNYYYYYYYNYN
It shows a 76% success rating out of 100, and is perfectly valid for each item having an 85% chance of being a success.
This string alone has only a 76% success rating (it's the first half of the above string). YYYYYYYYYYNNYYNYYYYNYYYYYYYYYNNYYNNYYYYNYYYYNYYNN
If I were only doing 50 loops rather than plucking it off the end, it is entirely possible for this chain to appear.
Again, that's not to say your concerns are not valid, Beamdog should probably look into it if the output is as low as you say it is, but my best guess is it's just the luck of the draw. You could have a string of Ns or a whole string of Ys.
For example, this a 92% success rate from one of my earlier tries (I copied it down since replicating it exactly is almost impossible): YYYYNYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYNYYYYYYYNYYYYYNYYYYYYYYYYYNNY
This is also valid as an output.
Multiple times I have gotten percentages that were higher or lower than the 85% that the game says. My point is that, although improbable, the output could very well still be entirely valid.
Yeah, but those are strings of only 50. And your error is less than 10%, a far cry from 50 successes out of 100 you mentioned earlier. I'm not debating your results, just the concept of statistics you are putting forward. Your results support my point. With 3000 trials, your result was 85.1% chance of success, a 0.1% error. So with a larger sample size, the result gets more accurate, even if each trial is calculated independently. And that is what should be expected.
If you say that each trial is independent, and 85% is just a threshold, then what is the point of the specific number 85%? There should be no difference between 51% and 99% chances of success. Each trial is calculated independently, and each trial has a higher chance of success than failure whether it be 51% or 99%. The only way the actual specific number has any significance is if we can see the success rate being close to 85% over multiple trials.
@Awong124, 85 being the percentage chance for a character at int 18 to learn a new spell. Since there's a percentage chance (that's not the random bit), it has to be taken into account when determining whether or not a particular go is a success or failure, if above 85% then it fails, if below 85% then it falls into the success range. A random roll of the dice, and the integer value is then compared, to determine if a number falls within that 85% range.
A more recent test involving 100s:
The total positive result of all tests yields a 60 percent success rate.
Maybe the random number generator sucks and its seeding based off something that isn't changing drastically enough second to second and this occurs more when you try to learn a ton of spells in a row?
They might want to look at the RNG function and consider a rewrite, given that most games in the 90s tended to not be true random numbers.
Then that just means 100 is not a large enough sample size. If the chance of success is 85%, then over the course of many trials, the rate of success should be close to 85%. Your own results support this. Chance of success shouldn't be confused with rate of success, but at the same time with a large enough sample size they should approach one another. That is what statistics is all about. Each trial being independent is a characteristic of combinations, which is still a part of statistics, and should have statistically consistent results. Statistics is the only way of verifying that 85%.
Maybe the random number generator sucks and its seeding based off something that isn't changing drastically enough second to second and this occurs more when you try to learn a ton of spells in a row?
They might want to look at the RNG function and consider a rewrite, given that most games in the 90s tended to not be true random numbers.
I have the same impression about the random number generator being bugged. Changing too slow or something. It is not unusual at all to get several strings of constant failure. Specially when copying spells in quick succession.
@Awong124, the chance of success is defined by a file, that's the only statistic we need to worry about. The actual rate (or probability) of success is a different thing. Each individual spell has a (85% in my case) chance of being written to the spell book. It also has a (15% in my case) chance of failing.
While being statistically lower and thus less likely to have a failure -- on an individual basis, if one fails they could all fail (if I can get 100% failure on five, that trend could continue, it's not probable, but it could (that's where our margin of error comes in), which is why I can do a test of 10 and have a whole string of Ns or a whole string of Ys with 100% failure of success based on something that should only be 85%), meaning that you could have a success probability (or rate) of 0% and it still be (10, for the same of an example) valid 'chances.' That's why only statistical probability can be worked out in this case, and that probability is between 60% (being the lowest I've gotten) and 92% (being the highest) in my case, based on empirical (observed) evidence. The actual chance per spell is 85%. The actual, programmed chance is a variable that can contain any value within the file mentioned on the previous page, depending on the intelligence of the character.
Clearly, the frequency of success is not 85% with a character of int 18. The evident, or empirical, rate (from what I can see,) is a median of 77% (using only the highest and lowest values that I have been able to get from the game and my program, actual median is about 84%, with deviation of 3%) with a deviation of ~16%. After repeating the tests several more times by running my own program to get near-identical results at a quicker pace, the results are still too far apart to really hit the exact 85 mark (even running the program with a sample of 1,000,000 only get 84.96 and 85.1, so it's still too much deviation, given the actual size of the sample, to say that it's 85 as a definite frequency. Even at a sample size of 10,000,000 the closest I can get is 85.03, which is about as close as anyone is ever going to get).
Meaning the chance to learn a spell can be 85% success, and 15% failure (that could be 15 failures followed by 85 success, or on a larger scale, ~144 failures followed by ~856 successes). A series of failures can happen in groups of 2 and 3 (even groups of 5 and 6), as some of the copy/pasting shows. Saying it is broken because you get more failures in one go is not representative of the whole.
The thing is, the game doesn't state frequency or rate, it states chance to learn spell, a spell, not more than one, and that chance is apparently 85 for my character (and 75 for Neera and so on), that chance is reset with every spell, and while statistically more likely to get successes than failures, no one can rule out that failures are going to happen. The average person won't learn any more than about 150 spells during a fresh playthrough (I'm not even sure what the maximum number of spell scrolls there are (without mods)), so working with a sample higher than that is pointless when trying to determine if what you are experiencing is normal behaviour for the game. General rules of probability say it shouldn't be too far away from the 85% listed as a chance over a long period of time, but clearly it can due to the deviation I'm getting at 'normal' rates; thus we can only realistically work out empirical data from our results, mostly as a mean or median average percent. We can work out the rate or frequency without any trouble, provided we always take the deviation into account. A normal rate is around 77% with some deviation, and that is with a chance of 85. I got as low as 60 and 63, so it wouldn't be that much of a stretch to get something similar to the results you got on the previous page, improbable though they might appear in such a small sample.
Unfortunately, if either my program or the game is bugged, that's probably an issue in Microsoft's (the OS in general, doesn't even have to be Microsoft) headers, and not in the game itself. Since rand uses a seed from srand, which gets the current time in seconds from your computer, and generates a random number based on that seed, it could very well be less effective than writing a random number generator themselves...
Of course, I could be the one who is confused, it's been a long day.
I agree with @Aosaw, though, the only way to properly know how the game is working it out would be to examine the code itself. Which is undoubtedly never going to happen. But if it is working elsewhere in the game, it should be working just fine here. If the devs say it is working as it should (or only way they can have it without doing a complete re-write of most random 'dice-throwing' statements or functions), then fine. I'm only explaining why, programmatically, it seems sound enough. I wouldn't have the faintest idea who to tag or ask. Like I said at the start, in all my years of playing BG, I've had possibly 50 spells fail (but then again, I usually import a mage from one of my other playthroughs, so the learning is already done).
If it is broken, then the Overhaul team should absolutely know about it (e-mail support).
@Troodon80 It is reasonable to assume a low sample rate for the reason you've given, that throughout the course of the game you're not going to write more than a certain number of spells. But if the median you obtained from multiple trials is 77% with an 85% chance of success, then there is something wrong. The system clearly favors failure more than the % chance it's using. Whether or not the system is working properly programming-wise, it's not working as intended in practice. If the system really works as intended, you might get a lot of deviation due to the small sample size, but you should get more or less equal occurrences of both high and low, with a median much closer to 85%. Random "luck" should work both ways. From what you said from your trials, you're getting results that favor low. Maybe the mechanics work properly as programmed, but it was poorly designed. Or as moopy said, maybe it's just not a very good RNG that's been implemented.
It could be that the RNG is slightly favoring high rolls over low ones, as a way to reduce the number of catastrophic failures that come from rolling a 1 on an attack roll (i.e. a catastrophic failure is going to be felt more painfully by the player than a critical success will feel good), which has the side-effect of also altering percentage chances for things like this, where a high number is actually bad.
I'm personally still inclined to trust it as it is; any time I'm not facing a 100% success rate, I prepare myself for the possibility of failure, even if it's only 15%.
@Awong124 Should i get out the 20 sided die and show you that there is a chance of having less (as well as more) than 85% success rate if rolling a 17 or above= failure and rolling below = success.
Then again you'd probably just claim i have faulty dice and that they were not made properly
@Awong124 Should i get out the 20 sided die and show you that there is a chance of having less (as well as more) than 85% success rate if rolling a 17 or above= failure and rolling below = success.
Then again you'd probably just claim i have faulty dice and that they were not made properly
I don't think you understand what @Awong124 is saying. No one here is saying that you should always get a success rate of exactly 85% when you have a probability of success of 85%. But you should expect to see your % success rate converge to your % probability of success as your number of trials increases; that's the definition of probability.
my experience with this (in BG:EE) was slightly bizzare. i played on core and reloaded every time a spell write failed. in the beginning i had what seemed to me like abysmal luck, i would always have to reload multiple times (in average 6-7) in order for the spell to be written successfully. which when compared to successes (adding the first time ones) felt like the chance of failiure was greater than the chance of success for my three arcane spellcasters (so that's a lot of writing of spells), or 50/50 at best
i didn't have an impression that *first times* i tried to learn a spell, the chance of failiure would be greater than the chance of success tho - but when i did fail, i'd have to reload a lot
so eventually i came to a weird idea that some kind of counter might start when you press the button to write the spell and ends when you release the button (yeah, sounds lame). so i tried to hold the button longer aaand - i never had to reload multiple times again... srsly - the character still failed occasionally, but whenever i reloaded, i'd hold the button a bit longer than normal and the success rate felt like 100% basically.
then i also tried it with HP rolls whenever i rolled badly (holding the button longer to get a higher roll, dunno? ...) and uhm, i played with almost maximal hp for the rest of the game (because i like cheese) with just one reload per bad HP roll - i'd always get the highest roll minus 1-2 hp. when doing that (hold the button longer), i'd start to trip i have a power to guess what the exact amount will be and i overwhelmingly guessed with a small margin of error +/-1.
i really don't like posting this because it sounds extremely superstitious and if i'm wrong and there's no connection between those things it'll be pretty embarassing (i already feel stupid), but i still felt like i had to because had a huge impression that there was something there.
So I'll just put this here, because I thought I was on to something which would affect the accuracy of tests should you be scribing spells of levels higher than 1, but did an in-game test which debunked that.
I was under the impression that for every spell level higher (for the spell you're learning) than the highest spell level you know, there is an incremental 10% increased failure chance.
So I took a very smart level 1 mage down to the map below Nashkel, obtained that cloudkill scroll and got well aquainted with the quickload and inventory keys. I noticed markedly more successful scribes than not, so that debunks that, no idea why I thought that was the case. I'll just leave this here, in case anyone else has the same strange thought for the same strange reason.
On the default difficulty you write spells 100%, which has always worked. I play on core rules and have just been using potions of genius or such before writing spells.
Also keep in mind that you receive a penalty if you try to write spells that are higher level than you can cast.
It could be that the RNG is slightly favoring high rolls over low ones, as a way to reduce the number of catastrophic failures that come from rolling a 1 on an attack roll (i.e. a catastrophic failure is going to be felt more painfully by the player than a critical success will feel good), which has the side-effect of also altering percentage chances for things like this, where a high number is actually bad.
I'm personally still inclined to trust it as it is; any time I'm not facing a 100% success rate, I prepare myself for the possibility of failure, even if it's only 15%.
That is a possibility. Though I'd be surprised that they would originally go through such lengths to do something like this. If that really was the case, if they wanted to give the player a slight edge instead of a slight disadvantage when it comes to learning spells, all they'd have to do is invert the % chance and make it so that any 'dice roll' above that inverted % is a success.
Lets say: x = % chance of success y = die roll then make the condition for success: y > (100 - x)
In any case, I suppose it doesn't really matter. Reloading isn't really a huge deal. I just find it psychologically annoying when video games seem to be 'rigged' to give the player a disadvantage, whether that actually is the case or not.
Like when I played Governor of Poker, there were so many times I got a really good hand (like a full house) and I go all in, then my opponent has an even better hand. Many people on forums claim the game is rigged.
Or like Gran Turismo 5, in which it's blatantly obvious, not hidden in the coding. On races where there is no qualifying session you automatically start last. Even though AI in the game is stupidly easy, it still annoys me that I am disadvantaged from the start.
@bob_veng That's an interesting observation. I'll try that sometime.
I don't think anybody here is claiming something like that should be happening. I don't think anybody is saying that after a string of bad rolls you should be more likely to get a string of good rolls.
There's a difference between mathematical probability and statistical probability.
Statistical probability suggests that the likelihood of a certain outcome diminishes or increases depending on the results of previous trials. In other words, if your first toss comes up heads, then your second toss is statistically more likely to come up tails, because the statistical probability of it coming up heads twice in a row is lower than the statistical probability of it coming up heads and tails in equal turns.
That is only a part of statistics, the permutation part. Combination, which deals with separate events that have no effect on previous trials is also a part of statistics. Also, your example is incorrect. Tossing two heads in a row, two tails in a row, head then tail, or tail then head all have the same statistical probability of 25%. Coin tosses are combinations, each toss doesn't affect successive tosses. However, getting a head and a tail once each (with no regard to their order) has a higher statistical probability than two heads or two tails. So if you already tossed a head, it's still equal chance that you will get either a head or a tail on the next toss, you're not statistically more likely to toss a tail.
Since you don't seem to be understanding what my point is, and continually misconstrue what I'm saying. I just found this page on wikipedia. It's an actual proper theorem.
Comments
Here's what I've done so far. Out of 3000, 30 blocks of 100. When I first started doing it, I worked out the percentage for the first two boxes, I thought something was wrong when the answer was 83.5% success - while close to 85%, it's still not 85%. After adding all the numbers up for all the boxes in the spreadsheet, I ended up with a total of 85.1% success (or 447 failures out of 3000), where the highest failure rate was 18% and lowest was 11%; 82% success and 89% success respectively. I am almost certain that if I were to do it again (I am not going to), the overall success rate would differ. I used a pure class mage, male, 18 int, playing on insane, and Colour Spray was my scroll of choice, while using CLUAConsole:CreateItem("scrl70",100) for each block of 100, I didn't reload at any time during the test, all I did was erase the spell and continued the trial.
This is not "proof" that either side is right or wrong, this is just the results of my testing over 5 hours. Perhaps someone else can work out the probability of how accurate 85% is as a number to aim for, offsetting these numbers with the numbers previously mentioned in other posts.
You said everything I tried to convey, but much more eloquent and clear than I ever could.
Wow.
Now, see, that's a lot of trials. And it does make it seem as though the percentage is working correctly. But unless we look at the code and see exactly how it's being calculated, we'll just be tossing out theories. It's not empirical to say "I did this a thousand times and this is what happened", because the behavior you're describing is only your experience, and someone else who recreates that same experiment (or a more thorough one) may very well come up with different results.
@Awong124
That's what I mean by anecdotal. Not that your trials never happened, but that they don't necessarily prove anything except that you happened to end up with those results.
srand(time(0)); int i; int ranum; float y = 0; float n = 0; for (i = 1; i <= 100; i++){ ranum = 1 + rand() % 100; if (ranum > 85){ n++; } if (ranum <= 85 && ranum > 0){ y++; } }After doing the tests earlier, I spent a couple minutes putting together a random number generator (which gave similar results to that of the game). Generate number, if number is lower than X (85, in this case, though there's obviously an array in Overhauls case that gets the values from the INTMOD.2da file), then the spell has been learned successfully. If not, then spell has not been learned.My total results range from 82% to 92% success, which means that the 85% chance to learn is still on a spell-by-spell basis (and only really visible in large numbers of tests), as you said before. Each spell is the 'roll' of a 'dice' (rand();), and in the long run, regardless of sample size, you're not likely to actually get 85% since that's only the number to work out individual spells.
Of course, not knowing what their code looks like, one can only guess. It would probably be worth Beamdog's time to take a quick look and make sure that it is actually correctly rolling the values, just in case. For me, they seem to be fine.
As an example: if I want to learn two spells, the first goes into the spell book first time, the second fails. According to those statistics, that's a 50/50 and nowhere near the 85% first thought. If both spells fail, then the probability looks to be about 0%, as in the current rate is 100% failure. The same applies in a much larger sample, if I want to learn 100 spells and the rand(); function keeps throwing back values above 85, I fail to learn the spell. I could learn 50 spells out of the hundred, and the result would still be entirely valid.
That's the role of the dice, so to speak.
Well, you did the work, and your results show that the success rate is reasonably close to 85%. So I suppose there's nothing wrong. My sample size just wasn't large enough.
But the thing is, that's not the meaning of the terms empirical and anecdotal evidence. Collecting data is empirical evidence by definition. Anecdotal evidence is something like a story of somebody's experience (which I don't really see how that can be considered evidence at all). Empirical evidence isn't necessarily going to prove anything, it can suggest something to be true to a certain extent. Empirical evidence is always going to be gathered by somebody or something, so it's always going to be somebody's experience. Whether or not you trust the integrity of the empirical evidence is another matter.
That, @Aosaw, is a perfect example of anecdotal evidence. And I've already addressed enough times the concept of statistics and probability in this thread, so I'll just leave it alone.
Except that I pointed out that most of that post is incorrect. Sorry Aosaw, but I do think most of the points in that post were incorrect, not just a matter of differing opinions. You're welcome to show me otherwise though.
This point I disagree with you. I don't disagree with your results, but I disagree with you on the concept of statistics. The examples in the first couple of sentences are meaningless, because if you just look at two or three trials it's obviously not going to mean anything. The same doesn't apply to a much larger sample, that is a major part of statistics, sample size is extremely important. Sure, if you get 50 successes out of a hundred with 85% chance of success it could be valid, but can you do it over and over again? You already showed in your own test that it's not likely, and certainly not repeatable.
I maintain that the best way to find an answer to the question would be to look at the code. But only the developers can do that, so...
YYYYYYYYYYNNYYNYYYYNYYYYYYYYYNNYYNNYYYYNYYYYNYYNNYYYNYNYYYYYNYYYYYYYYYYNYNNNYYYYYYYYNYNNYYYYYYYNYN
It shows a 76% success rating out of 100, and is perfectly valid for each item having an 85% chance of being a success.
This string alone has only a 76% success rating (it's the first half of the above string).
YYYYYYYYYYNNYYNYYYYNYYYYYYYYYNNYYNNYYYYNYYYYNYYNN
If I were only doing 50 loops rather than plucking it off the end, it is entirely possible for this chain to appear.
Again, that's not to say your concerns are not valid, Beamdog should probably look into it if the output is as low as you say it is, but my best guess is it's just the luck of the draw. You could have a string of Ns or a whole string of Ys.
For example, this a 92% success rate from one of my earlier tries (I copied it down since replicating it exactly is almost impossible):
YYYYNYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYNYYYYYYYNYYYYYYYNYYYYYNYYYYYYYYYYYNNY
This is also valid as an output.
Multiple times I have gotten percentages that were higher or lower than the 85% that the game says. My point is that, although improbable, the output could very well still be entirely valid.
If you say that each trial is independent, and 85% is just a threshold, then what is the point of the specific number 85%? There should be no difference between 51% and 99% chances of success. Each trial is calculated independently, and each trial has a higher chance of success than failure whether it be 51% or 99%. The only way the actual specific number has any significance is if we can see the success rate being close to 85% over multiple trials.
A more recent test involving 100s:
The total positive result of all tests yields a 60 percent success rate.
NNYYYNYYYNNYYNNYNNYYYNYYYYYNYNNYYNYNYYYNNNNYYYYYYYYNNNYYYYYYNYNYNYYNNYYYNYNNYNYNYYNNYYNYYNNYNNNYYYYY
I can't say if something is repeatable character-for-character due to it being random, but the result is repeatable with enough tries.
In quick succession, I have this one 63%:
NYNNYYNYNNNYNNYYNYYYYYNYNYYNYYYNYNYYNYYYNYYYYNYYYNYNYNYYYYNNNNYNYYYYYYYNYYNYYNNNYYYNYYYYYYYNNNYYYYNN
Now, I'm getting 90%+.
They might want to look at the RNG function and consider a rewrite, given that most games in the 90s tended to not be true random numbers.
Then that just means 100 is not a large enough sample size. If the chance of success is 85%, then over the course of many trials, the rate of success should be close to 85%. Your own results support this. Chance of success shouldn't be confused with rate of success, but at the same time with a large enough sample size they should approach one another. That is what statistics is all about. Each trial being independent is a characteristic of combinations, which is still a part of statistics, and should have statistically consistent results. Statistics is the only way of verifying that 85%.
I wouldn't be surprised if that was the case.
While being statistically lower and thus less likely to have a failure -- on an individual basis, if one fails they could all fail (if I can get 100% failure on five, that trend could continue, it's not probable, but it could (that's where our margin of error comes in), which is why I can do a test of 10 and have a whole string of Ns or a whole string of Ys with 100% failure of success based on something that should only be 85%), meaning that you could have a success probability (or rate) of 0% and it still be (10, for the same of an example) valid 'chances.' That's why only statistical probability can be worked out in this case, and that probability is between 60% (being the lowest I've gotten) and 92% (being the highest) in my case, based on empirical (observed) evidence. The actual chance per spell is 85%. The actual, programmed chance is a variable that can contain any value within the file mentioned on the previous page, depending on the intelligence of the character.
Clearly, the frequency of success is not 85% with a character of int 18. The evident, or empirical, rate (from what I can see,) is a median of 77% (using only the highest and lowest values that I have been able to get from the game and my program, actual median is about 84%, with deviation of 3%) with a deviation of ~16%. After repeating the tests several more times by running my own program to get near-identical results at a quicker pace, the results are still too far apart to really hit the exact 85 mark (even running the program with a sample of 1,000,000 only get 84.96 and 85.1, so it's still too much deviation, given the actual size of the sample, to say that it's 85 as a definite frequency. Even at a sample size of 10,000,000 the closest I can get is 85.03, which is about as close as anyone is ever going to get).
Meaning the chance to learn a spell can be 85% success, and 15% failure (that could be 15 failures followed by 85 success, or on a larger scale, ~144 failures followed by ~856 successes). A series of failures can happen in groups of 2 and 3 (even groups of 5 and 6), as some of the copy/pasting shows. Saying it is broken because you get more failures in one go is not representative of the whole.
The thing is, the game doesn't state frequency or rate, it states chance to learn spell, a spell, not more than one, and that chance is apparently 85 for my character (and 75 for Neera and so on), that chance is reset with every spell, and while statistically more likely to get successes than failures, no one can rule out that failures are going to happen. The average person won't learn any more than about 150 spells during a fresh playthrough (I'm not even sure what the maximum number of spell scrolls there are (without mods)), so working with a sample higher than that is pointless when trying to determine if what you are experiencing is normal behaviour for the game. General rules of probability say it shouldn't be too far away from the 85% listed as a chance over a long period of time, but clearly it can due to the deviation I'm getting at 'normal' rates; thus we can only realistically work out empirical data from our results, mostly as a mean or median average percent. We can work out the rate or frequency without any trouble, provided we always take the deviation into account. A normal rate is around 77% with some deviation, and that is with a chance of 85. I got as low as 60 and 63, so it wouldn't be that much of a stretch to get something similar to the results you got on the previous page, improbable though they might appear in such a small sample.
Unfortunately, if either my program or the game is bugged, that's probably an issue in Microsoft's (the OS in general, doesn't even have to be Microsoft) headers, and not in the game itself. Since rand uses a seed from srand, which gets the current time in seconds from your computer, and generates a random number based on that seed, it could very well be less effective than writing a random number generator themselves...
Of course, I could be the one who is confused, it's been a long day.
I agree with @Aosaw, though, the only way to properly know how the game is working it out would be to examine the code itself. Which is undoubtedly never going to happen. But if it is working elsewhere in the game, it should be working just fine here. If the devs say it is working as it should (or only way they can have it without doing a complete re-write of most random 'dice-throwing' statements or functions), then fine. I'm only explaining why, programmatically, it seems sound enough. I wouldn't have the faintest idea who to tag or ask. Like I said at the start, in all my years of playing BG, I've had possibly 50 spells fail (but then again, I usually import a mage from one of my other playthroughs, so the learning is already done).
If it is broken, then the Overhaul team should absolutely know about it (e-mail support).
I'm personally still inclined to trust it as it is; any time I'm not facing a 100% success rate, I prepare myself for the possibility of failure, even if it's only 15%.
Then again you'd probably just claim i have faulty dice and that they were not made properly
in the beginning i had what seemed to me like abysmal luck, i would always have to reload multiple times (in average 6-7) in order for the spell to be written successfully.
which when compared to successes (adding the first time ones) felt like the chance of failiure was greater than the chance of success for my three arcane spellcasters (so that's a lot of writing of spells), or 50/50 at best
i didn't have an impression that *first times* i tried to learn a spell, the chance of failiure would be greater than the chance of success tho - but when i did fail, i'd have to reload a lot
so eventually i came to a weird idea that some kind of counter might start when you press the button to write the spell and ends when you release the button (yeah, sounds lame). so i tried to hold the button longer aaand - i never had to reload multiple times again... srsly - the character still failed occasionally, but whenever i reloaded, i'd hold the button a bit longer than normal and the success rate felt like 100% basically.
then i also tried it with HP rolls whenever i rolled badly (holding the button longer to get a higher roll, dunno? ...) and uhm, i played with almost maximal hp for the rest of the game (because i like cheese) with just one reload per bad HP roll - i'd always get the highest roll minus 1-2 hp.
when doing that (hold the button longer), i'd start to trip i have a power to guess what the exact amount will be and i overwhelmingly guessed with a small margin of error +/-1.
i really don't like posting this because it sounds extremely superstitious and if i'm wrong and there's no connection between those things it'll be pretty embarassing (i already feel stupid), but i still felt like i had to because had a huge impression that there was something there.
I was under the impression that for every spell level higher (for the spell you're learning) than the highest spell level you know, there is an incremental 10% increased failure chance.
So I took a very smart level 1 mage down to the map below Nashkel, obtained that cloudkill scroll and got well aquainted with the quickload and inventory keys. I noticed markedly more successful scribes than not, so that debunks that, no idea why I thought that was the case. I'll just leave this here, in case anyone else has the same strange thought for the same strange reason.
Also keep in mind that you receive a penalty if you try to write spells that are higher level than you can cast.
Lets say:
x = % chance of success
y = die roll
then make the condition for success: y > (100 - x)
In any case, I suppose it doesn't really matter. Reloading isn't really a huge deal. I just find it psychologically annoying when video games seem to be 'rigged' to give the player a disadvantage, whether that actually is the case or not.
Like when I played Governor of Poker, there were so many times I got a really good hand (like a full house) and I go all in, then my opponent has an even better hand. Many people on forums claim the game is rigged.
Or like Gran Turismo 5, in which it's blatantly obvious, not hidden in the coding. On races where there is no qualifying session you automatically start last. Even though AI in the game is stupidly easy, it still annoys me that I am disadvantaged from the start.
@bob_veng That's an interesting observation. I'll try that sometime.
I am claiming this: http://en.wikipedia.org/wiki/Law_of_Large_Numbers
Although I've never heard of the term "Gambler's Fallacy", I've already somewhat described it earlier in the thread:
And to a lesser extent @Bytebrain
Since you don't seem to be understanding what my point is, and continually misconstrue what I'm saying. I just found this page on wikipedia. It's an actual proper theorem.
http://en.wikipedia.org/wiki/Law_of_Large_Numbers