Chance to learn a spell (with numbers)
Ariathor
Member Posts: 8
Background story (skip ahead if you want hard numbers): I'm a fairly new player, I bought and completed the original Baldur's Gate less than a month ago and I'm in the middle of my first playthrough of the enhanced edition. I planned to dual-class Imoen to a mage today, so I had saved a bunch of scrolls (around 20) to get her started.
I gave her a potion of genius to bring her intelligence to 21, which should give her a 97% spell learning success rate. However, I she failed learning way too many spells on my first try (around 3-4). I wanted to avoid reloading, but I was a bit perplexed, so I thought I'd give it another try. This time I counted and she failed 6 out of 20 spells. I made some more research and that's how I learned that you get a penalty for trying to learn spells above your current casting level.
I am already a bit skeptic about the Baldur's Gate RNG, since on my first playthrough I had a tendency to reload level-ups and I noticed that it was really hard to get the maximum number sometimes (like 20+ times when it should have been 1 out of 6). So I was not very convinced and I remember reading somewhere in an old post that people often complain about the RNG but never give proof (I think it was Elminster that said that?), so I thought I'd do some more testing on my own. I used EEkeeper to give Imoen ~120.000 more XP, which brought her to level 8 (I gave her XP instead of changing her level because I wanted to do everything as "naturally" as possible).
So I used this save to test again, and again she failed 5 times. I repeated the process five times and came up with the following number of failures:
5 out of 21
4 out of 20
6 out of 21
4 out of 20
4 out of 20
Some simple math: The number of spell scribing failures should follow the binomial distribution. Out of 20 trials, we would expect to have 0.6 failures!. Instead we have 4-6 every time. The probability of having 4 or more failures out of 20 trials is is 0.3%!!. As you can imagine, the probability of having 5 or 6 is even smaller. The probability of having 23 or more failures out of 102 tries is 4*10^-14, in other words 0.000000000004%.
This is obviously not very likely (an understatement), so I made this post here to ask what could be causing this. Am I missing something? Is there some other rule that would reduce the probability of learning a spell? Could this be a bug?
PS. I recorded my testing in case I need to provide video proof for a bug report. I didn't post this in a bug report forum in case I'm missing some obscure rule (After all, none of the FAQs I had read nor playithardcore mentioned the penalty for trying to learn spells above your level, so this is a possibility) and because I did edit the save file to make the testing possible and technically that is not allowed for bug reports (I guess I could try to reproduce this using only the console if this seems to be a bug).
EDIT: Forgot to mention the obvious, no mods.
I gave her a potion of genius to bring her intelligence to 21, which should give her a 97% spell learning success rate. However, I she failed learning way too many spells on my first try (around 3-4). I wanted to avoid reloading, but I was a bit perplexed, so I thought I'd give it another try. This time I counted and she failed 6 out of 20 spells. I made some more research and that's how I learned that you get a penalty for trying to learn spells above your current casting level.
I am already a bit skeptic about the Baldur's Gate RNG, since on my first playthrough I had a tendency to reload level-ups and I noticed that it was really hard to get the maximum number sometimes (like 20+ times when it should have been 1 out of 6). So I was not very convinced and I remember reading somewhere in an old post that people often complain about the RNG but never give proof (I think it was Elminster that said that?), so I thought I'd do some more testing on my own. I used EEkeeper to give Imoen ~120.000 more XP, which brought her to level 8 (I gave her XP instead of changing her level because I wanted to do everything as "naturally" as possible).
So I used this save to test again, and again she failed 5 times. I repeated the process five times and came up with the following number of failures:
5 out of 21
4 out of 20
6 out of 21
4 out of 20
4 out of 20
Some simple math: The number of spell scribing failures should follow the binomial distribution. Out of 20 trials, we would expect to have 0.6 failures!. Instead we have 4-6 every time. The probability of having 4 or more failures out of 20 trials is is 0.3%!!. As you can imagine, the probability of having 5 or 6 is even smaller. The probability of having 23 or more failures out of 102 tries is 4*10^-14, in other words 0.000000000004%.
This is obviously not very likely (an understatement), so I made this post here to ask what could be causing this. Am I missing something? Is there some other rule that would reduce the probability of learning a spell? Could this be a bug?
PS. I recorded my testing in case I need to provide video proof for a bug report. I didn't post this in a bug report forum in case I'm missing some obscure rule (After all, none of the FAQs I had read nor playithardcore mentioned the penalty for trying to learn spells above your level, so this is a possibility) and because I did edit the save file to make the testing possible and technically that is not allowed for bug reports (I guess I could try to reproduce this using only the console if this seems to be a bug).
EDIT: Forgot to mention the obvious, no mods.
Post edited by Ariathor on
8
Comments
One more detail needs confirmation: were all the spells you were trying to scribe level 4 or lower? (Because scribing a level 5 spell without penalty needs a level 9 Mage, and you're doing this with Imoen at level 8.)
If that's so, then ... you've avoided distortion by the little-documented penalty, you've conducted enough attempts-to-scribe to be a non-trivial sample, your maths looks right ... I think you've provided fairly good statistical evidence of an engine bug.
However, even so that doesn't prove that it's specifically the RNG which is faulty. For example, it could also be a fault in the level-penalty for spell-scribing, if it's applying incorrect or spurious penalties. (In fact, I reckon the latter is more likely.)
Just a thought: your observed results would make sense if the failure probability is actually 10 times what it ought to be, so it makes me wonder if there might simply be a missing 0 in the engine source code, e.g. "... /10" instead of "... /100" somewhere.
In the interest of honesty, I did have one Cloudkill scroll and I was too lazy to re-edit the save file to get to lvl9. Funnily enough that one only failed once out of 5 times, but even discounting it we would have 18 failures out of 98 attempts, which is still a ridiculously low probability (around 10^-11). I also repeated the experiment a couple of times with a lvl9 Imoen after starting this thread and the results were similar.
If this is indeed a bug there could be a number of explanations, from the RNG being tied to some environmental/save variables that lead to similar results after reloading to incorrect implementation of the level penalties (as you mentioned) or something completely different.
I don't think level penalties are the only factor. I tend to do major spell scribing at level 10, with scrolls below level 5, and I've seen the same patterns.
This is why I always stack Potions of Genius for 25 Intelligence in my no-reload runs, because anything less is extremely unreliable.
Nevertheless, in the interest of avoiding cheesiness, I'd prefer to keep difficulty constant and use a potion to keep the risk small ... if only we could trust the game to deliver a correct small risk instead of the observed high risk.
In the meantime what exact spells were you using for these tests Ariathor? Also I assume when you EEkeepered her you kept her as a non-specialist mage?
Infravision
Ghost Armor
Dispel Magic
Color Spray
Magic Missile
Armor x2 (used the second one if first one failed)
Burning Hands x2 (used the second one if first one failed)
Identify
Chromatic Orb
Sleep
Find Familiar
Glitterdust
Cloudkill (see my explanation above)
Dire Charm
Charm Person
Lightning Bolt
Stinking Cloud
Stoneskin
Mirror Image
I dual-classed Imoen naturally (so no no specialty). I only used EEkeeper to give her the extra XP, because I didn't want to edit my baldur.ini to enable cheats at that time.
Ok, I did some further testing.
I took the pre-generated mage NPC (17 Intelligence) and used the console to transport her to Sourcerous Sundries, give her 100.000 gold, 160.000XP and a scroll case. I leveled her up to level 9 and bought a single copy of every learnable scroll available to me (around 67 scrolls) in addition to a potion of genius. I drank the potion, bringing me up to 21 Intelligence learned every single scroll with 0 failures.
I was surprised, so I repeated the process by creating a Thief with 17 Intelligence, that I brought to level 6, dual classed to mage and then gave enough XP to reach level 9 (while staying under the XP cap). Again, I had 0 failures.
I was surprised, because the probability of that happening assuming a 97% success rate is quite low (~1.6%), but not enough to be of statistical significance.
Now for the good part: I started yet again another game, going out of Candlekeep to recruit Imoen. I repeated the same process as above with her, i.e. made her a level 6 thief/level 9 mage with 21 Intelligence from a potion of genius. For about the same number of scrolls, she had a ridiculously high failure rate (around 15), sometimes chaining two or three failures in a row.
Obviously there is something wrong with that. I have two theories:
1. There is something wrong with Imoen or NPCs in general.
2. The RNG is unbelievably streaky. Not having any failures in my first two trials made me a bit suspicious, so I repeated the process from a save of the first character. In two more trials, I still had 0 failures (total probability of 0 failures out of ~265 attempts is 0.03% if each attempt has a 3% chance to fail)
Further testing (and probably some input from a dev/digging through code) would be required to pinpoint the exact problem.
P.S. I have most of these tests recorded.
Well there's definitely something wrong with spell-scribing (although it's not evident exactly where the cause lies), since your tests are using decent-sized samples. However, it's bound to be relevant that this apparently applies to some Mages and not others.
Unfortunately, I can't recall details of my own less-systematically-tested impression that scribing fails too often, such as which Mages were involved, protagonist or NPC, etc., since I gave up struggling with it (and resorted to manipulating the difficulty setting instead) some while ago. Nevertheless, I'm pretty sure it's not just Imoen who suffers from excessive scribing-failure.
Come to think of it, some of my experience of excessive failure was in BG2ee, not just BG1ee. That may also be relevant.
If the starting seed is the same, which could happen after a reload.
Computers and "random" is a very tricky subject
Also, @semiticgod, 24 (not 25) INT is already enough for 100%.
Although in SoD (and I guess in BGEE and BG2EE with the subsequent patches) it all will be changed, with specialist mages getting -15% chance of learning spells not from their school.
Note that Potions of Genius are significantly less expensive than Potions of Mind Focusing.
If you're interested in doing a couple of tests yourself, I'll post the relevant console commands and my methodology at the end of these post, it might be useful to have someone else try it out to confirm that it isn't happening only for me.
@Feytor
I agree, as I mentioned in my first post I don't have the tools to deduce what could be the cause for it just through empirical testing. Hopefully someone with access to the code and more experience will be able to shed some light into it.
@Troodon80
I've played a lot of games in my life that include randomness (like Magic the Gathering and Hearthstone) and I understand that people constantly complain about randomness, even when it is truly random (such is the human nature, truly random results are rarely perceived as random). However I have a basic understanding of both mathematics and programming and I don't make such claims lightly.
I imagine you wrote this short function to prove to people that they shouldn't expect to get a 70/30 split with 16 Intelligence every time (actually the probability of a 70/30 split out of 100 scrolls is 1%!! Certainly not intuitive). But even in that very simple function, your numbers are not truly random. According cplusplus it does not produce uniformly distributed numbers (the modulo operation slightly skews the results in favor of lower numbers). Obviously that doesn't matter much for a small sample program, but it shows how hard it is to make a variable that behaves like a truly random variable. Randomness in programming is a huge topic and I certainly do not pretend to be an expert in it.
My conclusion is that there is a problem with how NPCs handle spell scribing (that would explain why you got the expected results in your own trial). I repeated my test with Edwin and Neera. In total, I got the following results. Each one was in a new game to avoid possible bad seeding issues:
Imoen with 21 Intelligence: 15 failures out of 68 scrolls (Probability=1.4*10^-9)
Neera with 21 Intelligence: 18 failures out of 62 scrolls (Probability=2*10^-13)
Edwin with 22 Intelligence: 11 failures out of 60 scrolls (Probability=2.8*10^-8)
That's without accounting for my initial testing with the edited save game. As you can see we're talking about ridiculously low probabilities here. I think we can say with quasi certainty that there is a problem here. Human error is much more likely than a probability in the order of 10^-30. Admittedly I could be the cause of the human error, so I uploaded my videos on youtube here for people to scrutinize (that's why I didn't post this in the bug report forum too). If not, it means there is indeed a faulty implementation of how the game handles spell scribing by NPCs.
To anyone interested in trying to replicated my results, here is my methodology:
Start a new game, recruit Imoen, Edwin or Neera give them enough XP to reach level 9 as mage (in the case of Imoen dual class her first), 100.000 gold and a scroll case (so I can buy all the scrolls at once) and teleport them to Sorcerous Sundries. There, I buy every learnable scroll (slight variations because of different starting spells and Edwin's specialization) and a potion of genius and count how many failed in core difficulty.
CLUAConsole:MoveToArea("AR3300") Beregost for Neera (you can use ctrl+Y to install kill the assailants)
CLUAConsole:MoveToArea("AR4800") Nashkel for Edwin
CLUAConsole:CreateItem("BAG03") scroll case
CLUAConsole:AddGold("100000")
CLUAConsole:SetCurrentXP("160000")
CLUAConsole:MoveToArea("AR0703")
Apart from the fact that several people are talking utter crap about the interpretation of probability and statistics, a couple of valid facts do emerge.
1) Your own extensive test demonstrated that for a particular protagonist Mage in particular environmental circumstances, spell-scribing was working correctly.
2) Other people testing NPC Mages or other protagonist Mages or other environmental circumstances consistently found very different results, but hadn't conducted systematic tests on the scale of yours.
3) There are a couple of indications in that thread that BG may be using a buggy seeding mechanism for whatever pRNG it's using.
What seems to be emerging from this present thread, however, is confirmation of what was alleged in the previous thread, i.e. that results are wildly different depending upon exactly which Mage is trying to scribe and under what circumstances.
Wildly different in a trivial sample means nothing, of course, but wildly different in a significant sample is evidence that something is wrong. (Obviously the larger sample, the stronger the evidence.) Er ... as an example of what? That code will just make calls to (the GCC library's version of) the rand() function to produce some (pseudo-)random numbers, which neither tells us anything about how the BG engine sources its random numbers nor tells us how the BG engine calculates the level-penalties for scribing and then calculates the success or failure of scribing.
Therefore I deduce that there is a specific bug which has an identifiable effect:
either the circumstances at the time of attempting to scribe a scroll are such that the bug doesn't trigger, and scribing is then calculated correctly;
or the circumstances are such that the bug does trigger, and the failure rate is incorrectly increased by a factor of about 10.
This implies that it's probably not an issue in the RNG itself (because that'd probably produce much more variable errors), instead it's probably an error in the code calculating success or failure.
I recently started BG2EE and thought I might as well repeat my tests.
With the same procedure Imoen failed 9 out of 55 scrolls. I used the same save to test the pre-made mage PC with a potion of genius (23 Int) and she only failed once. Then I used the same save for Imoen again and she failed 8 times. Needless to say that the probability of 19 or more failures out of 110 trials is a ridiculously small number.
Up until this point this was a completely clean BG2EE install, no mods, no NearInfinity, no shadowkeeper.
I decided to dig a bit deeper with NearInfinity and noticed something strange: In intmod.2da that @Mathsorcerer mentioned, the chance of success with 24 and 25 INT is set to 150 in both EEs (it's set to 100 in my BG1 original GoG version). So I edited it down to 100 (which should be enough) and gave Imoen another potion of genius to reach 25 INT. She failed scribing again!. Even though she should have 100% success rate according to intmod.2da
This points back to @Gallowglass's suggestion that it's more likely some kind of penalty being applied incorrectly rather than bad RNG.
I've been busy lately with personal life the holidays (and I assume it's the same for the developers), but since I still can't come up with a legitimate reason for my results I will make a thread in the bug report forum whenever I find an opportunity.
"Easy Spell Learning
Previously known as 100% Learn Spells
This component makes it easier for arcane spellcasters to scribe scrolls to their spellbook. The player has a choice between two implementations:
100% Learn Spells. Scribing scrolls is always successful.
100% Learn Spells and unlimited spells per level. In addition to the above, this implementation removes the limits on the maximum number of spells learned per level."
Give characters 150% chance of succes instead of 100%, maybe because you can still fail with 100%.
To dispel any doubts that something was wrong with my personal installation (even though I didn't have any mods), I completely deleted BG1:EE (including saves and baldur.ini) and re-installed it.
With the same test (without editing intmod.2da) I got yet again similar results (Imoen failed 10 scrolls out of 67) with 21 INT. For fun I thought of repeating my test with the edited intmod.2da (I set it to 110 at 25 INT this time) and made Imoen drink a second potion of genius to raise her to 25 INT. I got 2 failures out of 67 attempts (with what I presume to be an 110% chance of success!).
EDIT: And yes, @Mathsorcerer, if you could do some testing that would be very useful, I would like to confirm that I'm not the only one with abnormal results.
I hope there are plans to fix this problem. I checked the release notes for 2.0 and 2.1 and I couldn't find any mention of this.
Also, I am not sure but, there may be a penalty with trying to scribe higher level scrolls than you can cast. A lvl 1 mage may fail scribing a lvl 9 spell more often than a lvl 1 scroll.
Specialists suffer a -15% penalty to scribe any spell that is outside their specialty school. There is another 15% penalty to attempt to scribe a spell that is higher than the level you can currently cast.
Let's say your specialist has Intelligence 18. That's an 85 percent base chance to scribe. You attempt to scribe a spell that is outside your specialty. Your chance goes down to 70 percent. The spell is of a higher level than you can currently cast. Your chance goes down to 55 percent. That's barely higher than a 50-50 chance to scribe that spell, with Intelligence 18.
The results of the scribing tests in this thread make perfect sense when those penalties are subtracted. There's nothing wrong with the spell scribing RNG, it turns out. It's just that most people don't know how it actually works, which is understandable given how many years the penalties went undocumented.
I also used to believe the spell scribing RNG was bugged, until I found out this new information. Now I always play with Core Rules without cheating, making Intelligence score and Potions of Genius relevant again for my mages.
Also, 20 generations isn't nearly a large enough sample size to draw any conclusions. He may have just been having a run of bad luck. The RNG gods are very fickle.
EDIT: First attempt at this post had a dumb mistake in it.
Also, this isn't really worth arguing about. I mainly just wanted to point out information that might be helpful.
I did some 'not nearly as scientific' testing with a bunch of scrolls from High Hedge and came to the same conclusion. I understand the whole 'specialist scribing scrolls outside their school' penalty, but that doesn't account for CHARNAME mages (non specialists). I literally said to myself, "If I'm THIS (un)lucky I need to go buy a lottery ticket because I'm sure to win!"
Thanks for all the testing and confirming what I suspected all along. It's NOT a random chance based on the INT % chance to learn spells. At least not with all characters and NPC's, all the time.
Thank you, @BelgarathMTH, you have made my game a bit more interesting