Slot Statistics - Critiques Requested

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Jufo,

First of all, I sincerely apologize for obviously really pissing you off. I don't know what else to say. I'm sorry.

I said "having fun with that sim" because I assumed that the work which you outlined in your last paragraph:



would be long and tedious otherwise. Being long and tedious you would thus be disinclined to pursue it. I then assumed that the availability of your sim would make this work much easier, to the extent that generating statistical results which I agree would be very interesting would be "fun". Thus - "having fun with that sim". That's all I meant by that.

BTW, Eliot is in Cambodia right now, attending/speaking at a conference on Casino security.

Again, I apologize for making you so angry, or feeling so insulted.

Chris

Ok I see, thanks for clearing this up. It just felt like I am spending so much time and doing so much effort in trying to explain things and bring new things to the table only to receive sarcastic remarks as a result. But if that's what you meant by "having fun with the sim" then I took it the wrong way and I am fine.
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
I am sorry Chris for being quite tempered in my previous post, and saying things I didn't mean to. I would completely re-write that post but I can't edit it anymore. There were some other things in my life, not related to this thread or you, that had made me very frustrated and annoyed, and when I came back to this thread I became even more frustrated because it felt like the discussion was going nowhere. I felt like trying to swim upstream and only going backwards. So I lost my cool there a bit and I apologize to you. I hope you accept it.

I re-read the last few posts and I understand better now what you were getting at with those questions. I was wrong when I wrote that the SD or Variance doesn't apply to the RTP. It's very definition is to measure the amount of spread around the mean (and here mean = RTP). So the part in the help-file "The Standard Deviation values are associated with the Theoretical RTP values. " is entirely correct.

The only part where we didn't reach consesus is how to express the "relative to unit bet" best. The concept is trivial, like I tried to express with the money exchange example
but it sounds more complex when you try to explain it. I see now that the page reads:

"Please Note: The Standard Deviation and Variance values for each slot were calculated based on a total bet per spin of 1 unit (1 dollar), and on using the maximum payline count per slot."

and to me this seems just fine.

When describing variance, the page could also say something like: "Variance measures the amount of spread of the results. The larger the variance the more spread out the outcomes will be." This would be a description that is easy to understand. But it kind of says the same thing where describing Low, Medium and High variance slots. At the moment I have no other suggestions.

A player wrote to me by PM that inspired by this thread he had played only 3 lines on a high-variance slot. I wish to stress that this thread is for theoretical discussion and in no way advocating people to use such high risk betting styles.
 
Last edited:

GrandMaster

Dormant account
Joined
Jan 21, 2004
Location
UK
Jufo,

For ease of access, Link Removed ( Old/Invalid) to the page in question.

Note my Definition of Terms for Standard Deviation. What you have just described completely contradicts everything in here. I'm OK with the contradiction. My goal is an accurate publication.

(Technically, we did not calculate the SD. We calculated the Variance (using the equation in my first post), and then just took the square root of that for the SD.)

Assuming SD as it applies to a normal distribution curve, what are your X and Y axis labels?

Given your example #1 below:

20 lines, $5 bet per spin ($0.25 per line): SD = $5*4.7 = $23.5 per spin. Given that 1 SD is "66% of the area under a normal curve", are you saying that 66% of the time your win on this spin will fall between what? 0 and $23.5?


Chris
There is an error in this page:

"For example - using the Crazy 8 Line slot, with a Theoretical RTP of 97.47%, and a Standard Deviation (SD) of 4.8:
Let's say that we have 100 Players, each of whom has played 2,000 rounds of Crazy 8 Line. The Player RTP results that we might expect to see are:

About 68 Players will have an RTP in the range 97.47 +/- 1 SD = 92.7% to 102.3%.
About 95 Players will have an RTP in the range 97.47 +/- 2 SD = 87.9% to 107.1%.
About 99 Players will have an RTP in the range 97.47 +/- 3 SD = 83.1% to 111.9%.
Roughly 1 Player out of 100 will have an RTP below 83.1% or above 111.9%."

The standard deviation is proportional to the square root of the number of games, so the SD of 2000 spins is about 214.6 (units). As a percentage of the amount wagered, it is about 10.7%.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
Jufo, your apology is immediately accepted. :)


There is an error in this page:

The standard deviation is proportional to the square root of the number of games, so the SD of 2000 spins is about 214.6 (units). As a percentage of the amount wagered, it is about 10.7%.

GrandMaster,

Thanks for the input. As I said in an earlier thread to Jufo, I appreciate people pointing out any errors. My goal is an accurate publication.

I'm assuming that you got your 214.6 units by the formula: Square Root of 2000 = 44.7 * published SD of 4.8 = 214.6. 214.6 / 2000 ~= 10.7%.

Given the back-and-forth that Jufo and I have been doing, I was also curious as to what minimum sample size (other than the 2,000 rounds currently used) would be required in order to approximate the breakdown presented in the Help file's example.

(I ran these experiments and crunched these numbers yesterday, before I read your post. Jufo's sim has the reel layout for my Hot Peppers slot, so I did the following experiment using this slot rather than the Crazy 8 Line slot referenced in the Help file. However, please note: the SD for Crazy 8 Line, at 4.8, is almost exactly the same as Hot Peppers at 4.7.)

If you've read through this thread, you're aware that I have a table in my database containing 20 million rounds of Hot Peppers. For no reason in particular, I picked 5,000 rounds as a place to start. So, I did what was needed to get the RTP for the first 5,000 rounds, then the next 5,000 rounds, and like that.

(This was kind of tedious, so I did it for only 50 sets rather than the 100 mentioned in the Help file's example.)

Here is the data: Hot Peppers has a Theoretical RTP of 97.48 and a Standard Deviation (SD) of 4.7.

-3 SD-2 SD+/-1 SD+2 SD+3 SD
83.38%88.08%92.78% to 102.18%106.88%111.58%
86.5188.1493.23104.16108.64
86.5588.6694.22106.48
87.0290.0594.26106.86
87.6990.2994.38
87.8790.5194.40
90.6294.51
91.0594.65
91.1094.74
91.1194.75
92.0895.39
92.1095.62
92.2296.33
92.2896.42
96.57
97.26
97.37
98.08
98.28
98.63
99.21
99.37
100.03
101.18
101.21
101.50
101.86
101.88

There was one outlier - it was in the +4 SD range at 112.85.

So, a sample size of 5,000 falls reasonably well into the +/3 SD range using an SD of 4.7.

I am going to use my admittedly very limited understanding of your previous post, and perform a similar calculation.

Square Root of 5000 = 70.7 * published SD of 4.8 = 339.4. 339.4 / 5000 ~= 6.8%.

Backing your formula up would result in a sample size of 10,000 yielding an SD of 4.8. However, although 10,000 rounds gives the more precise result, 5,000 rounds is not exactly out of the ballpark either.

I know that when the issue of Theoretical RTP comes up, often the response is "Oh, you need to play a gazillion games before you'll ever see anything approaching that value.", and like that. I'm being facetious, but I'm sure you get my point. A Player can begin to see the statistics come into play with significantly less rounds than a gazillion.

I am thus left to conclude that where you state "There is an error in this page:", you are referring to the sample size of 2,000. If true, I agree. I'll perform some calculations, and run some verification tests, but this number needs to be changed.

Chris
 

3Dice

Accredited Casino Representative
Joined
Feb 14, 2007
Location
-
Chris,

I think grandmaster meant that the variance for x spins is not the same as the variance for 1 spin.

The statement he quoted from the original page is indeed completely wrong .. like applying oranges to apples so to speak.

If the SD of a single game is 4.8 (or 480%)

then the SD of 2000 games is sqrt(2000)*4.8/2000 = 0.1073 (or 10.7%)

So the numbers given on that page (68% will ... ) .. are wrong.

(they should be :
About 68 Players will have an RTP in the range 97.47 +/- 1 SD = 86.77% to 108.17%.
About 95 Players will have an RTP in the range 97.47 +/- 2 SD = 76.07% to 118.87%.
About 99 Players will have an RTP in the range 97.47 +/- 3 SD = 65.37% to 129.57%.
Roughly 1 Player out of 100 will have an RTP below 65.37% or above 129.57%.
)

i.e.

if you bet $1 and win $5 that would be just over 1 SD from average
if you bet 2000 spins of $1 and walk away with $10000 that would be almost 50 SD from average
(note - both these sessions have the same RTP .. )

Note that although the variance tells you much more about a slot than the RTP does, it is just a single number with
not much more meaning than the average a win will be away from the RTP. It doesn't tell you whether or not this
is with one huge win, or a couple big ones or .. etc ..

in fact, using the normal distribution to approximate the actual distribution of your typical slotmachine is a practice
that requires a lot of insight. For example, on the left side of the curve, this is a fairly valid approximation,
however on the right side (i.e. the winners) .. you will see a lot of exceptions since most bigger wins will be many,
many SD's away from average ..

Determining how many spins is needed for a 'fair' representation is even more difficult and the answer will
be different for each machine. A machine with 99.999 losing spins and one big winner would need a very large
amount of spins .. In general you could use a number of 'rules of thumb', one I use would be 30 times the
feature frequency .. since the feature often includes a very large amount of the payout, any representative
sample would need to include enough features. So if the feature gets hit once every 100, then 3000 spins
would be my rule-of-thumb answer.

Enzo
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
There is an error in this page:

"For example - using the Crazy 8 Line slot, with a Theoretical RTP of 97.47%, and a Standard Deviation (SD) of 4.8:
Let's say that we have 100 Players, each of whom has played 2,000 rounds of Crazy 8 Line. The Player RTP results that we might expect to see are:

About 68 Players will have an RTP in the range 97.47 +/- 1 SD = 92.7% to 102.3%.
About 95 Players will have an RTP in the range 97.47 +/- 2 SD = 87.9% to 107.1%.
About 99 Players will have an RTP in the range 97.47 +/- 3 SD = 83.1% to 111.9%.
Roughly 1 Player out of 100 will have an RTP below 83.1% or above 111.9%."

The standard deviation is proportional to the square root of the number of games, so the SD of 2000 spins is about 214.6 (units). As a percentage of the amount wagered, it is about 10.7%.

A very good spot. In fact I spotted this yesterday too, when I started to make calculations to compare the normal distribution approximation to actual distribution of results.

It looks like the above values were obtained by simply assuming that 1 SD range equals the unit standard deviation (4.8) value as a percentage: (102.3% - 92.7%) / 2 = 4.8%. Chris, this is obviously incorrect. First of all SD values are not percentages (4.8 != 4.8 %) and like Enzo pointed out the range of results for +/- 1 SD always depends on the number of spins. Now I see the confusion from our previous discussions where you thought that standard deviations are ratios like RTP. They are not.

To clarify, below is the calculation you need to do to get the +/- 1 SD range for 2000 spins:

SD_TOTAL = SD_UNIT*BET_SIZE*SQRT(SPINS)

2000 spins, $1 bet size, SD_UNIT = 4.8, gives:

SD_TOTAL = 4.8*$1*SQRT(2000) = $214.67 (1 standard deviation for 2000 spins)

House edge across 2000 spins is 2000*$1*(1-0.9747) = $50.6

So the 1 SD range would be:

-HE-SD_TOTAL .... -HE+SD_TOTAL
-$50.6-$214.67 ... -$50.6+$214.67 = -$265.26 ... +$164.06

When you divide the amounts by total wagered ($2000) you get the RTP range for 1 SD as:

86.74% ... 108.20%

You can work out the rest of the values the same way.

Enzo said:
(they should be :
About 68 Players will have an RTP in the range 97.47 +/- 1 SD = 86.77% to 108.17%.
About 95 Players will have an RTP in the range 97.47 +/- 2 SD = 76.07% to 118.87%.
About 99 Players will have an RTP in the range 97.47 +/- 3 SD = 65.37% to 129.57%.
Roughly 1 Player out of 100 will have an RTP below 65.37% or above 129.57%.
)

Putting the numbers to Excel and avoiding any rounding I get the following numbers which, apart from the last row, are very close to yours:

About 68 Players will have an RTP in the range 97.47 +/- 1 SD = 86.74% to 108.20%.
About 95 Players will have an RTP in the range 97.47 +/- 2 SD = 76.00% to 118.94%.
About 99 Players will have an RTP in the range 97.47 +/- 3 SD = 65.27% to 129.67%.
Roughly 1 Player out of 100 will have an RTP below 72.50% or above 122.44%

The last row disagrees with yours because +/- 3 SD isn't the same as "1 Player out of 100". Instead you need the calculate the RTP at +/- 2.32635 SD,
because in Excel: NORMDIST(-2.32635) = 1% and NORMDIST(2.32635) = 99%.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
Enzo,

Thank you for this considered response. As I mentioned, my objective here is to provide an accurate publication.

As best I can determine, those with knowledge that exceeds my own on this subject are OK with everything on this page except for the number of sample spins defined in the Standard Deviation Definition of Terms section.

I believe that the above statement is True, and would appreciate anyone posting "False" if otherwise.


Using The equation referenced in both your post and in GrandMaster's post:

(sqrt(X) * 4.8) / X = 4.8.

resulted in a spin count of ~10,000. So, I just modified the file to use this number.

For Hot Peppers, which has a Free Spin Frequency of 1 every 75 spins, this would result in a sample count of 2,250 rounds. I would be reluctant to use that sample count, given that the previous table of data was for this slot and a sample count of 5,000 could perhaps be most accurately described as "marginal".

Again Enzo, thanks for the input.

I bring my first statement to everyone's attention and repeat my request that if it is not True then please post "False". (Actually, if you could give me a clue as to what is "False", that would also be appreciated.)

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Enzo,

Thank you for this considered response. As I mentioned, my objective here is to provide an accurate publication.

As best I can determine, those with knowledge that exceeds my own on this subject are OK with everything on this page except for the number of sample spins defined in the Standard Deviation Definition of Terms section.

I believe that the above statement is True, and would appreciate anyone posting "False" if otherwise.


Using The equation referenced in both your post and in GrandMaster's post:

(sqrt(X) * 4.8) / X = 4.8.

resulted in a spin count of ~10,000. So, I just modified the file to use this number.

Yes, your RTP ranges are correct for 10,000 spins because SQRT(10000)*4.8/10000 = 100/10000*4.8 = 4.8% so the standard deviation parameter transforms into a percentage.

The last row is still not correct: "Roughly 1 Player out of 100 will have an RTP below 83.1% or above 111.9%."

You took those RTPs from +/- 3 SD range above but 3 Standard deviations equals a chance of 1 in 740, not 1 in 100. The Standard deviation point for 1 in 100 chance equals 2.32635 SD, not 3 SD.

For Hot Peppers, which has a Free Spin Frequency of 1 every 75 spins, this would result in a sample count of 2,250 rounds. I would be reluctant to use that sample count, given that the previous table of data was for this slot and a sample count of 5,000 could perhaps be most accurately described as "marginal".

What do you mean by sample count of 2250 rounds above? Free spins are included in the payout of the triggering spin, so 10 000 spins doesn't include free spins, you need to have 10 000 spins where free spins are not counted towards the total.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
OK guys, I'm trying hard to keep up here.

Because of Jufo's sim and the Hot Peppers reel layouts, most of the specific data in this thread has been about Hot Peppers.

So, I just modified the SD section of the file to reference Hot Peppers.

And, as I mentioned in my earlier post, I changed the sample count from 2,000 rounds to 10,000 rounds.

At the top of this section of the Definition of Terms, I underline the words "given a reasonably large sample size". This is your generic "out clause" for someone that doesn't want to get too specific about exactly what kind of sample size is required and how to calculate it.

I included this "out clause" because I had no freakin' idea how to calculate it. :)

Now that I have been provided with some specific recommendations on how to do that, my concern in providing this additional data is making this file any larger and more dense than it already is. (I think I mentioned in some earlier thread that it was already weighing in at about 500 pounds.)

But, then again, the probability that anyone is actually going to read this page is so vanishingly small, going from 500 pounds to 550 pounds may not be an issue. (Just as with the previous post on Game RTPs, ultimately this thread too will begin its eventual decay into CM's historical thread pile, doing its best impression of "lining the bottom of the bird cage".)

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
If you've read through this thread, you're aware that I have a table in my database containing 20 million rounds of Hot Peppers. For no reason in particular, I picked 5,000 rounds as a place to start. So, I did what was needed to get the RTP for the first 5,000 rounds, then the next 5,000 rounds, and like that.

Just a heads up, that you can also do this more efficiently with the simulator. Set it to play 5000 rounds and to repeat this 10 000 times, recording the RTP from each simulation of 5000 spins. Then check how the RTPs from the simulation fall into the percentage ranges in your help file. Ok, I can do it...
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
The last row is still not correct: "Roughly 1 Player out of 100 will have an RTP below 83.1% or above 111.9%."

You took those RTPs from +/- 3 SD range above but 3 Standard deviations equals a chance of 1 in 740, not 1 in 100. The Standard deviation point for 1 in 100 chance equals 2.32635 SD, not 3 SD.

What do you mean by sample count of 2250 rounds above? Free spins are included in the payout of the triggering spin, so 10 000 spins doesn't include free spins, you need to have 10 000 spins where free spins are not counted towards the total.

Jufo,

I was trying to avoid referring to a fractional person. That is "Approximately 0.3 out of 100 Players ...". I agree, the way that it is currently written may not be statistically accurate, but at least it does not involve any cutting implements.

Enzo, in his "rule of thumb" suggestion, indicated that 30 times the feature frequency (which for Hot Peppers is the Free Spin frequency at 1 in every 75 spins = 2,250 rounds) is one method of calculating the minimum sample size required to approximate the slot's statistics.

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Jufo,

I was trying to avoid referring to a fractional person. That is "Approximately 0.3 out of 100 Players ...". I agree, the way that it is currently written may not be statistically accurate, but at least it does not involve any cutting implements.

Why not then use the RTP that correndspons to 1 in 100 event, which is the point 2.32635 in standard deviation curve. To me it sounds misleading to say that unluckiest 1% ends up with RTP lower than 83.1% when such low RTP is almost ten times rarer than that. I thought you wanted the page to be accurate and perfect?

Enzo, in his "rule of thumb" suggestion, indicated that 30 times the feature frequency (which for Hot Peppers is the Free Spin frequency at 1 in every 75 spins = 2,250 rounds) is one method of calculating the minimum sample size required to approximate the slot's statistics.
Chris

I see. I doubt 2,250 rounds is going to be nearly enough like I will show you in a short while. In Video Poker the results will be normally distributed once you have hit the top payout (Royal Flush) enough times, so with a 1:40 000 frequency for Royal Flush, it takes ~200 000 rounds until the distribution of results is normally distributed. It might be that you need similar very large number of spins in slots too, to expect to hit the largest payouts at least a few times.
 

GrandMaster

Dormant account
Joined
Jan 21, 2004
Location
UK
Note that although the variance tells you much more about a slot than the RTP does, it is just a single number with
not much more meaning than the average a win will be away from the RTP. It doesn't tell you whether or not this
is with one huge win, or a couple big ones or .. etc ..

in fact, using the normal distribution to approximate the actual distribution of your typical slotmachine is a practice
that requires a lot of insight. For example, on the left side of the curve, this is a fairly valid approximation,
however on the right side (i.e. the winners) .. you will see a lot of exceptions since most bigger wins will be many,
many SD's away from average ..

Determining how many spins is needed for a 'fair' representation is even more difficult and the answer will
be different for each machine. A machine with 99.999 losing spins and one big winner would need a very large
amount of spins .. In general you could use a number of 'rules of thumb', one I use would be 30 times the
feature frequency .. since the feature often includes a very large amount of the payout, any representative
sample would need to include enough features. So if the feature gets hit once every 100, then 3000 spins
would be my rule-of-thumb answer.
You may need the exact probabilities of the various outcomes to get a decent estimate, but I will ask one of the statisticians at work if I remember. They may have some kind of approximation for this.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
Why not then use the RTP that correndspons to 1 in 100 event, which is the point 2.32635 in standard deviation curve. To me it sounds misleading to say that unluckiest 1% ends up with RTP lower than 83.1% when such low RTP is almost ten times rarer than that. I thought you wanted the page to be accurate and perfect?

LOL. Aw c'mon Jufo, give me a break, huh? I'm still actually trying (although at this point I concede that it is probably a futile effort) to write a file that can actually be understood by someone whose knowledge of all of this at least approaches that of the average person. :)

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
I will now compare how well the normal distribution interpretation mentioned in the help file and referenced here matches with the actual distribution of results. I'll choose the Hot Pepper slot (as there is a simulator for it) with max. lines and 2000 spins for each result. I use 2000 here because this was the value originally mentioned in the help file.

Hot Pepper slot has SD of 4.7 and T-RTP of 97.48% so below is what the normal distribution would predict the results to be.

Code:
1 SD range (2000 rounds): 10.51% 

+/- 1 SD RTP Range: 86.97% to 107.99%
+/- 2 SD RTP Range: 76.46% to 118.50%
+/- 3 SD RTP Range: 65.95% to 129.01%

RTP of Median player: 97.48%

Probability of being ahead (RTP >100%) after 2000 spins: 40.52%

RTP of the worst 1%: 73.03% or lower
RTP of the highest 1%: 121.93% or higher

Next I will run a simulation of 2000 spins with the same settings, which is repeated 10 000 times. The RTPs from those 10 000 results can be then compared with the above values.

The stats from the 10 000 simulated runs of 2000 spins are:

Code:
+/- 1 SD RTP Range: 87.92% to 106.68%
+/- 2 SD RTP Range: 80.18% to 120.33%
+/- 3 SD RTP Range: 73.50% to 174.42%

RTP of Median player: 96.69%

Probability of being ahead (RTP >100%) after 2000 spins: 36.51%

RTP of the worst 1%: 77.70% or lower
RTP of the highest 1%: 127.28% or higher

Compare these numbers with the ones above.

Below is the plot of the results. The x-axis is RTP (1% increments) and y-axis is the frequency count for that RTP. In red are the frequencies expected by the normal distribution assumption and in blue are the actual frequencies obtained by simulation.

normal_compare.jpg

A few notable differences from normal distribution:

The median and mode of actual results are less than what is predicted by normal distribution.

Normal distribution expects higher frequencies for low RTPs than what actually occurs. The lowest RTP in the sample data was 69.91%. Normal distribution approximation expects there to be 44 samples with lower RTP than that (out of 10,000).

In contrast at the high RTP end there were many more results than what normal distribution expects. Note that the X-axis changes to 10% increments at 140% RTP to save space (the right side of vertical line). According to normal distribution, the highest possible RTP in a sample of 10,000 is 136.57%. In the actual data there were 52 samples with total RTP higher than that. In addition the standard deviations of the top 10 results were (the top 1/1000th):

7,62
7,68
7,72
7,79
7,79
8,31
8,45
8,63
8,93
9,05

Obviously it is not possible have results with standard deviations more than Seven in a sample of 10,000, let alone 10 such results. This means that normal distribution doesn't handle rare big wins well, even within 2000 spins.

Let's check the statistical tests for normality. I chose the
You do not have permission to view link Log in or register now.
normality test as it should be quite lenient in accepting the null hypothesis for normality. The result was:

QQplot.JPG

Normality test very clearly rejects the null hypothesis of normal distribution. The plot above (Q-Q plot) shows the deviation from normality very clearly. If the results were normally distributed they would stay close to the red straight line. Now there is considerable deviation at both the negative SD (low RTP) and the high SD (high RTP) ends.

Based on the above study, I'd conclude that: 2000 spins isn't nearly enough to estimate the range of results by normal distribution. Given that this was a lower variance slot, this applies even more more to higher-variance slots. The differences between actual probabilities and estimated probabilities are several percent even in the parts where the data matches normal distribution well. Normal distribution vastly underestimates the frequency of big wins and high RTPs, such as the several +7 SD ... +9 SD outcomes in the data. It also overestimates the frequency of low RTPs.
 
Last edited:

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
LOL. Aw c'mon Jufo, give me a break, huh? I'm still actually trying (although at this point I concede that it is probably a futile effort) to write a file that can actually be understood by someone whose knowledge of all of this at least approaches that of the average person. :)
Chris

My beef was that I hate it when the casino underestimates the odds of a rare loss. So I hope at least you put it right.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
Jufo,

First of all, your post #92 in this thread is the most impressive post that I have ever seen at Casinomeister, both in content and in presentation.

The 2 charts/graphs at the bottom present a much more grim picture of the data than the 2 "blocks" of text data nearer the top. Just going by the "text blocks", the less knowledgeable reader (that is, me) might say, "Well, they're certainly off, but the differences are not overwhelmingly different". But the charts present a more grim picture.

It's interesting that your data supports something mentioned by Enzo - significant deviations from a normal distribution will occur at the high end, the winning end, of the curve because of the rare but significant "big winners." However, it also shows a significant deviation, in the other direction, at the lower end of the curve.

So, 2,000 as a sample size for a slot with an SD of 4.7 is clearly a problem. I'm left to wonder whether any further conclusions can be drawn from the data because of this.

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Jufo,

First of all, your post #92 in this thread is the most impressive post that I have ever seen at Casinomeister, both in content and in presentation.

Lol, thanks. Given the amount of work and research I put in to be able to produce that post, I'd respond: "Yeah, it better be" ;)

The 2 charts/graphs at the bottom present a much more grim picture of the data than the 2 "blocks" of text data nearer the top. Just going by the "text blocks", the less knowledgeable reader (that is, me) might say, "Well, they're certainly off, but the differences are not overwhelmingly different". But the charts present a more grim picture.

It's interesting that your data supports something mentioned by Enzo - significant deviations from a normal distribution will occur at the high end, the winning end, of the curve because of the rare but significant "big winners." However, it also shows a significant deviation, in the other direction, at the lower end of the curve.

Yes, I recognized from Enzo's post that he clearly knows a lot about the subject. I'd guess that he has studied the same things as I did in the above post. Yes, the tails of the distribution curve are the problematic part. It's actually very common in science that the middle part of the data follows normal distribution well but deviates from it at the tails. The problem is that the tails are usually the most interesting part: in this case large losses and large wins. It's important to know that normal distribution cannot handle these well and considers the largest wins to be nearly +10 SD outcomes.

A paranoid casino manager (who is paranoid of winners) might incorrectly apply normal distribution to slot results and think there is something wrong their games because of those frequent +10 SD outcomes :D

So, 2,000 as a sample size for a slot with an SD of 4.7 is clearly a problem. I'm left to wonder whether any further conclusions can be drawn from the data because of this.

I'll redo the previous analysis with 10,000 spins to see if there if there is improvement and report here shortly.
 

binary128

Dormant account
Joined
May 18, 2009
Location
Vancouver, BC
Jufo,

Understood and agreed about the tails, both tails.

However, if I understand Enzo correctly, he indicated that it was the high-end tail that would experience the greater number of "outliers" with a slot RTP distribution, because of the "big wins".

Losses, the low-end tail, are more controlled, in a sense, because of the relative limitation in the range of the available bet amounts. Wins, however, especially those rare and really big wins, are going to screw with the high end.

HOWEVER, all that said, the gross impact might be small in the grand scheme of things. That is, you plotted 10,000 samples, each of 2,000 games. You wound up with 52 samples whose RTP exceeds expectations, or 0.5% of the samples, which in the grand scheme of things is not that large a number.

I'll be interested in seeing your 10,000 sample size runs. The value of your sim is obvious. It would take me 4 or 5 days, maybe more, to do 10,000 sample sets. I don't have that time.

BTW, I modified the sample size referenced in the Help file from 100 to 1,000. The final line now reads:

"Only 3 Players in a thousand ..."

Chris
 

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Ok, I now re-did the sim with 10,000 runs of 10,000 spins (this equals 100 million spins in total so the simulator took about half an hour to play all those spins...)

The most visible improvement with increase to 10,000 spins is cutting off the most exreme high RTPs. In the previous simulation of 2000 spins the highest RTP in the data (out of 10,000 samples) was ~194%. Since the range for 1 SD was 10.51% (=4.7*SQRT(2000)/2000), this RTP equals (194% - 97.48%)/10.51% = 9.2 SDs. Nine standard deviations is of course impossible.

With 10,000 spins the highest RTP (out of 10,000 samples) was 125.43%. Since the range for 1 SD is now 4.7% (=4.7*SQRT(10000)/10000), this RTP equals (125.43% - 97.48%)/4.7% = 5.9 SDs.

5.9 SD equals odds ~1 in 750 million, so the normal distribution still fails to set correct odds for the biggest wins but at least this is an improvement from 9 SD.

Below are distribution plot and QQ-plot for 10,000 spins:

hist.jpg
qqplot2.JPG

If you compare these plots with the previous 2000 spins result, you will notice that the distribution is getting closer and closer to normal distribution. But the normality test (Kolmogorov-Smirnov) still very clearly rejects normality hypothesis. It's not even close.

So, my opinion of the text below in the help file:

For example - using the Hot Peppers slot, with a Theoretical RTP of 97.48%, and a Standard Deviation (SD) of 4.7:
Let's say that we have 1000 Players, each of whom has played 10,000 rounds of Hot Peppers. The Player RTP results that we might expect to see are:

About 682 Players will have an RTP in the range 97.47 +/- 1 SD = 92.7% to 102.3%.
About 954 Players will have an RTP in the range 97.47 +/- 2 SD = 87.9% to 107.1%.
About 997 Players will have an RTP in the range 97.47 +/- 3 SD = 83.1% to 111.9%.
Only 3 Players in a thousand will have an RTP below 83.1% or above 111.9%.


is that normal distribution approximation works very badly for slots. If the above snippet of text was related to Blackjack or even money bets on roulette (each of which has SD close to 1) then even a sample as small as 20 spins/rounds would be perfectly normally distributed. But it doesn't work when there are high payouts involved. Therefore I would either remove the above part of text completely or add a more informative disclaimer, which says something like:

"The number of spins required for the results to be even moderately close to normal distribution is very high and even then it will not be accurate for low or high return percentages. The result will also be more unreliable for higher variance slots."
 
Last edited:

Jufo

Dormant Account
Joined
Sep 21, 2007
Location
Finland
Understood and agreed about the tails, both tails.

However, if I understand Enzo correctly, he indicated that it was the high-end tail that would experience the greater number of "outliers" with a slot RTP distribution, because of the "big wins".

Losses, the low-end tail, are more controlled, in a sense, because of the relative limitation in the range of the available bet amounts. Wins, however, especially those rare and really big wins, are going to screw with the high end.

Yes, it's pretty much like you said above. Both tails deviate from normal distribution but the right tail deviates from it more because of those rare big hits. The left tail deviates from normal distribution because the maximum you can lose in every round is your bet. However normal distribution doesn't know this: the SD of 4.7 per spin indicates that a +/- 1 SD range is either a win or loss of 4.7 units, but you can never lose 4.7 units by betting 1 unit, can you? So like you said this means that the losses are "more controlled" than what the normal distribution estimates them to be.

In other words, normal distribution assumes wins and losses to be symmetrical where they are inherently not (it's possible to lose only 1 unit but to win XX units in every spin). The result will be normally distributed only after you have played so many spins that this asymmetry has "dissipated" under the large number of spins.

You could describe the right tail like this: Normal distribution becomes valid only when there are enough spins such that a player who has had a single big hit is indistinguishable from all other players in terms of his overall RTP. In other words the peak caused by the big win gets "dissipated" in the large number of spins.

HOWEVER, all that said, the gross impact might be small in the grand scheme of things. That is, you plotted 10,000 samples, each of 2,000 games. You wound up with 52 samples whose RTP exceeds expectations, or 0.5% of the samples, which in the grand scheme of things is not that large a number.

52 samples out of 10,000 exceeded the maximum possible payout estimated by normal distribution. But even below that point there were discrepancies with actual and expected frequencies. I checked that they started at around 130% RTP mark. This equals that around ~1% of top payouts are problematic.

I'll be interested in seeing your 10,000 sample size runs. The value of your sim is obvious. It would take me 4 or 5 days, maybe more, to do 10,000 sample sets. I don't have that time.

Yeah, when in post #81 you started to calculate RTPs of samples of 5,000 one at a time, I was like: don't do that, my sim can do it 10 000 times in one click. Also, doing only 50 sets of 5,000 and seeing that they happen to fall within +/- 3 SD range is not very convicing, ie. you can't draw the conclusion that the data matches the hyporhesis. That's why I made the "heads up" post #87.

BTW, I modified the sample size referenced in the Help file from 100 to 1,000. The final line now reads:

"Only 3 Players in a thousand ..."

Ok, that sounds fine.

However, I checked that the true odds to end up with RTP lower than 83.1% in 10k spins is of the order ~1/10000 rather than ~1/667 implied by statement "3 in a thousand (divided by two)". Normal distribution is again very inaccurate here as this is a probability related to a tail.
 
Last edited:

Users who are viewing this thread

Top