spiderlegz - I don't know what "facepalm" means.
Also, see below.
Nifty29 - there are 2 different types of audits that are included in the previous discussion.
The first is the typical Monthly Audit, which usually results in a report certifying RTP values for various game groups.
The second is a System Audit. This is more in line with what eCOGRA does, and what agencies like the Isle of Man Gambling Supervision Commission, the Alderney Gambling Control Commission, and the UK Gambling Commission are putting in place.
Both your question and spiderlegz question deal with this first type of audit, the Monthly RTP Report.
So, I will give you what I've got there. Please Note: This is what I've got, what Galewind does. I don't have a clue what other people/companies do (and it is none of my business either).
First of all, let me preface all of what follows with this: I am absolutely, unshakably committed to ensuring that the Player, and the Casino, receive not one penny more, and not one penny less, than what the game's Theoretical (Maximum) RTP provides. Not one penny more, not one penny less.
On the 3rd of each month, at about 6AM, an application "wakes up" on the Casino's database server. It starts executing what are called "queries" into the Casino database to get game data.
For example, the query for Keno for the month of August 2011 looks like this:
SELECT createdate, playeridno, betamount, gameresult, pickcount, catchcount, random1, random2, random3, random4, random5, random6, random7, random8, random9, random10, random11, random12, random13, random14, random15, random16, random17, random18, random19, random20, pick1, pick2, pick3, pick4, pick5, pick6, pick7, pick8, pick9, pick10 FROM kenogames WHERE createdate BETWEEN 40756 AND 40786.9999999999 ORDER BY createdate ASC
This query pulls all of the Keno game for August out of the database. The application then writes this data to a text file using commas to separate all of the game fields, and line breaks to separate all of the game rows.
The application creates one of these text files for each of the games in the Casino, and then goes back to sleep for another month.
Galewind then ZIPs up all of the files into a big pile, and FTPs the pile over to Certified Fair Gambling (CFG).
CFG unZIPs the pile, and starts running pre-defined "analysis packages" against each of the game files.
I can't speak to the details of exactly what these analysis packages do. To ensure integrity of the analysis, CFG doesn't want me to know, so that I can't manipulate the data to pass their tests. However, I do know that, for Keno as an example, the analysis include at a minimum a statistical analysis of the ball distribution across all of the 20 random fields, and across all 10 of the pick fields.
I can also speak to the professional working relationship that I have developed with CFG. When CFG and Galewind first started working together back in early 2009, we spent a lot of time running tests, dumping data into Excel worksheets, analyzing the results, talking about other tests, different tests we might run to check for various aspects of bias and correlation: step tests, group tests, run tests.
Suffice it to say, we just did a shit load of tests, and generated a shit load of Excel worksheets, to set everything up at the beginning.
Certified Fair Gambling Details
(In this linked post you will note the statement "CFG ran its standard suite of tests, as well as some additional tests by specific request of Galewind Software.")
Also as part of this analysis, CFG sums the bets for a game group, sums the win/loss, and calculates the game group's Reported RTP.
Finally, they issue a PDF report which Galewind publishes.
CFG brings to my attention anything which requires further analysis. Technically, this is usually a result with what is known as a "p-value" that is less than 0.05, or a result which falls outside of a 3-sigma (3 Standard Deviations) expectation.
Every time this has happened, the specific problem "factors out" when a larger sample size is analyzed.
In addition, CFG has combined these monthly files from Galewind into:
- a 6 month analysis.
- a 12 month analysis.
- an 18 month analysis.
- a 24 month analysis.
So, for Galewind and Certified Fair Gambling, this has been a LOT of work, and continues to be a lot of work. There is no rubber stamping going on here.