So, one of the cooler videos I did for Poker VT a while back was on variance. I don’t think most people truly appreciate how crazy variance can get. In one of my earliest videos I showed off a program written by RVG (who later went on to create HEM) that was an ROI simulator. It got the job done for my purposes for the games I was playing, but it had some limitations. The biggest one of which was that it only supported 7 payout spots. So the program was fine for looking at variance in 9 man games, but wouldn’t cut it for larger MTSNGs and MTTs.
After I had finished making that video on variance I decided to write my own Monte Carlo simulator. Since it was just a little project for my own usage I didn’t do anything fancy, make it nice for user input, or create any executables. It’s sloppy, obfuscated, and has no real documentation either. You can grab it here if you like. If you can find RVG’s old ROI simulator program you can use that for games with fewer than 7 payout spots and the results will agree with what my program produces.
For those of you who have seen my variance megapost on Poker VT, a lot of this post is straight from it.
Before giving you the output for various runs of this program I’ll explain what the output means and how to interpret it:
$6.00+0.50 9-man game
Payout distribution: 50.0% 30.0% 20.0%
Finish distribution: 12.7% 12.5% 12.6%
Theoretical ROI = 4.84%
100000 simulations of 1000 games:
90% CI for ROI: -2.88% to 12.65%
90% CI for downswing: 21.82 buy-ins to 72.35 buy-ins
90% CI for lowest drop: 0.00 buy-ins to 54.80 buy-ins
The top part of this describes the type of simulation we’re running. A standard $6.50 SNG with normal payouts and finishing in 1st place 12.7% of the time, 2nd place 12.5% of the time, and 3rd place 12.6% of the time. This corresponds to a 4.84% ROI. It also prints out the ROI and how many buy-ins it represents. It then tells the number of games in the sample size and how many simulations were run.
Below that we have confidence intervals (CI) for ROI, the biggest downswing and the lowest drop from the starting point. So, if you know you are a 4.84% ROI player, in a 1000 game sample 90% of the time your actual ROI will be between -2.88% and +12.65%. Remember that confidence intervals are centered over the median, so 5% of the time the actual ROI will be worse than -2.88% and 5% of the time it will be better than +12.65%. If someone said to me “I believe I am a 4.84% ROI player but my results over my last 1000 games are -3% ROI” then I could confidently say to them that there is less than 5% chance that they are actually as good as being a 4.84% ROI player.
Likewise, 90% of the time we can expect to have a downswing in that 1000 game sample between 21.82 and 72.35 buy-ins. 5% of the time it will be worse, 5% of the time it will be better. If you knew you were a 4.84% ROI player and you had a downswing of 80 buyins over a 1000 game sample, for example, you could say you are running exceptionally bad.
The lowest drop represents the largest drop below the starting bankroll. This is, by definition, going to be equal to or less than the downswing value. This is a good way to know a risk of ruin for a particular bankroll size. Remember that because the CI is centered on the median, that means that the risk of ruin over 1000 games with a bankroll of 54.8 buy-ins is 5% (100% – the “middle” 90% – the “top” 5%). You can look at the downswing values this way as well. You could say that 95% of the time we are going to have a downswing of more than 21.82 buyins.
Now, compare this data and interpretation for the 50% CI:
50% CI for ROI: 1.69% to 8.00%
50% CI for downswing: 29.55 buy-ins to 48.89 buy-ins
50% CI for lowest drop: 5.38 buy-ins to 26.82 buy-ins
What does this mean? It means that as our confidence goes down we are able to get closer to the median. For ROI this will start to converge on the true ROI. For downswing and lowest drop it will start to center on the most “common” downswing. From these numbers we can make interpretations such as “75% of the time our downswing will be 29.55 buy-ins or more” and “75% of the time our downswing will be 48.89 buy-ins or less”.
When we finally get down to 0% CI, we’re at the median:
0% CI for ROI: 4.84% to 4.84%
0% CI for downswing: 37.58 buy-ins to 37.58 buy-ins
0% CI for lowest drop: 13.60 buy-ins to 13.60 buy-ins
As expected the ROI converged to the theoretical ROI, and it always will for a large enough number of simulations. We also see the exact medians of downswing and lowest drop. The 13.60 buy-ins would correspond to a risk of ruin of 50% (half the time the drop will be better than the median, half the time the drop will be worse than the median).
OK, so that information gives you an idea of how to consider variance as you’re playing, but what if you’re sitting there saying “I’m a new player and I have played X games and my ROI is Y, how can I use this data?” This is a much more difficult question to answer. Per the comment about ROI claims above if you had an ROI of -10% after 1000 games at the $6.50 level, you could pretty confidently say that your actual ROI is not 4.84% or close to it. If you look at the difference between the high and low ROI, that may help though. For example consider the two outputs:
Theoretical ROI = 4.84%
99% CI for ROI: -7.29% to 16.97%
Theoretical ROI = -0.31%
99% CI for ROI: -12.02% to 11.74%
You’ll notice the difference between the high and low ROI in the first case is 24.26 and in the second case is 23.76. So, with 99% confidence we’re about +/- 12% ROI over 1000 games with these parameters from the theoretical. You could make an educated guess that over 1000 games if your ROI is -10% that your true ROI is probably in the -22% to +2% range with a high degree of accuracy.
With all this data be sure to keep in mind that you don’t play a certain way constantly. You should continuously be getting better. As your game changes, if you want to estimate your ROI try to give more weight to recent games.
Below are some more interesting runs of the program.
These first two show off the difference in variance between a 1000 game sample and a 5000 game sample with all other parameters equal:
$15.00+1.00 9-man game
Payout distribution: 50.0% 30.0% 20.0%
Finish distribution: 12.5% 12.5% 12.5%
Theoretical ROI = 5.47%
100000 simulations of 1000 games:
99% CI for ROI: -6.60% to 17.79%
99% CI for downswing: 17.06 buy-ins to 99.25 buy-ins
99% CI for lowest drop: 0.00 buy-ins to 84.88 buy-ins
95% CI for ROI: -3.81% to 14.83%
95% CI for downswing: 19.88 buy-ins to 79.19 buy-ins
95% CI for lowest drop: 0.00 buy-ins to 62.69 buy-ins
90% CI for ROI: -2.29% to 13.32%
90% CI for downswing: 21.62 buy-ins to 70.22 buy-ins
90% CI for lowest drop: 0.00 buy-ins to 52.00 buy-ins
75% CI for ROI: 0.07% to 10.95%
75% CI for downswing: 24.94 buy-ins to 57.72 buy-ins
75% CI for lowest drop: 2.00 buy-ins to 37.09 buy-ins
50% CI for ROI: 2.26% to 8.68%
50% CI for downswing: 28.97 buy-ins to 47.75 buy-ins
50% CI for lowest drop: 5.00 buy-ins to 25.16 buy-ins
25% CI for ROI: 3.95% to 6.99%
25% CI for downswing: 32.72 buy-ins to 41.44 buy-ins
25% CI for lowest drop: 8.47 buy-ins to 17.91 buy-ins
0% CI for ROI: 5.47% to 5.47%
0% CI for downswing: 36.69 buy-ins to 36.69 buy-ins
0% CI for lowest drop: 12.62 buy-ins to 12.62 buy-ins
$15.00+1.00 9-man game
Payout distribution: 50.0% 30.0% 20.0%
Finish distribution: 12.5% 12.5% 12.5%
Theoretical ROI = 5.47%
100000 simulations of 5000 games:
99% CI for ROI: 0.04% to 10.92%
99% CI for downswing: 33.12 buy-ins to 150.59 buy-ins
99% CI for lowest drop: 0.00 buy-ins to 107.97 buy-ins
95% CI for ROI: 1.32% to 9.64%
95% CI for downswing: 37.56 buy-ins to 120.75 buy-ins
95% CI for lowest drop: 0.00 buy-ins to 74.94 buy-ins
90% CI for ROI: 1.98% to 8.96%
90% CI for downswing: 40.34 buy-ins to 107.59 buy-ins
90% CI for lowest drop: 0.00 buy-ins to 60.31 buy-ins
75% CI for ROI: 3.02% to 7.92%
75% CI for downswing: 45.44 buy-ins to 90.56 buy-ins
75% CI for lowest drop: 2.19 buy-ins to 41.84 buy-ins
50% CI for ROI: 4.03% to 6.90%
50% CI for downswing: 51.44 buy-ins to 77.16 buy-ins
50% CI for lowest drop: 5.34 buy-ins to 27.50 buy-ins
25% CI for ROI: 4.78% to 6.14%
25% CI for downswing: 56.66 buy-ins to 68.72 buy-ins
25% CI for lowest drop: 9.00 buy-ins to 19.25 buy-ins
0% CI for ROI: 5.45% to 5.45%
0% CI for downswing: 62.25 buy-ins to 62.25 buy-ins
0% CI for lowest drop: 13.47 buy-ins to 13.47 buy-ins
Here are some numbers for 180 man games:
$11.00+1.00 180-man game
Payout distribution: 30.0% 20.0% 11.9% 8.0% 6.5% 5.0% 3.5% 2.6% 1.7% 1.2% 1.2% 1.2% 1.2% 1.2% 1.2% 1.2% 1.2% 1.2%
Finish distribution: 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8% 0.8%
Theoretical ROI = 32.00%
100000 simulations of 1000 games:
99% CI for ROI: -11.33% to 82.36%
99% CI for downswing: 45.65 buy-ins to 240.96 buy-ins
99% CI for lowest drop: 0.00 buy-ins to 195.42 buy-ins
95% CI for ROI: -2.04% to 69.59%
95% CI for downswing: 52.64 buy-ins to 192.69 buy-ins
95% CI for lowest drop: 0.00 buy-ins to 139.01 buy-ins
90% CI for ROI: 2.89% to 63.14%
90% CI for downswing: 57.05 buy-ins to 171.02 buy-ins
90% CI for lowest drop: 1.49 buy-ins to 113.20 buy-ins
75% CI for ROI: 11.14% to 53.35%
75% CI for downswing: 65.28 buy-ins to 141.29 buy-ins
75% CI for lowest drop: 5.00 buy-ins to 79.49 buy-ins
50% CI for ROI: 19.41% to 44.03%
50% CI for downswing: 75.06 buy-ins to 118.41 buy-ins
50% CI for lowest drop: 10.81 buy-ins to 53.05 buy-ins
25% CI for ROI: 25.73% to 37.33%
25% CI for downswing: 84.01 buy-ins to 104.10 buy-ins
25% CI for lowest drop: 17.79 buy-ins to 37.39 buy-ins
0% CI for ROI: 31.44% to 31.44%
0% CI for downswing: 93.36 buy-ins to 93.36 buy-ins
0% CI for lowest drop: 26.24 buy-ins to 26.24 buy-ins
Lastly, someone asked me about MTTs. Because field sizes change a lot it’s hard to do this sort of analysis. But if your average field size was about 1000 people, these would be the sorts of numbers you could expect if you knew you were a decent winner:
$55.00+5.00 1090-man game
Theoretical ROI = 53.29%
250000 simulations of 1000 games:
99% CI for ROI: -24.83% to 159.20%
99% CI for downswing: 66.20 buy-ins to 371.91 buy-ins
99% CI for lowest drop: 0.00 buy-ins to 328.46 buy-ins
95% CI for ROI: -10.02% to 130.44%
95% CI for downswing: 78.10 buy-ins to 302.12 buy-ins
95% CI for lowest drop: 1.00 buy-ins to 242.34 buy-ins
90% CI for ROI: -1.65% to 116.34%
90% CI for downswing: 85.17 buy-ins to 269.61 buy-ins
90% CI for lowest drop: 3.00 buy-ins to 200.65 buy-ins
75% CI for ROI: 12.63% to 95.42%
75% CI for downswing: 98.54 buy-ins to 223.18 buy-ins
75% CI for lowest drop: 8.90 buy-ins to 141.96 buy-ins
50% CI for ROI: 27.74% to 76.28%
50% CI for downswing: 114.68 buy-ins to 185.61 buy-ins
50% CI for lowest drop: 19.71 buy-ins to 95.56 buy-ins
25% CI for ROI: 39.86% to 62.73%
25% CI for downswing: 129.08 buy-ins to 162.34 buy-ins
25% CI for lowest drop: 32.33 buy-ins to 67.66 buy-ins
0% CI for ROI: 51.07% to 51.07%
0% CI for downswing: 144.54 buy-ins to 144.54 buy-ins
0% CI for lowest drop: 47.76 buy-ins to 47.76 buy-ins
). I think there’s potential for adding natural hands (7-straight, 5flush-2flush, maybe 3pair?) and the royalty system probably still needs more tweaks to be balanced, but even as is, I highly recommend it as it’s a lot of fun gambling game to add into a shorthanded home game. Try it out and let me know if anyone develops any cool tweaks/variants!
Subscribe to the RSS feed
