I programmed two mathematical distributions—Binomial and Normal—in Go with the intent of using programming to reinforce my understanding of mathematics.

Far from being a text about data science, this is about some time I took for myself, dedicating it to revisiting topics I enjoy.

Let’s talk about mathematics and programming.

The Beginning of the Mathematical Journey

It all starts with NNT—Nassim Nicholas Taleb—of whom I have been a fan since reading The Black Swan.

It was while reading Fooled by Randomness that I found myself captivated by the way Taleb makes use of Monte Carlo simulations.

The Talebian way of writing makes the subject accessible:

“The name Monte Carlo brings to mind the image of a tanned, urban europlayboy type entering a casino amidst a Mediterranean breeze. He knows how to ski and play tennis, but he can also hold his own in chess and bridge. He drives a gray sports car, wears well-pressed, made-to-measure Italian suits, and speaks carefully and smoothly about mundane and real subjects that a journalist could easily describe to the public in short sentences. Inside the casino, he counts cards with astuteness, calculates probabilities, and bets in a studied manner, his mind producing precise calculations of the perfect bet. He could be the lost, smarter brother of James Bond.

When I think of the Monte Carlo method, I happily think of a combination of both things: the realism of the Monte Carlo man, without the superficiality, mixed with the mathematician’s intuition, without the excessive abstraction.”

At this point, I admit: I’m hooked! Days of subway and train commutes to work went by following this stream of consciousness from Taleb.

And that led me to the probable exercise of probability in programming.

The Hook

NNT has a blast with his Monte Carlo simulations in the book, and I want to have some fun with them too 🙂

It is not uncommon to hear from a programmer that they are going to use Monte Carlo when what they really mean is that they will use randomness to test software in an uncertain scenario—or perhaps one not so uncertain after all. (Sometimes, the difficulty of collecting a significant sample from the entire population of logs leads to the use of Monte Carlo without taking into account that the result is different).

Using log samples is about looking at what has already happened, while Monte Carlo helps simulate what could happen.

Monte Carlo is often used to solve deterministic problems through statistics. It uses randomness to obtain numerical answers for problems that may be difficult to solve analytically. To simplify, you could say it involves using randomness to generate inputs (samples) for a system to understand its behavior from a probabilistic standpoint.

Heads or Tails

It was this game, heads or tails, that led me to mathematical distributions.

The coin toss game is common when talking about Monte Carlo because it has a random input and allows us to infer the probability of events, which is a combination of results. It is a classic start point.

There is a 50% chance of getting heads or tails when flipping the coin once, but what is the chance of getting 3 heads if I flip it 5 times? This is the interesting question that Monte Carlo helps answer:

What is the chance of getting 3 heads if I flip 5 times?

We can obtain the result through the analytical method (mathematics).

Formula: P(X = k) = ( n ) . p^k . q^(n-k)
( k )
where:
n = 5 (total) p = 0,5 (success)
k = 3 (desired) q = 0,5 (fail)

1) Binomial Coefficient (5 / 3):
5! / 3!(5-3)! = (5.4.3!) / (3!.2.1) = 20/2 = 10

2) Potências:
(0,5)³ = 0,125
(0,5)² = 0,25

3) Resultado:
P(X=3) = 10 . 0,125 . 0,25
P(X=3) = 0,3125

Or we can estimate an answer using the computational Monte Carlo method, as in the Go code at the link flipCoin3on5.go (see the execution examples below).

^{Note: The code below has not been optimized. Another version using atomics and goroutines can be found in flipCoin3on5_bignumbers.go, and even that version doesn’t exhaust all potential performance improvements.}

#A few rounds don't provide a good estimate. For example, with only 1 round, the probability will be either 0% or 100%.
# Results with with 4-decimal precision only.

./flipCoin3on5 1
Simulated 1 rounds of 5 flips...
Cases with exactly 3 heads: 0
Estimated Probability: 0.0000 (0.00%)

./flipCoin3on5 1
Simulated 1 rounds of 5 flips...
Cases with exactly 3 heads: 1
Estimated Probability: 1.0000 (100.00%)

# With an insufficient amount of Rounds, the data is not reliable.
./flipCoin3on5 50
Cases with exactly 3 heads: 17
Estimated Probability: 0.3400 (34.00%)

./flipCoin3on5 50
Cases with exactly 3 heads: 17
Estimated Probability: 0.3400 (34.00%)

# With too many rounds we have a great estimate but waste time.
# 1 Billion rounds => 1 minute
time ./flipCoin3on5 1000000000   
Simulated 1000000000 rounds of 5 flips...
Cases with exactly 3 heads: 312500797
Estimated Probability: 0.3125 (31.25%)
real 1m8.066s
user 1m8.147s
sys 0m0.080s

# With enough rounds we have a great estimate
# with 8/10 executions having an Estimated Probability identical to the Theoretical Probability.
# Two Hundred Million rounds => 13 sec
time ./flipCoin3on5 200000000
Simulated 200000000 rounds of 5 flips...
Cases with exactly 3 heads: 62506172
Estimated Probability: 0.3125 (31.25%)
real 0m13.721s
user 0m13.732s
sys 0m0.012s

time ./flipCoin3on5 200000000
Simulated 200000000 rounds of 5 flips...
Cases with exactly 3 heads: 62489409
Estimated Probability: 0.3124 (31.24%)
real 0m13.547s
user 0m13.557s
sys 0m0.024s

Some interesting points so far:

• An event is a combination of results (e.g., 3 heads occurring in 5 tosses).
• In the simulation, we saw evidence of a fundamental theorem of probability: the Law of Large Numbers.
• The simulation proved a calculated theoretical probability, and the question remains: How would we be sure of the accuracy of the answer if we didn’t have the theoretical probability calculated?
• The Heads or Tails game uses the Binomial Distribution in the analytical solution, and I will talk about it in the next article.
• The coin toss game brings up another interesting theme: The Gambler’s Fallacy.
• The Monte Carlo example used is simple. Monte Carlo is a method for obtaining numerical approximations of complex functions where it is sometimes impossible to obtain results in a deterministic way.I will return to Monte Carlo with more interesting examples.

Beyond Averages

Applying probability to technology helps bring more security to decision-making, ranging from the approval of a new checkout screen (an A/B test) or a new API endpoint (load testing) to noise treatment in data such as GPS data (signal reflections). I intend to visit these cases in future texts.

What Comes Next?

Here I shared that, while performing these exercises inspired by NNT, I ended up becoming even more interested in mathematical distributions. I decided to implement the Binomial first, because it appears in the coin tossing exercise, and then the Normal distribution, which has a direct relationship with what we have seen so far.

The repository for these implementations is godist, and I will talk about it in the article on binomial distribution.

As a next article, I think it is interesting to address the following question: How to be sure of the accuracy of the simulation response without comparing it with theoretical probability?

Godist on GitHub: https://github.com/iannsp/godist