Generating list of probabilites

Question

I am trying to create a loop that will generate a series of probabilities against which a list of values can be compared. I need the probabilities to be in list form and the length of the list can vary. For example, if the list is lst = [1, 2, 3] and I want to set the probability of calling lst[2] at 80% with equal likelihood of the other two values, I would use p = [0.1, 0.1, 0.8].

I want to be able to cycle through all of the possible probabilities (with a definable step, in this case step = 0.1). My logic has got stuck at this point:

lst = [1, 2, 3]
step = 0.1

for p in probability_gen_function(lst, step):
    print(p)

loop_1 -> p = [0.1, 0.1, 0.8]
loop_2 -> p = [0.1, 0.2, 0.7]
---
loop_n -> p = [0.8, 0.1, 0.1]

I can see ways for my probability_gen_function to generate random lists of probabilities, but not how to systematically cycle through them. Any suggestions would be appreciated.

Answer 1

You can use a recursive generator function:

lst = [1, 2, 3]
step = 0.1
def prob_gen(d, s, c = []):
   if len(c) == len(d):
      yield [round(i, 2) for i in c]
   else:
      r = 10-sum(int(k*10) for k in c)
      for i in ([r] if len(c)+1 == len(d) else range(1, r)):
         yield from prob_gen(d, s, c+[s*i])


print(list(prob_gen(lst, step)))

Output:

[[0.1, 0.1, 0.8], [0.1, 0.2, 0.7], [0.1, 0.3, 0.6], [0.1, 0.4, 0.5], [0.1, 0.5, 0.4], [0.1, 0.6, 0.3], [0.1, 0.7, 0.2], [0.1, 0.8, 0.1], [0.2, 0.1, 0.7], [0.2, 0.2, 0.6], [0.2, 0.3, 0.5], [0.2, 0.4, 0.4], [0.2, 0.5, 0.3], [0.2, 0.6, 0.2], [0.2, 0.7, 0.1], [0.3, 0.1, 0.6], [0.3, 0.2, 0.5], [0.3, 0.3, 0.4], [0.3, 0.4, 0.3], [0.3, 0.5, 0.2], [0.3, 0.6, 0.1], [0.4, 0.1, 0.5], [0.4, 0.2, 0.4], [0.4, 0.3, 0.3], [0.4, 0.4, 0.2], [0.4, 0.5, 0.1], [0.5, 0.1, 0.4], [0.5, 0.2, 0.3], [0.5, 0.3, 0.2], [0.5, 0.4, 0.1], [0.6, 0.1, 0.3], [0.6, 0.2, 0.2], [0.6, 0.3, 0.1], [0.7, 0.1, 0.2], [0.7, 0.2, 0.1], [0.8, 0.1, 0.1]]

Answer 2

We'll use numpy and itertools.

import numpy as np
from itertools import product

def probability_gen_function(lst, step):
    # Enumerate range of probabilities. 
    # This array is like [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
    probs = np.arange(step, 1, step)
    # Run through all permutations
    for p in product(probs, repeat=len(lst)):
        # Check if sum is 1
        # We check with some small number because of floating point errors 
        if np.abs(np.sum(p) - 1) > 1e-9:
            continue
        # Yield number
        yield np.array(p)

lst = [1, 2, 3]
step = 0.1

for p in probability_gen_function(lst, step):
    print(p.round(5))
>>>
[0.1 0.1 0.8]
[0.1 0.2 0.7]
...
[0.7 0.2 0.1]
[0.8 0.1 0.1]

There're some points to notice. First of all this function does not check step size. You should provide proper values. And, because we generate them with np.arange, there may be some floating point errors, you would want to normalize or round the probabilities. But I think this is enough to start.

Answer 3

This is perfect for recursiveness. I found it easier to work with a constant step size of 1 and make the normalisation of the probability vary inside the loop, but it doesn't change the output.

I also explicitly check that the step size you give allows valid solutions.

def normalised_prob_gen(num_outcomes, step_size):
    num_steps = int(1 / step_size)
    assert num_steps*step_size == 1, "Step size does not divide into one"

    yield from probability_gen_function(num_outcomes, num_steps, num_steps)


def probability_gen_function(num_outcomes, num_steps, prob_left):
    if num_outcomes == 1:
        yield [prob_left / num_steps]
    else:
        for x in range(1, prob_left-num_outcomes+2):
            for rest in probability_gen_function(num_outcomes-1, num_steps, prob_left-x):
                yield [first / num_steps] + rest


for p in normalised_prob_gen(3, 0.1):
    print(p)

Note that the items in lst don't matter, only the number of them, so I use that as the input.