Let's compute the expected value of a single roll first.

A fair six-sided die has outcomes 1 through 6 each with probability $1/6$. So the expected value of a single roll is:

$\dfrac{1}{6} \cdot 1 + \dfrac{1}{6} \cdot 2 + \dfrac{1}{6} \cdot 3 + \dfrac{1}{6} \cdot 4 + \dfrac{1}{6} \cdot 5 + \dfrac{1}{6} \cdot 6 = 3.5$

Now, for the strategy where we can roll a second time, the optimal strategy is to roll again when the first roll is below the expected value (i.e., 1, 2, or 3) because on average, the second roll should be higher.

So, if we roll a 1, 2, or 3, we should roll again, and the expected value of that roll is again 3.5. If we roll a 4, 5, or 6, we should keep it.

Therefore, the expected value of the game under this optimal strategy is:

$\dfrac{1}{6} \cdot 3.5 + \dfrac{1}{6} \cdot 3.5 + \dfrac{1}{6} \cdot 3.5 + \dfrac{1}{6} \cdot 4 + \dfrac{1}{6} \cdot 5 + \dfrac{1}{6} \cdot 6$

$=\dfrac{25.5}{6} = 4.25$

So, the expected payout under the optimal strategy of re-rolling when the first roll is 1, 2, or 3 is $4.25.

---

Follow up question: What if you are allowed to roll the dice as many times as you want? What is the expected value of the game?

Answer: 6

Solution: If the number is not 6, you can request a retry. Stop only when the value is equal to 6.