Sequential decision problems and Stochastic discrete-time control problem

There are three Questions. For all problems, you can use any software to code the dynamic programming equations and solve. However, you cannot use an optimization solver to solve the problems. Please, provide printouts for your solution.