Markov Decision Process (MDP) is a decision-making framework solved by dynamic programming. This powerful mathematical tool optimizes decisions in situations where the state of the system dynamically evolves and the decision maker is not in full control of the outcome of her actions. This course is divided in three parts. The first part will focus on modelling business and engineering situaitons via MDPs. Problems such as inventory managemen, healthcare and medical decision-making, revenue management and production planning and control will be discussed and modelled as MDP. The second part discusses popular and effective solution algorithms such as linear programming, value iteration and policy iteration. Finally, in the third part scientific literature on various application of MDPs is reviewed and open problems are discussed.