Markov Decision Processes (MDP) is a decision-making framework solved by dynamic programming. This powerful mathematical tool optimizes decisions in situation where the state of the system dynamically evolves and the decision maker is not in full control of the outcome of her actions. This course is divided in three parts. The first part will focus on modelling business and engineering situaitions via MDPs. Problems such as inventory management, healthcare and medical decision-making, revenue management and production planning and control will be discussed and modelled as MDP. The second part discusses popular and effective solution algorithms such as linear programming, value iteration and policy iteration. Finally, in the third part scientific literature on various applications of MDPs is reviewed and open problems are discussed.