---

Introduction and Goals

For years, I have been joking with my students that I would teach probability with the same level of excitement even if I were woken up in the middle of the night and asked to teach it. Years later, as a new father, I started writing this book when it became clear to me that I would not be sleeping at night for the foreseeable future.

This book is intended for undergraduate and first-year graduate-level courses in probability, statistics, and random processes. My goal has been to provide a clear and intuitive approach to these topics while maintaining an acceptable level of mathematical accuracy.

I have been teaching two courses on this subject for several years at the University of Massachusetts Amherst. While one of these courses is an undergraduate course taken by juniors, the other is a graduate-level course taken by our first-year Masters and PhD students.

My goal throughout this process has been to write a textbook that has the flexibility to be used in both courses while sacrificing neither the quality nor the presentational needs of either course. To achieve such a goal, I have tried to minimize the dependency between different sections of the book. In particular, when a small part from a different section of the book is useful elsewhere within the text, I have repeated said part rather than simply referring to it. My reasoning for doing so is twofold. Firstly, this format should make it easier for students to read the book and, secondly, this format should allow instructors the flexibility to select individual sections from the book more easily.

Additionally, I wanted the book to be easy to read and accessible as a self-study reference. It was also imperative that the book be available to anyone in the world, and as such the book in its entirety can be found online at www.probabilitycourse.com.

The book contains a large number of solved exercises. In addition to the examples found within the text, there is a set of solved problems at the end of each section. Detailed and step-by-step solutions to these problems are provided to help students learn problem-solving techniques. The solutions to the end-of-chapter problems, however, are available only to instructors.

Lastly, throughout the book, some examples of applications$-$such as engineering, finance, everyday life, etc.$-$are provided to aid in motivating the subject. These examples have been worded to be understandable to all students. As such, some technical issues have been left out.

Coverage

After a brief review of set theory and other required mathematical concepts, the text covers topics as follows:

• Chapters 1 and 2: basic concepts such as random experiments, probability axioms, conditional probability, law of total probability, Bayes' rule, and counting methods;
• Chapters 3 through 6: single and multiple random variables (discrete, continuous, and mixed), as well as moment-generating functions, characteristics functions, random vectors, and inequalities;
• Chapter 7: limit theorems and convergence;
• Chapters 8 and 9: Bayesian and classical statistics;
• Chapters 10: Introduction to random processes, processing of random signals;
• Chapter 11: Poisson processes, discrete-time Markov chains, continuous-time Markov chains, and Brownian motion;
• Chapter 12: basic methods of generating random variables and simulating probabilistic systems (using MATLAB);
• Chapter 13: basic methods of generating random variables and simulating probabilistic systems (using R);
• Chapter 14: basic methods of generating random variables and simulating probabilistic systems (using Python);
• Chapter 15: recursive methods;

All chapters are available at www.probabilitycourse.com. Chapters 12 through 15 are available as PDFs and are downloadable from the textbook website. Chapters 12 through 14 cover the same material. The difference is that the codes in chapter 12 are provided in MATLAB, the codes in Chapter 13 are provided in R, and the codes in Chapter 14 are provided in Python. The reason for this again is to give flexibility to instructors and students to choose whichever they prefer. Nevertheless, students who are unfamiliar with MATLAB, R, and Python should still be able to understand the algorithms.

Required Background

The majority of the text does not require any previous knowledge apart from a one-semester course in calculus. The exceptions to this statement are as follows:

• Sections 5.2 (Two Continuous Random Variables) and 6.1 (Methods for More Than Two Random Variables) both require a light introduction to double integrals and partial derivatives;
• Section 6.1.5 (Random Vectors) uses a few concepts from linear algebra;
• Section 10.2 (Processing of Random Signals) requires familiarity with the Fourier transform.

Acknowledgements

This project grew out of my educational activities regarding my National Science Foundation CAREER award. I am very thankful to the people in charge of the Open Education Initiative at the University of Massachusetts Amherst. In particular, I am indebted to Charlotte Roh and Marilyn Billings at the UMass Amherst library for all of their help and support.

I am grateful to my colleagues Dennis Goeckel and Patrick Kelly, who generously provided their lecture notes to me when I first joined UMass. These notes proved to be very useful in developing my course materials and, eventually, in writing this book. I am also thankful to Mario Parente$-$who used an early version of this book in his course$-$for very useful discussions.

Many people provided comments and suggestions. I would like to especially thank Hamid Saeedi for reading the manuscript in its entirety and providing very valuable comments. I am indebted to Evan Ray and Michael Miller for their helpful comments and suggestions, as well as to Eliza Mitchell and Linnea Duley for their detailed review and comments. I am thankful to Alexandra Saracino for her help regarding the figures and illustrations in this book. I would also like to thank Ali Rakhshan, who coauthored the chapter on simulation and who, along with Ali Eslami, helped me with my LaTeX problems. I am grateful to Sofya Vorotnikova, Stephen Donahue, Andrey Smirnov, and Elnaz Jedari Fathi for their help with the website. I am thankful to Atiyeh Sakaei-Far for managing the development of the website and its maintenance. I would also like to thank Elnaz Jedari Fathi for designing the book cover.

I am indebted to all of my students in my classes, who not only encouraged me with their positive feedback to continue this project, but who also found many typographical errors in the early versions of this book. I am thankful to all of my teaching assistants who helped in various aspects of both the course and the book.

Last$-$but certainly not least$-$I would like to thank my family for their patience and support.