Fundamental Probability A Computational Approach

Probability is a vital measure in numerous disciplines, from bioinformatics and econometrics to finance/insurance and computer science. Developed from a successful course, Fundamental Probability provides an engaging and hands-on introduction to this important topic. Whilst the theory is explored in detail, this book also emphasises practical applications, with the presentation of a large variety of examples and exercises, along with generous use of computational tools.

Based on international teaching experience with students of statistics, mathematics, finance and econometrics, the book:

• Presents new, innovative material alongside the classic theory.
• Goes beyond standard presentations by carefully introducing and discussing more complex subject matter, including a richer use of combinatorics, runs and occupancy distributions, various multivariate sampling schemes, fat-tailed distributions, and several basic concepts used in finance.
• Emphasises computational matters and programming methods via generous use of examples in MATLAB.
• Includes a large, self-contained Calculus/Analysis appendix with derivations of all required tools, such as Leibniz' rule, exchange of derivative and integral, Fubini's theorem, and univariate and multivariate Taylor series.
• Presents over 150 end-of-chapter exercises, graded in terms of their difficulty, and accompanied by a full set of solutions online.

This book is intended as an introduction to the theory of probability for students in biology, mathematics, statistics, economics, engineering, finance, and computer science who possess the prerequisite knowledge of basic calculus and linear algebra.

A note to the student (and instructor).

A note to the instructor (and student).

Acknowledgements.

Introduction.

PART I: BASIC PROBABILITY.

1. Combinatorics.

1.1 Basic counting.

1.2 Generalized binomial coefficients.

1.3 Combinatoric identities and the use of induction.

1.4 The binomial and multinomial theorems.

1.4.1 The binomial theorem.

1.4.2 An extension of the binomial theorem.

1.4.3 The multinomial theorem.

1.5 The gamma and beta functions.

1.5.1 The gamma function.

1.5.2 The beta function.

1.6 Problems.

2. Probability spaces and counting.

2.1 Introducing counting and occupancy problems.

2.2 Probability spaces.

2.2.1 Introduction.

2.2.2 Definitions.

2.3 Properties.

2.3.1 Basic properties.

2.3.2 Advanced properties.

2.3.3 A theoretical property.

2.4 Problems.

3. Symmetric spaces and conditioning.

3.1 Applications with symmetric probability spaces.

3.2 Conditional probability and independence.

3.2.1 Total probability and Bayes’ rule.

3.2.2 Extending the law of total probability.

3.2.3 Statistical paradoxes and fallacies.

3.3 The problem of the points.

3.3.1 Three solutions.

3.3.2 Further gambling problems.

3.3.3 Some historical references.

3.4 Problems.

PART II: DISCRETE RANDOM VARIABLES.

4. Univariate random variables.

4.1 Definitions and properties.

4.1.1 Basic definitions and properties.

4.1.2 Further definitions and properties.

4.2 Discrete sampling schemes.

4.2.1 Bernoulli and binomial.

4.2.2 Hypergeometric.

4.2.3 Geometric and negative binomial.

4.2.4 Inverse hypergeometric.

4.2.5 Poisson approximations.

4.2.6 Occupancy distributions.

4.3 Transformations.

4.4 Moments.

4.4.1 Expected value of X.

4.4.2 Higher-order moments.

4.4.3 Jensen?s inequality.

4.5 Poisson processes.

4.6 Problems.

5. Multivariate random variables.

5.1 Multivariate density and distribution.

5.1.1 Joint cumulative distribution functions.

5.1.2 Joint probability mass and density functions.

5.2 Fundamental properties of multivariate random variables.

5.2.1 Marginal distributions.

5.2.2 Independence.

5.2.3 Exchangeability.

5.2.4 Transformations.

5.2.5 Moments.

5.3 Discrete sampling schemes.

5.3.1 Multinomial.

5.3.2 Multivariate hypergeometric.

5.3.3 Multivariate negative binomial.

5.3.4 Multivariate inverse hypergeometric.

5.4 Problems.

6. Sums of random variables.

6.1 Mean and variance.

6.2 Use of exchangeable Bernoulli random variables.

6.2.1 Examples with birthdays.

6.3 Runs distributions.

6.4 Random variable decomposition.

6.4.1 Binomial, negative binomial and Poisson.

6.4.2 Hypergeometric.

6.4.3 Inverse hypergeometric.

6.5 General linear combination of two random variables.

6.6 Problems.

PART III: CONTINUOUS RANDOM VARIABLES.

7. Continuous univariate random variables.

7.1 Most prominent distributions.

7.2 Other popular distributions.

7.3 Univariate transformations.

7.3.1 Examples of one-to-one transformations.

7.3.2 Many-to-one transformations.

7.4 The probability integral transform.

7.4.1 Simulation.

7.4.2 Kernel density estimation.

7.5 Problems.

8. Joint and conditional random variables.

8.1 Review of basic concepts.

8.2 Conditional distributions.

8.2.1 Discrete case.

8.2.2 Continuous case.

8.2.3 Conditional moments.

8.2.4 Expected shortfall.

8.2.5 Independence.

8.2.6 Computing probabilities via conditioning.

8.3 Problems.

9. Multivariate transformations.

9.1 Basic transformation.

9.2 The t and F distributions.

9.3 Further aspects and important transformations.

9.4 Problems.

Appendix A. Calculus review.

Appendix B. Notation tables.

Appendix C. Distribution tables.

References.

Index.