Bayesian Model Evaluation and Criticism
Ravin Kumar Colin Carroll
canyon289.github.io colindcarroll.com
@canyon289 @colindcarroll

Good statisticians are able to explain their choices, justify their numbers, evaluate their own models, and share their results in a reproducible fashion! This tutorial demonstrates how to do the above using ArviZ.

Contents:

  1. Optional Prereading
  2. Slides
  3. Installation
  4. Acknowledgements
  5. Feedback

Optional Prereading

No prior work is required for this tutorial but if you'd like to prepare we've provided a list of resources

Slides

The material is all covered in notebooks, but slides motivating MCMC and introducing trace plots are available here.

Installation

  1. Clone the repository locally

    In your terminal, use git to clone the repository locally:

    git clone git@github.com:arviz-devs/bayesian-model-evaluation.git
    

    Alternatively, you can download the zip file of the repository at the top of the main page of the repository. If you prefer not to use git or don't have experience with it, this a good option.

  2. Download Anaconda (if you haven't already)

    If you do not already have the Anaconda Distribution of Python 3, go get it (note: you can also set up your project environment w/out Anaconda using pip to install the required packages; however Anaconda is great for data science and we encourage you to use it).

  3. Set up your environment
    1. conda users

      If this is the first time you are setting up your compute environment, use conda to create an environment:

      conda env create -n bayes-eval

      To activate the environment, use the conda activate command:

      conda activate bayes-eval

      If you get an error activating the environment, try to use the older source activate command:

      source activate bayes-eval

      Then follow the instructions for pip users.

    2. pip users

      Install all of the packages listed in the requirements.txt file:

      pip install -r requirements.txt
    3. Docker users

      An image can be built from the root directory of the repository using the following command. This will build an image on your computer with all dependencies and environment installed:

      ./scripts/container.sh --build

      Once an image is built a container can be started with the command:

      ./scripts/container.sh --notebook

      In your terminal a URL for the notebook server will be displayed. Copy and paste that into a browser. With that you'll have Jupyter in a container! If you're using docker you can skip step 4.

    4. Do not want to mess with dev ops

      You can click to get a Binder session on which you can compute and code.

  4. Open Jupyter Lab!

    In the terminal, navigate to this directory and execute jupyter lab

    Navigate to the notebooks directory, then 1_BayesianWorkflow, and open the notebook 1_Ins_BayesRefresher.ipynb.

    1. Want to view static notebooks

      If you are only interested in viewing the static versions of the notebooks, you can view them on github.

Acknowledgements

We would like to thank the whole Bayes community for being open with learnings and material. For this tutorial in particular we'd like to thank Ari Hartikainen, Osvaldo Martin, and Eric Ma for providing feedback and content.

Feedback

We have a Google form we would love you to use here. We'll use this information to help improve the teaching and delivery of the material. Issues and pull requests are also welcomed and encouraged if you would like!