chapter-04/4.4.3-quadratic-approximations-and-inference.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4.4.3 Quadratic Approximations and Inference\n",
    "\n",
    "The maximum-likelihood parameter estimates $\\hat{\\beta}$ satisfy a self-consistency relationship: they are the coefficients of a weighted least squares fit, where responses are (4.29):\n",
    "$$\n",
    "z_i = x_i^T\\hat{\\beta} + \\cfrac{(y_i - \\hat{p}_i)}{\\hat{p}_i(1-\\hat{p}_i)}\n",
    "$$\n",
    "and the weights are $w_i=\\hat{p}_i(1-\\hat{p}_i)$. This connection has more to offer:\n",
    "\n",
    "- The weight RSS is the familiar Pearson chi-square statistic (4.30)\n",
    "$$\n",
    "\\sum_{i=1}^N\\cfrac{(y_i-\\hat{p}_i)^2}{\\hat{p}_i(1-\\hat{p}_i)}\n",
    "$$\n",
    "\n",
    "  a quadratic approximation to the deviance.\n",
    "\n",
    "- Asymptotic likelihood theory says that if the model is correct, then $\\hat{\\beta}$ converges to the true $\\beta$.\n",
    "\n",
    "- A central limit theorem then shows that the distribution of $\\hat{\\beta}$ converges to $N(\\beta, (\\mathbf{X}^T\\mathbf{XW})^{-1})$. (Can be derived from weighted least squares fit)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}