Logit regression

If one assumes that the probability is $P(i\to j)$ so that actor $i\hspace{0.33em}=\hspace{0.33em}1..n$ chooses alternative $j\hspace{0.33em}=\hspace{0.33em}1..k$ is proportional (within the set of alternative choices) to the exponent of a linear combination of $p\hspace{0.33em}=\hspace{0.33em}1..p$ data values $X_{ijp}$ related to $i$ and $j$, one arrives at the logit model, or more formally:

Assume $P(i\to j)\sim w_{ij}{w}_{ij}:=exp(v_{ij})v_{ij}:=\sum _p{\beta }_p{X}_{ijp}$

Thus $L(i\to j)\hspace{0.33em}:=\hspace{0.33em}log(P(i\to j))\hspace{0.33em}\sim \hspace{0.33em}{v}_{ij}$.

Consequently, $w_{ij}\hspace{0.33em}>\hspace{0.33em}0$ and $P(i\to j):=\frac{w_{ij}}{\sum _{j^{\prime }}{w}_{i{j}^{\prime }}}$, since $\sum _j{P}_{ij}$ must be $1$.

Note that:

• $v_{ij}$ is a linear combination of $X_{ijp}$ with weights ${\beta }_p$ as logit model parameters.
• the odds ratio $\frac{P(i\to j)}{P(i\to j^{\prime })}$ of choice $j$ against alternative $j\prime$ is equal to $\frac{w_{ij}}{w_{i{j}^{\prime }}}=exp(v_{ij}-v_{i{j}^{\prime }})=exp\sum _p{\beta }_p(X_{ijp}-X_{i{j}^{\prime }p})$
• this formulation does not require a separate beta index (aka parameter space dimension) per alternative choice $j$ for each exogenous variable.

observed data

Observed choices $Y_{ij}$ are assumed to be drawn from a repreated Bernoulli experiment with probabilites $P(i\to j)$.

Thus $P(Y)=\prod _{ij}\frac{N_i!\times P(i\to j{)}^{Y_{ij}}}{Y_{ij}!}$ with $N_i:=\sum _j{Y}_{ij}$.

Thus $L(Y)\hspace{0.33em}:=\hspace{0.33em}log(P(Y))$

$=log\prod _{ij}\frac{N_i!\times P(i\to j{)}^{Y_{ij}}}{Y_{ij}!}$

$=C+\sum _{ij}(Y_{ij}\times log(P_{ij}))$

$=C+\sum _i\left[\sum _j{Y}_{ij}\times L(i\to j)\right]$

$=C+\sum _i\left[\sum _j{Y}_{ij}\times (v_{ij}-log\sum _{j^{\prime }}{w}_{i{j}^{\prime }})\right]$

$=C+\sum _i\left[(\sum _j{Y}_{ij}\times v_{ij})-N_i\times log\sum _j{w}_{ij}\right]$

with $C=\sum _i{C}_i$ and $C_i:=[log(N_i!)-\sum _{j}log(Y_{ij}!)]$, which is independent of $P_{ij}$ and ${\beta }_j$. Note that: $N_i\hspace{0.33em}=\hspace{0.33em}1\hspace{0.33em}⟹\hspace{0.33em}{C}_i\hspace{0.33em}=\hspace{0.33em}0$

specification

The presented form $v_{ij}\hspace{0.33em}:=\hspace{0.33em}{\beta }_p\hspace{0.25em}\times \hspace{0.25em}{X}_{ij}^p$ (using Einstein Notation from here) is more generic than known implementations of logistic regression (such as in SPSS and R), where $X_i^q$, a set of $q\hspace{0.33em}=\hspace{0.33em}1..q$ data values given for each $i$ ($X_i^0)$ is set to $1$ to represent the incident for each $j$) and $(k-1)\hspace{0.25em}\times \hspace{0.25em}(q+1)$ parameters are to be estimated, thus $v_{ij}\hspace{0.33em}:=\hspace{0.33em}{\beta }_{jq}\hspace{0.25em}\times \hspace{0.25em}{X}_i^q$ for $j\hspace{0.33em}=\hspace{0.33em}2..k$ which requires a different beta for each alternative choice and data set, causing unnecessary large parameter space.

The latter specification can be reduced to the more generic form by:

• assigning a unique $p$ to each $jq$ combination, represented by $A_{jq}^p$.
• defining $X_{ij}^p\hspace{0.33em}:=\hspace{0.33em}{A}_{jq}^p\hspace{0.25em}\times \hspace{0.25em}{X}_i^q$ for $j\hspace{0.33em}=\hspace{0.33em}2..k$, thus creating redundant and zero data values.

However, a generical model cannot be reduced to a specification with different $\beta$'s for each alternative choice unless the latter parameter space can be restricted to contain no more dimensions than a generic form. With large $n$ and $k$, the data values $X_{ijk}$ can be huge. To mitigate the data size, the following tricks can be applied:

• limit the set of combinations of $i$ and $j$ to the most probable or near $j$'s for each $i$ and/or cluster the other $j$'s.
• use only a sample from the set of possible $i$'s.
• support specific forms of data:

#	form	reduction	description
0	${\beta }_p{X}_{ij}^p$		general form of p factors specific for each i and j
1	${\beta }_p{A}_{jq}^p{X}_i^q$	$X_{ij}^p\hspace{0.33em}:=\hspace{0.33em}{A}_{jq}^p{X}_i^q$	q factors that vary with i but not with j.
2	${\beta }_p{X}_i^p{X}_j^p$	$X_{ij}^p\hspace{0.33em}:=\hspace{0.33em}{X}_j^p{X}_i^p$	p specific factors in simple multiplicative form
3	${\beta }_{jq}{X}_i^q$		q factors that vary with j but not with i.
4	${\beta }_p{X}_j^p$	$X_{ij}^p\hspace{0.33em}:=\hspace{0.33em}{X}_j^p$	state constants Dj
5	${\beta }_j$		state dependent intercept
6	${\beta }_p(J_i^p==j)$		usage of a recorded preference

regression

The ${\beta }_p$'s are found by maximizing the likelihood $L(Y|\beta )$ which is equivalent to finding the maximum of $\sum _i\left[\sum _j{Y}_{ij}\times v_{ij}-N_i\times log\sum _j{w}_{ij}\right]$

First order conditions, for each $p$: $0=\frac{\partial L}{\partial {\beta }_p}=\sum _i\left[\sum _j{Y}_{ij}\times \frac{\partial v_{ij}}{\partial {\beta }_p}-N_i\times \frac{\partial log\sum _j{w}_{ij}}{\partial {\beta }_p}\right]$

Thus, for each $p$: $\sum _{ij}{Y}_{ij}\times X_{ijp}=\sum _{ij}{N}_i\times P_{ij}\times X_{ijp}$ as $\frac{\partial v_{ij}}{\partial {\beta }_p}=X_{ij}^p$ and

$(\frac{\partial log\sum _j{w}_{ij}}{\partial {\beta }_p}\times \frac{\sum _j\partial w_{ij}/\partial {\beta }_p}{\sum _j{w}_{ij}}\times \frac{\sum _j{w}_{ij}\times \partial v_{ij}/\partial {\beta }_p}{\sum _j{w}_{ij}}\times \frac{\sum _j{w}_{ij}\times X_{ijp}}{\sum _j{w}_{ij}}\times \sum _j{P}_{ij}\times X_{ijp})$

example

logit regression of rehousing logit_regression_of_rehousing "wikilink".