# Linear Regression via MLE

Basic MLE-Programming and margins

## Introduction

Stata has a very useful command that can be used for the estimation of almost any linear and nonlinear models using maximum likelihood. This command is -ml-.

Properly speaking, this command can be used to obtain M-type estimators, however, I’ll concentrate on maximum likelihood models.

In this small tutorial, I’ll provide a small example of how to use -ml- in combination with -margins- to estimate marginal effects for a linear model, as long as one identifies the outcome of interest.

## LR as MLE

As I mentioned earlier, the ml command in Stata is a powerful tool for obtaining maximum likelihood estimators, although it can be used to find solutions for any m-type estimators. The one limitation I have encountered with this command is that it can be resource-intensive when estimating complex models on large datasets. For instance, If you have a dataset with one million observations but only use 10% of it for modeling, dropping the unused data before estimation can speed up the process. ml will not do that for you.

There are several ways to program ml, such as using the lf, df0, df1, or df2 options. The main difference among them is that you must define the objective function, its gradient, and its Hessian. However, for most purposes, I find that lf is the only one you will ever need.

When using ml with the lf option, you only need to declare the loglikelihood function contributed by each observation in the sample. To illustrate this concept, let’s assume that we want to estimate a simple linear regression using MLE. For this, we need to assume that either the error follows a normal distribution or that the outcome follows a conditionally normal distribution.

\begin{aligned} y_i &= X_i'\beta + e_i \\ e_i &\sim N(0,\sigma^2) \ or \ y_i \sim N(X_i'\beta,sigma^2) \end{aligned}

This implies that the Likelihood ($$L_i$$) and Log-Likelihood ($$LL_i$$) of a single observation is given by: \begin{aligned} L_i &= \frac{1}{\sigma \sqrt{2\pi}} exp\left( -\frac{1}{2} \left(\frac{y_i -X_i'\beta }{\sigma} \right)^2 \right) \\ LL_i &= -log(\sigma) - \frac{1}{2} log (2\pi) -\frac{1}{2} \left(\frac{y_i -X_i'\beta }{\sigma} \right)^2 \end{aligned}

So we just need to create a program that defines this log-likelihood function.

program myols_mle
args lnf xb lnsigma
local sigma exp(lnsigma')
*qui:replace lnf' = -lnsigma' - 1/2 * log(2*_pi) - 1/2 *(($ML_y1-xb')/sigma')^2 qui: replace lnf' = log(normalden($ML_y1,xb',sigma'))
end

Notice that this program has 3 arguments (which come after args) (see line 2)

• lnf: Will store the log-Likelihood for each observation
• xb: Will store the linear combination of variables and their coefficients
• lnsigma: Will store the $$log(\sigma)$$.

We do not estimate $$\sigma$$ directly, because its numerically more stable to estimate the $$log(\sigma)$$. Also, we require $$\sigma$$ to be strictly positive, which can only be done by using the log transformation.

In line 3, I specifically impose the transformation to obtain $$\sigma$$.

In line 4, I leave the comment of how I would write the full Loglikelihood using the formula I provided before, but for simplicilty I use the built-in normalden (line 5)

## Conclusions

The purpose of this small article was to walk you through how to use -ml- for the estimation of a simple linear regression.

In addition, it also introduces you to creating a program that will allow you to use margins, when you have a specific outcome in mind, that is not currently available in the command you are interested in using.

Hope you find it useful.