# Logistic regression statistics assignment

Logistic regression statistics assignment

Implement the logistic regression algorithm using Gradient Descent in

analogy with neural networks (as described in the lectures, Chapter 4, or

[1], Sections 11.3–11.5) for p attributes. The objective function, which

your program should minimize, can be either the training MSE

MSE := 1

n

Xn

i=1

(yi − p(xi))2

(1)

(cf. “Brier loss” on slide 99 in Chapter 4) or the negative log-likelihood

(NLL)

NLL := −

X

i:yi=1

log p(xi) −

X

i:yi=0

log(1 − p(xi))

(as described on slide 44 of Chapter 4); the choice is yours. (Be careful

not to confuse p as the number of attributes and p(x) as the predicted

probability of 1.) Your program can be written either in R or in MATLAB. You are not allowed to use any existing implementations of logistic

Logistic regression statistics assignment

regression or Gradient Descent in R, MATLAB, or any other language,

and should code logistic regression from first principles. However, you are

allowed to set the number of attributes p to a specific value that allows

you to do the following tasks.

2. Use the Auto data set. Create a new variable high that takes values

high := (

1 if mpg ≥ 23

0 otherwise (i.e., if mpg ≤ 22).

Apply your program to the Auto data set and new variable to predict

high given horsepower, weight, year, and origin. (In other words,

high is the label and horsepower, weight, year, and origin are the

attributes.) Since origin is a qualitative variable, you will have to create

appropriate dummy variables. Normalize the attributes, as described in

Lab Worksheet 4, Section 5 or Exercise 9 (or Section 4.6.6 of [2]).

3. Split the data set randomly into two equal parts, which will serve as the

training set and the test set. Use your birthday (in the format MMDD)

as the seed for the pseudorandom number generator. The same training

and test sets should be used throughout this assignment.

4. Train your algorithm on the training set using independent random numbers in the range [−0.7, 0.7] as the initial weights. Find the MSE on the

test set (it is defined by (1) except that the average should be over the

test rather than training set). Try different values of the learning rate η

and of the number of training steps (so that your stopping rule is to stop

after a given number of steps). Give a small table of test and training

MSEs in your report.

Optional: Try different stopping rules, such as: stop when the value of

the objective function (the training MSE) does not change by more than

1% of its initial value over the last 10 training steps.

6. Run your logistic regression program for a fixed value of η and for a fixed

stopping rule (producing reasonable results in your experiments so far)

100 times, for different values of the initial weights (produced as above,

as independent random numbers in [−0.7, 0.7]). In each of the 100 cases

compute the test MSE and show it in your report as a boxplot.

7. Optional: Redo the experiments in items 4–5 modifying the training

procedure as follows. Instead of training logistic regression once using

Gradient Descent, train it 4 times using Gradient Descent with different

values of the initial weights and then choose the prediction rule with the

best training MSE.

Place an order with us on https://assignmentsproficient.com/ and expect the best grades and help from qualified writers