Assignment 3

Big Data代写 Consider the two variables in the dataset Assign3.csv. We are interested in predicting the second variable Y given the first variable X.

Machine Learning and Big Data for Economics and Finance

Consider the two variables in the dataset Assign3.csv.Big Data代写

We are interested in predicting the second variable Y given the first variable X.

1. Fit a linear regression model to the data. Show the data scatter plot on the samefigure with the values predicted by the linear model.
2. Fit a quadratic regression model to the data. Show the data scatter plot on the same figure with the values predicted by the quadratic.
3. We are interested in constructing a step function learner asfollows:

First draw a random number U uniformly on the interval spanned by the minimum and maximum values of the inputs (x₁; :::; x_n) and then use it to construct the following function whose purpose is to give the prediction of Y given X = x:

f(x) = a₁I(U 6 x) + a₂I(U > x);Big Data代写 

where a₁ and a₂ are just unknown constants to be learned. It goes without saying that I(some statement) is the indicator function that equals 1 when the statement is true and 0 otherwise.

a.Usetwo  different  methods  to  compute the  estimate  f^(x) = a^₁I(U 6x)+a^₂I(U > x).  Is f^ a strong learner?

b.Use one of the previous two methods to write an Rfunction that takes as input x and the data (x₁; :::; x_n; y₁; :::; y_n) and gives as output f^(x).Make sure the function is capable of dealing with the case where x conatains more than one number.Big Data代写

c.Usingthree different runs of the previous function, create three dif- ferent plots where, on each, f^ is shown together with the scatter plot of the

4.Write an R function that applies boosting to the previous step function learner.Big Data代写

That R function should take as inputs: the data, B the number of boosting iterations, L the learning rate and an optional argument indicating thesize of the test subsample in case a validation set approach is needed.

As output the function should give:  f^boost  the boosted learner evaluated

at the training data and the training mean squared error evaluated for each iteration b = 1; :::; B of the boosting algorithm. Also, in case the size of the test subsample is greater than zero, the function should output:  f^boost  evaluated at the test sample and the test MSE evaluated for each iteration b = 1; :::; B.Big Data代写

a.Use that  function  to  plot  f^boost   on  top  of  the  data  scatter  plot  for

L = 0.01 and for B = 10000. Show the same with different values of B.

b.Plot the training MSE vs. the number of

c.Was there overfitting when B =10000?Big Data代写

Even though the algorit

NOTE:

hm is described in detail in both the slides and textbook, for the sake of making the implementation easier, its special case per- taining to the questions in the assignment is presented here.

Boosting algorithm:

1. Inputs:

• A sample of covariates (i.e. inputs) x₁; :::; x_n and responses (i.e. out- puts) y₁; :::; y_n.
• A (weak) learnerf^.
• A learning rate L >0.

2. Initialize:
   • Set f^boost(x)   0.

• Compute  the  first  learner  f^0(x) = a^₁I(U 6 x) + a^₂I(U > x)  on  the original data.
• Setr_i   y_i ¡ Lf^0(x_i) for i = 1; :::; n.Big Data代写

3. Do the following for b = 1; :::;B:

a.Givenx₁; :::; x_n as covariates and r₁; :::; r_n as responses, fit a learner f^b by first sampling U and then estimating f^b(x) = a^₁I(U 6 x) + a^₂I(U > x).

b.Set f^boost(x)   f^boost(x) + Lf^b(x).

c.Setr_i   r_i ¡ Lf^b(x_i).

4.Output: f^boost(x).