Log of Streaming.

## 2018-01-27 Saturday

### Notes

• at least 6 models explored
• final model: a mix of 6 models on same features

• 1x lightgbm
• 5x nn

procedure (loop 6 times)

1. 6 models (i.e., 6 times)

1. fit 1 models
2. get the predicted value

results: 6 predicted values from 6 mdoels respectively

2. average 6 predicted values (all weights=1)

• #1 + #2 + #3 + #4 + #5 + #6: .2965
• #1 + #2: 0.29502 on private
denoising autoencoder hidden activations

Nonlinear stacking

• Bootstrap Method: statistic varible of statistic variable

Example: a sample of 100 values (x)

goal: get an estimate of the mean of the sample.

• old way: calculate the mean directly from the sample: mean(x) = 1/100 * sum(x)

• bootstrap:

1. define sample size in each sub-samples,such as 1000
2. define number of sub-samples, e.g. 3

i.e., 3 sub-samples with 1000 samples each

3. Create random sub-samples of dataset with replacement
4. Calculate the mean of each sub-sample.
5. Calculate the average of all of collected means
6. use that as estimated mean for the data.

Bootstrap Aggregation (Bagging): combine the predictions from multiple machine learning algorithms together to make more accurate predictions than any individual model.

• reduce the variance for those algorithm that have high variance. An algorithm that has high variance are decision trees, like classification and regression trees (CART). lower variance, increase bias

Example of CART: a sample dataset of 1000 instances (x) and we are using the CART algorithm. Bagging of the CART algorithm would work as follows.

1. Create many (e.g. 100) random sub-samples of our dataset with replacement.
2. Train a CART model on each sample.
3. Given a new dataset, calculate the average prediction from each model.

### the way to share notes

I will post the notes on my blog.

1. easy search <= search bar
2. comment section for each note