代写Neural Nets、代做MASS留学生、R程序设计代写、代做R代写

日期：2019-04-19 10:27

Machine Learning, Homework 9, Neural Nets

April 15, 2019

Contents

Boston Housing with a Single Layer and R package nnet 1

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Digit Recognition with R package h2o 5

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Boston Housing with a Single Layer and R package nnet

Let’s do a very simple example with single layer neural nets.

We’ll do the Boston housing data with x=lstat and y =mdev so that we have one numeric x and a numeric y.

We’ve used this classic data set a few times so we are very familiar with it.

Let’s get the data, pull off x and y and standardize x.

library(MASS) ## a library of example datasets

attach(Boston)

## standardize lstat

rg = range(Boston$lstat)

lstats = (Boston$lstat-rg[1])/(rg[2]-rg[1])

##make data frame with standardized lstat values sorted for plotting

ddf = data.frame(lstats,medv=Boston$medv)

oo = order(ddf$lstats) #order the data by x, convenient for plotting

ddf = ddf[oo,]

head(ddf)

## lstats medv

## 162 0.000000000 50.0

## 163 0.005242826 50.0

## 41 0.006898455 34.9

## 233 0.020419426 41.7

## 193 0.031456954 36.4

## 205 0.031732892 50.0

And here is the familiar plot:

plot(ddf)

1

0.0 0.2 0.4 0.6 0.8 1.0

10 20 30 40 50

lstats

medv

Let’s fit a simple neural net.

One hidden layer with 5 units (neurons).

library(nnet)

set.seed(14)

nn1 = nnet(medv~lstats,ddf,size=5,decay=.1,linout=T,maxit=1000)

## # weights: 16

## initial value 274435.143486

## iter 10 value 14655.902880

## iter 20 value 13675.210318

## iter 30 value 13618.543249

## iter 40 value 13593.167670

## iter 50 value 13548.561442

## iter 60 value 13545.520754

## iter 70 value 13544.330448

## iter 80 value 13541.583759

## iter 90 value 13540.386199

## iter 100 value 13539.604916

## iter 110 value 13536.860853

## iter 120 value 13535.643158

## iter 130 value 13535.589069

## final value 13535.578458

## converged

summary(nn1)

## a 1-5-1 network with 16 weights

## options were - linear output units decay=0.1

## b->h1 i1->h1

## 1.06 0.69

## b->h2 i1->h2

## 2.38 -38.17

## b->h3 i1->h3

## 2.49 -7.61

## b->h4 i1->h4

## 2.05 0.55

## b->h5 i1->h5

## 2.53 -7.60

## b->o h1->o h2->o h3->o h4->o h5->o

## 4.67 3.64 21.22 9.19 3.48 8.93

Now let’s plot the fit:

yhat1 = predict(nn1,ddf)

plot(ddf)

lines(ddf$lstats,yhat1,lty=1,col="red",lwd=3)

2

0.0 0.2 0.4 0.6 0.8 1.0

10 20 30 40 50

lstats

medv

Notice that you understand exactly how the single layer neural fit did this !!!

Now let’s fit the 5 unit neural net for a set of decay values.

Let’s do this in parallel using the R parallel package. This is simple enough that we don’t really need to

speed it up, but we can illustrate the approach. You may want to use if for some of the more complicated

model fits!

library(doParallel) #library for parallel computing

## Loading required package: foreach

## Loading required package: iterators

## Loading required package: parallel

registerDoParallel()

cat("number of workers is: ",getDoParWorkers(),"\n")

## number of workers is: 4

#you could pick the number of workers with:

# registerDoParallel(cores=num) where num is the number of workers.

Now we will use the function foreach to fit neural net models in parallel. First we set up a vector of decay

values to try. Then we use foreach to run the neural net fits. foreach will return a list, with the i

th list

element corresponding to the results obtained in the i

th loop iteration.

decv = c(.5,.1,.01,.005,.0025,.001,.0001,.00001)

#do a parallel loop over decay values

modsL = foreach(i=1:length(decv)) %dopar% {

library(nnet) #I did not have to do this when I was not in Rmarkdown.

set.seed(5*i) #I did have to to this.

nnfit = nnet(medv~lstats,ddf,size=5,decay=decv[i],linout=T,maxit=10000)

nnfit

}

is.list(modsL)

## [1] TRUE

length(modsL)

## [1] 8

The function foreach will launch a bunch of R processes so things like random number seeds may have to be

reset for each process.

Now we can plot all the fits by looping over the list of models.

plot(ddf)

for(i in 1:length(modsL)) {

yhat = predict(modsL[[i]],ddf)

3

lines(ddf$lstats,yhat,col=i,lty=i,lwd=2)

}

0.0 0.2 0.4 0.6 0.8 1.0

10 20 30 40 50

lstats

medv

Problem

fit the neural net model with size=100 and decay=.001, plot the fits. How does it look? Try running

the fit at least twice to see that it changes.

Redo the the loop over decay values with size=100. How does it look now? Do we need 100? Will

decay be more important with 100 than it was with 5 units?

4

Digit Recognition with R package h2o

First, let’s fire up h2o.

print(date())

## [1] "Tue Apr 16 16:22:12 2019"

library(h2o)

##

## ----------------------------------------------------------------------

##

## Your next step is to start H2O:

## > h2o.init()

##

## For H2O package documentation, ask for help:

## > ??h2o

##

## After starting H2O, you can use the Web UI at http://localhost:54321

## For more information visit http://docs.h2o.ai

##

## ----------------------------------------------------------------------

##

## Attaching package: 'h2o'

## The following objects are masked from 'package:stats':

##

## cor, sd, var

## The following objects are masked from 'package:base':

##

## &&, %*%, %in%, ||, apply, as.factor, as.numeric, colnames,

## colnames<-, ifelse, is.character, is.factor, is.numeric, log,

## log10, log1p, log2, round, signif, trunc

h2o.init()

##

## H2O is not running yet, starting it now...

##

## Note: In case of errors look at the following log files:

## /tmp/RtmpzaPmRq/h2o_root_started_from_r.out

## /tmp/RtmpzaPmRq/h2o_root_started_from_r.err

##

##

## Starting H2O JVM and connecting: . Connection successful!

##

## R is connected to the H2O cluster:

## H2O cluster uptime: 1 seconds 203 milliseconds

## H2O cluster timezone: America/Phoenix

## H2O data parsing timezone: UTC

## H2O cluster version: 3.20.0.8

## H2O cluster version age: 6 months and 25 days !!!

## H2O cluster name: H2O_started_from_R_root_jrw534

## H2O cluster total nodes: 1

## H2O cluster total memory: 6.84 GB

## H2O cluster total cores: 8

## H2O cluster allowed cores: 8

## H2O cluster healthy: TRUE

## H2O Connection ip: localhost

## H2O Connection port: 54321

## H2O Connection proxy: NA

## H2O Internal Security: FALSE

## H2O API Extensions: XGBoost, Algos, AutoML, Core V3, Core V4

## R Version: R version 3.5.1 (2018-07-02)

## Warning in h2o.clusterInfo():

## Your H2O cluster version is too old (6 months and 25 days)!

## Please download and install the latest version from http://h2o.ai/download/

Now we can read in the data.

In order to make things run faster I’ll down sample to just ns=10,000 observations.

train60D = read.csv("http://www.rob-mcculloch.org/data/mnist-train.csv")

train60D$C785 = as.factor(train60D$C785)

n = nrow(train60D)

5

set.seed(99)

ns = 10000

trainDS = train60D[sample(1:n,ns),]

trainS = as.h2o(trainDS,"trainS")

##

|

| | 0%

|

|=================================================================| 100%

testD = read.csv("http://www.rob-mcculloch.org/data/mnist-test.csv")

testD$C785 = as.factor(testD$C785)

test = as.h2o(testD,"test")

##

|

| | 0%

|

|=================================================================| 100%

x=1:784;y=785

print(ls())

## [1] "ddf" "decv" "i" "lstats" "modsL" "n"

## [7] "nn1" "ns" "oo" "rg" "test" "testD"

## [13] "train60D" "trainDS" "trainS" "x" "y" "yhat"

## [19] "yhat1"

print(h2o.ls())

## key

## 1 test

## 2 trainS

Let’s run h2o.deeplearning at settings similar to the ones that were found to work in the lecture notes. I

dropped the layer/node architecture down to (50,50) so it would run faster. On my laptap in took about 90

seconds to run the one below.

I don’t know how long it will take on your machine.

fp = file.path("./files","mDNNdrop")

if(file.exists(fp)) {

mDNNdrop = h2o.loadModel(fp)

} else {

tm = system.time({

mDNNdrop = h2o.deeplearning(x,y,training_frame = trainS,

hidden=c(50,50),

activation="TanhWithDropout",

hidden_dropout_ratios=c(.1,.1),

l1=1e-4,

epochs=2000,

model_id="mDNNdrop",

validation_frame=test)

})

}

## Warning in .h2o.startModelJob(algo, params, h2oRestApiVersion): Dropping bad and constant columns: [C646, C645, C644, C365, C760, C51, C53, C52, C55, C54, C57, C56, C59, C58, C533, C253, C60, C703, C702, C701, C700, C1, C422, C2, C784, C3, C420, C783, C4, C782, C5, C143, C781, C6, C142, C780, C7, C141, C8, C9, C674, C673, C672, C393, C84, C83, C86, C85, C88, C87, C729, C728, C727, C726, C169, C561, C281, C11, C10, C12, C15, C617, C616, C17, C16, C19, C18, C699, C732, C731, C730, C450, C170, C20, C22, C21, C24, C23, C26, C25, C28, C505, C27, C29, C589, C225, C588, C31, C30, C32, C35, C759, C758, C757, C756, C755, C754, C115, C753, C477, C113, C112, C111, C197].

##

|

| | 0%

|

| | 1%

|

|= | 1%

|

|= | 2%

|

|== | 2%

|

|== | 3%

|

|=================================================================| 100%

cat("the time is: ",tm,"\n")

## the time is: 0.617 0.005 80.098 0 0

6

print(h2o.confusionMatrix(mDNNdrop,valid=TRUE))

## Confusion Matrix: Row labels: Actual class; Column labels: Predicted class

## 0 1 2 3 4 5 6 7 8 9 Error Rate

## 0 959 0 2 1 0 9 6 1 2 0 0.0214 = 21 / 980

## 1 0 1111 1 7 0 1 5 3 7 0 0.0211 = 24 / 1,135

## 2 20 3 954 14 7 1 10 7 16 0 0.0756 = 78 / 1,032

## 3 0 1 18 946 0 15 2 16 9 3 0.0634 = 64 / 1,010

## 4 1 0 3 0 938 1 11 4 3 21 0.0448 = 44 / 982

## 5 6 2 5 33 10 775 15 11 29 6 0.1312 = 117 / 892

## 6 10 4 5 1 9 12 911 3 3 0 0.0491 = 47 / 958

## 7 3 5 19 8 8 0 2 964 1 18 0.0623 = 64 / 1,028

## 8 11 6 7 23 12 13 10 11 874 7 0.1027 = 100 / 974

## 9 7 6 3 12 33 8 2 22 4 912 0.0961 = 97 / 1,009

## Totals 1017 1138 1017 1045 1017 835 974 1042 948 967 0.0656 = 656 / 10,000

missclass = h2o.performance(mDNNdrop,valid=TRUE)@metrics$mean_per_class_error

cat("the mean per class error is: ",missclass,"\n")

## the mean per class error is: 0.06676157

## if you like it, keep it

#h2o.saveModel(mDNNdrop,path="./files")

print(date())

## [1] "Tue Apr 16 16:25:01 2019"

Problem

I always used dropout. Is that a good idea? Change the settings to not use dropout. Is it worse or

better? Do a couple of runs.

look at h2o.deeplearning. Pick another option and try changing it to see if you can improve the

prediction.

7