Summary of oct 19, 2014 experiments with

double-layer ANNs

11111110

22222220

33333330

chosen number of RBM epochs in

1st hidden layer

30

20

20

chosen number of RBM epochs in

2nd hidden layer

10

20

20

0.027000

0.000000

0.000000

validation cost (lowest)

1.471139

1.457574

1.448290

validation set accuracy

67.394444

70.700000

71.144444

chosen number of RBM epochs in

1st hidden layer

40

20

30

chosen number of RBM epochs in

2nd hidden layer

20

20

10

0.000000

0.000000

0.000000

validation cost (lowest)

1.125226

1.288496

1.291019

validation set accuracy

77.305556

71.605556

72.216667

chosen number of RBM epochs in

1st hidden layer

30

30

30

chosen number of RBM epochs in

2nd hidden layer

10

10

10

0.003000

0.000111

0.000000

validation cost (lowest)

1.106896

1.025061

1.022945

validation set accuracy

77.577778

81.900000

80.355556

chosen number of RBM epochs in

1st hidden layer

30

20

30

chosen number of RBM epochs in

2nd hidden layer

20

10

20

0.000111

0.003000

0.000000

validation cost (lowest)

0.861608

1.096372

0.919526

validation set accuracy

85.788889

79.961111

84.350000

chosen number of RBM epochs in

1st hidden layer

30

20

40

chosen number of RBM epochs in

2nd hidden layer

10

10

20

0.000000

0.000000

0.000000

validation cost (lowest)

0.834839

0.922774

0.812600

validation set accuracy

88.000000

84.977778

87.900000

chosen number of RBM epochs in

1st hidden layer

40

30

30

chosen number of RBM epochs in

2nd hidden layer

30

20

20

0.000000

0.000000

0.000333

40 hidden
chosen lambda
nodes

50 hidden
chosen lambda
nodes

60 hidden
chosen lambda
nodes

70 hidden
chosen lambda
nodes

Number of nodes in each hidden layer

ID

80 hidden
chosen lambda
nodes

90 hidden
chosen lambda
nodes

90 hidden
nodes

100
hidden
nodes

110
hidden
nodes

120
hidden
nodes

validation cost (lowest)

0.808818

0.567312

0.780851

validation set accuracy

87.277778

92.944444

89.100000

chosen number of RBM epochs in

1st hidden layer

50

30

40

chosen number of RBM epochs in

2nd hidden layer

20

40

20

chosen lambda

0.001000

0.000000

0.001000

validation cost (lowest)

0.551953

0.572878

0.583373

validation set accuracy

92.772222

93.594444

91.577778

chosen number of RBM epochs in

1st hidden layer

40

40

40

chosen number of RBM epochs in

2nd hidden layer

10

20

20

chosen lambda

0.003000

0.000000

0.003000

validation cost (lowest)

0.318331

0.356740

0.375993

validation set accuracy

97.361111

96.638889

96.738889

chosen number of RBM epochs in

1st hidden layer

30

30

30

chosen number of RBM epochs in

2nd hidden layer

10

10

20

chosen lambda

0.001000

0.000333

0.000000

validation cost (lowest)

0.385072

0.401459

0.453609

validation set accuracy

97.011111

96.000000

94.783333