NLP Platform Chatbot Ask Ubuntu Web Applications Overall

NLP.js 0.9905660377358 0.9444444444444 0.796610169491525 0.9304029304029

Watson 0.9716981132075 0.9174311926606 0.830508474576271 0.9197080291971
LUIS 0.9811320754717 0.8990825688073 0.813559322033899 0.9124087591241
Botfuel 0.9811320754717 0.8940092165899 0.796610169491525 0.9067641681901
Recast 0.9905660377358 0.8623853211009 0.745762711864407 0.8868613138686
RASA 0.9811320754717 0.8623853211009 0.745762711864407 0.8832116788321
DialogFlow 0.9268292682927 0.8532110091743 0.803418803418803 0.8703703703704
 NLU Evaluation
1.05

0.95

0.9

0.85

0.8

0.75

0.7
NLP.js Watson LUIS Botfuel Recast RASA DialogFlow

Chatbot Ask Ubuntu Web Applications Overall

Column1 intent true + false - false + precision
chatbot FindConnection 70 0 1 0.98591549295775
chatbot DepartureTime 35 1 0 1
askUbuntu Software Recommendation 35 1 4 0.8974358974359
askUbuntu None 3 1 2 0.6
askUbuntu Shutdown Computer 14 2 0 1
askUbuntu Make Update 37 2 0 1
askUbuntu Setup Printer 13 0 0 1
webApp Find Alternative 16 3 0 1
webApp Delete Account 9 4 1 0.9
webApp Export Data 2 3 1 0.66666666666667
webApp Sync Accounts 4 0 2 0.66666666666667
webApp None 0 0 4 0
webApp Change Password 3 0 3 0.5
webApp Filter Spam 13 1 1 0.92857142857143
webApp Download Video 0 1 0
Total 254 19 19 0.93040293040293
Total Chatbot 105 1 1 0.99056603773585
Total askUbuntu 102 6 6 0.94444444444444
Total webApp 47 12 12 0.79661016949153
recall f1
1 0.9929078014184
0.972222222222 0.9859154929577
0.972222222222 0.9333333333333
0.75 0.6666666666667
0.875 0.9333333333333
0.948717948718 0.9736842105263
1 1
0.842105263158 0.9142857142857
0.692307692308 0.7826086956522
0.4 0.5
1 0.8

1 0.6666666666667
0.928571428571 0.9285714285714
0
0.930402930403 0.9304029304029
0.990566037736 0.9905660377358
0.944444444444 0.9444444444444
0.796610169492 0.7966101694915
Column1 intent true + false - false + precision
chatbot FindConnection 70 1 2 0.972222222222222
chatbot DepartureTime 33 2 1 0.970588235294118
askUbuntu Software Recommendati 35 5 3 0.921052631578947
askUbuntu None 1 4 1 0.5
askUbuntu Shutdown Computer 14 0 0 1
askUbuntu Make Update 37 0 4 0.902439024390244
askUbuntu Setup Printer 13 0 1 0.928571428571429
webApp Find Alternative 15 1 1 0.9375
webApp Delete Account 9 1 3 0.75
webApp Export Data 2 1 2 0.5
webApp Sync Accounts 5 1 0 1
webApp None 0 4 1 0
webApp Change Password 5 1 0 1
webApp Filter Spam 13 1 2 0.866666666666667
webApp Download Video 0 0 1 0
Total 252 22 22 0.91970802919708
Total Chatbot 103 3 3 0.971698113207547
Total askUbuntu 100 9 9 0.917431192660551
Total webApp 49 10 10 0.830508474576271
recall f1
0.985915492958 0.97902097902098
0.942857142857 0.95652173913043
0.875 0.8974358974359
0.2 0.28571428571429
1 1
1 0.94871794871795
1 0.96296296296296
0.9375 0.9375
0.9 0.81818181818182
0.666666666667 0.57142857142857
0.833333333333 0.90909090909091
0
0.833333333333 0.90909090909091
0.928571428571 0.89655172413793

0.919708029197 0.91970802919708
0.971698113208 0.97169811320755
0.917431192661 0.91743119266055
0.830508474576 0.83050847457627
Column1 intent true + false - false + precision
chatbot FindConnection 70 1 1 0.985915492957747
chatbot DepartureTime 34 1 1 0.971428571428571
askUbuntu Software Recommendation 36 4 5 0.878048780487805
askUbuntu None 0 5 0
askUbuntu Shutdown Computer 14 0 0 1
askUbuntu Make Update 36 1 4 0.9
askUbuntu Setup Printer 12 1 2 0.857142857142857
webApp Find Alternative 14 2 2 0.875
webApp Delete Account 8 2 0 1
webApp Export Data 3 0 1 0.75
webApp Sync Accounts 5 1 0 1
webApp None 3 1 8 0.272727272727273
webApp Change Password 3 3 0 1
webApp Filter Spam 12 2 0 1
webApp Download Video 0 0 0
Total 250 24 24 0.912408759124088
Total Chatbot 104 2 2 0.981132075471698
Total askUbuntu 98 11 11 0.89908256880734
Total webApp 48 11 11 0.813559322033898
recall f1
0.985915492958 0.9859154929577
0.971428571429 0.9714285714286
0.9 0.8888888888889
0
1 1
0.972972972973 0.9350649350649
0.923076923077 0.8888888888889
0.875 0.875
0.8 0.8888888888889
1 0.8571428571429
0.833333333333 0.9090909090909
0.75 0.4
0.5 0.6666666666667
0.857142857143 0.9230769230769

0.912408759124 0.9124087591241
0.981132075472 0.9811320754717
0.899082568807 0.8990825688073
0.813559322034 0.8135593220339
Column1 intent true + false - false + precision
chatbot FindConnection 69 2 0 1
chatbot DepartureTime 35 0 2 0.945945945945946
askUbuntu Software Recommendation 33 7 1 0.970588235294118
askUbuntu None 1 4 0 1
askUbuntu Shutdown Computer 14 0 2 0.875
askUbuntu Make Update 37 0 6 0.86046511627907
askUbuntu Setup Printer 12 1 2 0.857142857142857
webApp Find Alternative 16 0 4 0.8
webApp Delete Account 9 1 3 0.75
webApp Export Data 2 1 2 0.5
webApp Sync Accounts 4 2 0 1
webApp None 0 4 0
webApp Change Password 4 2 1 0.8
webApp Filter Spam 12 2 1 0.923076923076923
webApp Download Video 0 0 1 0
Total 248 26 25 0.908424908424908
Total Chatbot 104 2 2 0.981132075471698
Total askUbuntu 97 12 11 0.898148148148148
Total webApp 47 12 12 0.796610169491525
recall f1
0.971830985915 0.985714285714
1 0.972222222222
0.825 0.891891891892
0.2 0.333333333333
1 0.933333333333
1 0.925
0.923076923077 0.888888888889
1 0.888888888889
0.9 0.818181818182
0.666666666667 0.571428571429
0.666666666667 0.8
0
0.666666666667 0.727272727273
0.857142857143 0.888888888889

0.905109489051 0.90676416819
0.981132075472 0.981132075472
0.889908256881 0.89400921659
0.796610169492 0.796610169492
Column1 intent true + false - false + precision
chatbot FindConnection 70 1 0 1
chatbot DepartureTime 35 0 1 0.9722222222222
askUbuntu Software Recommendation 29 11 1 0.9666666666667
askUbuntu None 1 4 0 1
askUbuntu Shutdown Computer 14 0 6 0.7
askUbuntu Make Update 37 0 7 0.8409090909091
askUbuntu Setup Printer 13 0 1 0.9285714285714
webApp Find Alternative 15 1 3 0.8333333333333
webApp Delete Account 9 1 5 0.6428571428571
webApp Export Data 2 1 4 0.3333333333333
webApp Sync Accounts 3 3 0 1
webApp None 1 3 1
webApp Change Password 4 2 1 0.8
webApp Filter Spam 10 4 1 0.9090909090909
webApp Download Video 0 0 0
Total 243 31 31 0.8868613138686
Total Chatbot 105 1 1 0.9905660377358
Total askUbuntu 94 15 15 0.8623853211009
Total webApp 44 15 15 0.7457627118644
recall f1
0.985915492958 0.9929078014184
1 0.9859154929577
0.725 0.8285714285714
0.2 0.3333333333333
1 0.8235294117647
1 0.9135802469136
1 0.962962962963
0.9375 0.8823529411765
0.9 0.75
0.666666666667 0.4444444444444
0.5 0.6666666666667
0.25
0.666666666667 0.7272727272727
0.714285714286 0.8

0.886861313869 0.8868613138686
0.990566037736 0.9905660377358
0.862385321101 0.8623853211009
0.745762711864 0.7457627118644
Column1 intent true + false - false + precision
chatbot FindConnection 70 1 1 0.985915492957747
chatbot DepartureTime 34 1 1 0.971428571428571
askUbuntu Software Recommendation 33 7 4 0.891891891891892
askUbuntu None 0 5 1 0
askUbuntu Shutdown Computer 14 0 6 0.7
askUbuntu Make Update 34 3 2 0.944444444444444
askUbuntu Setup Printer 13 0 2 0.866666666666667
webApp Find Alternative 15 1 8 0.652173913043478
webApp Delete Account 9 1 5 0.642857142857143
webApp Export Data 0 3 0
webApp Sync Accounts 3 3 0 1
webApp None 0 4 1 0
webApp Change Password 4 2 0 1
webApp Filter Spam 13 1 0 1
webApp Download Video 0 0 1 0
Total 242 32 32 0.883211678832117
Total Chatbot 104 2 2 0.981132075471698
Total askUbuntu 94 15 15 0.862385321100918
Total webApp 44 15 15 0.745762711864407
recall f1
0.985915492958 0.98591549296
0.971428571429 0.97142857143
0.825 0.85714285714
0
1 0.82352941176
0.918918918919 0.93150684932
1 0.92857142857
0.9375 0.76923076923
0.9 0.75
0
0.5 0.66666666667
0
0.666666666667 0.8
0.928571428571 0.96296296296

0.883211678832 0.88321167883
0.981132075472 0.98113207547
0.862385321101 0.8623853211
0.745762711864 0.74576271186
Column1 intent true + false - false + precision
chatbot FindConnection 60 11 0 1
chatbot DepartureTime 35 0 4 0.8974358974359
askUbuntu Software Recommendation 28 12 2 0.93333333333333
askUbuntu None 2 3 8 0.2
askUbuntu Shutdown Computer 14 0 2 0.875
askUbuntu Make Update 36 1 3 0.92307692307692
askUbuntu Setup Printer 13 0 1 0.92857142857143
webApp Find Alternative 16 0 2 0.88888888888889
webApp Delete Account 10 0 2 0.83333333333333
webApp Export Data 1 2 2 0.33333333333333
webApp Sync Accounts 4 2 0 1
webApp None 2 2 1 0.66666666666667
webApp Change Password 4 2 1 0.8
webApp Filter Spam 10 4 3 0.76923076923077
webApp Download Video 0 0 0
Total 235 39 31 0.88345864661654
Total Chatbot 95 11 4 0.95959595959596
Total askUbuntu 93 16 16 0.85321100917431
Total webApp 47 12 11 0.81034482758621
recall f1
0.845070422535 0.9160305343511
1 0.9459459459459
0.7 0.8
0.4 0.2666666666667
1 0.9333333333333
0.972972972973 0.9473684210526
1 0.962962962963
1 0.9411764705882
1 0.9090909090909
0.333333333333 0.3333333333333
0.666666666667 0.8
0.5 0.5714285714286
0.666666666667 0.7272727272727
0.714285714286 0.7407407407407

0.857664233577 0.8703703703704
0.896226415094 0.9268292682927
0.853211009174 0.8532110091743
0.796610169492 0.8034188034188