# PRESENTED BY

LEARNING AGENTS LABORATORY GEORGE MASON UNIVERSITY

Disciple-RKF/COG: Agent Teaching by Subject Matter Experts

4. Specifying a training scenario Modeling expert’s reasoning Task and rule learning Rule refinement Exception handling Problem solving Tools for the knowledge engineer 1. 3. 6. Viewing the solutions generated by Disciple 3. Specifying a scenario 2. Generating a center of gravity analysis report Teaching Disciple-COG by a subject matter expert 1.The Summary of the DEMO About Disciple-COG Students use of Disciple-COG as a learning assistant 1. Ontology development tools 2. 2. 5. Ontology import tools .

Disciple is a theory, methodology, and learning agent shell for rapid development of knowledge bases and agents, by subject matter experts, with limited assistance from knowledge engineers

Disciple learns from the expert, building, verifying and improving its knowledge base The expert teaches Disciple in a way that resembles how the expert would teach a person.

DISCIPLE SYSTEM Problem Solving Learning

Interface

Ontology + Rules

Demo overview

First we will demonstrate how a developed Disciple agent helps the students at the US Army War College to learn about Center of Gravity analysis. Then, in the main part of the demo, we will show how this Disciple agent was developed by being taught the problem solving method of a military expert. Finally, we will demonstrate additional tools that are used by the knowledge engineer to perform knowledge base development tasks that are currently beyond the capabilities of a subject matter expert.

Students use of Disciple-COG as a learning assistant
In the “Case Studies in Center of Gravity Analysis” course students learn to identify the centers of gravity (COG) of the opposing forces in military conflicts. The COG of a force is its main source of strength, power, and resistance. Each student has to study a historic scenario (such as the World War II invasion of the island of Sicily by the Allied Forces) and has to develop a center of gravity analysis report. Disciple guides the students to specify the relevant aspects of the assigned war scenario. Then it identifies and tests the strategic center of gravity candidates for that scenario, and generates a center of gravity analysis report. The students study and critique the solutions generated by Disciple and finalize the report.

1. Scenario Elicitation First the student is guided by the scenario elicitation tool to specify the relevant aspects of the war scenario. .

Scenario Elicitation The tool shows: A table of contents that will be developed during scenario elicitation.1. . A list of questions that have to be answered by the student.

Scenario Elicitation The student is asked to provide a short English description of the scenario. .1.

.1. Scenario Elicitation The student is asked to provide a short English description of the scenario.

1. . Disciple assists by providing clarifications for each requested piece of information. Scenario Elicitation Then the student has to name the opposing forces of the scenario.

.1. Scenario Elicitation Each opposing force is automatically introduced into the table of contents When the student selects one of these forces. Disciple asks more specific questions about it.

Scenario Elicitation Because the student has characterized Allied_Forces_1943 as a multi state force.1. . Disciple expands the table of contents with the aspects that are relevant for such a force.

. Disciple asks more specific questions about it.1. Scenario Elicitation When the student selects one of these aspects.

1. Scenario Elicitation In this way the student is guided to specify the relevant aspects of the scenario. .

Viewing the solutions generated by Disciple After the scenario is specified Disciple can identify and test the corresponding strategic center of gravity candidates .2.

2. Viewing the solutions generated by Disciple The tool shows: A description of the selected candidate and the rationale for its identification and testing The strategic center of gravity candidates for each of the opposing forces .

. Viewing the solutions generated by Disciple The solution includes a summary of the specific candidate’s solution.2.

2. . Viewing the solutions generated by Disciple A justification for the entities selection as a candidate.

Viewing the solutions generated by Disciple And a summary of the candidates testing.2. .

Viewing the solutions generated by Disciple The justification may be presented in a more abstract form .2.

2. Viewing the solutions generated by Disciple Or it may be presented in a more detailed form .

2. Viewing the solutions generated by Disciple Or it may be presented in a more detailed form .

.3. Generating a center of gravity analysis report Then Disciple generates a draft center of gravity analysis report.

The student may further improve it by using a text editor. Generating a center of gravity analysis report The first part of the report is a description of the scenario which is based on the student’s specification.3. .

3. The student has to critique Disciple’s reasoning and specify his own justification when in disagreement with Disciple. . Generating a center of gravity analysis report The second part of the report is a list of COG candidates and their justifications.

First a knowledge engineer has used the tools of Disciple to build an object ontology that represents the concepts from the COG analysis domain. The teaching process consists of the sequence of displayed steps and results in a set of problem solving rules learned by Disciple. with only limited assistance from the knowledge engineer.Teaching Disciple-COG by a subject matter expert Now we will discuss how this Disciple agent was developed. After that an expert can teach Disciple his own reasoning in center of gravity analysis. He has also defined elicitation scripts to guide the user in defining a specific scenario. <object> Scenario Force Force_goal Strategic_COG_relevant_factor Strategic_ Operational_ goal goal Other_relevant_factor Demographic_factor Civilization_factor Psychosocial_factor Historical_factor Geographical_factor International_factor Military_factor Political_factor resource_ or_ infrastructure_element Economic_factor Initial object ontology Scenario elicitation Modeling Task learning Rule learning Rule refinement Problem solving <object> Scenario Force Force_goal Strategic_COG_relevant_factor Strategic_ Operational_ goal goal Other_relevant_factor Demographic_factor Civilization_factor Psychosocial_factor Historical_factor in sta n ce_o f resource_ or_ infrastructure_element IF the task is IF thetas k is Economic_factor Scenarios (elicited) Geographical_factor International_factor i nsta nce_ of Brita in _1 94 3 com pon en t_ st a te Political_factor S icily _1 94 3 ha s as o pp o sin g fo rce ins tan ce _o f ins tan ce _o f A ng lo _a lli es _1 94 3 h as _a s_ pr im ary_ fo rc e_ el em en t b ri ef_ descrip tion in sta nce_ of ty pe _o f_ op era ti on s “W W II A lli ed i n vasi on o f S i ci ly in 19 43 ” Military_factor insta nce_ of US _1 9 4 3 ha s_ as_ in dust ri al_ fa ct or ins ta nce_ of U S _7 th _A rmy _ (Fo rce_ 34 3) i nsta nce_o f in st a nce_ of B r_ 8 th_ A rm y_ (F orce _5 45 ) co mpo ne nt_ sta t e h as _a s_ su bg ro up A ll i ed _f orce s_ op er ati on _Hu sk y i n st an ce_o f ha s_a s_ su bg ro up h as_ as _sub gro up W e stern_ N a va l_ TF h as _a s_s ubg roup East ern_ N av al _ TF h as_a s_ su bg ro up ins ta nce_ of ha s_a s_ su bg ro up U S_ 9 t h_ Ai r_ Fo rce i nsta nce_ of N o rt hw est_A f ric a_ A ir_Fo rce Identify t he strategic COG cand idat es withrespect to the Identif y the strate gic COGcandidateswith re spe ct to the industrial civilization of a statewhich isa m em ber of aforc e ind ust rial civilization of a state whichis amem ber of a force IF thetas k is IF the task is Thestate is ?O2 T he stateis ? O2 Id entify the strate gic COGca ndidates with re spe ct to th e Iden tifythestrateg ic C OGcandidateswith respec t tothe Theforce is ?O1 T he forceis ? O1 in dustrial civilization of a state whichisa mem ber of a force industrial c ivilization of a state wh ich is a m em berof a force IFstate thetas k is IF the task is expla nat ion explanatio n T he is ?O2 Th e state is ?O 2 Identify the strategic C OGcandidateswith respect to the Id t h e strategic COG cand idates with respect to th e ?O2 has_as _ind ustrial_factor ?O3 ?O 2 h as_as_industri l_factor T he force is ?O1 Th eentify force a is ?O 1 ?O3 ind ustrial civilization r of a fo rce in dustrial civilization e ?O3 is_a_m ajor_genera tor_of ?O4 ?O 3 is_a_m ajor_generator_o fk ?O4 IF th e task isof a statewhichis amembe IFstate thetas isof a state which isa member of aforc explanatio n exp lana tion Th e stateis ? O2 Thessential_go is ?O2 ?O4 IS strategic ally_essenti al_goods_or_m ateriel ?O 4 ISstrateg ically_e ods_or_m aterie l Iden tifythe strategic COG candidates with resp ect to the Identif y is the strate gic COG candidates with re spe ct to the ?O 2 h as_as_industri a l_factor O3 ?O2 has_a s_industrial_factor ?O 3 Th e force is ? O1 ? The force ?O1 industrial civiliz ation em berof aforce ind ust rIF ialthe civiltas ization ?O 3 is_a_m ajor_gen erator_o f ?O4 ?O3 is_a_m ajor_g enerator_of ?O4 Plausible Up per Bou ndn Con dition Plausible Upper Bound Condition IF the task isof a state which isa m k isof a state whichis amember of a force exp lanatio expla nation Thessential_go state is?O 2 T he stateis ? O2 ds_or_m ?O 4F IS strateg ically_e ods_or_m ateriel ?O4Force IS strategi cally _essential_goo ateriel ?O 1 IS ?O1 IS orce Id entify the strate gic COG ca ndidates with respect t o th e Identify thestrateg ic C OG can didateswith respec t tothe ?O 2 has_as_industrial_factor ?O2 has_as _ind ustrial_factor O3 Theforce is?O 1 ?O3 T he force is ?O1 ? 2 IS Force ?O2 IS F orce in dustrial civilization of a state which isa m em ber of aforc e ?O ind ustrial c ivilizati on of a state wh ich is a m em ber of a fo rce ?O 3 is _a_m ajor_g enerator_of ?O4 ?O3 is_a_m ajor_gen era tor_of ?O4 Plausible Upper Bound Condition Plausib le U pper Bound Co ndition IF thetas k is IF the task is explanatio n has_as_industrial _fac tor ? O3 has_as_ind ustrial_factor ?O3 T he state is ?O2 Th e state is ?O2 ?O 4 Force ISexplanation strateg icall y _essen tial_go ods_or_m aterie l strategically_essenti a l_goods_or_m ateriel ?O 1 IS ?O1 ?O4 IS IS Forc e Identif y the strategic C OGcandidateswith re spe ct to the ?O Iden tify the strategi c COG cand idates withrespect to the ?O2 has_as_industrial_factor ?O3 ?O 2 h as_as_industri l_factor ?O3 ?O3 IS Industrial_fa ctor 3 IS Ind ustrial_factor T he force is ?O1 Th e fo rcea is ?O1 ?O 2 IS r_generato Force IS Forc e ind ust rndi ial civilization r of a fo rce ?O2 industrial civiliz atio n of a statewhich is a m em ber of aforce ?O3 is_a_m ajor_generator_of ? O4 ?O 3 is_a_m ajor_generator_o f ?O4 is_a_majo r_of ?O4 is_a_m ajor_ge nerator_of ?O4 Pla usib le U pper Bound Co tion Plaus ible Up per Bou nd Con dition IF the task isof a state whichis amembe IF the task is nation exp lanatio n has_as_ind ustrial_factor ?O3 h as_as_i n du stria l_factor ?O3 Th e state is ? O2 The state is ?O2 ISexpla strategically _essent i al_g oo ds_o r_m ateriel ?O IS strateg ically_e ssential_go ods_or_m aterie l dates with respect t 4 IS Strategical l4 y_esse ntial_goods_o_materiel ?O4 IS Strategically ood s_o _m ateriel ?O 1 I?O4 S _essential_g Force ?O1 IS F orce Iden tify thestrategic C OG cand idateswithrespect ?O to the Id entify the strate gic3 COG ca ndi o th e ?O2 has_as_ind ustrial_factor O3 ?O 2 has_a l_factor ?O 3 IS Ind ustrial_factor ?O3 IS Indu strial_facto r s_industria Th e force is ? O1 ? Theforce is ?O1 ?O ?O 2 IS Force IS F orce industri a l civiliz atio no fa s tatewh ich is amem ber of a force ?O2 dustrial civilization of a state which isa m em ber of aforc e ?O3 is_a_m ajor_gen erator_of ?O4 ?O 3 is_a_ major_g enerato ?O4 is_ a_m ajor_ge nerator_of ?O4 is_a_m ajor_gen erator_o f in ?O4 Plausibl e Up per Condition Plausible Upper Bound Condition Plau sible Lo wer Bo und Con diti o n Bound IF the task is PlausibleLower Bound Condition IF ther_of tas k is explanatio n explanation h as_as_indu strial_factor ?O3 has_as_industrial _ fac tor ?O3 The state is ?O2 T he state isriel ?O2 ?O4 F IS strategically_essential_goods_or_m ateriel ?O 4Force ISsen strateg ically_essential_goods_or_mat eriel IS Strategically_esse ntial_goods_o_ma teriel IS Stra tegically_es tial_ goo ds_o_m ate IS orc e ?O 1 Industrial_factor IS ? O1 ?O IS 4 Anglo_allies ?O1 ?O4 IS An glo_allies_1943 Id entify t h strategic COG cand idates with respect to th e Identify strategic C OGcandidateswith re spect to the ?O 2 h as_as_industrial_factor ?O3 ?O2 has_as _in dustrial_fac tor ?O3 ?O 3 I?O1 S _1943 Indu strial_factor ?O3 IS The force i se ?O1 T he forcethe is ?O1 ?O2 IS F orc e 2 IS Force ? O2 IS US_1943 ?O2 ISaforc US _1943 in dustrial civilization e ?O ind ustrial civilization r of a fo rce ?O 3 is _a_m ajor_g enerator_o fk ?O4 ?O3 is_a_m ajo r_generator_of ?O4 is_a_ma jor_ generator_of ?O4 is_a_m ajo r_generato r_of ?O4 Plaus ible Upper Bou nd Condition Plausib le U pper Bound Co ndition Plausibledustrial_factor Lower Bou nd Condition IF the tas isof a state which isa member of Plausib le L ower Bound Co ndition IF the task isof a state whichis amembe exp lanation expla nation has_as_in dustria l_factor ? O3 has_as_ind ustrial_factor ?O3 has_as_in ? O3 has _as_industrial_ factor ?O3 The state is ?O2 Th e state is ? O2 ds_or_m ?O 4 ISstrateg ically_e ssential_go ods_or_m aterie l ates with respect IS strategic ally_essential_goo ateriel ?O IS Strategically_essential _ goods_o_m ateriel ?O4 IS lo_allie Strate gically_essential_g ood s_o _m ateriel ?O1 InIS Force ?O 1 I?O4 S Force Id entify the st r ategic COG candid to the Iden tify the strategi c COG cand idates withrespect to the ? O1Industrial_capacity_of_U IS 4 An glo _allies_1943 ?O1Ind IS Ang s_19 43 ?O2 has_as_in dus trial _ fac tor ?O3 ?O2 has_as_ind ustrial_factor O3 ?O3 IS du strial_factor ?O 3 IS Ind ustrial_factor ? O3 IS S_1943 ?O3 IS ust r ial_cap acity_o f_US_1943 The force ?O1 Th e force is ? O1 ? ?O2 IS Force ?O 2 IS Force IFis the isof astate which is a memb IF task is ? O2 maj IS or_g US _1943 ?O2 IS US_1943 in dustrial civitask lization er of a force industri a lthe civiliz n of a s tatewhich is a m em ber of aforce ?O3 is_a_m r_generator_of ?O4 ?O3 is_a_m ajor_gen erator_of ?O4 is _a_m ajor_g enerato r_o f ajo ? O4 is_ a_m ajor_ge nerator_of ?O4 is_a_ enerato r_o f Pla ?O4 is_a_m ajor_generator_of ?O4 usible U pper Bound Condition Plausible Up per Bo und Con diti on atio Plausible Lower Bound Condition Plau sible Lo wer Bou nd Con dition expla nation exp lanatio n has_as_ind ustrial_fac tor ?O3 h as_ as_indu strial_facto r ?O3 Iden tify the strategic C OG candidates with respect to the Id entify the strategic COG ca ndidates with respect to th e has _as_industrial_ factor ?O3 has_as_industri a l_factor ?O3 The state is ?O2 The state is ?O2 ?O4 IS strategically_essen tial_ goo ds_ or_m ateriel ?O4 I S strategically_essential_goods_or_m ateriel ?O4 IS S trategically_es sential_ goo ds_o_m ateriel ?O 4 IS Strategically_esse ntial_goods_o_materiel ? O4 IS War_ materiel_and_transports_of_US_1943 ?O4 IS Wa r_m ateriel_ and _transp orts _194 3 ?O 1 Industrial IS Force IS_of_US F orce ?O1 IS Ang lo_allies_1943 ? O1 IS Anglo_al lies _1943 ?O2 has_as _ind ustrial_factor ? O3 ?O 2 h as_as_industrial_factor ?O3 IS _factor ?O 3 I?O1 S Indu stri a l_facto r The The industrial force iscivili ?O1zatio no fa s tatewh ichis amem of a fo rce in dustrial force iscivilization ?O1 ?O3 of a state which isa m em ber of aforc e ? O3 IS In dustrial_cap aci ty_o f_U S_1943 ?O3 ISber Industrial_capacity _of_US_1943 ?O 2 IS Force ?O2 IS F orce ?O2 IS US_1943 ? O2_major_ IS US_1943 ?O3 is_a_m ajor_gen era tor_of ?O4 ?O 3 is _a_m ajor_g enerator_o ?O4 is_ a_m ajo r_generator_of ?O4 is_a_m ajor_gen erator_of ?O4 THEN TH EN Th e state is ?O2 The state isf?O2 is_a_m ajor_generator_of ? O4 is_a gen erator_o f Plaus ?O4 Plausib le U pper Bo und Co ndition ible Upper Bou nd Condition Plau sible Lower Bound Condition Plausible Lower Bou nd Condition n explanation has_as_ind ustrial_factor ?O3 has_a s_industrial_factor ? O3 tial_go has_as_ industrial_facto r IS ?O3 has_as_in dustrial_factor O3 ?O4 ISexplanatio strategically_essenti a l_goods_or_m ateriel ?O 4 IS strateg ically_e ssen ods_or_m aterie l ?O4 ISilo_allies_19 Strate gically_essenti al_g ood s_o _ma teri el ?O 4 IS Stra tegicall y _essen tial_goo ds_o_m ate Th e force i s ?O1 The force isriel ?O1 Conclud e that an econom ic factor s_transp a strategic COG nclude that an eco nom ic factor is_allies_1943 a strategic C OG ?O1 Force ?O1 IS? Force ? O4 IS War_m orts _of_US _1943 ?O4 IS War_materi e l_and_tran spo rts_of_U S_1943 ?O1 ateriel_and IS A ng 43 ?O1 IS An glo ?O 2 h as_as_indu strial_facto r ?O3 Co ?O2 has_as_in dus trial_fac tor ?O3 ?O 3 IS Ind ustrial_factor ?O3 IS Indu strial_fa ctor ?O3 IS Ind ustrial_capacity_of_US_1943 ? O3 IS In dustrial_capacity_of_U S_1943 candidate fo r a state wh ich is aU mem be r of a fo rceForce atefor a state whic h isa m em ber of aforc ?O2 IS ?O2 IS e Force ?O2 IS S_1943 ?O2 IS US _1943 exp lanation expla nation ?O 3 is _a_ major_gen erator_of ?O4 candid ?O3 is_a_m ajo r_generator_of is_a_m ajor_generator_of ?O4 is_a_major_generato r_o f ?O4 is_a_major_gen ?O4 is_a_ major_g enerato r_o f ? O4 TH EN is ?O2 Plaus ible Up per Bou nd Con dition THEN Pla usib le U pper Bound Co ndi tion ?O4 Plausibleerator_of Lo wer Bou nd Con dition Plausib le L ow er B oun d Co ndi tustrial_fac ion The state has_as_in dustrial_factor ? O3 has _as_industrial_ factor ?O3 T he stateis ? O2 h as_a s_indu strial_factor ? O3ntial_goods_or_materiel has_as_ind tor ?O3 ?O2 has_as_in dus trial_fac tor ?O3 ?O2 has_as_ind ustrial_factor ? O3 ateriel ?O 4 IS strateg ically_esse ?O4 IS strategic ally_essential_ goo ds_ or_m ?O 4 IS Strategical l y_esse ntial_goods_o_materiel ?O4 IS Strategically _essential_goo ds_o_m ateriel IS Wa r_m ateriel_and_tran rts_of_ US_194 3 ? O4 IS materiel_and_transports_of_US_1943 ?O1 ISCOG F orce ?O 1 IS COG Force Co nclu?O4 de t an eco nom ic isa strategic Con clude an War_ econ omic factor is aallie strategic ? O1 ISfactor Anglo _all ispo es_1943 ?O1 IS lo_ The force istha ?O1 he force isthat ? O1 ?O3 IS Industrial_capacity_of_US_1943 ?O3 IS Ind ust rAng ial_cap acity_o f_US_1943 ?O3 IS Indu strial_factor ?O3 IS s_1943 Industrial _factor ?O3 is_a_m ajo r_g enerator_of ?O4 T ?O3 is_a_m ajor_gen era tor_of ?O4 ?O2 IS e F orce ?O 2a fo IS Force candid atefor a state whic h isaUS_1943 m em ber of aforc can didate for astate w hich is aUS_1943 m emb er of rce ? O2 IS ?O2 IS The eco nom ic facto r is ?O3 T he economi c factor is ?O3 is_a_major_ generator_o f ?O4 is_ a_m ajo r_generator_of ?O4 is_a_maj o r_generato r_o f Pla ?O4 is_a_m ajor_generator_of ?O4 TH EN usible Upp er B oun dC ond ition THEN Plausible U pper Bo und Con dition Pla usi bledustrial_factor Lower Bound Condition Plau sible Lower Bo und Co ndition ?O4 ISstrategically_essen tial_ goo ds_ or_m ateriel ?O4 IS strategically_essential_goods_or_m ateriel has_as_industrial _ fac tor ?O3 h as_as_indu strial_factor ?O3 T he state is ?O2 has_as_in ?O3 Th e state is ?O 2 War_m has_as_industrial_factor ?O3 ?O4 IS W ar_materiel_and_transports_o f_U S_1943 ds_o_m ?O4 IS ateriel_ and _transp orts _of_US _1943 ?O4 IS Strategi c ally_essential_goo ateriel ?O4 IS Strate gically_essenti al_g ood s_o _ma teriel Co nclude an econ om ic factor is a strateg ic C OG Conclude an econom ic fact o r is COG ?O 1 Industrial_factor IS Fo rce ?O1 IS F orce ?O1 IS An glo_allies_1943 ?O1Ind IS Ang lo_allies T he force isthat ?O1 Th e force isthat ?O 1 ?O 3a strategic I S _1943 Ind ustrial_factor ?O3 IS ? O3 IS In dustrial _capacity_o f_U S_1943 ?O3 IS ustrial_capacity _of_US_1943 Plausib le U pper Bo undCo ndition Plaus ible Up perBou nd Condition can did atefor astate which isaUS m em ber o f2a force candidate fo r a state wh ich is aUS_1943 mem ber of a force ?O IS Fo rce ?O2 IS F orce ?O2 _1943 ?O2 IS is_a_ma jor_generator_of ?O4 is_a_m ajo r_generato r_of ?O4 T HEN TH EN is_a_m ajor_g enerato r_of ? O4 is_a _major_gen ?O 4 T he econom ic factor is ?O3IS Th e econom icfactor is ?O3 Plausible Lower Bo und Co ndition Plausibleerator_of Lo wer Bou nd Con dition ?O1 IS Force factor ?O3 ?O1 IS? Force has_as_industrial_ has_a s_industrial_factor ? O3 Th e stateis ? O2 Thestate is ?O2 has _as_ industrial_ factor ?O3 has_as_in du strial_factor O3 ?O4 IS Strate gically_essential_g ood s_o _m ateriel ?O 4 I S Strategical l y_essential _ goods_o_materiel C on clud e th at an ec ono mic factor i s a strategic COG Co nclude that an eco nom ic factor is a strategic C OG ? O4 IS War_m ateriel_and_transports_of_US _1943 ?O4 IS War_m ateriel_and_tran spo rts_of_US_ 1943 ?O1Ind IS Ang lo_allies_1943 ? O1Industrial_capacity_of_US_ IS Anglo ies_1943 ?O2 Industrial_ IS Force ?O2 Force Th e forceis ? O1 ?O 3 acity_of_US_1943 IS factor Theforce is ?O1 ?O3_all IS InIS du strial_fa ctor ?O3 IS ustrial_cap ?O3 IS 1943 cand idate fo rfactor astate which is aUS_1943 m emberis_a_m of a fo rce ndu candid atefor a state whic h isaUS_1943 m em beris_a_major_generato of aforc e ?O2 ? O2 IS h as_as_i strial_factor _ tor ajor_genera tor_of ?O4 ?O3 r_o ffac ?O4 ?O3 Th e econo mic is ?O3IS The eco nom ic facto r is ?O3 is_a_m ajor_generator_of ?O4 is_a_ maj or_generato r_o fhas_as_industrial ?O4 THEN THEN Plau sibleLo werBou nd Con dition Plausi ble dustrial_factor Lower Bound Condition The state is?O 2 T he stateis ? O2 has_as_indu strial_facto r ?O3 has_as_in ?O3 ?O3 IS Indu strial_factor ?O3 ISsports_of_US_1943 Industrial _factor ?O 4 Anglo_al IS Strateg ically_essential_go _o_ materiel ?O4 IS Strategically _essential_goo ds_o_m ateriel ?O4 IS Wa r_m ateriel_ and _transp orts _of_ US_194 3 ods ?O4 IS War_materiel_and_tran Conclude t an eco nom ic is a strategic COG Conclude an econ omi c fa ctor is a strategic C OG ? O1 ISfactor lies _1943 ?O1 IS An glo_allies_1943 The force istha ?O 1 T he force is that ?O1 ?O3 IS Industrial_capacity_of_US_ 1943 ? O3 IS In dustrial_capacity_o f_U S_1943 r_of ?O4 is_a_ma jor_ generator_o f ?O4 is_a_m ajo r_generato candi d atefor a state which is aUS_1943 m em ber of aforce can did atefor astate w hich isaUS m emb er of a force ? O2 IS ?O2 IS _1943 The eco nom icfactor is ?O3 he econom ic factor is ?O3 TH EN THEN is _a_major_gen erator_o f ?O4 is_a_m ajor_g enerato ? O4 Plausible Low er B ou nd C ond itio n Plau sible Lower Bo und Co ndition al_g ?O4 IS Strategi c ally_essential_goo ds_o_m ateriel T ?O4 ISr_of Strate gically_essenti ood s_o _ma teriel T hestate is ?O2 Th e state is ?O2 has_as_in dustrial_factor O3 has_as_ industrial_facto r ?O3 Co nclude that an econ om ic factor isa strateg ic? C OG Con clud e that a n econom ic factor is astrategic COG _1943 ?O4 IS War_materi e l_and_tran spo rts_of_US_1943 ? O4 IS War_m ateriel_and_transports_of_US ?O1 IS Anglo_allies_1943 ?O1 IS Ang lo_allies_1943 T heforce is ?O1 Th e fo rce is ?O1 ? O3 IS In dustrial_capacity_of_U S_1943 ?O3 IS Ind ustrial_cap acity_of_US_1943 candid atefor a state whic h Plausible isaUS_19 m em Lower ber f Bound aforc eCo ndition candidate fo r astate wh ich Plausible is a mem Lo be wer r of Bou a fo rce ndCon dition ?O2 IS 43 o ?O2 US_1943 T heeconom ic facto r is ?O3 Th e econom icfac tor is ?O3IS is_a_ major_g enerato r_o f ? O4 is_a_m ajor_generator_of ?O4 THEN THEN Th e stateis ? O2 ?O1 IS Ang lo_allies_1943 The state is ?O2 ? O1 IS Anglo _allies_1943 has_as _ind ustrial_factor ?O3 has_as_indu strial_factor ?O3 ? O4 IS War_ materiel_and_transports_of_US_1943 ?O4 IS r_m ateriel_ and _transpo rts_of_ US_194 3 Con clud e that a n econom ic factor is astrategic COG 43 Conclude t an Wa eco nom ic is a strategic COG Th e force is ? O1 The force istha ?O1 ?O2 Ind IS US_1943 ? O2 ISfactor US_1943 ?O3 IS ustrial_capacity_o f_US_19 ?O3 IS Industri al_capacity_of_US_ 1943 can didate for astate which is amem ber of a fo rcer ?O3 candi d atefor a state which is a m em ber of aforce ? stri al_facto has_as_in dustrial_factor Th e economi c factor is ?O3has_as_indu The eco nom icfactor is is_a_m ajor_generator_of ?O 4 is?O3 _a_ major_gen erator_o f ?O4 O3 THEN TH EN Thestate is ?O2 T hestate is ?O2 ?O3 IS Industrial_capacity_of_US_1943 ? O3 IS om In dustrial_capacity_of_U S_1943 ?O4 IS War_m ate riel_and _transports_of_US_1943 ?O4 IS War_materi el_and_tran spo rts_of_US_1943 Conclude that an econom ic fact o r is a strategic COG Co nclude that an econ ic factor is a strateg ic C OG Theforce is ?O1 T heforce is ?O1 is_a_major_gen erator_of ?O4 enerato r_of ? O4 candidate fo r a state wh ich is amem ber of a force can did atefor astateis_a_m whic h ajor_g isa m em ber o f aforc e THEN THEN Theecono micfactor is?O3 T heeconom ic facto r IS is?O3 materiel_and_transports_of_US_1943 ?O4 IS War_materiel_and_tran spo rts_of_ US_1943 ? O4 Thestate is ?O2 Th e state is ? O2 War_ Concl ude ic facto r is a st rategic C OG Con clud e that n econom icfactor is astrategic COG The force isthat ?O1an econom Th e force is ? O1a TH EN THEN candid ate for a state em berof aforce candidate fo rfactor astate ich is amem be r of a fo rce The eco nom ic facto r iswhich ?O3 isa m Th e econo mic iswh ?O3 Con clude om ic factor is astrateg ic C OG Conclude t an eco nom icfact or isa strategicCOG The state isthat ?O2an econ The state istha ?O2 can didate astate which is amem ber o f a force candidate a statewhich is amem ber of aforce The force isfor ?O1 The force is for ?O1 Th e state is ?O2 The state is ?O2 The econ om ic factor is ? O3 Theeco no mic factor is?O3 Th e force is ?O1 Theforce is ?O1 Th e econo micfactor is ?O3 Theeco nom ic facto r is?O3 Problem solving rules (learned) “c omb in ed an d jo int o pera tio ns” . In the following we will demonstrate this teaching process.

In this illustration we will use the World War II invasion of the island of Okinawa by the US forces. as has been illustrated before. . by using the Scenario Elicitation tool.1. Specifying a training scenario First the expert has to specify a training scenario.

The internal representation may be viewed using the Association Browser . Specifying a training scenario Based on the information elicited from the expert Disciple creates a formal representation of the scenario.1.

I need to Identify and test a strategic COG candidate for the Okinawa_1945 scenario What kind of scenario is Okinawa_1945? Okinawa_1945 is a major theater of war scenario Therefore I need to Identify and test a strategic COG candidate for Okinawa_1945 which is a major theater of war scenario Which is an opposing force in the Okinawa_1945 scenario? Japan_1945 Therefore I need to Identify and test a strategic COG candidate for Japan_1945 … . Modeling expert’s reasoning Then the expert has to express his reasoning in center of gravity identification and testing for the Okinawa scenario.2. by using the task reduction paradigm.

Modeling expert’s reasoning The expert expresses his reasoning process in English.2. using the Modeling tool of Disciple: The modeling tool shows: The description of the node selected from the reasoning tree. The entire reasoning tree. .

Modeling expert’s reasoning The first steps of the reasoning process show how the expert has identified Emperor Hirohito as a strategic center of gravity candidate for Japan. .2. To reduce the current task the expert needs some information that is identified by asking a question. We will now show how the expert uses the Modeling tool to express his reasoning of testing whether Emperor Hirohito is a viable candidate.

2. . Modeling expert’s reasoning Then the expert must provide an answer to this question To reduce the current task the expert needs some information that is identified by asking a question.

Modeling expert’s reasoning Then the expert must provide an answer to this question .2.

2. Modeling expert’s reasoning Then the expert must provide an answer to this question .

Modeling expert’s reasoning .2.

2. . Modeling expert’s reasoning The question and its answer lead to this reduction of the top task.

2. Modeling expert’s reasoning

2. Modeling expert’s reasoning

2. Modeling expert’s reasoning

2. Modeling expert’s reasoning .

2. Modeling expert’s reasoning .

Modeling expert’s reasoning .2.

2. Modeling expert’s reasoning .

Modeling expert’s reasoning .2.

.2. Modeling expert’s reasoning This process continues in the same way until the expert has enough information to either eliminate or not eliminate the COG candidate.

2. Modeling expert’s reasoning .

Task and Rule Learning We will now demonstrate how Disciple learns general tasks and rules from the expert’s reasoning.3. .

3. Task and Rule Learning First the expert and Disciple have to formalize the English statements of the tasks. . This is done in the Formalization mode.

Task and Rule Learning In the formalization mode the tool shows: The modeling in English The formalized tasks .3.

3. Task and Rule Learning Disciple will propose a formalization of the task .

Task and Rule Learning Disciple will propose a formalization of the task The expert may accept it or he may edit it .3.

3. which will learn a rule from it . Task and Rule Learning After the tasks are formalized the expert may explain the example to Disciple.

. Task and Rule Learning The Rule Learning tool helps the expert to explain the example to Disciple The tool shows the English form of the example Disciple uses analogical reasoning and other heuristics to propose plausible explanations pieces that justify the task reduction step.3.

Task and Rule Learning The expert selects those explanation pieces that correspond to the meaning of the question-answer pair from the task reduction example .3.

3. Task and Rule Learning The expert selects those explanation pieces that correspond to the meaning of the question-answer pair from the task reduction example .

3. Task and Rule Learning .

3. Task and Rule Learning .

.3. Task and Rule Learning The expert may direct Disciple to generate explanation pieces related to certain objects from the example When the expert is satisfied with the identified explanation he can click on “End learning”. Disciple will then create a general rule corresponding to this example and its explanation.

3. Notice that it has a plausible upper bound condition and a plausible lower bound condition. Task and Rule Learning This is the general task reduction rule learned by Disciple. During rule refinement the two conditions will converge toward one another. . ultimately leading to a rule with a single condition.

Task and Rule Learning This is the general task reduction rule learned by Disciple. The informal structure is used in the communication with the user.3. which is used in problem solving and learning. In addition to the formal structure of the rule. Disciple maintains also an informal structure of the rule. .

Task and Rule Learning After learning a rule from the current task reduction step Disciple returns to the formalization mode.3. .

Disciple will learn another rule from the next task reduction step: . Task and Rule Learning Following the same procedure.3.

3. Task and Rule Learning .

3. Task and Rule Learning .

3. Task and Rule Learning .

3. Task and Rule Learning .

3. Disciple will learn another rule from the last task reduction step: . Task and Rule Learning Following the same procedure.

Rule Refinement Disciple uses the partially learned rules in problem solving and refines them based on expert’s feedback.4. .

. The “?” preceding a reasoning step indicates that the step is less plausible.4. Rule Refinement Disciple uses the rules learned from the expert’s identification and testing of Emperor Hirohito to identify and test President Truman as a US strategic center of gravity candidate. because it is based on the plausible upper bound condition of a learned rule.

4. Rule Refinement The expert has to examine this step and has to indicate whether it is: correct but incompletely explained by selecting “Explain Example” incorrect correct and completely explained by selecting “Correct Example” by selecting “Incorrect Example” .

4. to cover this example. . Rule Refinement The expert has indicated that the reasoning step is correct and Disciple has generalized the plausible lower bound condition of the corresponding rule.

. as has been illustrated before. • In all the cases the rule is automatically refined by Disciple. Rule Refinement • In the case of a correct but incompletely explained example the expert may further explain the example. • For an incorrect example the expert has to help Disciple to identify an explanation of why the example is incorrect.4.

it may not contain the knowledge pieces that explain why a certain rule example is incorrect. . In such a case the expert has two options: • End Rule Refinement (the incorrect example is kept as a negative exception of the rule. the knowledge base will be extended latter.5. Exception Handling Because the object ontology of Disciple is incomplete. We will demonstrate the handling of exceptions by the subject matter expert. • Invoke Exception Handling (the expert attempts to extend the object ontology by himself ). with the help of the knowledge engineer).

The expert and Disciple attempt to find an explanation of why this reasoning step is incorrect. Exception Handling Disciple generates a task reduction step which is rejected by the expert.5. However. there is no ontology piece that explains why this is an incorrect reasoning step. and Disciple will keep the incorrect reasoning step as a negative exception to the rule that generated it. Or. The expert may end Rule Refining. the expert may start Exception Handling .

Exception Handling The Exception Handling module shows: The incorrect reasoning step (the negative exception of the rule) An object from the positive example of The the rule left and right buttons used to view the other objects from the rule’s examples The corresponding object from the negative exception of the rule .5.

g.5. or new objects) This is a more detailed description of the candidate highlighted above The negative exception may be removed by providing a value for the property “is_mission_oriented” of Military_of_Japan_1945 which is different from the value for Military_of_US_1945 The expert selects this candidate to remove the exception . new features. Exception Handling Disciple generates an ordered list of candidate hypotheses that may distinguish between the positive example and the negative exception of the rule These candidate hypotheses are new ontology pieces (e. new feature values.

Exception Handling Disciple guides the expert in the definition of new knowledge about the selected candidate .5.

5. Exception Handling The expert defines the value of the property “is_mission_oriented” for “Military_of_Japan_1945” This property value differentiates between the object from the positive example (Military_of_US_1945) and the object from the negative exception (Military_of_Japan_1945) in absolute terms and disregard of preserving the lives of its soldiers .

5. Exception Handling Disciple refines the object ontology and the rule with the elicited feature value .

5. Exception Handling .

Problem Solving The automated problem solver presented at the beginning of this demonstration was customized for the Center of Gravity domain. .6.

Solution Justification .6. Problem Solving The Disciple shell also contains a general automated problem solver.

Tools for the knowledge engineer The Disciple shell contains also a suite of knowledge base development tools for the knowledge engineer: •Ontology browsers and editors to develop the object ontology. . •More complex exception handling tools for the knowledge engineer. •Script editor to define elicitation scripts. •Ontology import tools to import ontological knowledge for reusable knowledge repositories. They will be briefly demonstrated in the following. such as CYC.

Ontology Development Tools Object Viewer Object Browser .1.

1. Ontology Development Tools Object Editor .

Ontology Development Tools Script Editor Object Browser .1.

•Mixed-initiative import from the intermediate Disciple ontology into the destination Disciple ontology.2. such as CYC. Ontology import tools Ontology import tools perform a three step import process: •Mixed-initiative retrieval of potentially relevant ontological knowledge from external knowledge repositories. The following is a brief demonstration of ontology import from CYC. •Automatic translation of the retrieved ontological knowledge into an intermediate Disciple ontology. .

Ontology import tools The expert looks for relevant ontological knowledge in CYC The expert specifies a searching string .2.

2. Ontology import tools The expert looks for relevant ontological knowledge in CYC The expert specifies a search string CYC returns all the matching names The expert chooses to import “NATO” and the knowledge related to it .

Ontology import tools The knowledge retrieved from CYC is automatically translated into an intermediate Disciple ontology.2. .

Ontology import tools The expert and the knowledge engineer import knowledge from the intermediate Disciple ontology into the destination Disciple ontology The objects selected to be imported from the intermediate Disciple ontology into the destination Disciple ontology Fragment of the intermediate Disciple ontology translated from CYC The expert and the knowledge engineer select the entities to be imported into the destination Disciple ontology .2.

2. Ontology import tools The expert and the knowledge engineer import knowledge from the intermediate Disciple ontology into the destination Disciple ontology “NATO” is being imported from the intermediate Disciple ontology into the destination Disciple ontology The user specifies the additional knowledge to be imported together with “NATO” The selected superconcepts of “NATO” will also be imported The selected features of “NATO” will also be imported .

its super-concepts. Ontology import tools The ontology fragment represented by “NATO”. while a + denotes an existing term The user specifies the position of the imported concepts in the destination Disciple ontology .2. and its features is copied into the destination Disciple ontology A + denotes a term that does not yet exist in the ontology.

2. Ontology import tools The user updates the feature values .

. Ontology import tools “NATO” is marked as imported in the intermediate Disciple ontology The import of “NATO” has led to the partial import of these additional objects. Disciple suggests the user to complete their import.2.