[20220119] INFOMAIS - Adaptive interactive

systems - 1-GSH - UITHOF

Course: Adaptive interactive systems (INFOMAIS)

Retake Exam for INFOMAIS – Adaptive Interactive Systems

for academic year 2021/22, period 1, 19 January 2022

Regular exam time: 90 minutes

The exam includes multiple choice questions, combined questions of multiple choice
and open text questions for explaining the answer, and open text questions.
Answer all questions (including sub-questions).
With the multiple choice questions, one or more of the answer options may be correct.
Each part of combined questions is graded separately.
Answer the open text questions as precisely as you can.
You can freely navigate between the questions as you like (thus, you can also navigate
back).

Number of questions: 9

You can score a total of 48 points for this exam, you need 25.01 points to pass
the exam.

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 1 of 12
41097-69108 1-GSH - UITHOF
 1 Which statements are true?
2 pt.
Tick all statements that are true.
For each correctly answered item, you receive one point (+1 point). For each incorrect
answered item, one point is deducted (-1 point). You can receive a maximum of 2 points. No
negative points will be given in the answer's total points; so you can reach between 0 and 2
points in total.

a. User models are dynamic and may evolve over time.

b. With a passive context-aware system the user remains passive while the system
monitors the environment and acts autonomously.

c. If a platform’s users want “more of the same” (e.g., similar items), then a
content-based filtering approach is typically more suitable than a collaborative
filtering approach.

d. A record of a user’s clicking behavior on a platform is a suitable input for an explicit

user model.

e. A context-aware system offers appropriate options to users but does not act
autonomously.

f. None of the above options is correct.

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 2 of 12
41097-69108 1-GSH - UITHOF
 2 Harry and Sally have developed two new algorithmic approaches for recommender
systems: AlgoHarry and AlgoSally. They want to know how their approaches perform in
comparison. They conduct an offline evaluation following a leave-n-out approach. They
choose Root Mean Square Error (RMSE) as their metric of choice.
Besides being interested in how well the approaches work overall with RMSE, they are also
interested in the performances for specific user groups. Thus, they split their user sample in
three equally sized subgroups based on age (young, middle-aged, old) and also calculate
the RMSE for each user group separately.
You find the results of their offline evaluation in the table below.

Table with Root Mean Square Error (RMSE) values

AlgoHarry AlgoSally

overall user 6.07 4.72

sample

young users 7.02 8.55

middle-aged 5.31 6.22

users

old users 5.93 4.11

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 3 of 12
41097-69108 1-GSH - UITHOF
 2 pt. a. Considering these results, which statements are true?
Tick all statements that are correct.
For each correctly answered item, you receive one point (+1 point). For each
incorrect answered item, one point is deducted (-1 point). You can receive a
maximum of 2 points. No negative points will be given in the answer's total points; so
you can reach between 0 and 2 points in total.

a. According to the RMSE results, AlgoHarry works better for old users than
AlgoSally works for the overall user sample.

b. For young users, AlgoHarry has a lower error rate than AlgoSally.

c. Considering the RMSE values for old users, AlgoHarry outperforms AlgoSally.

d. According to the RMSE results, AlgoHarry works better for old users than
AlgoSally works for middle-aged users.

e. The RMSE values suggest that middle-aged users would be better off with
AlgoSally.

f. None of the above statements apply.

2 pt. b. Given the RMSE results given in the table, Harry and Sally do not come to an
agreement which algorithm they should choose to employ in their real-life
recommender system. They reach out to you to help them outlining the advantages
(pros) and disadvantages (cons) of both algorithms given the RMSE results.
Give an explanation about the advantages (pros) and disadvantages (cons) of
the two algorithms based on the results presented in the table above.
(about 3-4 sentences) 2P

1 pt. c. Give a clear advice which algorithm to employ in practice (AlgoHarry or AlgoSally).
And give a concise and reasonable explanation for this choice. (about 1-2 sentences)
1P

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 4 of 12
41097-69108 1-GSH - UITHOF
 Case text 1

JobMatch
Same text for all questions related to JobMatch!
JobMatch is a platform that recommends jobs to candidates (job2candiate) as well as
candidates to potential jobs (candidate2job). They solve this two-way matching task
for around two million candidate users per year. JobMatch offers jobs for and from
236,100 companies. For example, John has registered on the JobMatch platform to
find a new job. He is shown five fitting job offers from three different companies based
on his user profile. On the other hand, SuperOrg needs ten new employees for a
project. They, thus, insert a description for each job, and the system shows them a list
of fitting candidates for this description. Both sides can ignore a recommendation or
contact the respective other side within the system.

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 5 of 12
41097-69108 1-GSH - UITHOF
 3 For this question, you can use case text 1 from page 5

1 pt. a. Please read the case description on the left.

Which recommendation approach is best suited for the above-described scenario
both for candidate2job and for job2candidate, if only one approach can be used? 1PT

a. Collaborative

b. Knowledge-based

c. Content-based

2 pt. b. Explain this choice in detail! (about 3-4 sentences) 2PT

1 pt. c. They now want to switch to a hybrid solution. Which hybridization design is ideal in
this scenario? 1PT

a. Weighted

b. Switching

c. Meta-level

d. Feature Combination

e. Cascade

2 pt. d. Explain this choice in detail! (about 3-4 sentences) 2PT

1 pt. e. Which issue common in recommender systems do you expect in the

above-described scenario? 1PT

a. Privacy and trust

b. Cold start and latency problems

c. Diversity of recommendations

d. Changing user interests

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 6 of 12
41097-69108 1-GSH - UITHOF
 4 For this question, you can use case text 1 from page 5

Please read the case description on the left.

JobMatch has already tried to solve their task using a somehow suboptimal algorithm. Here
is an example of their approach and dataset:
They used similarity in tags between a candidate and a job (Content-based, Dice
Coefficient).

Dice Coefficient:

The tags are the following:

CANDIDATES:
Alice: old, master, logic, mathematics
Jane: female, language, diploma, dutch
Tom: male, bartender, driver, young
JOBS:
Teaching: education, master, diploma, dutch
Cleaning: young, old, strong, driver
Managing: old, mathematics, diploma, leadership
1 pt. a. Given this information, which candidate would the system recommend for the
teaching job? 1PT

a. Alice

b. Jane

c. Tom

5 pt. b. Explain your recommendation by naming the relevant steps of the algorithm and the
predicted matching scores! 5PT

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 7 of 12
41097-69108 1-GSH - UITHOF
 5 For this question, you can use case text 1 from page 5

Please read the case description on the left.

1 pt. a. The company JobMatch has the idea to improve their recommender system by
providing explanations to the users. Here is an example of the explanation provided:
This job offer matches you because your user data contains the following keywords:

Which metric did the company focus on when creating this explanation? 1PT

a. Efficiency

b. Trust

c. Persuasiveness

d. Satisfaction

e. Scrutability

f. Transparency

g. Effectiveness

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 8 of 12
41097-69108 1-GSH - UITHOF
 3 pt. b. Please define this explanation on the three dimensions of XAI elements below. Note
that each dimension pair is mutually exclusive (local XOR global), (post XOR ante),
and (specific XOR agnostic). You will receive 1 point for each correct answer and -1
point for each wrong answer. No negative points will be given in the answer's total
points, so you can reach between 0 and 3 points in total. 3PT

a. Global

b. Local

c. Post-Hoc

d. Ante-Hoc

e. Model-Specific

f. Model-Agnostic

2 pt. c. What problems do you see with the visual component of the explanations in terms of
effectiveness? (about 3-4 sentences) 2PT

6 For this question, you can use case text 1 from page 5

Please read the case description on the left.

After the epic failure of the explanation extension to their system, the company JobMatch
decides that they need a better systematic way of evaluating their system. You are brought
in as a research expert to conduct a proper evaluation. The target of the evaluation is an
adaptive explanation of a match towards both the candidate and the company.
3 pt. a. How would you evaluate the "Decide upon Adaptation" layer of the explanation
feature? Name the method used, describe how it generally works, and explain your
choice in detail! (about 3-4 sentences) 3 PT

4 pt. b. Describe in detail how you need to instantiate the chosen method. In your answer,
describe the requirements of the respective layer, of the explanation feature, and the
metrics you want to test in this layer. (about 5-7 sentences) 4PT

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 9 of 12
41097-69108 1-GSH - UITHOF
 Case text 2

Smart Home Scenario

Same case text for all questions related to the smart home scenario!
Kiara and Lucas bought a new house that is equipped with various sensors. When
they move in, they set up rules to automate their home using the “smart home app”.
The app can access the available sensors and can control applications or appliances
(e.g., switch on the lights) based on the configured rules.
The accessible sensors and appliances are:
▪ local time
▪ access to Kiara’s and Lucas’ digital calendar
▪ room thermostat
▪ light on/off per room
▪ washing machine on/off
▪ coffee machine in kitchen on/off
▪ smoke sensor in every room.

For appliances (those with “on/off”), the app can (i) sense whether it is on or off and
(ii) activate an on- or an off state.
(Note, for simplification, “off” is used as a synonym for turning off a machine and sleep
mode alike.)
In the following examples, you can use any of the available sensors and appliances
(which means only one, or several, or all).

For each example, create an IF-THEN-ELSE house rule. In your answer, you can
use full sentences or use pseudocode (whatever you prefer).

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 10 of 12
41097-69108 1-GSH - UITHOF
 7 For this question, you can use case text 2 from page 10
4 pt.
Please read the case description and instructions on the left.
Lucas wants that the coffee machine is already warmed up (from the machine’s sleep
mode) when he arrives at the kitchen for his morning coffee.
After 17:00, he does not drink coffee anymore. Thus, from 17:00 onwards until he gets up in
the morning, he won’t start using the coffee machine and it should turn off automatically.
Kiara does not drink coffee at all. Thus, when Lucas is away for attending a
2-day-conference (or longer), the coffee machine should remain in sleep mode.

8 For this question, you can use case text 2 from page 10
3 pt.
Please read the case description and instructions on the left.
In their prior home, Kiara was annoyed when the washing machine started spinning during
an online meeting for work or when she had friends over at her place in the evening.
Help her configure the app to avoid these incidences in the new house.

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 11 of 12
41097-69108 1-GSH - UITHOF
 9 Jordan and Ali work at the music streaming platform StreamSongs. As a team, they are in
charge of evaluating a new recommender approach to meet the company’s goal to provide
users with a diverse (i.e., broad) set of music.
The overall goal is to increase the diversity of the items that are recommended.
Addressing this overall goal, the evaluation design needs to address the following
evaluation objectives:
▪ Increasing the diversity of music genres (e.g., heavy metal, pop, jazz, folk)
▪ Increasing the diversity of language of the song lyrics
▪ Increasing the gender diversity of artists

At the same time, the user satisfaction should be increased or at least maintained. Jordan
and Ali define to measure user satisfaction in terms of “the average daily time a user
spends on the platform”.
You help Jordan and Ali to create a multi-method evaluation design (according to
Creswell) to appropriately address all constraints.

1 pt. a. Which multi-method evaluation design do you suggest for addressing this evaluation
scenario? Only mention the technical term for the multi-method design.
(Note, you only get a point for this part if it corresponds with your description in the
question below.) 1P

7 pt. b. Explain the sequence of steps that need to be done.

Explicitly mention which steps are done concurrently and which follow one another.
Describe for each step, what is done, what is measured, and how. Do not forget to
address, when “interpretation” is happening as a step in the process.

[20220119] INFOMAIS - Adaptive interactive systems -

Questions - Page 12 of 12
41097-69108 1-GSH - UITHOF