IMP 3 :

In order to perform the assignment, the Chair needs to know both the conference topics selected
for each submitted paper and the topics that better describe the reviewers expertise. In the
following we show how it is possible to automatically acquire such a knowledge by means of the
Latent Semantic Indexing (LSI) technique. As regards the papers, a system for the automatic
processing of the submitted documents will be presented. It will be exploited in order to
automatically extract the significant components, i.e. title and abstract, from the paper without
the author do it manually. As concern the reviewers, the information needed from the
application of the LSI was extracted from the online repository of their publications (at the
moment this task is carried out manually). Successively, an expert system that automatically
performs the assignments based on the extracted knowledge about the papers/reviwers topics
will be presented.

IMP4:
There are various ways of stating competences and describing papers. One of them is the
“keywords mechanism”. Authors and reviewers are required to select keywords (usually from a
previously defined by the PC chairs set of keywords) that best describe their papers, respectively
competences. Then a compatibility (similarity) factor between i-th paper and the j-th reviewer
can be easily calculated by applying some basic operations to the sets of keywords selected by
the author of i-th paper and the j-th reviewer. So, instead of browsing and rating all submitted
papers PC members just specify their interests and competences. The disadvantage of the
keywords mechanism is that the common set of all keywords has to be both finite (having a
reasonable number of elements – 20 to 30) and complete in respect to the area of science. This is
not easily achievable. Situations where an author can not find any relevant keyword to his/her
paper should be avoided. Otherwise they will result in less accuracy. Another way of expressing
interests is by using conference topics together with an Iterative Rating Method [2] as used by
Philippe Rigaux in the MyReview System [3]. The idea here is to suggest a small set of papers
(based on the conference topics) to each PC member. Then he/she rates the papers from his/her
set and a collaborative filtering algorithm is run to obtain a new set of papers that will be
suggested for either further rating or assignment to the specified PC member. The collaborative
filtering algorithm predicts the compatibility factors between reviewers and the papers that have
never been rated by those reviewers. In this way PC members are not required to rate all
submitted papers but just a small amount of them. The disadvantage of this method is that for
achieving a high enough accuracy, PC members have to rate papers several times.

There is one thing in common between these two ways of expressing reviewers’ interests and
preferences – it is that both ways compute a compatibility (similarity) factor for each pair <paper,
reviewer>. Thus a matrix of similarity factors can be built and used as an input for the assignment
algorithm.
This paper is a continuation of the one entitled “An algorithm for automatic assignment of
reviewers to papers” [1] published in CompSysTech 2006 Conference Proceedings. The main aim
of the present paper is to outline the results from the analysis and the experimental study of the
suggested in [1] algorithm. The latter has been compared in terms of accuracy and running time
to the maximum-weighted matching algorithm of Kuhn and Munkres (also known as the
Hungarian Algorithm) implemented in The MyReview System [3, 2]. The paper describing the
algorithm [1] can be downloaded from the CompSysTech virtual library – please see references
section.

IMP5:
IMP6

An essential part of an expert-finding task, such as matching reviewers to submitted

papers, is the ability to model the expertise of a person based on documents. We
evaluate several measures of the association between a document to be reviewed
and an author, represented by their previous papers. We compare language-model-
based approaches with a novel topic model, Author-Persona-Topic (APT). In this
model, each author can write under one or more “personas,” which are represented
as independent distributions over hidden topics. Examples of previous papers written
by prospective reviewers are gathered from the Rexa database, which extracts and
disambiguates author mentions from documents gathered from the web. We
evaluate the models using a reviewer matching task based on human relevance
judgments determining how well the expertise of proposed reviewers matches a
submission. We find that the APT topic model outperforms the other models.

IMP7
Manual keyword assignment of keywords by authors and reviewers.

An important feature of the solution we find is that each paper is sent to at least one
reviewer who is “as expert as possible” for that paper

The problem we study is the following. We are given a set [ 1, . . ., r) of reviewers and
a set { 1, . . ., p} of papers; we wish to assign the papers to the reviewers so as to
satisfy the following conditions.
(1.1) Each reviewer should be assigned at most three papers;
(1.2) Each paper should be assigned to exactly three reviewers;
(1.3) As much as possible, each paper should be assigned to reviewers who are
experts for that paper.

IMP8

IMP9 :
Contributions In this paper, we formally define the problem of expertise matching
and propose a constrain-based optimization framework to solve the problem.
Specifically, the expertise matching problem is cast as a convex cost flow problem
and the objective is then to find a feasible flow with minimum cost under certain
constraints. We theoretically prove that the proposed framework can achieve an
optimal solution under various constraints and develop an efficient algorithm to
solve it. By extending our preliminary work [14], we re-formalize our framework
considering “multi-topic coverage” matching, which is very important to paper-
reviewer assignment problem. We show that topic coverage measures (e.g.
Coverage and Confidence) [3] can also be added as constraints to our framework.
To evaluate whether our approach can achieve bettermulti-topic coverage,we
compared our framework with several state of the art approaches on an additional
dataset. Experimental results substantiate the effectiveness and efficiency of the
proposed approach. We have applied the proposed method to help assign
reviewers to papers for a top conference. Feedbacks from the conference
organizers confirm the usefulness of the proposed approach. The rest of the paper
is organized as follows: Section 2
reviewstherelevantliteratures;Section3formallyformulatesthe problem; Section 4
explains the proposed optimization framework. Section 5 gives experimental
results that validate the effectiveness and the computational efficiency of our
methodology. Finally, Section 6 concludes.

In general, existing methods for expertise matching mainly fall into two categories:
probabilistic model and optimization model. The probabilistic model tries to improve
the matching accuracy between experts and queries based on various probabilistic
models such as keyword matching [1], latent semantic indexing [2], probabilistic
topic modeling [3][4]. However, most of these methods do not consider the various
constraints or simply consider the constraints by heuristics.

Most previous works cast expert finding as an information retrieval problem. These
methods mainly focus on two points: how to define the matching score between a
query and a document; and how to represent each expert[15][16]. For
example,Dumais and Nielsen[2]use Latent Semantic Indexing(LSI) as the retrieval
method, and abstracts provided by reviewer as expertise documents. Yu et al. [17]
represent experts by analyzing text content and extracting related information. Basu
et al. [1][18][19] integrate different sources of information for recommendation (e.g.
publications, research interests, etc.). Yarowsky and Florian [20] assign a paper by
computing its cosine similarity with a reviewer and choosing the one with the highest
rank.
In addition, different language models [11][12][13][21][22] and topic models [23] are
used for expert matching/finding problem. In all of the language models, the
matching score is the probability of a query given an expertise document i.e. p(q|d),
but its definition varies. Mimno and McCallum [4] improve the matching accuracy by
proposing a novel topic model Author-Persona-Topic (APT), in which experts are
represented as independant distributions over topics. Karimzadehgan et al. also
consider matching experts on multiple aspects of expertise [3]. Unlike the previous
probabilistic models in which a query is matched as a whole unit, it tries to find
“comprehensive” matchings to cover all subtopics of a query. New measures
Coverage and Confidence are defined to evalute multiaspect expertise matching
results.

However most of the aforementioned works treat each query independently and
ignore the certain constraints (e.g. load balance), thus they cannot be directly
adapted to an expertise matching system. In the real world, expertise matching is a
highly constrained problem, and some existing works study this constrained
optimization problem using various methods. For example, Guerv´os et al. [24]
combine a greedy and an evolutionary algorithm [25][26] to assign papers to
reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as an Integer Linear
Programming (ILP) problem so approximate solutions can be found by any ILP solver.
Recently, a few systems [8][9][10][28][29] have also been developed to help
proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution. However most of the aforementioned
works treat each query independently and ignore the certain constraints (e.g. load
balance), thus they cannot be directly adapted to an expertise matching system. In
the real world, expertise matching is a highly constrained problem, and some existing
works study this constrained optimization problem using various methods. For
example, Guerv´os et al. [24] combine a greedy and an evolutionary algorithm [25]
[26] to assign papers to reviewers. Karimzadehgan et al. [5] and Taylor [27] cast it as
an Integer Linear Programming (ILP) problem so approximate solutions can be found
by any ILP solver. Recently, a few systems [8][9][10][28][29] have also been
developed to help proposalreviewer and paper-reviewer assignment. However, the
expertisematchingproblemisstilltreatedasaninformationretrieval problem, which
obviously cannot result in an optimal solution. In this paper, we aim to formalize the
problem of expertise matchinginaconstrain-basedoptimizationframeworkandpropose
an efficient algorithm to solve the framework. The differences of our work from
existing work are: (a) we offer an optimization framework that incorporates the
expertise matching and various constraints together; (b) the framework can be easily
extended since new constraint can be combined into the
optimizationframeworkbysimplydefininganew(hardorsoft) constraint; and (c) the
framework can guarantee an optimal solution.

IMP10 :

Expert search, in which given a query a ranked list of experts instead of documents
is returned, has been intensively studied recently due to its importance in
facilitating the needs of both information access and knowledge discovery. Many
approaches have been proposed, including metadata extraction, expert profile
building, and formal model generation. However, all of them conduct expert search
with a coarse-grained approach. With these, further improvements on expert
search are hard to achieve. In this paper, we propose conducting expert search with
a fine-grained approach. Specifically, we utilize more specific evidences existing in
the documents. An evidence-oriented probabilistic model for expert search and a
method for the implementation are proposed. Experimental results show that the
proposed model and the implementation are highly effective.