Professional Documents
Culture Documents
In mathematics, a submodular set function (also known as a submodular function) is a set function
whose value, informally, has the property that the difference in the incremental value of the function that a
single element makes when added to an input set decreases as the size of the input set increases.
Submodular functions have a natural diminishing returns property which makes them suitable for many
applications, including approximation algorithms, game theory (as functions modeling user preferences)
and electrical networks. Recently, submodular functions have also found immense utility in several real
world problems in machine learning and artificial intelligence, including automatic summarization, multi-
document summarization, feature selection, active learning, sensor placement, image collection
summarization and many other domains.[1][2][3][4]
Definition
If is a finite set, a submodular function is a set function , where denotes the power set of
, which satisfies one of the following equivalent conditions.[5]
A nonnegative submodular function is also a subadditive function, but a subadditive function need not be
submodular. If is not assumed finite, then the above conditions are not equivalent. In particular a function
defined by if is finite and if is infinite satisfies the first condition above, but the
second condition fails when and are infinite sets with finite intersection.
Monotone
then f is monotone.
Budget-additive functions
budget additive.[6]
Coverage functions
Let be a collection of subsets of some ground set . The function
Non-monotone
Symmetric
Graph cuts
Let be the vertices of a graph. For any set of vertices let
denote the number of edges such that and . This can be
generalized by adding non-negative weights to the edges.
Mutual information
Let be a set of random variables. Then for any we have that
is a submodular function, where is the mutual
information.
Asymmetric
Directed cuts
Let be the vertices of a directed graph. For any set of vertices
let denote the number of edges such that and . This can
be generalized by adding non-negative weights to the directed edges.
Continuous extensions
Definition
In the context of submodular functions, there are a few examples of continuous extensions that are
commonly used, which are described as follows.
Examples
Lovász extension
This extension is named after mathematician László Lovász.[9] Consider any vector
such that each . Then the Lovász extension is defined as
where the expectation is over chosen from the uniform distribution on the
interval . The Lovász extension is a convex function if and only if is a submodular function.
Multilinear extension
Consider any vector such that each . Then the multilinear extension is
defined as .
Convex closure
Consider any vector such that each . Then the convex closure is
Concave closure
Consider any vector such that each . Then the concave closure is
defined as .
Properties
1. The class of submodular functions is closed under non-negative linear combinations.
Consider any submodular function and non-negative numbers .
Then the function defined by is submodular.
Optimization problems
Submodular functions have properties which are very similar to convex and concave functions. For this
reason, an optimization problem which concerns optimizing a convex or concave function can also be
described as the problem of maximizing or minimizing a submodular function subject to some constraints.
The hardness of minimizing a submodular set function depends on constraints imposed on the problem.
Unlike the case of minimization, maximizing a generic submodular function is NP-hard even in the
unconstrained setting. Thus, most of the works in this field are concerned with polynomial-time
approximation algorithms, including greedy algorithms or local search algorithms.
Apart from submodular minimization and maximization, there are several other natural optimization
problems related to submodular functions.
1. Minimizing the difference between two submodular functions[24] is not only NP hard, but
also inapproximable.[25]
2. Minimization/maximization of a submodular function subject to a submodular level set
constraint (also known as submodular optimization subject to submodular cover or
submodular knapsack constraint) admits bounded approximation guarantees.[26]
3. Partitioning data based on a submodular function to maximize the average welfare is known
as the submodular welfare problem, which also admits bounded approximation guarantees
(see welfare maximization).
Applications
Submodular functions naturally occur in several real world applications, in economics, game theory,
machine learning and computer vision.[4][27] Owing to the diminishing returns property, submodular
functions naturally model costs of items, since there is often a larger discount, with an increase in the items
one buys. Submodular functions model notions of complexity, similarity and cooperation when they appear
in minimization problems. In maximization problems, on the other hand, they model notions of diversity,
information and coverage.
See also
Supermodular function
Matroid, Polymatroid
Utility functions on indivisible goods
Citations
1. H. Lin and J. Bilmes, A Class of Submodular Functions for Document Summarization, ACL-
2011.
2. S. Tschiatschek, R. Iyer, H. Wei and J. Bilmes, Learning Mixtures of Submodular Functions
for Image Collection Summarization, NIPS-2014.
3. A. Krause and C. Guestrin, Near-optimal nonmyopic value of information in graphical
models, UAI-2005.
4. A. Krause and C. Guestrin, Beyond Convexity: Submodularity in Machine Learning, Tutorial
at ICML-2008
5. (Schrijver 2003, §44, p. 766)
6. Buchbinder, Niv; Feldman, Moran (2018). "Submodular Functions Maximization Problems"
(https://www.taylorfrancis.com/chapters/edit/10.1201/9781351236423-42/submodular-functio
ns-maximization-problems-niv-buchbinder-moran-feldman). In Gonzalez, Teofilo F. (ed.).
Handbook of Approximation Algorithms and Metaheuristics, Second Edition: Methodologies
and Traditional Applications. Chapman and Hall/CRC. doi:10.1201/9781351236423 (https://
doi.org/10.1201%2F9781351236423). ISBN 9781351236423.
7. "Information Processing and Learning" (https://www.cs.cmu.edu/~aarti/Class/10704_Spring1
5/lecs/lec3.pdf) (PDF). cmu.
8. Fujishige (2005) p.22
9. Lovász, L. (1983). "Submodular functions and convexity". Mathematical Programming the
State of the Art: 235–257. doi:10.1007/978-3-642-68874-4_10 (https://doi.org/10.1007%2F97
8-3-642-68874-4_10). ISBN 978-3-642-68876-8.
10. Vondrák, Jan. "Polyhedral techniques in combinatorial optimization: Lecture 17" (https://theo
ry.stanford.edu/~jvondrak/CS369P/lec17.pdf) (PDF).
11. Grötschel, M.; Lovasz, L.; Schrijver, A. (1981). "The ellipsoid method and its consequences
in combinatorial optimization". Combinatorica. 1 (2): 169–197. doi:10.1007/BF02579273 (htt
ps://doi.org/10.1007%2FBF02579273). hdl:10068/182482 (https://hdl.handle.net/10068%2F
182482). S2CID 43787103 (https://api.semanticscholar.org/CorpusID:43787103).
12. Cunningham, W. H. (1985). "On submodular function minimization". Combinatorica. 5 (3):
185–192. doi:10.1007/BF02579361 (https://doi.org/10.1007%2FBF02579361).
S2CID 33192360 (https://api.semanticscholar.org/CorpusID:33192360).
13. Iwata, S.; Fleischer, L.; Fujishige, S. (2001). "A combinatorial strongly polynomial algorithm
for minimizing submodular functions". J. ACM. 48 (4): 761–777. doi:10.1145/502090.502096
(https://doi.org/10.1145%2F502090.502096). S2CID 888513 (https://api.semanticscholar.or
g/CorpusID:888513).
14. Schrijver, A. (2000). "A combinatorial algorithm minimizing submodular functions in strongly
polynomial time" (https://ir.cwi.nl/pub/2108). J. Combin. Theory Ser. B. 80 (2): 346–355.
doi:10.1006/jctb.2000.1989 (https://doi.org/10.1006%2Fjctb.2000.1989).
15. Z. Svitkina and L. Fleischer, Submodular approximation: Sampling-based algorithms and
lower bounds, SIAM Journal on Computing (2011).
16. R. Iyer, S. Jegelka and J. Bilmes, Fast Semidifferential based submodular function
optimization, Proc. ICML (2013).
17. U. Feige, V. Mirrokni and J. Vondrák, Maximizing non-monotone submodular functions, Proc.
of 48th FOCS (2007), pp. 461–471.
18. N. Buchbinder, M. Feldman, J. Naor and R. Schwartz, A tight linear time (1/2)-approximation
for unconstrained submodular maximization, Proc. of 53rd FOCS (2012), pp. 649-658.
19. Nemhauser, George; Wolsey, L. A.; Fisher, M. L. (1978). "An analysis of approximations for
maximizing submodular set functions I". Mathematical Programming. 14 (14): 265–294.
doi:10.1007/BF01588971 (https://doi.org/10.1007%2FBF01588971). S2CID 206800425 (htt
ps://api.semanticscholar.org/CorpusID:206800425).
20. Williamson, David P. "Bridging Continuous and Discrete Optimization: Lecture 23" (https://pe
ople.orie.cornell.edu/dpw/orie6334/lecture23.pdf) (PDF).
21. G. Calinescu, C. Chekuri, M. Pál and J. Vondrák, Maximizing a submodular set function
subject to a matroid constraint, SIAM J. Comp. 40:6 (2011), 1740-1766.
22. M. Feldman, J. Naor and R. Schwartz, A unified continuous greedy algorithm for submodular
maximization, Proc. of 52nd FOCS (2011).
23. Y. Filmus, J. Ward, A tight combinatorial algorithm for submodular maximization subject to a
matroid constraint, Proc. of 53rd FOCS (2012), pp. 659-668.
24. M. Narasimhan and J. Bilmes, A submodular-supermodular procedure with applications to
discriminative structure learning, In Proc. UAI (2005).
25. R. Iyer and J. Bilmes, Algorithms for Approximate Minimization of the Difference between
Submodular Functions, In Proc. UAI (2012).
26. R. Iyer and J. Bilmes, Submodular Optimization Subject to Submodular Cover and
Submodular Knapsack Constraints, In Advances of NIPS (2013).
27. J. Bilmes, Submodularity in Machine Learning Applications, Tutorial at AAAI-2015.
References
Schrijver, Alexander (2003), Combinatorial Optimization, Springer, ISBN 3-540-44389-4
Lee, Jon (2004), A First Course in Combinatorial Optimization, Cambridge University Press,
ISBN 0-521-01012-8
Fujishige, Satoru (2005), Submodular Functions and Optimization, Elsevier, ISBN 0-444-
52086-4
Narayanan, H. (1997), Submodular Functions and Electrical Networks, ISBN 0-444-82523-1
Oxley, James G. (1992), Matroid theory, Oxford Science Publications, Oxford: Oxford
University Press, ISBN 0-19-853563-5, Zbl 0784.05002 (https://zbmath.org/?format=complet
e&q=an:0784.05002)
External links
http://www.cs.berkeley.edu/~stefje/references.html has a longer bibliography
http://submodularity.org/ includes further material on the subject