CONCLUSION

For social network analysis, the convergence of straggling FEP may need to experience
significant numbers of iterations and also needs very large amounts of computation and
communication in each iteration, inducing serious load imbalance. However, for this problem,
current solutions either require significant overhead, or can not exploit underutilized computers
when some features converged in early iterations, or perform poorly because of the high load
imbalance among initial tasks. This paper identifies that the most computational part of
straggling FEP is decomposable. Based on this observation, it proposes a general approach to
factor straggling FEP into several sub-processes along with a method to adaptively distribute
these sub-processes over workers in order to accelerate its convergence. Later, this paper also
provides a programming model along with an efficient runtime to support this approach.
Experimental results show that it can greatly improve the performance of social network analysis
against state-of-the-art approaches. As shown in Section 5, the master in our approach may
become a bottleneck. In future work, we will study how to employ our approach in a hierarchical
way to reduce the memory overhead and evaluate its performance gain.