Memory and Selectivity in Evolving Scale-Free Immune Networks

P. Tieri1,2 , S. Valensin1,2 , C. Franceschi1,2 , C. Morandi3 , and G.C. Castellani1,3

L.Galvani Interdipartimental Center for Biophysics, Bioinformatics and Biocomplexity University of Bologna, Bologna 40100, Italy 2 Department of Experimental Pathology, University of Bologna, Bologna 40100, Italy 3 DIMORFIPA, University of Bologna, Ozzano E 40100, Italy gasto@alma.unibo.it
1

Abstract. In this paper we examine the impact of graph theory and more particularly the scale-free topology on Immune Network models. In the case of a simple but not trivial model we analyze network performances as long term selectivity properties, its computational capabilities as memory capacity, and relation with Neural Networks. A more advanced Immune Network model is conceptualized and it is developed a scaold for further mathematical investigation.

Introduction

Network models and related equations represent a topic of growing interest in modern interdisciplinary research. One of the present challenges is a better understanding of how microscopic relations between and within network elements can lead to macroscopic phenomena such as adaptation, learning and memory, and how genetic and environmental information uxes interact. Modern graph theory (small-world models, scale-free networks theory) is one of the most effective and advantageous mathematical technique applied for understanding the evolution of network systems and to comprehend the basic principles of their structural organisation and evolution. Our work is focussed on the application of such technique to the formalisation of Immune System (IS) molecular-cellular networks in order to depict their topological features and attributes aecting their functionality. This approach, proceeding from the formalisation of elements of the network and their interactions as nodes and links, respectively, allows to structure a topology whose characterising features can be derived from analytical and numerical solutions. Various IS network topologies have been envisaged by us as relevant by both, an immunological and a structural point of view, that exploit dierent mechanisms of IS element interacting features, and show a progressively increase of immune complexity. In fact, a network model being reasonably adherent to the real IS should include combinations of ingredients like weighted links, fast evolving connections, interchange and average life span of nodes, appropriate number of nodes, topologically dierent functions of nodes,
J. Timmis et al. (Eds.): ICARIS 2003, LNCS 2787, pp. 93101, 2003. c Springer-Verlag Berlin Heidelberg 2003

94

P. Tieri et al.

and some more, which are all factors that rise the mathematical analysis complexity. Thus, our aim is to proceed by deploying models at increasing levels of structural and mathematical complexity. At present, two dierent types of IS networks are under our investigation, having two dierent levels of analysis completion. The rst is a growing, scale-free, idiotypic-inspired immune network (IN) model. The second is a network of heterogeneous immune cells constituting the nodes linked by soluble molecular mediators (cytokines and chemokines) that in the real IS appoint a molecular eld by means of which immune cells indirectly communicate. In this paper we focus on a well known IS network model [25,11,4] whose activation function is based on microscopic and experimental data. This is a simplied model, that neverthless allows to capture some fundamental features of the IS network properties. This work deals with the IS connectivity problem [7,26], and its extension to scale-free topology. The analysis on immune network can be extended also to the so called BCM (Bienenstock, Cooper and Munro) neural network model (BCM NN)[2,19,8] showing a interesting degree of similarity between their selectivity properties in term of stability of learning rule solutions. The relation between immune and nervous system looks very intriguing since both systems can be described, from a mathematical-systemic point of view, by similar equations based on a network structure [16,20]. Recent studies suggest that the similarity between the two systems is not purely formal [17]. Among others, a surprising result was [1] that some fundamental molecules for learning and memory in the nervous system (like CaMK II and Calcineurin) play a fundamental role also in the IS, leading to a new scenario in which molecules and pathways are shared by the two systems and conserved during evolution. One point of dierence between IN and NN models is that for the IN we dont have a explicit learning so that we can think the connectivity changes as a prototipical learning rule or, more generally, as a physiological change that mark the IS behaviour and development throughout individual life. The major aim of this paper is a better understanding of the development of memory in immune network models when the network size is growing in a scale-free way. This study is motivated by the increased availability of experimental data on the evolution of the immune memory during lifespan (from birth to older age), that shows an age related increase in the ratio between memory and virgin cells [6] as well as by computational applications of immune networks in problems of feature extraction, dimensionality reduction and nonlinear separability.

Idiotypic-Inspired Scale-Free Immune Network

The IS network model that we consider describes the dynamics of a system of interacting lymphocyte clones. This interaction is described by the following system of nonlinear dierential equations: dCi = Si + Ci (pi fi (hi ) i ) i = 1, . . . , n, (1) dt where Ci is the concentration of the ith clone (that obviously must be taken as nonnegative: Ci 0, i), n is the number of clones, Si is the inux term

Memory and Selectivity in Evolving Scale-Free Immune Networks

95

Fig. 1. Semilogarithmic plot of the bell-shaped dose-response curve. The points L and H represent respectively the low and high stable states for the clone concentration, given a specic value for , represented by the horizontal line.

(number of clones produced per unit of time), i is the decay term (related to the death rate of the cells), pi is the proliferation rate, while the function fi (hi ) describes the activation of the ith clone. We assume that fi (hi ) is given by a biphasic dose-response curve (Fig. 1), modeled by the following equation [3]: fi (hi ) = 2 i hi (1i + hi )(2i + hi ) i = 1, . . . , n. (2)

The parameters 1i and 2i represent two threshold values (0 < 1i < 2i ), respectively for proliferation and for suppression, while the argument hi represents the inuence of the other clones on the ith clone, dened as follows:
n

hi =
j =1

ij Cj

i = 1, . . . , n.

(3)

The coecients ij of the A matrix quantify the interactions between the i-th clone and the other ones; here we assume the symmetry property introduced by Homann [18]: ij = ji , i = j , ii = 0, since each clone does not interact with itself. Moreover, for the sake of simplicity, we assume that all the interactions have the same strength, ij = . A further assumption is that the parameters pi , i , 1i , 2i and Si have the same values for all the clones. In this way we have a simplied system1 : dCi = S + Ci (pf (hi ) ) dt i = 1, . . . , n. (4)

For n = 2 this system is similar to one previously analysed [12,4], while the case with S = 0 is equivalent to the model analysed by Neumann and Weisbuch [23] with a window automaton proliferation function.
1

System (4) admits the permutations over the indices {1, . . . , n} as a symmetry group, then solutions belonging to the same equivalence class by the group share the same dynamical properties (for instance stability). Hereafter, when treating some specic solutions, it is implicitly understood that our remarks about it are extended to all the elements of the equivalence class.

96

P. Tieri et al.

Fig. 2. Birth of a scale free graph.

The choice of the numerical values of the parameters in numerical computations, was performed on the basis of experimental data: = 1, p = 2, 1 = 0.1, 2 = 10, and = 0.1 [12]. Following the classication proposed by [12,23,25,4], based on linear stability analysis, the only asymptotically stable states are those having only one clone at high (H) concentration: (H, L, . . . , L and all permutations). We will refer to these solutions (observed for the completely connected network) as highly selective solutions. The IN is, in this case, able to recognize or memorize only one antigenic specicity at a time. 2.1 Preliminary Results

We have generated scale-free graphs by the growth process described in [31] and depicted in Fig (2) using the adjacency matrices as connectivity matrix for the IN model. The scale free mechanism is based on two assumptions: the network size grows and new edges are added to the network at each time step following a preferencial attachement rule that states that the probability for a new node to be connected to the existing nodes is proportional to the number of their connections (the most connected nodes have the maximun probability to be connected). In our preliminary analysis the equation (4) has been numerically integrated, and in the case of nonsingular adjacency matrix a comparison between analytical and numerical solutions has been performed [4], while in the general case of singular adjacency matrix the analytical solution is in progress. The numerical results show that by increasing the network size the number of stable states remains relatively constant. In particular we found that the most probable number of stable states is 3. These stable states are not selective for only one clonal specicity, as in the completely connected case, but they exhibit selectivity for more than one specicity. This result was expected on the basis our past studies that showed that the memory capacity of IN models changes drastically as function of the connectivity. This preliminary result that need to

Memory and Selectivity in Evolving Scale-Free Immune Networks

97

be conrmed, would suggest that it is possible to control the number of stable solutions as well as their selectivity properties, or in other word their capacity of recognizing foreign patterns.

Towards More Realistic Immune Network Models

Graph theory has already been successfully exploited for studies of the topological and connective features of existing real world networks [27,28], like for example citations of scientic papers and networks of collaborations [29], WWW and Internet [30,32], biological networks as neural networks, metabolic reactions network [33], genome and protein network [34], ecological and food webs, world web of human contacts [35], and languages [36], telephone call graphs, power grids, nets of small electronic components etc. As far as we know (beyond the undistinguishable clones hypothesis), a similar approach has never been applied before to the study of realistic IS network peculiarities [37]. A scale-up line of attack should be applied to take into consideration the facing problems, rstly dealing with the main basic attributes of the IS elements and successively arriving, step by step, to more realistic immunologic congurations (i.e. mirroring concepts more adherent to the real IS characteristics like weighted links, fast evolving connections, interchange and average life span of nodes, a more appropriate number of nodes, topologically dierent functions of nodes, among others). The rst phase of this strategy can consist in the representation and formalisation procedure of a small scale system (some tens of elements) in which nodes represent IS cells and links represent the immune role of cytokines and chemokines. In this simplied representation of the IS, cells (nodes) will be distinguished in source cell type (secreting cytokines and chemokines) and target cell type (on which secreted cytokines and chemokines act). We propose this rst oversimplied visualization of a part of the network of communication among the main types of IS cells. The graph (Fig 3) shows the network of heterogeneous immune cells constituting the nodes linked by soluble molecular mediators. Data are extracted and adapted from [40]. Sources (left side) and targets cells (right side) are linked by the various cytokine and chemokine signalling molecules. A source cell secretes dierent kinds of mediators that can act upon one or more target cells as well as on the source cell itself. The topology of this graph can be depicted by its adjacency matrix. The matrix of sources and mediators and the matrix of mediators and targets can be multiplied to obtain a square adjacency matrix showing how cells communicates with each others. Relevant coecients related to the network topology are: the mean degree of the network, i.e. the average number of the nearest connected neighbours of a node; the degree distribution; the clustering coecient of the network, i.e. the probability of one edge between two nearest neighbours of a randomly chosen node; the characteristic path length, i.e. the average of the shortest paths -number of steps- necessary to go from any two vertices calculated on all pairs of vertices of the network. This last coecient can be calculated both

98

P. Tieri et al.

Fig. 3. Graph representing the complex cytokine-mediated cell interaction within IS. Diamonds represent cytochines and chemokines, source cells are on the left side, while target cells are on the right side.

Memory and Selectivity in Evolving Scale-Free Immune Networks

99

with and without considering edge directions. We consider 26 types of cells, among which 14 are sources, 18 are targets, and 6 are both source and target. 36 types of chemical mediators are secreted. Among sources, T cells (and T CD4+ Th2 in particular), endothelial cells, epithelial cells, broblasts and macrofages are the most connected sources, while B cells, T cells, monocytes, basophils, neutrophils, eosinophils are the most linked targets. A complete mathematical formalization of the proposed model is in progress as well as a ne tuning of the model itself.

Conclusion

This analysis is one of the rst attempts to characterize a key property as the long term memory specicity in a evolving IN. We are condent that these preliminary results can be exported to the BCM NN model, where the eect of replacing an input pattern global mean with local mean has been investigated. A remaining interesting question is the relation between memory properties in BCM NN model and moving average procedures. The numerical results obtained for the NN model suggest that the memory size and selectivity (in terms of recognized pattern) strongly depend on network topology, and that in a scale-free context these two properties can be separated so that we can control the composition of immune response in term of pattern of activity. Once an IS-representing small scale graph is fully and coherently formalised, its topological characterising features can be derived from analytical and numerical solutions. Our attack strategy foresees later on a migration to medium scale (103 -105 nodes-elements) representation, in which we will try to take into consideration the weights of the links, the direction of action ows, the fast evolving connections, interchange and average life span of nodes, topologically dierent functions of nodes and other peculiar more realistically detailed attributes of IS elements. The resulting complex graph will be again mathematically analysed in order to derive all the necessary measures and eventually a mesoscale model, that will serve as a rst basis for IT architecture design. Indeed in approaching a more realistic formalisation of the IS network we will face problems relative to the complexity of the interrelationships among cells, due to: i. the diverse mechanisms of interaction (biochemical signalling, signal duration, chemical concentration, cascade activation and secretion, activation thresholds, i.e. the quantity of activated receptors per time unit, sterical recognition of peptides and antigens by surface receptors, locality of the interaction, migration of the cell in another immune district, systemic eects); ii. the miscellaneous nature of each cell type; iii. their rapid mobility; iv. the evanescence and alteration of the biochemical connection among cells; v. the dierent function, weight and relevance of each exchanged signal.

Acknowledgments
This work has been supported by Italian MIUR COFIN 2001 grant Human Immunosenescence: an integrated experimental and theoretical approach to C.F. and a Bologna University grant (ex 60 %) to G.C.C.

100

P. Tieri et al.

References
1. J. D. Bui, S. Calbo, K. Hayden-Martinez, L. P. Kane, P. Gardner, and S. M. Hedrick. A role for camkii in t cell memory. Cell, 10 (2000):457467. 2. E. L. Bienenstock, L. N. Cooper, and P. W. Munro. Theory for the development of neuron selectivity: orientation specicity and binocular interaction in visual cortex. J. Neurosci., 2:(1982)3248. 3. F. Celada. The cellular basis of immunological memory. Immunol. Rev., 110: (1971)6387. 4. G. C. Castellani, C. Giberti, C. Franceschi, and F. Bersani. Stable state analysis of an immune network model. Int. J. Chaos Bif., 8(6):(1998) 12851301. 5. G. C Castellani, N. Intrator, H. Shouval, and L. Cooper. Solution of the bcm learning rule in a network of lateral interacting nonlinear neurons. Network, 10: (1999) 111121. 6. A. Cossarizza, C. Ortolani, R. Paganelli, D. Barbieri, D. Monti, P. Sansoni, U. Fagiolo, G. Castellani, F. Bersani, M. Londei, and C. Franceschi. Cd45 isoforms expression on cd4+ and cd8+ t cells throughout life, from newborns to centenarians: Implications for t cell memory. Mechanisms of Aging and Development, 86:(1996)173195. 7. A. Coutinho. Beyond clonal selection and network. Immunol. Rev., 110:(1989)63 87. 8. GC. Castellani, EM. Quinlan, LN Cooper, and HZ. Shouval. A biophysical model of bidirectional synaptic plasticity: Dependence on ampa and nmda receptors. PNAS, 98(22): (2001) 1277212777. 9. L. N. Cooper and C. L. Scoeld. Mean-eld theory of a neural network. PNAS, 85:(1988) 19731977. 10. S. M. Dudek and M. F. Bear. Homosynaptic long-term depression in area CA1 of hippocampus and the eects on NMDA receptor blockade. PNAS, 89: (1992) 43634367. 11. R. J. De Boer. Symmetric idiotypic networks: Connectance and switching stability and suppression. Bull. Math. Biol., 55:(1993) 745780. 12. R. J. De Boer, I.G. Kewrekidis, and A.S. Perelson. Immune network behaviour from stationary states to limit cycle oscillations. Bull. Math. Biol., 55:(1993) 745 780. 13. R. J. De Boer and A. Perelson. Size and connectivity as emergent properties of a developing immune network. J. Theor. Biol., 149:(1991)381424. 14. C. Franceschi, S. Valensin, M. Bonaf e, P. Paolisso, A. Yashin, and G. De Benedictis. The network and the remodelling theories of aging. Exp. Gerontol., 35:879896, 2000. 15. C. Franceschi, S. Valensin, F. Fagnoni, C. Barbi, and M. Bonaf e. Biomarkers of immunosenescence: the challenge of heterogeneity and the role of antigenic load. Exp. Gerontol., 34:(1999) 911921. 16. G. W. Homann, T.A. Kion, R.B. Forsyth, K.G. Soga, and A. CooperWillis. The n-dimensional network. In A.S. Perelson, editor, Theoretical Immunology, Part 2, page 291. Addison-Wesley, 1988. 17. K. Hayden-Martinez, L. P. Kane, and S. M. Hedrick. Eects of a constitutively active form of calcineurin on t cell activation and thymic selection. J. Immunol., 165:(2000) 37133721. 18. G. W. Homann. A neural network model based on the analogy with the immune system. J. Theor. Biol., 122:(1986) 3367.

Memory and Selectivity in Evolving Scale-Free Immune Networks

101

19. N. Intrator and L. N Cooper. Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions. Neural Networks, 5:(1992) 317. 20. N. K. Jerne. Towards a network theory of immune system. Annu. Immunol., 125:(1974)373389. 21. A. Kirkwood, H. K. Lee, and M. F. Bear. Co-regulation of long-term potentiation and experience-dependent synaptic plasticity in visual cortex by age and experience. Nature, 25;375(6529):(1995) 328331. 22. J. A. Lisman. A mechanism for the hebb and the anti-hebb processes underlying learning and memory. PNAS, 86:(1989) 95749578. 23. A.U. Neumann and G. Weisbuch. Window automata analysis of population dynamics in the immune system. Bull. Math. Biol., 54:(1992) 2144. 24. A.S. Perelson. Immune network theory. Immunol. Rev., 110:(1989) 536. 25. A.S. Perelson. Mathematical approaches in immunology. In S. I. Andersson, editor, Theory & Control of Dynamical Systems, pages 200230. World Scientic, 1992. 26. J. Stewart and F. J. Varela. Exploring the meaning of connectivity in the immune network. Immunol. Rev., 110:(1989) 3761. 27. S.H. Strogatz: Exploring complex networks. Nature, 410 (2001) 268276. 28. D.J.Watts, S.H. Strogatz: Collective dynamics of small-world networks. Nature, 393 (1998) 440-442. 29. M.E.J. Newman: Scientic collaboration networks. I. Network construction and fundamental results, and II. Shortest paths, weighted network and centrality. Physical Review E, 64 (2001) 016131016132. 30. A.L. Barabasi, R. Albert: Emergence of scaling in random networks. Science, 286 (1999) 509-512. 31. A.L. Barabasi, R. Albert: Statistical mechanics of complex networks, Rev of Mod Phys, 74, 2002 48-94 32. R. Albert: Diameter of the World-Wide Web. Nature, 401 (1980) 130-131. 33. H. Jeong, B.Tombor, R.Albert, Z.N.Oltvai and A.L.Barabasi et al.: The large scale organization of metabolic networks. Nature, 407 (2000) 651-654. 34. H.Jeong, S.P.Mason, A.L.Barabasi and Z.L.Oltvai.: Lethality and centrality in proteins network. Nature,411 (2001) 41-42. 35. F.Liljeros: The web of human sexual contacts. Nature, 411 (2001) 907-908. 36. R. Ferrer i Cancho, R.V. Sole: Least eort and the origins of scaling in human language. Proc Natl Acad Sci U S A , 100 (2003) 788-791. 37. Editorial article: Making connections. Nature Immunology , 3 (2002) 883. 38. S.H. Yook,H. Jeong, A.L. Barabasi,Y. Tu: Weighted evolving networks Physical Review Letters, 86 (2001) 58355838 39. P. Kourilsky, P. Trua-Bachi: Cytokine elds and the polarization of the immune response. Trends Immunol 22 ,(2001) 502-509. 40. A.K. Abbas, A.H. Lichtman, J.S. Pober: Cellular and Molecular Immunology. Saunders (2000) 4th ed.