Artificial Immune Systems: A New Computational Intelligence Approach

New Trends in Intelligent Information Processing and Web Mining. Zakopane, Poland, June 2-5, 2003
Jonathan Timmis Computing Laboratory University of Kent CT2 7NF. UK. J.Timmis@kent.ac.uk http:/www.cs.kent.ac.uk/~jt6

Novel paradigms are proposed and accepted not necessarily for being faithful to their sources of inspiration, but for being useful and feasible

What do I want to achieve?
Give you a taster of what AIS is all about
Define an AIS Why do we find the immune system useful? Explain what AIS are Show you where they are being used Some high level case studies Comments for the future

I won¶t
Talk about all areas of AIS and applications Talk too much about how AIS relate to other bioinspired ideas (although I will mention it) Go into too much detail: this is an introduction

Outline
What are AIS? Useful immunology Thinking about AIS Application Areas and Case Studies The Future

Why the Immune System?
Recognition
Anomaly detection Noise tolerance

Robustness Feature extraction Diversity Reinforcement learning Memory; Dynamically changing coverage Distributed Multi-layered Adaptive

which are applied to complex problem domains .A Definition AIS are adaptive systems inspired by theoretical immunology and observed immune functions. principles and models.

mid 1990¶s ± Machine learning . Suggested we µmight look¶ at the IS 1990 ± Bersini first use of immune algos to solve problems Forrest et al ± Computer Security mid 1990¶s Hunt et al.Some History Developed from the field of theoretical immunology in the mid 1980¶s.

Bentley¶00¶01. de Castro ¶02) Adaptive Control (Bersini¶91) . Gibert¶94. Lamont¶98¶01.02.Scope of AIS: Computer Security(Forrest¶94¶96¶98.02) Anomaly Detection (Dasgupta¶96¶01¶02) Fault Diagnosis (Ishida¶92¶93. Ishiguro¶94) Data Mining & Retrieval (Hunt¶95¶96. Timmis¶99¶01. Kephart¶94. ¶02) Pattern Recognition (Forrest¶93. Dasgupta¶99¶01.

Timmis ¶02) Autonomous Systems (Varela¶92. ¶01. ¶02) Chemical Pattern Recognition (Dasgupta¶99) Robotics (Ishiguro¶96¶97. de Castro ¶02) Web Mining (Nasaroui¶02) Fault Tolerance (Tyrrell.Singh¶01) Optimization (DeCastro¶99. ¶01. Nunes¶00) And so on « .Ishiguro¶96) Engineering Design Optimization (Hajela¶96 ¶98. ¶02.Endo¶98.Scope of AIS (Cont««): Job shop Scheduling (Hart¶98.

Outline What are AIS? Useful immunology Thinking about AIS Application Areas and Case Studies The Future .

Role of the Immune System Protect our bodies from pathogen and viruses Primary immune response Launch a response to invading pathogens Secondary immune response Remember past encounters Faster response the second time around .

c e ll ) L y m p h o k in e s ( V I ) A c tiv a te d ( p la s m a B .c e ll ( V ) ( I V A c tiv a te d T .How does it work: A simplistic view M H C A P C p r o te in A n tig e n ( I ) P e p tid e ( I I ) T .c e ll ( I I I ) B .c e ll c e ll) ( V I I ) .

etc.Immune cells There are two primarily types of lymphocytes: B-lymphocytes (B cells) T-lymphocytes (T cells) Others types include macrophages. phagocytic cells. . cytokines.

therefore Some kind of positive selection Some element of negative selection .Self/Non-Self Recognition Immune system needs to be able to differentiate between self and non-self cells Antigenic encounters may result in cell death.

Antigen Substances capable of starting a specific immune response commonly are referred to as antigens This includes some pathogens such as viruses. fungi etc . . bacteria.

antigens might present several epitopes. Antibodies present a single type of receptor.Immune Pattern Recognition BCR or Antibody B-cell Receptors (Ab) Epitopes Antigen B-cell The immune recognition is based on the complementarity between the binding region of the receptor and a portion of the antigen called epitope. This means that each antibody can recognize a single antigen .

nti n Clonal deletion (negati e selection) ¦ ¨ ¥ ¦§ ¦ ¤¥ ¤ ¢  ¡ ¢ £ ¢ ¡     © ©  © Proliferation (Cloning) M M o r c lls c lls  © .nti n Antibody Selection Differentiation Pl s For i ti s S lf.Clonal Selection Clonal deletion (negati e selection) S lf.

1978) Elimination of self antigens Proliferation and differentiation on contact of mature lymphocytes with antigen Restriction of one pattern to one differentiated cell and retention of that pattern by clonal descendants. Generation of new random genetic changes.Main Properties of Clonal Selection (Burnet. subsequently expressed as diverse antibody patterns by a form of accelerated somatic mutation .

Immune Network Theory Idiotypic network (Jerne. 1974) B cells co-stimulate each other Treat each other a bit like antigens Creates an immunological memory P ra p a to e S p re s n u p s io N g tiv re p n e ea e sos Ag 1 2 Id to e io p A tib d n oy A tiv tio c a n P s ere p n e o itiv sos 3 .

secondary immune responses Remembers encounters No need to start from scratch Memory cells Continuous learning .Reinforcement Learning and Immune Memory Repeated exposure to an antigen throughout a lifetime Primary.

. .. Ag2 .. Res onse to Ag2 Res onse to Ag1 + Ag3 . Antigen Ag1 + Ag3 Ti e .Learning (2) Pri ary Res onse Secondary Res onse Cross-Reacti e Res onse Antibody Concentration Lag Lag Lag Res onse to Ag1 Res onse to Ag1 .. Antigen Ag1 Antigens Ag1.....

activate several defense mechanisms leading to its destruction (or neutralization). Overall behavior of the immune system is an emergent property of many local interactions. Subsequent exposure to similar entity results in rapid immune response.Immune System: Summary Define host (body cells) from external entities. So it is useful? . When an entity is recognized as foreign (or dangerous).

Outline What are AIS? Useful immunology Thinking about AIS Application Areas and Case Studies The Future .

principles and models.Artificial Immune Systems AIS are adaptive systems inspired by theoretical immunology and observed immune functions. which are applied to complex problem domains .

This Section General Framework for describing and constructing AIS models A short review of where AIS are used today Can not cover them all. far too many Also we are not experts in all application areas ! Where are AIS headed? .

we need: To be able to describe immune system components Be able to describe their interactions Quite high level abstractions Capture general purpose processes that can be applied to various areas . Therefore.What do want from a Framework? In a computational world we work with representations and processes.

General Framework for AIS Immune Algorithms Affinity Measures Representation Application Domain .

Representation ± Shape Space Describe the general shape of a molecule Antigen Antibody ‡Describe interactions between molecules ‡Degree of binding between molecules .

Ab2... .. Ag2.Representation Vectors Ab = £Ab1.. AbL³ Ag = £Ag1.. AgL³ Real-valued shape-space Integer shape-space Binary shape-space Symbolic shape-space . ..

Affinity Threshold .Define their Interaction Define the term Affinity Distance measures such as Hamming. etc. Manhattan etc.

Basic Immune Models and Algorithms Negative Selection Algorithms Clonal Selection Algorithm Immune Network Models Somatic Hypermutation .

Negative Selection (NS) Algorithms Forrest 1994: Idea taken from the negative selection of T-cells in the thymus Applied initially to computer security Split into two parts: Censoring Monitoring Self strings (S) D e te c to r S e t (R ) Generate random strings (R0) Match No Yes Reject Detector Set (R) Pro te c te d S trin g s (S ) M a tc h Y es N o n -s e D e te c te d No   .

do: 2.1 Antigenic binding: determine affinity to each P¶ 2.Clonal Selection Algorithm (de Castro & von Zuben. Initialisation: Randomly initialise a population (P) 2. then add new mutants to P 3.2 Affinity maturation: select n highest affinity from P and clone and mutate prop. Cycle: repeat 2 and 3 until stopping criteria .1 select highest affinity P to form part of M 3. 2001) 1. Antigenic Presentation: for each pattern in Ag.2 replace n number of random new ones 4. to affinity with Ag. Metadynamics: 3.

determine its stimulation level (based on antigenic and network interaction) 2. 2001) 1. 2.4 Somatic hypermutation: mutate each clone 2.1 Clonal selection and network interactions: for each network cell. Cycle: Repeat step 2 until termination condition is met . Initialisation: create an initial network from a sub-section of the antigens Antigenic presentation: for each antigenic pattern.5 Network construction: select mutated clones and integrate 3.3 Clonal Expansion: select the most stimulated network cells and reproduce them proportionally to their stimulation 2. do: 2.2 Metadynamics: eliminate network cells with a low stimulation 2.Discrete Immune Network Models (Timmis & Neal.

1 0 0 0 .9 1 V = 10 V = 20 D* .3 0 .9 0 .6 0 .5 0 .3 0 .7 0 .2 0 .Somatic Hypermutation Mutation rate in proportion to affinity Very controlled mutation in the natural immune system Trade-off between the normalized antibody affinity D* and its mutation rate E.4 0 . 1 0 .6 V = 5 E 0 .7 0 .8 0 .4 0 .2 0 .8 0 .1 0 .5 0 .

Case Study: Data Mining .

Data mining: Problem description More benchmark problem in this case Assume a set of labelled vectors Classification .

though found this did not work Resource allocation Somatic hypermutation Eventually Antibody/antigen binding .AIRS: (Artificial Immune Recognition System) Watkins 2003 Clonal Selection Based initially on immune networks.

AIRS: Mapping from IS to AIS Antibody Recognition Ball Antigens Immune Memory Feature Vector Combination of feature vector and vector class Training Data Memory cells²set of mutated ARBs .

Classification Stimulation of an ARB is based not only on its affinity to an antigen but also on its class when compared to the class of an antigen Allocation of resources to the ARBs also takes into account the ARBs¶ classifications when compared to the class of the antigen Memory cell hyper-mutation and replacement is based primarily on classification and secondarily on affinity .

AIRS Algorithm Data normalization and initialization Memory cell identification and ARB generation Competition for resources in the development of a candidate memory cell Potential introduction of the candidate memory cell into the set of established memory cells .

AIRS: Performance Evaluation Fisher¶s Iris Data Set Pima Indians Diabetes Data Set Ionosphere Data Set Sonar Data Set .

1 74.9 95.9 .0 84.7 96.2 Sonar 84.6 Diabetes 74.Classification Accuracy Important to maintain accuracy AIRS1: Accuracy AIRS2: Accuracy Iris 96.0 Ionosphere 94.

Features No need to know best architecture to get good results Default settings within a few percent of the best it can get User-adjustable parameters optimize performance for a given problem set Generalization and data reduction .

aiNET: Artificial Immune Network for Data Mining .

Problem description More benchmark problem in this case Assume a set of unlabelled vectors We can ask the questions: Is there a large amount of redundancy? Are there any groups or subgroups intrinsic to the data? What is the structural or spatial distribution? .

aiNET: Immune principles employed B-cells (antibodies) Antigens Antibody/antigen binding Clonal selection process Immune network theory Combined with statistical analysis tools .

5 Metadynamics: 2. Network suppression: 5. 2. . Diversity: 6.4 Clonal suppression: 2.3 Clonal interactions: 2.Data mining: Immune Network Algorithm 1.2 Affinity maturation: 2.6 Network construction: 3. Initialization: create an initial random population of network antibodies. do: 2. Network interactions: 4. Antigenic presentation: for each antigenic pattern.1 Clonal selection and expansion: 2. Cycle: repeat Steps 2 to 4 until a pre-specified number of iterations is reached.

Data mining: Mapping from IS to aiNET Immune System B-cell (antibody) Antigen Binding Cell cloning Somatic hypermutation Immune network Metadynamics aiNET Internal data vector Training data vector Calculation of Euclidean distance Duplication of internal data vectors Affinity proportional mutation Network of internal data vectors Removal and creation of internal data vectors .

6 0 .8 8 8 88 8 88 8 88 8 8 8 88 8 8 8 8 88 0 .2 1 Result immune network .2 0 7 7 777 8 8 7 7 7 8 8 8 7 88 8 7 7 8 7 77 7 7 888 88 8 7 7 7 8 7 7 8 8 7 88 8 7 8 77 7 88 8 8 77 77 88 88 8 8 77 8 8 77 77 7 7 8 7 7 0 .8 7 77 7 7 77 7 0 0 .4 0 .4 0 .Data mining: Clustering (aiNet) Limited visualisation Interpret via MST or dendrogram Compression rate of 81% Successfully identifies the clusters Training Pattern T ra in in g P a tte rn s 1 1 11 1 1 11 1 11 1 1 1 11 11 1 11 11 1 1 1 111 1 1 1 1 1 11 11 11 1 111 1 11 1 1 111 11 11 1 11 11 11 111 1 11 11 11 11 11 1 1 11 1 1 1 1 1 5 55 1 1 1 1 2 22 11 11 1 2 1 1 1 1 1 1 1 1 22 22 22 1 1 5 5 55 1 555 5 2 22 11 11 1 22222 2 2 5 555 1 1 5 55 2 22222 2 22 555 5 5 222 2 1 1 1 1 22 11 6 66 5555555 5 11 6 6 1 11 2222 6 5555 5 5 5 5 2 2 2 2 1 6 55555 5 555 5 2 1 1 22 22 1 1 1 55 5 5 5 5 5 22222 1 16 1 22222 2 22 2 22 55 5 5 5 5 55 1 1 1 1 1 22 1 22 2 2 1 11 1 1 1 5 5 55 1 5 1 5555 5 5 55 1 2 22 1 1 11 1 1 11 11 11 11 1 1 1 11 11 1 111 11 1 1 11 11 1 1 1 1 11 11 1 11 1 1 1 1 11 1 11 11 1 11 1 11 111 1 1 1 1 111 1 11 1 1 111 1 1 1 1 1 1 1 11 4 4 4 4 4 44 44 4 4 444 4 4 44 44 44 4 4 4444444 44 4 44444 4 44 4 4 444 4 44 4 4 4 44 4 4 44 4 444 4444 4 4 44 4 44 4 4 44 444 4 44 3 3 33 3 33 3 3 3 33 3 3 3 3 33 3 3 3 3 33 33 3333 3 3 3 33 3 33 3 33 3 33 33 3 33 3 3 3 33 333333 3 3 333 3 3 3 3333 3 3 3 3 3 3 3 33 3 3 3 33 33 3 0 .6 0 .

Data mining:Hierarchical Clustering (aiNET) 0 1-1 1-2 1-3 1-4 1-5 2-1 2-2 2-3 2-4 .

Other Interesting Applications Immune Network for continuous learning (Neal 2002) Track moving data over time Maintains clusters in absence of patterns Useful for dynamic environments Continuous Classification Email classification of interesting/non-interesting emails Changing profile of the user Maintain classification accuracy Comparable to Naïve Bayes .

This makes it context dependant Could this be useful for Web Mining? .New Trends Danger Theory Not self/non-self but Danger/Non-Danger Immune response is initiated in the tissues. Danger Zone.

Summary Covered much. but there is much work not covered (so apologies to anyone for missing theirs) Immune metaphors Antibodies and their interactions Immune learning and memory Self/non-self ‡ Negative selection Application of immune metaphors .

Self-Assertion may have something to say to AIS .The Future Rapidly emerging field Much work is very diverse Framework helps a little More formal approach required? Wide possible application domains What is it that makes the immune system unique? More work with immunologists Theories such as Danger theory.

The Future (2) ARTIST: A Network for Artificial Immune Systems (EPSRC funded network) Work towards: A theoretical foundation for AIS as a new CI Extraction of accurate metaphors Immune System Modelling Application of AIS Train PhD students Fund workshops/meetings Coordinate and Disseminate UK based AIS research (links to Europe) .

The Future (hopefully) IT IS: Information Technology Inspired by the Immune System FP 6 IP: 16 institutions across Europe Create a European Library of immune algorithms Theoretical analysis of AIS Application of AIS ‡ Autonomous boat ‡ Immunoinformatics ‡ Web Mining Modelling of Immune System .

Sokolova. de Castro. Springer erlag. Immunocomputing: Principles and Applications by Alexander O. Tarakanov. January 1999. Springer erlag. April 2003. ictor A. November 2002.AIS Resources: Books Artificial Immune Systems and Their Applications by Dipankar Dasgupta (Editor) Springer erlag. Jonathan Timmis. Artificial Immune Systems: A New Computational Intelligence Approach by Leandro N. Svetlana P. . Skormin.

Special Track on Artificial Immune Systems at Genetic and Evolutionary Computation Conference (GECCO). 2003. July 12-16. 2003. Special Session on Immunity-Based Systems at Seventh International Conference on Knowledge-Based Intelligent Information & Engineering Systems (KES). UK. The University of Nottingham. August 10. USA. UK. Napier University. Australia.AIS Related Events in 2003: Special Session on Artificial Immune Systems at the Congress on Evolutionary Computation (CEC). Mexico. Edinburgh. Canberra. 12 August 2003. Tutorial on Immunological Computation at International Joint Conference on Artificial Intelligence (IJCAI). University of Oxford. 2003. 2003 . December 8-12. Second International Conference on Artificial Immune Systems (ICARIS). 2003. UK. September 1-3. September 3-5. Tutorial on Artificial Immune Systems at 1st Multidisciplinary International Conference on Scheduling: Theory and Applications (MISTA). Acapulco. Chicago.

Sign up to vote on this title
UsefulNot useful