You are on page 1of 125

Evaluating compound cloud based solutions in an ecosystem context

Master thesis - Bernard Verhoeven #IKU-0307386

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Supervisors: Dr. S.R.L. Jansen First supervisor Utrecht University S.Jansen@cs.uu.nl Author: Bernard Verhoeven 0307386 bernard@bverhoeven.nl @b_verhoeven http://cloud.bverhoeven.nl

Dr. I. van de Weerd Second supervisor Utrecht University I.vandeWeerd@cs.uu.nl

L. Nguyen External supervisor Deloitte Consultancy LoNguyen@deloitte.nl

University: Master Business Informatics Department of Information Science Beta Faculty Utrecht University Company: Deloitte Consulting Technology - Emerging Solutions - Cloud computing Laan van Kronenburg 2, Amstelveen November 2010 - April 2011 Document: Filename: Final thesis 16-06-2011 - public.docx Last print: Thursday 16-06-11 14:31:00 Size: 5.023 KB Pages: 125 Word count: 47.217 Characters: 242.142 Dodo: The way a players behavior influences other players in the ecosystem plays an important role in this thesis. The dodo was a bird that could not fly and could only stroll. Its only method of defense was its sharp pecker. This wasnt a real problem, because it had no enemies in his ecosystem and ate seeds and fruit. The dodo lived on Mauritius, a small island off the east coast of Africa and was discovered by hungry Portuguese sailors. Because the bird could not run away, it became an easy meal and was quickly extinguished (Wikipedia, 2011). Hence the dodo is a representation of the weaker species in the ecosystem. Cover: By Michael Kutsche. http://michaelkutsche.com/
Chapter 1. Introduction - 1.1 Research introduction Page 2 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Abstract
In this research, the Cloud solution Risk Identification and Selection of Precautions (CRISP) method is created that allows customers to identify risks early on in the process of selecting or adopting a cloud solution that is delivered by multiple parties. The CRISP method is based on scientific literature and fine-tuned with the help of experts in the field of cloud computing and outsourcing. The method identifies eight different risks by systematically working through a cloud-based solution and takes into account the applicability of risks as well as their impact on the business. Based on the risks found, the method suggests precautions that can be taken to mitigate the identified risks. To be able to visualize cloud computing solutions, the Cloud Solution Modeling Method is also developed. The CSMM produces a Cloud Solution Supply Network and is an extension of the Software Supply Network by Boucharas et al. (2009). The CRISP method is validated by executing it on a project that Deloitte Consulting is currently carrying out at Canon. The results from this execution, as well as the experiences during execution were discussed with people involved in the project. The validation led to the conclusion that the method is capable of identifying risks before cloud solutions are implemented and that the identified risks were relevant. However, not all of the proposed precautions were easy to implement in the case study at Canon.

Due to the confidential nature of Deloittes work at Canon, large parts of chapter 7 of this public version of my master thesis had to be censored. Where possible, some generic conclusions that are drawn based on validation at Canon are available. Where text is removed, this is indicated as such.
Chapter 1. Introduction - 1.1 Research introduction Page 3 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Table of contents
Chapter 1
1.1 1.2 1.3 1.4 1.5 1.6 1.7

Introduction ............................................................................................... 10
Research introduction ............................................................................................ 10 Objective and Problem Statement ......................................................................... 11 Research Questions ................................................................................................ 11 Scientific relevance ................................................................................................. 11 Social relevance ...................................................................................................... 12 Definitions and Scoping .......................................................................................... 12 Contents of this thesis ............................................................................................ 14

Chapter 2
2.1 2.2 2.3

Research approach ..................................................................................... 15


Research Model ...................................................................................................... 15 Challenges ............................................................................................................... 15 Design Science Research ......................................................................................... 16

Chapter 3
3.1 3.2

Theoretical background .............................................................................. 23


Software ecosystems .............................................................................................. 23 Cloud computing ..................................................................................................... 28

Chapter 4
4.1 4.2 4.3

Cloud Solution Modeling Method ............................................................... 35


Requirements on the CSMM .................................................................................. 35 Detailed description of the CSMM ......................................................................... 36 Example of a Cloud Solution Supply Network ........................................................ 39

Chapter 5
5.1 5.2 5.3 5.4

Risks and precautions ................................................................................. 40


Requirements.......................................................................................................... 40 Literature study....................................................................................................... 42 Expert interview outcomes ..................................................................................... 45 Final risks and precautions ..................................................................................... 50

Chapter 6
6.1 6.2 6.3 6.4

CRISP Method ............................................................................................. 53


Model of the concepts in the CRISP method .......................................................... 53 High level overview of the CRISP method .............................................................. 55 Detailed view of the final CRISP method ................................................................ 57 Usability of the method .......................................................................................... 60

Chapter 7
7.1 7.2 7.3

Validation of the CRISP method .................................................................. 62


Introduction to the solution ................................................................................... 62 Execution of the CRISP method .............................................................................. 62 Validation of the results of the CRISP method ....................................................... 62

Chapter 8
8.1 8.2 8.3 8.4

Perception of risks and precautions ............................................................ 64


Interview setup ....................................................................................................... 64 Interview results regarding risks and precautions.................................................. 65 Interview results regarding customers vs. vendors ................................................ 72 Summary of the interview results .......................................................................... 75
Chapter 1. Introduction - 1.1 Research introduction Page 4 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 9
9.1 9.2 9.3 9.4 9.5

Conclusion and Discussion .......................................................................... 77


Conclusion............................................................................................................... 77 Results from interviews .......................................................................................... 79 Discussion and Limitations...................................................................................... 80 Further research ..................................................................................................... 81 Application of the CRISP method in consulting practice ........................................ 82

Chapter 10 Works Cited ................................................................................................ 85 Chapter 11 Special thanks............................................................................................. 92 Chapter 12 Appendices ................................................................................................. 93

Chapter 1. Introduction - 1.1 Research introduction Page 5 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Table of figures
Figure 1 - Conceptual model of cloud computing ecosystems. 13 Figure 2 - Research model. 15 Figure 3 - Information Systems Research Framework, adjusted for this thesis 21 Figure 4 - Design cycle by Takeda et al. (1990) 22 Figure 5 - Visualization of value capturing and value creation strategies in an ecosystem. 24 Figure 6 - Nice player strategies: Ecosystem influence vs. dependency. 27 Figure 7 - Ontology of cloud computing, based on Abdat et al. (2010) and Hagel et al. (2010) . 29 Figure 8 - SaaS Ecosystem based on Abdat et al. (2010). 31 Figure 9 - Task support vs. flexibility of cloud computing services. 34 Figure 10 - Example Software Supply Network of an on-premise solution. Modeled following the method by (Boucharas, Jansen, & Brinkkemper, 2009). 35 Figure 11 - Conceptual model of the Cloud Solution Modeling Method. 36 Figure 12 - Example Software Supply Network of a SaaS solution. Modeled following an adjusted method based on (Boucharas, Jansen, & Brinkkemper, 2009). 39 Figure 13 - Model of the concepts in the method's scope. 54 Figure 14 - High level overview of the final method. 56 Figure 15 - Detailed view of the final CRISP method. 58 Figure 17 - Visualization of difference in customers' and vendors' perception of risks. 73 Figure 18 - Visualization of possible explanation of differences between scientific context and real life. 75 Figure 19 - Example of raw expert interview results. 105

Chapter 1. Introduction - 1.1 Research introduction Page 6 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Table of tables
Table 1 - Concepts from the Conceptual model 14 Table 2 - Description of the concepts from the CSMM. 38 Table 3 - Entity and concept mapping to form method entity types 41 Table 4 - Risks based on literature. 42 Table 5 - Mapping of risks on entity types. 43 Table 6 - Precautions based on literature. 44 Table 7 - Precautions mapped on risks (based on literature) 45 Table 8 - Expert interview results 48 Table 9 - Risks suggested by experts that were not included in method. 49 Table 10 - Final risks based on literature and expert interviews 50 Table 11 - Mapping of risk on entity type, after expert interviews 51 Table 12 - Final precautions based on literature and expert interviews 52 Table 13 - Final risks - precautions mapping based on literature and expert interviews 52 Table 14 - Concept descriptions from the conceptual model of the method's scope (Figure 13). 55 Table 15 - Description of the activities from the high level overview of the final method. 56 Table 16 - Description of the concepts from the high level overview of the final method. 57 Table 17 - Description of the activities from the detailed view of the final CRISP method. 59 Table 18 - Description of the deliverable concepts from the detailed view of the final method. 59 Table 29 - List of (potential) cloud computing customers 65 Table 30 - List of cloud computing vendors 65 Table 31 - Description of the IT departments of the interviewed customers 66 Table 32 - Definitive risks in cloud solutions. 78 Table 33 - Definitive precautions to the risks in cloud solutions. 79 Table 34 - List of entities in CSSN. 94 Table 35 - V.1: Detailed description of the risk Solution lacks solid availability. 95 Table 36 - V.1: Detailed description of the risk Solution is insecure. 96 Table 37 - V.1: Detailed description of the risk Solution or ecosystem produces lock-in. 96 Table 38 - V.1: Detailed description of the risk Service or ecosystem becomes less popular 96 Table 39 - V.1: Detailed description of the risk Little or no influence on third parties. 97 Table 40 - V.1: Detailed description of the risk Entity lacks innovation. 97 Table 41 - V.1: Detailed description of the risk Lack of dedication to the ecosystem. 98 Table 42 - V.1: Detailed description of the precaution "Deploy escrow services". 99 Table 43 - V.1: Detailed description of the precaution "Make regular backups". 100 Table 44 - V.1: Detailed description of the precaution " Sign contracts on required service levels ". 100 Table 45 - V.1: Detailed description of the precaution " Choose an ecosystem of significant size". 101 Table 46 - V.1: Detailed description of the precaution " Choose solution that support open data formats ". 101 Table 47 - V.1: Detailed description of the precaution "Beware of trends in the market and your ecosystem". 101 Table 48 - V.1: Detailed description of the precaution "Choose Certified and Auditable partners". 102 Table 49 - V.2: Detailed description of the risk Solution lacks solid availability 106 Table 50 - V.2: Detailed description of the risk Solution is insecure 107
Chapter 1. Introduction - 1.1 Research introduction Page 7 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Table 51 - V.2: Detailed description of the risk Solution or ecosystem produces data lock-in 107 Table 52 - V.2: Detailed description of the risk Little or no influence on third parties 108 Table 53 - V.2: Detailed description of the risk Entity lacks innovation 108 Table 54 - V.2: Detailed description of the risk Lack of dedication to the ecosystem 109 Table 55 - V.2: Detailed description of the risk Small parties 109 Table 56 - V.2: Detailed description of the risk Product lifecycle 110 Table 57 - V.2: Detailed description of the precaution "Deploy escrow services" 111 Table 58 - V.2: Detailed description of the precaution "Make regular backups" 112 Table 59 - V.2: Detailed description of the precaution " Sign contracts on required service levels " 112 Table 60 - V.2: Detailed description of the precaution " Choose an ecosystem of significant size 113 Table 61 - V.2: Detailed description of the precaution "Choose Certified and Auditable partners" 113 Table 62 - V.2: Detailed description of the precaution "Certificates and auditability" 113 Table 63 - V.2: Detailed description of the precaution "Make use of an intermediary" 114 Table 64 - V.2: Detailed description of the precaution "Choose solutions on one shared platform 114 Table 65 - V.2: Detailed description of the precaution "Choose a mature and reputable company" 114 Table 66 - Example COLLECTION OF ACTORS. 115 Table 67 - Example COLLECTION OF FLOWS. 115 Table 68 - List of entities in CSSN and their type. 116 Table 69 - ENTITY-RISK MAPPING. 117 Table 70 - ENTITY-RISK MAPPING OF APPLYING RISKS. 118 Table 71 - Weighing the BUSINESS IMPACTS of APPLICABLE RISKS 119 Table 72 - Weighed ENTITY-BUSINESS IMPACT of RISK -mapping. 120 Table 73 - Most important risk using ordering method 1. 120 Table 74 - Actors ordered by weight of risks. 121 Table 75 - Risks ordered by weight of entities. 121 Table 76 - Top three risks from ordering method 3. 122 Table 77 - Relevant part of the risk-precaution mapping (Table 13) for use with this example. 122 Table 79 - Quantitative results from interviews with vendors and customers 125

Chapter 1. Introduction - 1.1 Research introduction Page 8 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Preface
Before you, lays the culmination of my life as a master student of the Master Business Informatics, taught at the Department of Information Science at the Utrecht University. My Master Thesis! As my thesis is finished, I can be honest, and to be honest: I wasnt really looking forward to the months of solitude, piles of scientific papers and abstract-academic-thinking. But actually, I had a really good time! This preface is the perfect place to honor the two main ingredients that made writing my thesis into a pleasant and challenging endeavor. The first ingredient is my first supervisor Slinger Jansen. Long before I begun with this research he aroused my interest for Software Ecosystems during a Capita Selecta I did on the Ruby on Rails Ecosystem. The numerous meetings I had with Slinger regarding my thesis were always full of positive energy and new ideas; a great way to move on with my research with a good view on where it should go. This had a very positive influence on my schedule and made it possible to finish my thesis within the limited timeframe that I had imposed on myself. The second ingredient was Deloitte Consulting, the company that dared to give me the opportunity to leverage their resources to complete my research. Not only is the environment at Deloitte very stimulating to work in, writing at Deloitte also allowed me to talk to their experts for the development of my method, opened doors to interesting parties in the cloud market, such as Microsoft, Eloqua and SalesForce and gave me access to their vast Knowledge Exchange, On-line learnings about cloud computing and Intellinet. A total of 22 meetings were scheduled to make this research possible and I would like to thank all the people who made time in their busy schedules to meet with me and answer all my tricky questions. There are many people to thank, so to be able to name them all I created a separate chapter to do so (Chapter 11). Not only were those meetings with people from the business very interesting from a scientific point of view (they really helped making my method less abstract and more business savvy), they also made me realize that cloud computing is a really interesting field of expertise that is still in its infancy and is bound to change a lot in the upcoming years. This thesis was written between the 1st of November 2010 and the 21st of April 2011 and is solely written by the author, all works that formed the foundation for this research are cited as such.

Enjoy the reading!

Bernard Verhoeven.

Chapter 1. Introduction - 1.1 Research introduction Page 9 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 1
1.1

Introduction

Research introduction

Cloud computing is a different way of delivering computing. Rather than installed on-site, the computing power or the software that runs on it, is offered as a service via the Internet. Although there are many different views on cloud computing, the consensus is that there are three forms of cloud computing: Software- , Platform- and Infrastructure- as a Service. A well-known example of Software as a Service is webmail. The customer does not have to install an email client, but uses the online software to send, receive and archive their mail. This way of delivering computing has some significant advantages. Large upfront investments are no longer necessary, as the service is paid for via a monthly or a pay-per-use based fee. In some way the software is rented (Abdat, Spruit, & Bos, 2010), (Levinson, 2007). The software runs on external servers, maintained by dedicated professionals, rather than by the in-house IT department. This makes cloud computing often more reliable than internal housed computing solutions (Dubey & Wagle, 2007), (Hofmann & Woods, 2010). With Software as a Service (SaaS), there is only one instance of the software, running on only one platform, which makes platform-related bugs scarce. If bugs do come up, they are often easily repaired because of the single code base (Kaplan, 2007), (Levinson, 2007). Centralizing the computational power needed for different organizations, allows for more efficient capacity management. The average in-house server is only used about 5 to 10% of the time (Vaquero, Rodero-Merino, Caveres, & Lindner, 2009), (Armbrust, et al., 2010).

Besides the above mentioned advantages, using cloud computing also has some disadvantages. The (perceived) security of the companys data is always a difficult point when the data is physically stored elsewhere (Hofmann & Woods, 2010). To add to this security issue, a SaaS solution often runs on a virtualized cloud computing platform, where it is often hard to tell where data exactly is (Marks, 2008). The users of cloud solutions are directly dependent on the performance of the cloud computing provider. If the solution becomes unavailable for any reason, the consequences could be far-reaching and might be hard to prevent (Fonseca, 2008), (Hofmann & Woods, 2010). Because of the single code base, customization of a service to suit individual needs is often expensive or only possible to some extent (Abdat, Spruit, & Bos, 2010). Not only is the user dependent on the SaaS provider, one of the characteristics of SaaS solutions is that they are often built from other services, not necessarily built or maintained by the same provider (Abdat, Spruit, & Bos, 2010). This can lead to complex multi-party dependencies where it might be hard to understand who is responsible for what part of the final solution.

Chapter 1. Introduction - 1.1 Research introduction Page 10 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

There are currently no methods available that facilitate the customer in the identification and assessment of risks in cloud based solutions that are delivered by multiple parties. Examples of such risks are What will happen to my cloud solution if provider X goes bankrupt/moves on to another market/etc? and How big is the chance of provider X leaving the market?. Methods exist to assess parts of these risks, but a comprehensive method is not yet available.

1.2

Objective and Problem Statement

The objective of this research is to create a method that supports (potential) users in identifying and assessing the operational risks and challenges ahead when taking a multi-party cloud solution into service. This method will focus on: The challenges that come from the ecosystem(s) a solution is part of. The complexities that arise from the multi-party-aspects of the solution. The specific cloud computing characteristics that make cloud solutions different from standard on premise solutions.

Being able to point out weak spots in the SaaS-suppliers business or ecosystem for a particular solution will enable better project management. As this method helps to analyze suppliers and ecosystems, it can of course also be used by those same suppliers to evaluate their propositions, allowing them to compose better propositions to potential customers. The formal problem statement of this project is: Complex Cloud solutions that consist of multiple services and companies seem too complex to properly identify and assess the risks involved.

1.3

Research Questions
How can a method be developed that identifies and assesses risks caused by service and company dependencies in cloud based solutions? a. How can service and company dependencies be identified and modeled? b. What kinds of risks exist in these dependencies? c. Can a method be developed that derives risk from these dependencies? d. Are risks better identified with the method than without it?

1.4

Scientific relevance

The field of SaaS Ecosystems is a relatively new field on the research agenda of IS/IT researchers (Jansen, Finkelstein, & Brinkkemper, 2009). This upcoming field contains new ways of doing business, creating partnerships and entering dependencies that were not often seen before on such a scale. These new ways of doing, entail new risks that have not yet been fully examined and described. This method will help to establish a new way of evaluating suppliers and their ecosystem. Although initially not intended as such, the outcomes of these evaluations could also be used to benchmark suppliers or ecosystems.

Chapter 1. Introduction - 1.2 Objective and Problem Statement Page 11 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

To summarize the scientific relevance: The method that will be created will bring new insights to the field of SaaS ecosystems and could be used in other research in the future. Not only will this research add to the body of knowledge on SaaS ecosystems, it will also expose new subjects that may evolve into interesting research possibilities in the future.

1.5

Social relevance

At the moment, there is not yet an integrated way to assess both the supplier and the ecosystem the supplier operates in. When such a method is constructed, it will help to identify problems early on. This saves future customers time, money and possibly also a lot of frustration. On the other hand, the method will allow companies who are part of an ecosystem to self-evaluate and improve their value proposition.

1.6

Definitions and Scoping

This section will further explain and narrow the scope used in the different parts of this research. Figure 2 shows the conceptual model of the scope of the method. The method is constructed around the PROBLEM. The PROBLEM is for example the USERs which wants to move his sales process into the cloud. A USER can be either a specific CUSTOMER or a POTENTIAL MARKET. The SOLUTION is het collection of SERVICES that are used by the INTEGRATOR to solve this PROBELEM and a CONSULTANT can be of help to tailor the SOLUTION to the business needs of the USER. A SERVICE is never an isolated entity, it always runs on a PLATFORM, and can make use of other SERVICEs to complete its task. PLATFORMs and SERVICEs are always built by a SERVICE VENDOR, but it is possible that the organization supplying the SERVICE to the USER is different from the SERVICE VENDOR. This SERVICE PROVIDER can for example be a cloud hosting company that is specialized in delivering services. The SERVICE VENDOR can be a software company that is specialized in building software and has little knowledge of hosting applications. To conclude, the SERVICE PROVIDER and VENDOR can be represented by a RESELLER.

Chapter 1. Introduction - 1.5 Social relevance Page 12 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

CUSTOMER USER POTENTIAL MARKET


1..* 1..* 0..* advises on
Builds has

0..* has

PROBLEM

1..* 1..*

SOLUTION

consists of

0..*

0..*

CONSULTANT

INTEGRATOR
0..*
uses

1..* 0..*

1..*

PLATFORM
0..*
builds

1..* runs on 1..* 1..*


builds

SERVICE
1..* 1..* 0..* uses

delivers

0..1

SERVICE PROVIDER
1..* 1..* represents

1..* delivers for 0..*

SERVICE VENDOR
1..* represents

0..*

0..*

RESELLER

0..*

sells

delivers

Figure 1 - Conceptual model of cloud computing ecosystems.

A complete description of the entities from Figure 2 can be found in Table 1.


Concept PROBLEM Description This concept represents the problem the SOLUTION will be made for. A PROBLEM can be a real problem of a CUSTOMER that must be solved We need a new CRM solution or a problem that corresponds to a specific demand from a POTENTIAL MARKET There is no specific CRM solution for the telephone business. Represents the (collections of) SERVICEs that together form the SOLUTION to the PROBLEM of the USER The customer of the SOLUTION. This could be a specific organization (CUSTOMER) or a POTENTIAL MARKET Specific customer of the SOLUTION. Non-specific customer of the SOLUTION. The INTEGRATOR is the entity that builds the SOLUTION out of the SERVICEs and PLATFORMs. It could be a separate entity, but it could well be that the INTEGRATOR and the CONSULTANT are actually the same organization. However, because the INTEGRATOR and the CONSULTANT perform very different tasks, it is important to distinguish between them.

SOLUTION USER CUSTOMER POTENTIAL MARKET INTEGRATOR

Chapter 1. Introduction - 1.6 Definitions and Scoping Page 13 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Concept CONSULTANT

Description The CONSULTANT is the advisor on the SOLUTION. It is in most cases responsible for requirement analysis, selecting SERVICE VENDORS, etc. SERVICE The SERVICE is the main component of the SOLUTION. A SERVICE can consist of or use other SERVICEs, is built by a SERVICE VENDOR and runs on a PLATFORM. A SERVICE can be a small component that only solves a small part of the PROBLEM (a SERVICE for real-time currency conversion in an financial accounting package), but it can also be a complete software package (SalesForce). PLATFORM A SERVICE needs a PLATFORM to run on. A PLATFORM could be another SERVICE (like the Force.com-platform), but it could also be a PaaS like Google App Engine. SERVICE VENDOR The SERVICE VENDOR is the entity that built the SERVICE. It can offer its SERVICE via a SERVICE PROVIDER or deliver the SERVICE directly to the USER. In the latter case, the SERVICE VENDOR also acts as the SERVICE PROVIDER SERVICE PROVIDER The SERVICE PROVIDER is the entity that delivers the service to the USER. This can be the same entity as the SERVICE VENDOR, but it could also be a separate entity. Separate SERVICE PROVIDERS often have more experience with datacenters then SERVICE VENDORS. RESELLER The RESELLER is a relatively easy entity that represents SERVICE VENDORS or SERVICE PROVIDERS and sells their service to customers. The RESELLER could for example sell extra support or help with the configuration of the SERVICE. Table 1 - Concepts from the Conceptual model

1.7

Contents of this thesis


Research Approach - How the research is performed and which steps it contains. Theoretical background - Contains the theoretical background on software ecosystems and cloud computing, the two main topics of this research. Cloud Solution Modeling Method - The method to create Software Supply Networks by Boucharas, Jansen & Brinkkemper (2009) was updated to support cloud solutions. Risks and precautions - Risks and ways to mitigate those risks are the two primary parts of the method. This chapter shows the risks that were found in literature and how they were influenced by experts opinions. Method - The construction of the method to identify risks and limitation on the method. Validation of the method - How the method was validated and what the results of the validation were. Risks and precautions in the real world - The results from the interviews with vendors and customers of cloud computing. Conclusion & Discussion - The final results of this research, the limitations, advice for further research and a note on how this research can be used in everyday consulting practice. Works Cited - Literature that was used to base this research on. Special thanks - The names of all the fantastic people that helped me during this research. Appendices - Everything else.

This thesis consists of eleven more chapters:

Chapter 1. Introduction - 1.7 Contents of this thesis Page 14 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 2
2.1

Research approach

Research Model

Figure 2 shows the basic steps of which this research will consist.

Discover problem area

RESEARCH PLAN

Study literature

SCIENTIFIC BACKGROUND input for

Create 1st version of the method

FIRST VERSION OF THE METHOD input for EXPERT OPINIONS ON THE METHOD input for

Interview experts
(evaluate method)

Create final version of he method

FINAL METHOD

Interview customers Interview vendors


Interview the business

VIEWS ON RISKS & PRECAUTIONS

Validate final method

VALIDATION RESULT

Conclude research

input for CONCLUSION & DISCUSSION

Figure 2 - Research model.

Figure 2 is based on methods for design research that will be further discussed in section 2.3.

2.2

Challenges

During the execution of this thesis-plan, several challenges will be met. In this part the most relevant challenges will be described.

Chapter 2. Research approach - 2.1 Research Model Page 15 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Too abstract - During the evaluation of a partner and his ecosystem, the focus will be on overall performance, position in the ecosystem, the ecosystems health, etc. The challenge is to keep this high-level focus, while withholding the method from becoming too abstract. A too abstract method would only come up with general findings of little use. A solution to this problem could be to develop a multi-level method. Developing a multi-level method would allow the user to stay on a high overall level and only go more in-depth on fields where this is necessary. For example, if a potential supplier also serves four mayor banks, it might not be necessary to put much as much energy into the financial analyses of this supplier, as it would when this supplier has had financial problems in the last few years.

Availability of information - The purpose of the method is to show what the possible problems could be when a particular partner or supplier is chosen. Whether or not this question can be answered depends on the availability of information about the entity under evaluation. In particular the information about the performance, future plans and intentions of an entity might be hard to gather. During the formation of the method, this informationvoid should be kept in mind. In an ideal situation, the method should be usable, even if such information is not at hand. On the other hand, if such information is not easily available, this could also be a bad sign in itself. Further research on this topic should shed more light on this topic.

Validation of use - The method will be validated by using a case. The method is used to assess the situation and the problems that might occur. After that, the outcomes are checked by interviewing people involved in the project. However, if the case did not work out, it might be hard or even painful to have those involved admit they did not see a certain problem coming.

2.3

Design Science Research

The result of this thesis will be a usable method. This research can therefore be classified as design research, research that delivers an artifact as meant by (Hevner, March, Park, & Ram, 2004).

2.3.1 Design-Science Research Guidelines


To make sure the research is carried out properly, the seven Design-Science Research Guidelines, as set out by Hevner et al (2004) will be applied. However, it has to be taken into account that the seven principles were designed to support Information System design research. As this research will deliver a method, some of the principles might be (partially) irrelevant. Guideline 1: Design as an Artifact - Design-science research must produce a viable artifact in the form of a construct, a model, a method, or an instantiation. The purpose of this research is to create a method that can be used to identify the risks in cloud computing solutions and propose precautions to counteract these risks. This method to be produced will be a viable artifact as described by Hevner et al. (2004). They state prior *+ it was not clear if such an *artifact+ could be constructed. It was not clear how to
Chapter 2. Research approach - 2.3 Design Science Research Page 16 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

describe or represent it, or how well it would perform. and this is exactly what this research will be about. At the moment, it is not clear how to describe or develop a method that can identify risks in cloud solutions. The goal of this research is to prove that developing such a method is possible and that the method will be able to identify those risks.

Guideline 2: Problem Relevance - The objective of design-science research is to develop technology-based solutions to important and relevant business problems. Cloud computing solutions are relatively young. The impacts of the risks businesses run when using these solutions are still unclear and so are the precautions that could be taken to counteract those risks. To add to the complexity, cloud solutions often do one thing very well, so integration of different solutions to tackle a business problem is no exception. These combined solutions bring extra dependencies and risks to the table. Businesses will need to look into these risks before taking cloud solutions into service. Not only for (potential) customers will a method to identify risks be useful. Also vendors of cloud computing could use the method to evaluate their own offerings and deliver less risky solutions to their customers.

Guideline 3: Design Evaluation - The utility, quality, and efficacy of a design artifact must be rigorously demonstrated via well-executed evaluation methods. The effectiveness of the method will be tested by using the method to evaluate an existing cloud solution that consists of multiple solutions. In an ideal situation, the method would be tested on more than one cloud solution, but due to the time available, this is not possible. However, during the development of the method there will be a round of expert interviews. These interviews will help to take their experience in multi-party cloud computing solution into account.

Guideline 4: Research Contributions - Effective design-science research must provide clear and verifiable contributions in the areas of the design artifact, design foundations, and/or design methodologies. Hevner et al. state Most often, the contribution of design-science research is the artifact itself. In this research, the method that is able to identify risks and select appropriate precautions is only part of the contribution. The research also aims to create a way to model cloud computing solutions. Lastly, the research focuses on discovering risks and precautions in literature and through expert interviews. Although these risks and precautions are a vital part of the method, they could also be used separately and are a contribution to the scientific community in itself.

Guideline 5: Research Rigor - Design-science research relies upon the application of rigorous methods in both the construction and evaluation of the design artifact.
Chapter 2. Research approach - 2.3 Design Science Research Page 17 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

During the design of the method, the design cycle by Takeda et al. (1990) will be used to add the necessary research rigor. How the principles from their design research cycle are used in this research will be described in section 2.3.3. Applying the design research cycle will make sure that all the important steps in the research are made.

Guideline 6: Design as a Search - The search for an effective artifact requires utilizing available process means to reach desired ends while satisfying laws in the problem environment. Design research is used when the nature of the problems is so complex that it is not possible to solve the solution by listing all the solutions and their characteristics and select the best solution mathematically. Design research is therefore regarded as a search in which a solution is adjusted by means of expert interviews, validation, etc. This research will be constructed as such and an example of this method can be seen in the way the literature research will be done. Rather than performing a strict literature research, where queries are determined beforehand and only those papers and books are included that match the exact search criteria, a more loose literature exploration is used to be able to react to new findings along the course of the exploration.

Guideline 7: Communication of Research - Design-science research must be presented effectively both to technology-oriented as well as management-oriented audiences. As the artifact that is to be developed is a method that allows the identification of risks and proposes precautions based upon those risks, there is not really a difference between the technology and the management-oriented audience. However, if the customer is regarded as the management-oriented audience and the consultant who executes the method is regarded as the technology-oriented audience, differences in communication are indeed necessary. The consultant will be interested in how the method is applied, how the risks are identified and what their possible impact on the business will be, whereas the customer will primarily be interested in what precautions he needs to take to be as risk-free as possible.

2.3.2 Information Systems Research Framework


Hevner et al (2004) have also developed the Information Systems Research Framework. How this thesis fits within this framework is shown in Figure 3. The elements and how they relate to the research are described below. 2.3.2.1 Environment The environment defines the problem space in which reside the phenomena of interest. (Hevner, March, Park, & Ram, 2004) Organizations - Strategies Strategies are of high importance to this research, as the strategy an organization follows gives a lot of information about how it treats its environment. There is a direct link between
Chapter 2. Research approach - 2.3 Design Science Research Page 18 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

the strategy an organization follows and the risks it inflicts on its customers. Organizations that are hard to reach or follow their own agenda, organizations that exhibit dominator behavior as a keystone, niche players that solely focus on one platform all form risks to their customers.

Organizations - Processes Cloud solutions support an organizations processes and whether this process is a supporting one or part of the organizations primary process is for a large part the determinant of how big the impact a risk on that particular process can be. This research will primarily focus on how certain risks can influence certain processes, not on how a specific disrupted process influences the organizations performance.

Technology - Infrastructure & Communications Architecture In the case of cloud solutions, the Infrastructure and Communications Architecture are closely connected. Cloud solutions are often focused on one task, so in order to solve an organizations problem; multiple solutions need to be connected. This is done over the internet, by means of APIs or comparable technologies. The multitude of systems on which a single solution to a customers problem can depend adds to the complexity and the risks involved in the solution. The infrastructure and communications architecture are therefore important factors in the description of the environment of the method.

Technology - Applications The fact that Hevner et al still talk about Applications is a sign that cloud computing is indeed a new phenomenon. Software as a Service (SaaS) is the new application. It is used in different ways and has many new pitfalls. One of the most important differences to take into account is that the availability and security are the responsibility of the vendor, rather than of the customer.

2.3.2.2 IS Research IS Research consists of two parts; development of an artifact or theory and the justification of this artifact by means of experiments, case studies, etc. Develop / Build - Artifacts As described before in section 2.3.1, the primary objective of this research is to create an artifact. The artifact is the method that will be constructed. It will be constructed based on the requirements from the Environment and the knowledge from the Knowledge Base. During and after its development it will be evaluated by interviews and a case study.

Chapter 2. Research approach - 2.3 Design Science Research Page 19 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Justify / Evaluate - Case Study The Case Study will be performed at an organization that is currently rolling out a multiparty cloud solution for their entire business in Europe. The method will be executed, the risks will be identified and the precautions will be suggested. The outcomes of this execution will be discussed with stakeholders in the project to see whether the outcomes are of value to them. Hevner et al. state that both truth and utility are the primary goals of design research. *+, our position is that truth and utility are inseparable. Truth informs design and utility informs theory. During the evaluation of the developed artifact, it will be determined to which extend it is contains utility and to which extend it is true.

Justify / Evaluate - Interviews Three kinds of interviews are planned. The first interviews will be held after the development of the first version of the method. The objective of these interviews will be to check the scientific findings with the everyday experience from industry experts. Based on the outcomes of these interviews, the method will be improved. The second set of interviews will be held with (potential) customers and vendors of cloud computing solutions. The method that will be developed will be made to support an objective identification of risks, based on scientific literature and expert opinions. Interviews with the business will shed light on how these objective risks are perceived in the real world and on what the best ways are to communicate these risks and precautions. The third and last series of interviews will focus on the outcomes of the case study. Did the method indeed add knowledge and identify risks that were not known before?

2.3.2.3 Knowledge Base The knowledge base provides the raw materials from and through which IS research is accomplished. The knowledge base is composed of foundations and methodologies. (Hevner, March, Park, & Ram, 2004) Foundations - Theories The theories that will be used in this research will come from the two research fields that this research relates to. The first is ecosystems, to be more specific, business ecosystems or better software ecosystems. Theories around software ecosystems (strive to) explain what kind of implications specific strategies within an ecosystem have. The second set of theories will be about cloud computing. It is the specific characteristics of cloud computing that make that it inflicts certain risks on its users.

Chapter 2. Research approach - 2.3 Design Science Research Page 20 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Foundations - Frameworks & Methods Methods and Frameworks that are used in this research are both related to the research itself (this research framework is a good example) and to the two main topics, ecosystems and cloud computing. Methods that are used can for example help in the visualization of cloud solutions.

Methodologies - Validation Criteria Validation will be performed by a case study and the criteria for this validation will be based on the methodologies on validation criteria.

Environment
Organizations Strategies Processes Technology Infrastructure Applications Communications Architecture

Relevance

IS Research
Develop / Build Artifacts

Rigor

Foundations

Knowledge base

Business Needs

Assess

Theories Frameworks Methods Applicable Knowledge Methodologies Validation Criteria

Justify / Evaluate Case Study Interviews

Refine

Applications in the Appropriate Environment

Additions to the Knowledge Base

Figure 3 - Information Systems Research Framework, adjusted for this thesis

Figure 3 shows how the elements of the Information Systems Research Framework that were described above relate to each other in this research.

2.3.3 Design Research Cycle


The five steps of the design research cycle by Takeda, Veerkamp, Tomiyama & Yoshikawa (1990) (bold in Figure 4) are used as the basis for the planning of this research. The steps are: (1) Awareness of the Problem, (2) Suggestion, (3) Development, (4) Evaluation and (5) Conclusion. (Takeda, Veerkamp, Tomiyama, & Yoshikawa, 1990)

Chapter 2. Research approach - 2.3 Design Science Research Page 21 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Object Level Awareness of problem Enumeration of problems

Action Level

Decision on a problem to be solved

Suggestion

Development

Evaluation to confirm the solution

Evaluation

Evaluation to know what is a problem in the soluiton

Decision on a soluiton to be adopted

Conclusion

Decision on an action to be done next

Figure 4 - Design cycle by Takeda et al. (1990)

These five steps are partly visible in the research model (Figure 2). The first step Awareness of the Problem has of course been taken, but is not explicitly modeled. How the awareness of the problem became reality can be read in section 1.1. The awareness of the problem is a hard to define process, awareness is often gradually obtained and it is difficult to pin down when the awareness developed exactly. The activities Discover problem area and Study literature are part of the Suggestion process. In these activities, the field of cloud computing is examined and the research plan and scientific background are constructed. Strictly taken, Discover problem area should come before Study literature, but in practice, these two activities overlap significantly. After the basis for the research has been laid, the design phase is entered. The design and evaluate steps are both taken twice. The first design step is the Create 1st version of the method-activity, in which the first version of the method is created. This design is evaluated in the Interview expertsactivity, which results in the design of the final method in Create final version of the method. However, these three activities combined (Create 1st version of the method, Interview experts and Create final version of the method) can also be regarded as one single development-step. The evaluation of the final method is completed in two ways. The primary evaluation of the final method is the Validate final method-activity. In this activity, the method is used in a real case to see whether it meets the requirements and adds knowledge to the user of the method. However, the Interview the business-activity can also be considered an evaluation. This activity is meant to put the design in context and see how the final method would fit into everyday business activities. The last step in the research is the Conclude research-activity, it produces the summary and conclusion of the research, answers the research questions and allows us to determine whether the research goals were met.

Chapter 2. Research approach - 2.3 Design Science Research Page 22 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 3

Theoretical background

In the previous chapter, the research approach has been laid out. This chapter provides a theoretical background to the research; it will be the Knowledge Base from Hevner et al.s Information System Research Framework (Figure 3). A sound theoretical background is needed to lay the foundation for the development of the method and to help answering the research questions. There are two topics that cover the most important aspects of this research. Software ecosystems - Companies that work together to deliver a product or service are part of a so called ecosystem. The composition, amount of players and connectedness of an ecosystem are only a few indicators of the ecosystems health. The health of an ecosystem, the health of the dependencies of the companies involved is critical when assessing risks. As the ecosystemstopic is very broad, it will primarily be discussed from a risk-identification-perspective. Ecosystem strategies for participants and other related ecosystem topics will also be covered from a risk-perspective. Cloud computing - To give insight in the possible risks in Software as a Service, it is best to take a look at the entire field of cloud computing. Different types of cloud computing (SaaS, PaaS and IaaS) will be discussed in general, with a special focus on the different kinds of risks that arise from the dependencies between services.

3.1

Software ecosystems

Almost all companies operate in networks of suppliers, partners and customers. Such networks are called business ecosystems. The term business ecosystems was first introduced by (Moore, 1993). Iansiti and Levien used his work to further develop the concept of business ecosystems (Iansiti & Levien, 2004). Jansen et al. (2009) used this work to construct a definition for software ecosystems, which are basically a more specified form of business ecosystems. We define a software ecosystem as a set of businesses functioning as a unit and interacting with a shared market for software and services, together with the relationships among them. (Jansen, Finkelstein, & Brinkkemper, A Sense of Community: A Research Agenda for Software Ecosystems, 2009)

Two terms in this definition should be further defined. Shared market - A shared market could either be a physical market, like the Apple app store, where all software is sold through one channel. It could however also refer to the more abstract meaning of market, as the sum of all the transactions in a specific domain. The shared market acts as a condition that helps to determine whether or not a company is member of an ecosystem, however it is generally the case that companies operate in different markets and thus in different ecosystems. Besides the iPhone and iPad ecosystem, Apple is also part of Mac OS ecosystem. A different ecosystem often comes with a different strategy. Apple follows the Keystone strategy on the Mac OS ecosystem, but follows more of a dominator strategy in the iPhone and iPad ecosystem.

Chapter 3. Theoretical background - 3.1 Software ecosystems Page 23 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Relationships - Signed contracts, customer-supplier relationships or joint ventures are examples of relationships between companies. When two companies compete with each other in the same ecosystem (on the same shared market), there is also a relationship between them. In the following sections the two roles in an ecosystem, the Keystone and the Niche Player, will be discussed. Both roles know three different strategies, which will be discussed to show the possible risks to the customer a certain strategy could cause.

3.1.1 Keystone
A keystone is the most important player in an ecosystem. The platform on which an ecosystem runs is generally property of the keystone. Through licensing (Microsoft) or through ownership of the only marketplace (Apple), the keystone has influence over what happens on its ecosystem. Keystones can follow three different strategies (Iansiti & Levien, 2004) and as they control the most important resource of the ecosystem, their strategy directly influences every other player in the ecosystem. Figure 5 gives an overview of how the different strategies relate to value capturing and value creation in an ecosystem.

High

Value creation

Niche players
(combined)

Dominators

Keystones

Hub landlords

Low Value capturing Low High


Figure 5 - Visualization of value capturing and value creation strategies in an ecosystem.

In the following three sections, the three different keystone strategies will be discussed, note that a Keystone can follow the Keystone-, Dominator- or Hub landlord- strategy. After a general description of the strategy, the implications of such a strategy to the customer will be dealt with. It is important to keep in mind that keystones can be (and often are) part of more than one ecosystem. Strategies across those ecosystems are often different. 3.1.1.1 The keystone strategy The keystone strategy can be seen as the default strategy a keystone should follow. The strategy describes a way of doing that strives to improve the total value of the ecosystem, rather than just improving the keystones own value. A keystone strategy is an operating strategy that improves the overall health of the ecosystem and, in so doing, benefits the sustained performance of the firm. (Iansiti & Levien, 2004)
Chapter 3. Theoretical background - 3.1 Software ecosystems Page 24 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Microsoft is a good example of a company that follows a keystone strategy. It puts a lot of effort into strengthening its ecosystem with extensive knowledge bases, conferences and new innovations. Few of these efforts directly turn into a higher turnover, but the effect that the increased ecosysteminvolvement of partners has on turnover is far greater. The keystones primary interest is (thus) improving the overall health of the software ecosystem (Bosch, 2009). Improving the health of the ecosystem can only be done by addressing the needs of the participants (Hagel & Seely Brown, 2008). The keystone strategy is a strategy that does not make investments convert into cash immediately. The niche players benefit first, which in turn benefits the keystone. This makes the keystone strategy vulnerable to managers with a more short-term focus. Short-term focus can turn companies from enabling keystones to oppressive dominators overnight, greatly reducing the potential value of their ecosystem. 3.1.1.2 The dominator strategy A dominator strategy often comes combined with the more open keystone strategy. A dominator is not only the provider of the platform the ecosystem runs on, but also actively participates as a player in certain niches on that platform. A dominator strategy is an operating strategy that integrates vertically or horizontally to manage and control an ecosystem or ecosystem domain. (Iansiti & Levien, 2004)

Three examples of companies that (partly) follow the dominator strategy in their ecosystem: Microsoft - The Windows platform is open for everyone to develop on, making Microsoft into a keystone, which supports the platform with the MSDN, a knowledgebase, etc. However, in the niche of office suites the Microsoft Office family is clearly the dominating party. SalesForce - The Force.com platform is an open platform that allows anyone to build their own web applications. The CRM niche on this platform has only one occupant: SalesForce.com CRM. Apple Apple (in the iOS-ecosystem) has complete control over which apps are installed on devices and asks a 30% premium over every app that is sold. In this regard, Apple takes a lot of value from their ecosystem. However, Apple also creates a lot of value for their ecosystem partners. Apple handles all the transactions, the installation and the updates of apps. The developers are supported with a dedicated IDE and tens of millions of users can instantly buy apps published in the app store.

An excellent example of how sudden dominator-behavior can have a direct negative influence on the ecosystem was recently shown by Twitter. As Twitter entered the niche of iPhone apps, by buying Tweety (Parr, 2010), investments in the Twitter ecosystem declined heavily (Ingram, 2010).
Chapter 3. Theoretical background - 3.1 Software ecosystems Page 25 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

This was due to the fact that niche players will never be able to compete with the keystone, a risk that was already recognized by (Hagel III, Seely Brown, & Davidson, 2008). This kind of keystone behavior could be a risk to customers when one of their service-suppliers becomes a direct competitor of the keystone. The keystones solution will probably be better integrated with the keystones platform, making the niche players solution superfluous. A keystone dominating some niches in an ecosystem is not always seen as a bad sign for the ecosystem. Iansiti and Levien (2004) state that *...+ Microsofts domination of office productivity software stabilizes this segment of the software industry, shifting diversity and ecosystem effort to other areas where it may be more appropriately applied. They find that a reliable, stable and predictable tool for something as fundamental to an OS as office functionality adds value to the entire ecosystem. Whether this is indeed a desirable situation in all ecosystems is doubtful (Jansen, Brinkkemper, & Finkelstein, 2010), but lies beyond de scope of this thesis. 3.1.1.3 The hub landlord strategy A company that follows the hub landlord strategy is taking too much value from its ecosystem, while not creating much value itself A hub landlord strategy is an operating strategy that extracts as much value as possible from an ecosystem or ecosystem domain without integrating forward to control it. (Iansiti & Levien, 2004) A hub landlord strategy is an inconsistent and never a sustainable strategy in the long run. To be able to stay on top of its ecosystem, the landlord consequently refuses to share information with its niche players. This makes it hard for niche players to build a stable business. As the rent for using the ecosystem is too high, niche players have a hard time making money and development of the ecosystem will lag behind. The hub landlord strategy is a strategy that delivers some success in the short run, but eventually the ecosystem will dry out. The risk of being customer of an ecosystem of which the keystone follows the hub landlord strategy, is primarily the uncertainty this brings (Hagel III, Seely Brown, & Davidson, 2008). This uncertainty makes that every risk assessment is difficult and all risks could apply.

3.1.2 Niche Players


A Niche player is a (usually small) entity that fills a small spot (or: niche) in the ecosystem. Niche players exist because the keystone of a platform cannot produce all possible functions that customers require (Bosch, 2009). An example of a Niche player could be an app developer for the Apple iPhone ecosystem. Hagel III et al. (2008) determine three different roles that a niche player (or: participant, as they call it) can play in an ecosystem, depending on influence and dependence on that ecosystem. Figure 6 shows these strategies on those axes.

Chapter 3. Theoretical background - 3.1 Software ecosystems Page 26 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

High

Ecosystem influence

Influencer

Hedger

Disciple

Low Ecosystem dependency Low High

Figure 6 - Nice player strategies: Ecosystem influence vs. dependency.

3.1.2.1 The influencer strategy An influencer is a niche player who not only commits himself to one ecosystem, but also tries to be as influential as possible in that ecosystem. An Influencer increases asset efficiency, builds capabilities and gains a strong market position by influencing the *keystone+. (Hagel III, Seely Brown, & Davidson, 2008) Dedicated niche players are very valuable to keystones as they can add unique value to the ecosystem, rather than hedgers that build on multiple ecosystems. Influencers are often invited to test pre-release versions of the platform and give their opinion before it goes live. The risk of their dependency on only one ecosystem is thereby lowered through their influence in that ecosystem. To the customer, the risk of doing business with an influencer lies in the fact that no matter how much influence the influencer has, he is still dependent on only one platform and in most cases the keystone is not obligated to keep the influencer in mind. 3.1.2.2 The hedger strategy A niche player that is active in more than one ecosystem is called a hedger. A hedgers eggs are spread across several baskets in several competing platforms. (Hagel III, Seely Brown, & Davidson, 2008) Primary advantage of being a hedger is that the niche player is not dependent on the situation in one particular ecosystem. If the health of one ecosystem deteriorates, there is still revenue from another. An example of a hedger is Layar, an Amsterdam-based startup that builds an augmented reality browser. Rather than building their software for only one platform, their browser is available for both the iPhone and Android platform (Layar, 2010). The risk of having a hedger as a supplier lies in the division of revenue between the different ecosystems. If the ecosystem your solution is built upon becomes less popular, the hedger might put less effort in keeping your service up to date or could even switch completely to the other ecosystems it is already using.
Chapter 3. Theoretical background - 3.1 Software ecosystems Page 27 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

3.1.2.3 The disciple strategy A disciple could also be called a follower. A disciple is part of only one ecosystem and has little or no influence over what the direction of the ecosystem or its platform will be. A disciple has a clear strategic focus and direction; it does not invest in competing *keystone platforms]. (Hagel III, Seely Brown, & Davidson, 2008) A lot of start-ups are committed to only one platform in the beginning, but this is often because of the cost of servicing two different ecosystems. Being committed to only one ecosystem brings more risks to the table than being a hedger, but being a hedger will generally make it harder to become an influencer. Instagram is an example of a startup that is committed to one platform only. Their photo sharing app is only available for the iOS-platform and will not be available for other platforms in the near future. Supposedly because: We are currently working on making the iPhone experience as solid as possible. Only then will we consider other platforms *+, however, cost is probably also a major reason why Instagram is still a disciple. Mash-up developers that combine two ecosystems to create a new service are still a disciple. When more than two ecosystems are combined, they become a hedger only when each ecosystem can fail without their solution becoming a failure or useless. Disciples are risky because they are dependent on only one ecosystem and could easily go out of business when the health of the ecosystem deteriorates.

3.2

Cloud computing

The term cloud computing is a popular buzzword that is often used in situations that do not comply with the definitions found in scientific literature such as (Armbrust, et al., 2010), (Buyya R. , Yeo, Venugopal, Broberg, & Brandic, 2009), (Durkee, 2010), (Mell & Grance, 2010), (Vaquero, Rodero-Merino, Caveres, & Lindner, 2009). Cloud computing has become an umbrella term for flexible, offsite, pay-per-use computing that is used in many different ways. As the term cloud computing is used in many different situations, many different definitions exist. The definition of the National Institute of Standards and Technology is used in this research.

Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models. (Mell & Grance, 2010) Some of the aspects of cloud computing need further clarification:

Chapter 3. Theoretical background - 3.2 Cloud computing Page 28 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Pay-per-use model - Cloud services are typically paid for using a pay-per-use model, as opposed to in-house IT-systems, which require upfront investments. This allows for more flexibility on the customers side and lowers the initial hurdle to use the service. Cloud computing changes capital expenses into operational expenses (Niesen & Slaats, 2010). This makes cloud computing especially suitable for start-ups (Wei & Brian Blake, 2010). Also, *+ customers believe they would have more control over the relationship if they simply paid monthly fees *+ (Dubey & Wagle, 2007). Variable load - Using multi-layered architectures that may include resource virtualization, allows cloud computing to easily adapt to growing or declining needs. Providers are able to shift capacity between demanding parties and reduce the need for large overheads (Dillon, Wu, & Chang, 2010). Infrastructure Provider - Although it is possible to create private cloud services, the majority of cloud services is hosted off-site. An infrastructure provider is contracted to deliver the service and make sure there is sufficient capacity for the proper execution of the service (Abdat, Spruit, & Bos, 2010). Service Level Agreements (SLAs) - A service level agreement (SLA) is an agreement between the provider of a service and its customer which quantifies the minimum quality of service which meets the business need (Hiles, 1994).

The above-mentioned definition will be the basis for the further exploration of the different parts of cloud computing, including Software as a Service, application- and enabling-Platform as a Service and Infrastructure as a Service. In the following subsections, these topics will be discussed and definitions will be given. To give further insight in how these different parts of cloud computing relate to each other, the ontology of cloud computing by (Abdat, Spruit, & Bos, 2010), (Figure 7) is shown below, with some examples of well-known services.

Application: - User interfaces - Business logic


aPaaS ePaaS

SaaS

Application supporting other applications

PaaS

Operating system and Application Server Stack

Virtualized infrastructures - Virtual machines - Virtual firewalls - Virtual routers

Figure 7 - Ontology of cloud computing, based on Abdat et al. (2010) and Hagel et al. (2010) .

IaaS

Chapter 3. Theoretical background - 3.2 Cloud computing Page 29 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

3.2.1 Software as a Service (SaaS)


SaaS is software that is not installed locally or on a shared network drive, but that is used over the internet, directly from the vendor. Typically the vendor has a so-called multi-tenant solution in place. This means that all the users share the same code bases and systems for their service. This shared code base and shared systems allow vendors to: Perform updates easier and more often - As SaaS solutions typically have only one code base, there is only one instance of the software to update. This takes away the need for extensive testing of every update or patch on different systems and ensures that all the customers are using the same version (Levinson, 2007). Cut support efforts for different versions - As all the customers use the same version of the software, the support only needs to be available in the latest version of the software (Kaplan, 2007), (Dubey & Wagle, 2007), (Levinson, 2007). Reduce overhead on available resources - SaaS solutions are multi-tenant (Kaplan, 2007), different companies use the same systems, but not all at the same time. This means that the supplier of the service can overbook his resources and save on the amount of resources needed (Armbrust, et al., 2010). Offer APIs to integrate with other SaaS solutions - Integration with other SaaS solutions is easier because both systems always work with only one version of the software. If specifications change due to updates, other vendors only need to patch one instance of their software and all their customers can continue to enjoy the connection (Dubey & Wagle, 2007).

Advantages for customers are, apart from the advantages like scalability and pay-per-use, already mentioned in the cloud computing subsection: No hidden maintenance costs - As all the costs of patching, updating and upgrading the software are covered by the monthly fee to the supplier of the service, there is no need for the customers IT departments to spend time on upgrades and testing (Levinson, 2007). Easy remote access - SaaS is typically delivered over the internet, this makes using the software location independent. Customers can use the software from anywhere just by logging in to the service (Dubey & Wagle, 2007), (Levinson, 2007), (Abdat, Spruit, & Bos, 2010).

The scientific definition of SaaS that will be used in this thesis is derived from 14 different definitions by Abdat et al. (2010): SaaS is a software delivery model that supports multi-tenancy in which the vendors host and operate their software on a data center (either independently or through third-party) and provide it to their customers over the Internet and typically on a subscription basis and/or pay-for-use basis. (Abdat, Spruit, & Bos, 2010).

Chapter 3. Theoretical background - 3.2 Cloud computing Page 30 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

3.2.1.1 SaaS delivery As is the case with ordinary products, supplier and vendor of a SaaS solution do not have to be the same company. Abdat et al. (2010) have made a schematic conceptual view of the SaaS ecosystem. Even though there could be some debate about whether or not it should be called an ecosystem, their scheme gives a good overview of the various ways in which SaaS can be delivered.

money service

SaaS Vendor

SaaS Supplier

money service

money service

money service

Reseller or Integrator

Enduser
money service

Figure 8 - SaaS Ecosystem based on Abdat et al. (2010).

In Figure 8, the combined SaaS ecosystem from Abdat et al (2010) is shown. SaaS Vendor - The SaaS Vendor is the developer of the software. It could very well be that the SaaS Vendor is also the Supplier of the software. This is the case with the Salesforce.com CRM software. The same company is both the developer of the CRM software, as it is supplier of the cloud platform the service runs on. SaaS Supplier - In short the SaaS supplier is the owner of the platform the service runs on. For example: Exact Software B.V., vendor of the financial account service Exact Online lets Rackspace Hosting host its service (Van der Beek, 2010). This is because Rackspace is an experienced hosting company, where Exact is specialized in making accounting software. Reseller or Integrator - Resellers sell white label versions of services under their own label. This might be done when the reseller has a brand that appeals more to a certain market, or when the software only works in a certain context. A good example is the Email Marketing Service Campaign Monitor. This SaaS solution can be customized so that (potential) customers that receive an email do not know it was send using Campaign monitor (http://www.campaignmonitor.com). An integrator is a company that helps to set up the SaaS solution to its customers needs. As SaaS solutions tend to be very generic, there might be quite some customization or even coding involved to make it suit the customers needs.

3.2.2 Platform as a Service (PaaS)


A platform is a collection of software components that together form a basis on which other software applications can run. Probably the best known platform is Microsoft Windows; however platforms can come in many different shapes and sizes. Platforms make development of applications easier, because they take care of common tasks like file storage, printing and network connections (Lawton, 2008).

Chapter 3. Theoretical background - 3.2 Cloud computing Page 31 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Platforms, whether delivered as a service or not, often create a lock-in. Applications built on one platform will not just work on another platform, so the choice of a platform for an application developer is somewhat definitive. This makes that successful platforms are valuable to the platform owner, as (also: keystone) (Iansiti & Levien, 2004). More on platforms and the ecosystem they foster are discussed in the section on Software ecosystems (p. 23) and Cloud computing (p. 28). Although many researchers and writers just speak of PaaS, there are different levels of PaaS that can be distinguished. In this thesis, a distinction will be made between two different levels. The enabling Platform as a Service (ePaaS) and the application Platform as a Service (aPaaS) (Hagel, Seely Brown, Arnowitz, Asnaani, Gillingham, & Sharma, 2010) differ not only in features and specifications, but also in how ecosystems are formed around these platforms. However, it might be possible that a platform is not strictly identifiable as ePaaS or aPaaS. 3.2.2.1 Enabling Platform as a Service (ePaaS) Enabling Platforms are the most basic type of platforms. They usually consist of an operating system with some low-level supporting application like a webserver and a scripting language like PHP or Python. When these platforms are offered as a service, they usually also include some kind of loadbalancing. Enabling platforms give developers a large amount of freedom as to how they want to develop their application, while taking care of basic tasks like OS and webserver updates. An example of this kind of solution is Microsoft Azure platform. Azure delivers an environment in which SaaS solutions can be developed without much knowledge of server hardware. It also delivers readymade solutions for common tasks like identification and database storage (Chappel, 2010). This makes ePaaS solutions ideal for vendors that want to focus themselves on software development, rather than hosting solutions. See also the Exact Rackspace example in the SaaS delivery section on p. 31 (Van der Beek, 2010). Once an application has been built for a certain platform; it will not be possible to run this application on another platform. When vendor and supplier are the same company and this company is the only supplier of the platform, this could lead to unwanted lock-in situations (Durkee, 2010), (Pring, 2010). 3.2.2.2 Application Platform as a Service (aPaaS) The Force.com platform can be considered a genuine application platform as a service according to Hagel et al. (2010). The definition below, given by Force.com might be considered somewhat commercial, but it gives a good overview of what aPaaS is. *application+ Platform as a Service (PaaS), such as Force.com, is an application-centric approach that abstracts the concept of servers altogether. PaaS lets developers focus on core application development from day one and to deploy an application with the push of a button. The provider never needs to worry about multi-tenancy, high-availability, loadbalancing, scalability, system backups, patches and security, and other infrastructure-related concerns (Weissman & Bobrowski, 2009). In this definition provider corresponds to vendor as described in Figure 8 on p. 31. Application Platforms are more advanced platforms that provide more support to developers. Examples of such support can be found on the Google App engine aPaaS (Google, 2010). On this
Chapter 3. Theoretical background - 3.2 Cloud computing Page 32 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

platform, APIs allow developers to easily add YouTube and Google Maps support to their applications. However, as their task support is higher (see also Figure 9 on p. 34) than of enabling platforms, their use is more restricted to a specific area. The Force.com platform is another example of such a platform. Developers are able to build their application on top of the platform, but as the platform is more directed towards building a business application (after all, the platform was built to support SalesForce.com CRM), the platform offers less freedom to what is built (Perry, Hatcher, Mahowald, & Hendrick, 2009). 3.2.2.3 Platform lock-in Platforms, especially those that are delivered as a service, often create a firm lock-in. The following characteristics of application platforms impose such a lock-in: Closed source and only one provider Platforms that are delivered as a service (like Force.com) have a closed and protected source code and often only have one supplier (Mietzner, Leymann, & Papazoglou, 2008). This is due to the fact that platforms often spring from successful SaaS solutions. As a SaaS vendor realizes that he is unable to build all the requested functionality himself, he opens up his application to third party developers (Bosch, 2009). As opposed to platforms that are not intended as a service, like operating systems, this makes it impossible to switch providers. Specific programming language and/or environment Force.com comes with its own programming language, called APEX (SalesForce.com, 2008). To allow developers to switch easily, it looks like Java, but it is specifically designed for the Force.com platform. Applications written in APEX cannot be run elsewhere. Even if platforms do employ standard programming languages (Googles App Engine allows the use of both Java and Python), applications are often built to the design choices of the supporting platform, making them hard to run stand alone (Mudumbai, 2010).

3.2.3 Infrastructure as a Service (IaaS)


Infrastructure as a Service (IaaS) is the delivery of hardware (server, storage and network), and associated software (operating systems virtualization technology, file system), as a service. It is an evolution of traditional hosting that does not require any long term commitment and allows users to provision resources on demand (Bhardwaj, Jain, & Jain, 2010). Infrastructure as a Service (IaaS) is the most basic form of cloud computing. It is also known as HaaS (Hardware as a Service) and in essence, IaaS is the renting of virtual hardware. IaaS is useful when full control over resources is needed, but when there is no room for large investments or when the required capacity is volatile. IaaS-solutions usually offer the possibilities to upload your own virtual machine image and run this image on as many virtual hosts as you need to serve your application. The client, not the platform supplier, is responsible for capacity provisioning. Amazons EC2 is a good example of an IaaS.

Chapter 3. Theoretical background - 3.2 Cloud computing Page 33 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

3.2.4 Task support in cloud computing


To give an overview of how different cloud computing services relate to each other, refer to Figure 9. This figure gives an overview of how different kinds of cloud services score on task support and flexibility.
High
SaaS aPaaS

Task support

ePaaS IaaS

Low Low Flexibility High

Figure 9 - Task support vs. flexibility of cloud computing services.

Chapter 3. Theoretical background - 3.2 Cloud computing Page 34 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 4

Cloud Solution Modeling Method

The previous chapter laid the foundation for this research by exploring the different theories behind cloud computing and software ecosystems. This chapter will focus on the visualization of cloud computing solutions and is included in this research because a proper way to visualize cloud solutions has yet to be developed and contributes to the value of this research. In order to be able to analyze (the risks of dependencies in) a solution that consists of multiple parties and solutions, it is necessary to be able to model the flow of services inside this solution. Boucharas et al. have developed a Formal Software Ecosystem Modeling method (FSEMM) (Boucharas, Jansen, & Brinkkemper, 2009), that enables one to visually represent a Software Supply Network. A Software Supply Network (SSN) is the network of vendors, suppliers, resellers and customers and the software products that flow between them. An example of a network that could be modeled is the network around an on-premise Exact Globe 2003 installation. This network is shown in Figure 10.

Microsoft

P.3 .3

.4

P.2

Exact Software

P.1 .1

Reseller

.2 P.1 S.1

Customer
Services S.1: Support

Products P.1: Exact Globe 2003 P.2: Microsoft Windows Server P.3: Microsoft SQL Server

Money .1: Fee for P.1 .2: Fee for P.1 & S.1 .3: Fee for P.3 .4: Fee for P.2

Figure 10 - Example Software Supply Network of an on-premise solution. Modeled following the method by (Boucharas, Jansen, & Brinkkemper, 2009).

Elements from the FSEM by Boucharas et al. (2009) were used in the composition of a method that is able to model cloud computing solutions. This method is called the Cloud Solution Modeling Method (CSMM) and should be able to visually represent solutions that include cloud computing components. The CSMM uses the SSN components from the FCEM of Boucharas et al, but does not include any of the Product Deployment Context (PDC) concepts. The aim of the CSMM is not to be able to capture entire ecosystems, but focuses on the modeling of specific solutions that solves a problem for a specific customer or market. The PDC is of a too detailed level to be included in this view. The type of model the CSMM delivers is called a Cloud Solution Supply Network, or a CSSN.

4.1

Requirements on the CSMM


Model the solution - The CSMM is designed to model cloud computing solutions rather than the ecosystem(s) around a specific entity. Further research will be necessary to check whether the CSMM is also useful to model the ecosystems in cloud computing.
Chapter 4. Cloud Solution Modeling Method - 4.1 Requirements on the CSMM Page 35 of 125

The CSMM was constructed to fit the following requirements:

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Focus on the relationships - The CSMM focuses on the relationships between the different actors in a solution and the services that flow between them. Boucharas et al (2009) also handle the technical aspects of software in their Product Deployment Context diagrams. The delivery part of cloud computing is not relevant to this research and will therefore not be part of this version of the CSMM. An addition to the method that is able to describe the technical context that support the deployment of the solution has yet to be made. Support services - The most distinguishable feature of cloud computing is that software is delivered as a service. A service is always delivered from a platform on which that service runs. The modeling method should be able to correctly show such a platform.

4.2

Detailed description of the CSMM


ECOSYSTEM

The conceptual model of the CSMM is shown in Figure 11 and is further elaborated on in Table 2.

0..1 KEYSTONE

1..* NICHE PLAYER

CUSTOMER

ACTOR 2
connects

RESELLER

VALUE ADDED RESELLER 0..*


owns

SUPPLIER 1
delivers

0..*

owns

1..* 1..* RELATIONSHIP 1..* 0..*

0..1 0..1 PLATFORM 1


originates from

0..* . COMPOSITE FLOW 2..*

0..* MONEY FLOW 0..* 0..* 0..* PRODUCT FLOW 0..* 0..* 0..* DATA FLOW connects 0..* 0..* SERVICE FLOW 1..*

Figure 11 - Conceptual model of the Cloud Solution Modeling Method.

The conceptual model of the Cloud Solution Modeling Method shows which entities exists within the method and how they relate to each other. Table 2 shows not only the description of the entities in the CSMM but also how they are visualized in the Cloud Solution Supply Network. The appearance of
Chapter 4. Cloud Solution Modeling Method - 4.2 Detailed description of the CSMM Page 36 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

the entities in the CSSN is made to fit in with the appearance of the entities from the Software Supply Model.
Concept ECOSYSTEM Description An ECOSYSTEM is a collection of KEYSTONES and NICHE PLAYERS and their RELATIONSHIPS. The ECOSYSTEM in the CSMM is used to show which underlying ECOSYSTEMS are active in the solution. Note that only the relevant ECOSYSTEMS are represented in the model of the cloud solution and that not all ACTORS of an ECOSYSTEM are part of the solution that is being modeled. A KEYSTONE is the most important ACTOR in an ecosystem and is usually owner of the PLATFORM or the main SERVICE of an ECOSYSTEM. In non-cloud ECOSYSTEMS, the PLATFORM can also be a software PRODUCT. Visualization An ECOSYSTEM is shown in the background of the model as a colored area. Different colors are used to distinguish between different ECOSYSTEMS.

E.1
The KEYSTONE is an ACTOR and either a (VALUE ADDED) RESELLER or a SUPPLIER. The KEYSTONE ACTOR is shown with a thick border to distinguish it from the other ACTORS

KEYSTONE

Keystone Supplier
The NICHE PLAYER is an ACTOR and either a (VALUE ADDED) RESELLER or a SUPPLIER. The NICHE PLAYER ACTOR is shown with a thin border.

NICHE PLAYER

A NICHE PLAYER delivers a PRODUCT, SERVICE or PLATFORM that fills a specific niche in the ECOSYSTEM.

Niche player Supplier


ACTOR An ACTOR is a player in an ECOSYSTEM. In this modeling method, it is a super concept for the RESELLER, VALUE ADDED RESELLER and the SUPPLIER. A RELATIONSHIP is a relationship between two ACTORS; it cannot exist without at least a MONEY-, PRODUCT-, SERVICE-, DATA- or COMPOSITE-FLOW that flows over it. There is no visual representation for the ACTOR, as it is only a super concept for three sub concepts. A RELATIONSHIP is represented by a line connecting two ACTORS

RELATIONSHIP

CUSTOMER

The CUSTOMER is the ACTOR that is the end-user of the solution.

The CUSTOMER is represented by the following yellow figure.

Customer
RESELLER A RESELLER is an ACTOR who resells PRODUCTS, SERVICES or COMPOSITE products or services. A RESELLER adds nothing to these PRODUCTS or SERVICES, but may provide additional SERVICES like support or contract negotiation. A VALUE ADDED RESELLER is a RESELLER that ADDS something to the SERVICES or PRODUCTS that it resells. These additions could for example be the platform on which a VALUE ADDED RESELLER hosts a PRODUCT that is then delivered as a SERVICE. The RESELLER is represented by the following in green shape.

Reseller
A VALUE ADDED RESELLER is represented by the shape of the RESELLER, but in the color of the SUPPLIER.

VALUE ADDED RESELLER

Value added reseller

Chapter 4. Cloud Solution Modeling Method - 4.2 Detailed description of the CSMM Page 37 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Concept SUPPLIER

Description A SUPPLIER is the vendor of PRODUCTS or SERVICES. A SUPPLIER can have SUPPLIERS itself, but most of the time a SUPPLIER is the beginning of a product flow.

Visualization A SUPPLIER is represented by the following orange shape.

Supplier
PLATFORM A PLATFORM is owned by a SUPPLIER or a VALUE ADDED RESELLER (VAR). It is a system that SERVICES can run on. An important aspect of a platform is that it is not part of a RELATIONSHIP and is not transferred to another ACTOR. PLATFORMS can be explicitly part of the solution, when for example a specific PLATFORM is chosen to develop an application on, but they can also be implicitly part of the solution. This is when the platform is there to host a SERVICE that adds to the solution. A PLATFORM is shown in the same way as FLOWS are. The PL.1 is short for Platform 1 and can later be clarified in the legend of the model.

PL.1
As PLATFORMS are not transferred between ACTORS, they stay with the VAR or the SUPPLIER that provides them.

Supplier
PL.1
PRODUCT FLOW The PRODUCT FLOW is the flow of a PRODUCT over a RELATIONSHIP between two ACTORS. A PRODUCT cannot exist in the model without a RELATIONSHIP. The SERVICE FLOW is the flow of a SERVICE over a RELATIONSHIP between two ACTORS. A SERVICE cannot exist in the model without a RELATIONSHIP. The COMPOSITE FLOW is a FLOW that consists of two or more PRODUCT- or SERVICE- FLOWS. Composite FLOWS are only used when two separate PRODUCTS or SERVICES are so tightly coupled that there is actually only one PRODUCT or SERVICE. The PRODUCT FLOW is represented by the shape below.
P.1

SERVICE FLOW

The SERVICE FLOW is represented by the shape below.


S.1

COMPOSITE FLOW

The COMPOSITE FLOW is represented by a FLOW that encompasses one or more other FLOWS. The abbreviation for the FLOW is shown under the FLOW.
S.2 S.3 S.1

DATA FLOW

The DATA FLOW is used when data is flowing between two ACTORS. The distinction with SERVICE is that only the data flows. The DATAFLOW is usually between two PLATFORMS that deliver SERVICES that use the same data. The DATA FLOW is the only FLOW that is bidirectional. This is used when data is synchronized for example.

The DATA FLOW is show unidirectional and bidirectional.


D.1 D.1

MONEY FLOW

The MONEY FLOW is interesting because it shows who is paid by whom and this might be different than how the other FLOWS run. The absence of MONEY FLOWS where is a SERVICE FLOW could indicate interesting dependencies

The MONEY FLOW is normally shown in the opposite direction as the other FLOWS, because it is the reward for the use of these FLOWS
.1

Table 2 - Description of the concepts from the CSMM.

Chapter 4. Cloud Solution Modeling Method - 4.2 Detailed description of the CSMM Page 38 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

4.3

Example of a Cloud Solution Supply Network

Figure 12 shows how the CSMM van be used and shows a possible use of the method to model the Exact Online solution. AeroPlus is company that delivers a separate service (S.3) from their own platform (PL.2) which uses the Exact Online API to synchronize data (D.1). Supplier X is an imaginary supplier that delivers an add-on (P.4) that integrates with Exact Online (P.1) and runs on the same platform Exact Online runs on (PL.1). Exact is currently working on this kind of integration, so a real suppliers name is not yet available.

E.2

AeroPlus
E.1

Microsoft

P.3 .3

PL.2

P.2 .4

D.1

3 S.

6 .

Supplier X

P.4 .7

Exact Software

.1 P.4
P.1

Rackspace
PL.1

S.4
S.1

Customer

Reseller

S . .2 5

2 .

Products P.1: Exact Online P.2: Microsoft Windows Server P.3: Microsoft SQL Server P.4: Add-on for P.1 Data D.1: Dataflow betw. S.3 & S.1

Money flows .1: Fee for hosting P.1 .2: Fee for S.1 .3: Fee for P.3 .4: Fee for P.2 .5: Fee for S.1 & S.2 .6: Fee for S.3 .7: Fee for P.4

Services S.1: Exact Online S.2: Support S.3: AeroPlus Flight Planning S.4: P.4 as a Service

Ecosystems E.1: Exact ecosystem E.2: Microsoft ecosystem

Platforms PL.1: Rackspace Hosting Platform PL.2: AeroPlus platform

Figure 12 - Example Software Supply Network of a SaaS solution. Modeled following an adjusted method based on (Boucharas, Jansen, & Brinkkemper, 2009).

When a CSSN is completed, it gives an overview of the entities from the solution it represents. See Appendix A. for an overview of the entities from the CSSN of the Exact Online solution. Further research will however be necessary to check whether the CSMM is able to model all the relevant parts of the solution. Because the CSMM only adds components to the FSEMM, the CSMM is also capable of visualizing on-premise entities. The simultaneous modeling and representation of cloud and on-premise entities is used successfully in the Canon case study, but further evaluation of this use is necessary. The next chapter will be about the risks and precautions that are applicable to the entities in cloud computing solutions, they form the basis for the method that will be developed.

Chapter 4. Cloud Solution Modeling Method - 4.3 Example of a Cloud Solution Supply Network Page 39 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 5

Risks and precautions

Where the previous chapter handled the visualization of cloud computing solutions, this chapter discusses the risks that can come up in these solutions. The first section is about the requirements that are used to identify these risks and their precautions (what can be done to mitigate the risks). The section also shows the different entities to which the risks can be applicable. The second section reviews the risks and precautions that came up in an extensive literature research. These risks and precautions are then discussed with experts in expert interviews. The results of these interviews and the impact experts opinions had on the collections of risks can be found in section three. Based on these interviews the risks and precautions are updated and the final version of them is put forward in the fourth and last part of this chapter.

5.1

Requirements

After the solution is properly visualized, the risks must be described that could be present in the solution. At first, a literature study was performed to find the most important risks and the ways to mitigate those risks (precautions). The risks are identified following three requirements: The risk has to be external; risks that evolve from internal affairs in the customers organization will not be included in this method. The risk must be inflicted by the supplying parties in the ecosystem or by the relationships between them. The risk must be operational and able to jeopardize the delivery of the service or the compatibility of the service with other services. Risks that can occur during the implementation phase are not taken into account.

Based on these requirements, the entity types from the CSSN (Chapter 4) and the concepts from the conceptual model (Figure 1 in section 1.5), four entity types are formed on which the risks can be applicable (Table 3). Note that there is no one-on-one mapping between the entities from the conceptual model and the CSSN and the entities that will be used in the method. Certain concepts from the aforementioned models are not related to the scope of the risks and precautions and are therefore not included in the creation of the method.

Chapter 5. Risks and precautions - 5.1 Requirements Page 40 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Related concepts from conceptual model SERVICE PLATFORM

Related concepts from CSSN

Entity type to be used in the method Technical cloud component

o o o o

RELATIONSHIP PRODUCT FLOW DATA FLOW SERVICE FLOW COMPOSITE FLOW PLATFORM ECOSYSTEM

The ecosystem is not shown in the conceptual model, because it is more a property than an entity in itself. SERVICE PROVIDER SERVICE VENDOR RESELLER SERVICE PROVIDER SERVICE VENDOR RESELLER INTEGRATOR CONSULTANT

Ecosystem

ACTOR KEYSTONE SUPPLIER VALUE ADDED RESELLER RESELLER ACTOR NICHE PLAYER SUPPLIER VALUE ADDED RESELLER RESELLER

Keystone

Niche Player

Table 3 - Entity and concept mapping to form method entity types

The next section will show the results of the literature study and the ones after that show what the expert interviews changed to these results and what the final risks and precautions will be for this method.

Chapter 5. Risks and precautions - 5.1 Requirements Page 41 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

5.2

Literature study

The initial risks and precautions that were found in literature are shown in Table 4 and Table 6. A detailed description of these risks and precautions based on that literature can be found in Appendix B. and Appendix C. . How these risks apply to the different entity types can be seen in Table 5. The way in which these precautions can be mapped onto the risks is found in Table 7.

5.2.1 Risks
Table 4 shows how the risks that are found in literature and a short description of those risks.
Based on (Armbrust, et al., 2010), (Dubey & Wagle, 2007), (Durkee, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Kaplan, 2007), (Kontogiannis, et al., 2007), (Olsen, 2009), (Wei & Brian Blake, 2010), (Van de Zande & Jansen, 2010) Services can become unavailable through a number of different reasons. The risk that is involved with a service becoming unavailable depends on how important this service is to the business.Appendix B. R2 Solution is insecure (Armbrust, et al., 2010), (Cloud Security Alliance, 2010), (Dillon, Wu, & Chang, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Hwang, Kulkarni, & Hu, 2009), (Levinson, 2007), (Penn, 2010), (Yildiz, Abawajy, Ercan, & Bernoth, 2009) Security threats come in all sorts. A data leak or unauthorized access to a system could be the result of a badly designed system, two services that do not collaborate properly or an employee that is dissatisfied. R3 Solution or ecosystem (Armbrust, et al., 2010), (Bosch, 2009), (Huizing, 2008), (Kindness, Whiteley, produces lock-in & Hayes, 2010), (Linthicum, 2009), (Mietzner, Leymann, & Papazoglou, 2008), (Mudumbai, 2010), (Van der Pluijm, 2010), (Pezzini, Driver, Feinberg, Natis, Perkings, & Scott, 2010) Being locked-in is the situation when leaving your current supplier or ecosystem is either too expensive, or impossible because of a lack of a suitable alternative. R4 Entity becomes (Adner, 2006), (Bosch, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), unpopular (Hagel & Seely Brown, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Ingthorsson, 2010), (Moore, 1993), (Musil, 2008) Services, companies or ecosystems which become less popular, for example because there is a better alternative available, pose a risk to the customer. As services become less popular, suppliers might try different things to make the service more attractive or keep revenue levels steady. Those changes can negatively impact the customer. R5 Ecosystem contains (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), power imbalance (Iansiti & Levien, 2004), (Jansen, Finkelstein, & Brinkkemper, 2009) Having influence on your supplier means that you could persuade him to change designs or move release dates to suit your situation better. Also the supplier could discuss new features with you. R6 Entity lacks innovation (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel & Seely Brown, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Iansiti & Levien, 2004), (Iansiti & Richards, 2006) Cloud computing is a relatively new field with new innovations coming up about every month. When a company stops preparing for the future, by neglecting these innovations, this is the beginning of the end. R7 Dedication to the (Bosch, 2009), (Hagel & Seely Brown, 2008), (Hagel III, Seely Brown, & ecosystem Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Jansen, Brinkkemper, & Finkelstein, 2010) The dedication a keystone or a niche player shows towards an ecosystem is the degree in which the player is dedicated to improve the ecosystem. It is important to note that being dedicated to an ecosystem is not the same as following the Disciple-strategy. Table 4 - Risks based on literature. Risk R1 Name Solution lacks solid availability

Table 4 shows the risks that are based on the literature study. For each risk, the following characteristics were described that can be found in Appendix B. .

Chapter 5. Risks and precautions - 5.2 Literature study Page 42 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Applicable to - The entity type this risk is related to. This can be Technical cloud component, Ecosystem, Keystone or Niche Player. A risk can be applicable to more than one type of entity. Type - The type of risk, business, legal, technical or a combination. This risk-characteristic is not a very strict categorization, but it might be useful in the future to group the risks into other categories than just by which entity type they are applicable to. Description - This is a general description of the risk. It explains why this is a risk and can also contain some references to sources that directly present the risk. Metrics - The metrics characteristic shows ways to identify if the risk is actually present in a specific situation. A risk could be applicable to a certain type of entity, but that does not necessarily mean that the entity in fact runs the risks. If a risk is dependent on company size, the metrics could be large companies. Note that the metrics are not always exact; a large company is an example of a metric that is not exact. Impact on business - The potential impact a risk has on the business when it becomes reality. A specific impact is very hard to give, because this is largely dependent on the kind of business that uses the solution. Example - A short example of the risk in a real-life situation. Some risks are hard to provide with a good example from a Cloud perspective. Cloud companies go to great lengths to shield their errors from the outside world. Sources - Sources that underpin the risk as described. Not all sources precisely quote the risk that is described. When a source states that it is vital to do X and Y, it can often be concluded that not doing X and Y may be a risk.

A complete overview of all the details of the risk based on the literature study can be found in Appendix B.

Entity type
R1 R2 R3 R4 R5 Technical cloud components V V V V V V Ecosystem Keystone Niche Player

R6 R7 Table 5 - Mapping of risks on entity types.

Risk

V V V

V V V V V

V V V V V

Table 5 gives an overview of which risks are applicable to which entity types. It might catch the eye that there is no difference in the risks that apply to Keystones and the risks that apply to Niche Players. This is bound to change when experts are able to comment on the risks and on the types they are applicable to.

Chapter 5. Risks and precautions - 5.2 Literature study Page 43 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

5.2.2 Precautions
Besides the seven risks mentioned in the section above, there are also seven precautions to be found in literature. The precautions are not based on a particular entity type, but are focused on preventing a certain risk. See Table 7 for which precautions helps to prevent which risk. Note that a precaution can never take away a risk completely and it might also be possible that in specific situation a precaution might cause some risks itself.

Precaution P1

Name Deploy escrow services

P2 P3

Make regular backups Sign contracts on required service levels

P4

Choose an ecosystem of significant size Choose solutions that support open data formats

P5

Beware of trends in the market and your ecosystem P7 Choose Certified and Auditable partners Table 6 - Precautions based on literature.

P6

Based on (Exact Salesdesk, 2010), (Krs, 2010), (Denson, 2002), (Herbert, Ferrusi Ross, Rose, & Karcher, 2010), (Spiotto & Spiotto, 2003), (Van de Zande & Jansen, 2010) (Bodle, 2010), (Emmerik, 2010), (Heiser, 2010), (Marks, 2008), (Weinberger, 2010) (Buyya, Yeo, & Venugopal, 2008), (Durkee, 2010), (Herbert, Speyer, Gaynor, & Schule, 2006), (Hiles, 1994), (Hofmann & Woods, 2010), (Vaquero, Rodero-Merino, Caveres, & Lindner, 2009) (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Richards, 2006) (Antonsen, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), (Subramanian, 2008), (Open Cloud Manifesto, 2009), (Zeller, et al., 2009) (Anderson, 2010) (Armbrust, et al., 2010), (Heiser, 2010), (Heiser, 2010), (Hofmann & Woods, 2010)

Table 6 shows the precautions that are based on the literature study. P6 and P7 were thought of shortly before the expert interviews would start and are therefore not extensively backed up with literature. P6 was only found in one source, P7 was found in a multitude of sources, but it was never stated that this is actually a precaution against any of the risks. This is something that will definitely be part of the expert interviews. For each precaution, the following characteristics were described. Counteracts - Shows which risks could be counteracted by taking this precaution. Some precautions can only partly mitigate a risk or might not be able to counteract a risk in every situation. This mapping is especially useful when the risks are identified and one is looking for measures to take against it. Type - The type of precaution, business, legal, technical or a combination. This precautioncharacteristic is not a very strict categorization, but it might be useful in the future to group the precautions into other categories than just by which risk they can counteract. Description - This is a general description of the precaution. It explains why this precaution is useful and can also contain some references to sources that directly present the precaution. Example - A short example of the precaution in a real-life situation, where the precaution was used or how it could be used. Explanation of counteracting - A short description of how this precaution counteracts the risks that are mentioned under Counteracts. It might not always be obvious how a precaution could counteract a risk.
Chapter 5. Risks and precautions - 5.2 Literature study Page 44 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Sources - Sources that underpin the precaution as described. As is the case with the risks, not every precaution is explicitly named in every source. Many sources only state that something would be a good idea, but do not name the risks that they solve.

Precautions
P1 P2 P3 P4 P5 P6 P7 V V V V V V R1 V V V R2 V V R3 V V V R4 V R5 V V R6 V R7 Table 7 - Precautions mapped on risks (based on literature)

Table 7 shows which precautions can counteract which risks. As written above, not every risk can be fully mitigated by a precaution, nor can precautions make sure that the business impacts of a risk will be nil. The sources that propose a precaution are not always posing it as a way to counteract a specific risk, the mapping between precautions and risks is formed based on the total of the different sources that describe a risk or a precaution.

5.3

Risks

Expert interview outcomes

The expert interviews were carried out following an interview protocol. This protocol can be found in Appendix D. . The purpose of this protocol was to make sure that: All the experts had the same introduction to the research and were familiar with the definition of the terms used. The interviews would all elapse in more or less the same order. It was clear what needed to be asked and what the purpose of the interview was. Supervisors were able to comment on the proposed interviews.

Table 8 gives an overview of the comments the experts gave on the topic and the influence that their comments had on the final method. The raw expert interview outcomes are shown in Appendix E. Individual experts are not shown by their name, but are numbered E1-E8. The names of these experts are known to the author. The quotations are the authors interpretation, translation (most interviews were in Dutch) and summary of what the experts said. These comments were then checked by a fellow researcher. In the column subject, R1-R7 refers to the risks from Table 4 and P1-P7 to the precautions from Table 6. NR* and NP* are new risks and precautions, proposed by the experts. Not all the new risks and precautions are included in the table, only those with multiple backing from experts.

Chapter 5. Risks and precautions - 5.3 Expert interview outcomes Page 45 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Experts had little or no comments on my method to identify and the risks and the proposed precautions. This may be because the method that I created is pretty straight forward. The experts comments on my method are therefore not shown in this section.
Subject R1 - Solution lacks solid availability Expert views Risks The consensus among the experts is that unavailability is indeed an important risk to keep into account. Reputation, uptime and maturity of the provider of a service are possible indicators of how a company might perform in the future. E5: The more platforms and services involved, the higher the chance a problem occurs. 4 *99% uptime = 96% uptime. E3: SaaS vendors generally perform better than on-premise solutions E1: It is their job to build stable services, so they ought to be good at it but E4: The perception is against them, if a SaaS vendor fails, thousands of companies are affected, if an internal network fails, the world wont notice. The ways in which experts saw security differed significantly. Security is a broad field, where compliance (E3, E4), data security (E1, E2, E6), privacy (E4) and protection from in- and out-side threats (E2) are to be treated differently. Influence on method R1 remains in the method.

R3 - Solution or ecosystem produces lock-in

Lock-in is a subject that clearly divides the experts into two camps. E2, E3, E4 and E6 did not see Lock-in as an important risk. Arguments were There is a short term contract, no large investment, so switching is less costly., The risk of lock-in is a temporal risk because there are few alternatives in this immature market, SaaS is more commoditized, so there are less specific features to stay with one particular vendor. On the other hand, E1, E5, E7 and E8 argued that Data might be exported, but is this data usable, s the structural integrity of the data assured? and point out to be completely sure that the client owns the data, not the service provider.

The consensus about popularity was that it is not so much for the diminishing popularity that poses a risk, but the almost inevitable reduction in innovations that comes from a lower popularity. It was suggested by multiple experts to merge R4 with R6. Specific risks that were associated with R4 are that a less popular platform loses experts to other platforms, making it more difficult to support services on that platform. Then there is of course also the risk that an organization leaves the market completely due to lack of interest in its platform. Multiple experts pointed out that a cloud vendors business model is built on the idea that he is able to deliver economies of scale because everyone uses the same solution. Influence on the vendor is therefore always complicated. E4 points out that a lot of services are far from mature, so that functionality will only be expanded in most cases, lowering the risk in the functional area. Most experts agree that the business impact of R5 is low.

R4 - Entity becomes unpopular

R2 is a broad risk that cannot be completely covered in this method. However, to give a first impression on how a provider is doing from a security perspective, one could use certifications and checklists from organizations as the Information Security Forum as an indicator. There are plenty arguments to remove R3 from the method. Instead of removing R3, it will be renamed to Solution or ecosystem produces data lock-in and the indicators will focus more on the ability to export data in such a way that it keeps its richness and metadata. Lock-in in general is a complicated phenomenon that is too divers to capture in a couple of indicators, that lay beyond the scope of this research R4 is merged with R6 and the focus will be on the consequences of a lack of innovation. The other elements in R4 are partly covered by R1 and are named only by one expert.

R2 - Solution is insecure

R5 - Ecosystem contains power imbalance

The indicators for R5 will be relative company-size and also the way in which the vendor has handled wishes from users in the past.

Chapter 5. Risks and precautions - 5.3 Expert interview outcomes Page 46 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Subject R6 - Entity lacks innovation

R7 - Dedication to the ecosystem

Expert views E3, E4 and E5 all pointed out individually that customers have a tendency to focus on the features that are shown in a services roadmap, rather than on the features that are in the service at the moment. This tendency makes the situation where innovation is lagging behind even more relevant to take into account. Lack of innovation is often a consequence of R4 (E4, E6, E7). E3 and E6 added that in a developing market, innovation is key to keep up with other vendors, also because as more solutions are coupled, it is important that they all keep the same pace. The experts were not convinced that the original description of R7 was relevant to commercial SECOs, they found it better suitable for open source SECOs. E4 interpreted dedication in a new way, which was confirmed by E5, E6, E7, E8. E4 saw dedication more as a stamp of approval. When a vendor is dedicated to his platform, he could certify that certain add-ons are safe to use and this would strongly improve the value of his platform and of those add-ons. Precautions When a provider offers an escrow service, this can be a good precaution to tackle risks like unavailability (through bankruptcy). The majority of the experts agreed on the importance of this precaution. E4 noted that some vendors might see it as a sign of weakness if they have an escrow agreement in place. However, this may become less of an issue as the market matures. All but one expert noted this as an important precaution. There were some specific remarks on this topic that are worth mentioning. E4, E8: important to know when and what is back upped, and is the result of the backup sufficient to restore the entire solution. E6: with combined solutions, it is important to have synchronized backups, especially when there are a lot of transactions between the services. All the experts acknowledge the importance of a proper SLA. However, they also point out that the SLA is often imposed by the vendor and leaves little room for specific wishes of the customer. E8: some of the conditions in an SLA may even impose a risk on the client. Another problem is pointed out by E6: If I have a multi-party solution, who am I having an SLA with? And how do I make sure that SLAs are not in conflict?. The safety a large ecosystem with lots of niche players has to offer is only partly recognized by the experts. There is an advantage in being able to choose among more solutions for the same problem, even after a platform is chosen. Another advantage lies in the fact that large ecosystems often are mature ecosystems. E3, E5, E6, E7 and E8 all state that a mature ecosystem with a good reputation is definitely preferable to a small ecosystem with small players. Open data formats could be a precaution to lock-in, but the main advantage of open data formats is that it makes services less complex to integrate with other services that also support these formats. This decreases the risk of failing integrations (E6 and E7).

Influence on method R6 is combined with R4, focusing on the risks of lack of innovations, with indications from both risks. One way to check how innovative a company is, is to see if there is a department in place, how much of their turnover they spend on innovation (E2) R7 is renamed to Keystone neglects ecosystem needs, and the interpretation is changed to match the dedication of the keystone to its niche players applications and focus on benefitting the entire ecosystem.

P1 - Deploy escrow services

Make the precautions more specific to cloud computing, no further changes.

P2 - Make regular backups

Keep P2 in the method and adjust the explanation based on some of the experts comments.

The SLA is in a lot of cases a vendor imposed contract, which offers little protection to the customer. P3 will therefore be removed from this method, as the SLA is not able to prevent risks for the customer. Although this precaution might overlap with NP3, they look at the size and maturity benefits from different angles. Therefore P4 will remain in the method

P4 - Choose an ecosystem of significant size

P3 - Sign contracts on required service levels

P5 - Choose solutions that support open data formats

Open formats are important and P5 is therefore kept in the method.

P6 - Trends

Most of the experts confirm the importance of watching trends in the market and the ecosystem; however, they do have their doubts as to how much of a precaution it is.

P6 will be excluded from the method, it is too vague.

Chapter 5. Risks and precautions - 5.3 Expert interview outcomes Page 47 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Subject

Expert views Although P7 does not give any guarantees (E2), it does show that an organization is paying attention to the aspects that are certified. Another aspect is that certifications do add to the reputation of the company. There are however still no certifications that are purely designed to certify cloud solutions (E7)

Influence on method Although only a few experts named this precaution from the top of their minds, certifications are an indicator of a companys maturity. And is therefore an important precaution in the method. NR1 will be added to the risks in the method.

NR1 - Small parties

P7 - Choose Certified and Auditable partners

NP1 - Intermediary

New risks E1, E2, E3, E4, E7 all came up with the risks Actor lacks size and stability impose on customers. E1 *they+ are often not able to supply a large (and growing) customer group with sufficient support. E2, E4: They are easily taken over, which makes their future unsure. E2: When they support large organizations and fail, there is not much to sue. E3: Large companies are more risk averse than small companies. E7: Reputation and track record is very important, small entities lack that. E3 - E8 all named (an unpredictable) product lifecycles as risk to the customer of a cloud solution. E3, E5: You do not know how the new releases will influence the functions you use most.. E4: New releases could affect current integrations with other services. E6: New features could mean retraining staff, more helpdesk capacity and retesting of custom code. E8: New releases do not necessarily have the same quality standard as the current version; there might be some bugs left.. These risks all come from the fact that services use shared code bases and there is no choice to update, you are obliged to come along with the new version. New precautions An intermediary could catch a lot of issues before they affect the customer (E2, E3, E4, E5, E7). This means that there is only one party to deal with and as the intermediary is experienced with the task of handling multiple parties, there are fewer problems to expect. An intermediary could act on (at least) two different levels and could come in three forms. The intermediary could act on a technical level, hosting a service that ties all the other services together, or on a business level, where it is primarily involved with managing the contracts between the different parties. The intermediary could be a third party, like a consultancy company, or it could be the platform vendor on which the solutions run. In some cases the platform vendor might also guarantee the workings of external services. A shared platform on which all the add-ons run forms a solid basis that makes integration easier (E6) and eventually allows the platform vendor to guarantee the workings of the entire solution through certification (see P7 and NP1) E4, E5. A shared platform could also provide a standard to connect services, limiting the finger-pointing when things go wrong E8.

NR2 - Solution has an unpredictable lifecycle

NR2 will be added to the risks in the method

The intermediary is named by five out of the eight experts that I have interviewed; this is an important precaution and will be included in the method.

Table 8 - Expert interview results

NP3 - Mature companies with good reputation

All experts have pointed out how larger companies with a good reputation and proven success in the market are to be preferred. It is hard to point out when a company is mature, or when it is large enough, but the tenor is that experience and reputation are considered important factors.

Six out of the eight experts came up with the fact that a shared platform has less risk than when a solution uses multiple platforms. The only thing about this precaution might be that it is just not usable when there is no single platform that can deliver all your functions. Although NP3 is a bit vague, this precaution is useful and could help decision making.

NP2 - Shared platform

Chapter 5. Risks and precautions - 5.3 Expert interview outcomes Page 48 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Not all the risks and precautions that were suggested by the experts during the interviews were directly added to the final version of the method. Table 9 shows the risks that did not make it into the final method, together with some explanation why this was decided. The table does not include any precautions, because there was more overlap in the precautions, it was possible to combine these separate precautions into precautions that could be added to the method.

Subject NR* - Contract issues

Expert views E4, E5 and E8 pointed out that bad contact management is often a cause for problems. This is due to the fact that the market is still immature and a lot of decisions are made based on expected future developments, not clearly written down in contracts. An example of this is that a supplier promises that some features will be implemented in the future, but does not meet this promise.

E4, E7 and E8 named compliancy with governance rules as a possible risk. This risk is relevant because there are no clear standards yet to be used in the compliance auditing of cloud services. Especially data location is a specific item that is often subject of debate.

E5 and E6 said that the fact that composed cloud solutions often have multiple user interfaces hinders the user acceptance of the solution. E8 pointed out that the acceptance of cloud systems is also a risk, but gave a different perspective on the matter. E8 said that with on-premise solutions, the current business processes are analyzed and are leading in the implementation and customization of the solution. Cloud solutions however, due to their single code-base, are often less configurable and take the system as the example as to how organizations processes may need to be transformed. E6 and E7 both named performance issues as a risk to the customer. E6 focused on the problem that if two or more complex systems have to deliver a single view, this can be very slow, as all the database transactions have to be performed in a serial process, over multiple systems. E7 said that for complex, high capacity, transactional systems, like usage based billing in the telecom sector, cloud solutions are still not a match to dedicated on-site solutions. Table 9 - Risks suggested by experts that were not included in method.

NR* - User acceptance

Not included in the method Contract Issues are not taken into account in this method, because of several reasons. First of all, contract issues are not only external risks; it has a lot to do with the st customer as well (see the 1 riskrequirement on p. 40). Furthermore, contract issues are often very specific to a case, making it hard to find generic precautions. Finding indications to whether contract-issues will come up is complicated and beyond the scope of this research. Governance is not a direct risk, but it should be a selection criterion for new suppliers. Also, to which rules a supplier must comply is completely dependent on the customers location, business, etc. This makes that this st risk is not compatible with the 1 riskrequirements as shown on p. 40 Although user acceptance is a viable risk in the implementation of cloud computing, it rd is out of the scope of this research (3 riskrequirement on p. 40).

NR* Governance

NR* - Performance

Performance can be considered a risk, but this risk is closely related to the more general risk R1 - Availability and somewhat to R7 - Keystone neglects ecosystem needs

Chapter 5. Risks and precautions - 5.3 Expert interview outcomes Page 49 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

5.4

Final risks and precautions

This section contains the final risks and precautions that will be used in the method. These risks (Table 10), precautions (Table 12) and their mapping (Table 13) are based on the results of the literature study in section 0 and the expert interviews in section 5.3. The updated description of the risks can be found in Appendix F. and the updated description of the precautions in Appendix G. .

5.4.1 Risks
Table 10 shows the risks that are used in this research, with a short version of the explanation from Table 8. The most important changes were: R4 was merged with R6, because experts thought that R4 was a cause for R6, and R4 was not a risk on its own (red). R8 and R9 were added based on the experts opinions and experiences (green). R7 kept its name, but its contents changed completely.

Although R4 was removed, the risks all kept their original number; this was done to simplefy the comparison between the versions from before and after the expert interviews.

Risk R1 R2 R3

Name Solution lacks solid availability Solution is insecure Solution or ecosystem produces data lock-in Entity becomes unpopular Ecosystem contains power imbalance Entity lacks innovation Keystone neglects ecosystem needs

R4 R5 R6 R7

Actor lacks size and stability Solution has an unpredictable lifecycle Table 10 - Final risks based on literature and expert interviews

R8 R9

Status Remains in the method. Remains in the method, but some indicators are added. Remains in the method, but was changed from Solution or ecosystem produces lock-in to Solution or ecosystem produces data lock-in. Removed from the method, partly overlapped with R6 and R1. Remains in the method, some changes in metrics. Remains in the method. Some characteristics from R4 were added. Description of this risk was changed to the way the keystone assures third party additions to his platform. Added, based on experts opinions. Added, based on experts opinions.

The new risks and the updated detailed descriptions can be found in Appendix F. and are described following the characteristics shown in section 5.2.1, Applicable to, Type, Description, Metrics, Impact on business, Example and Sources. Table 11 shows the new mappings of how certain entity types are subject to certain risks. It is the new version of Table 5 from p. 43 and reflects the changes in the risks as shown in Table 10, R4 is removed and R8 and R9 are added. The gray cells in the table indicate the mappings that were removed, based on the expert interviews. Below the table, these removed mappings are explained.

Chapter 5. Risks and precautions - 5.4 Final risks and precautions Page 50 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Entity type
R1 R2 R3 R5 Technical cloud components V V V Ecosystem Keystone Niche Player

V V V V V V V

R6 V R7 V R8 R9 V Table 11 - Mapping of risk on entity type, after expert interviews

R3 - Keystone and Niche Player. The name and scope of R3 was changed from Solution or ecosystem produces lock-in to Solution or ecosystem produces data lock-in and this risk is therefor only applicable to the services and products that can create data lock-in. A specific ecosystem might not be able to create physical data lock-in, but it can that a particular ecosystem uses a proprietary format, or is the only player supporting certain data structures. R5 - Technical cloud components. Expert interviews made it clear that a customer cannot have influence on the behavior of a technical component. Influence can be exerted on other people or organizations (Keystone, Niche Player). R6 - Ecosystem. Innovation might be stimulated or hindered by the keystone of an ecosystem, but it is rarely the ecosystem itself that lacks innovation. This is always the result from Keystone or Niche Player behavior with regard to a specific Technical cloud component. R7 - Niche Player. Dedication changed from a risk that was applicable to all the actors in the ecosystem, to a risk that was only applicable to the keystone or the ecosystem itself.

In the next section, the changes to the precautions, based on the interviews will be discussed.

5.4.2 Precautions
Table 12 shows the precautions that are used in this research, with a short version of the explanation from Table 8. The most important changes to the precautions were: P6 was a precaution that was thought to be important, but found little backup in literature. This lack of backup was confirmed by the experts. This precaution is removed because it is not a real precaution (red). P7 did not have any literature backup as well, but the experts gave significant backup to this precaution to include it in the method. P8, P9 and P10 were added based on the experts opinions and experiences (green).

Although P6 was removed, the precautions all kept their original number; this was done to simplify the comparison between the versions from before and after the expert interviews.
Precaution P1 P2 P3 Name Deploy escrow services Make regular backups Sign contracts on required service levels Status Remains in the method. The precautions was extended a bit, to better suit cloud computing. Remains in the method. The explanation was extended a bit, based on experts comments. Remains in the method, but with the addition that a SLA is most of the time a static document.

Risk

Chapter 5. Risks and precautions - 5.4 Final risks and precautions Page 51 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

P4 P5 P6

Removed from the method, P6 is too vague and is not a real precaution. P7 Choose Certified and Auditable partners Remains in the method, with some good backup from experts. P8 Make use of an intermediary Added, based on experts opinions. P9 Choose solutions on one shared platform Added, based on experts opinions. P10 Choose a mature and reputable company Added, based on experts opinions. Table 12 - Final precautions based on literature and expert interviews

Choose an ecosystem of significant size Choose solutions that support open data formats Beware of trends in your ecosystem

Remains in the method. Remains in the method.

The new precautions and the updated detailed descriptions can be found in Appendix G. Appendix F. and are described following the characteristics shown in section 5.2.2, Counteracts, Type, Description, Example, Explanation of counteracting and Sources. Table 13 shows the mappings of how the precautions can help to counteract certain risks. Note that P6 and R4 are removed from the method and are therefore not shown in this mapping. There are no changes in this from the original mapping (Table 7), based on the expert interviews, only extra mappings in P8-P10 and R8 and R9.

Precautions
P1 P2 P3 P4 P5 P7 P8 P9 V P10 V V

R1 V V V V V R2 V V V R3 V V V V R5 V V R6 V R7 V R8 V V R9 V Table 13 - Final risks - precautions mapping based on literature and expert interviews

Risks

V V

V V

Table 13 plays an important part in the method. The precautions that can help to counteract the risks that are identified in the solution will be selected using this mapping. It must be stated that the precautions are never completely able to prevent a specific risks. As each solution and customer is different, the effectiveness of a precaution is different in every situation. In the next chapter, the risks and precautions from this chapter will be used in the creation of the method, the goal of this research.

Chapter 5. Risks and precautions - 5.4 Final risks and precautions Page 52 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 6

CRISP Method

This chapter uses the input from the precious three chapters (theoretical background, Cloud Solution Modeling Method and risks and precautions) to create a method that is able to identify risks and suggest precautions in cloud computing solutions. The method is named the CRISP Method, The Cloud solution Risk Identification and Selection of Precautions method. The final version of the method is constructed based on the following ingredients. The structured way to model a CSSN. The risks found in literature. The way in which risks can be mitigated by taking precautions, based on literature. The experts additions and comments to all of the above. The model of how the different concepts in the problem area related to each other (next section). Risks that jeopardize the solutions have a realistic probability of having impact on a Customers business (the system is of importance to the customer).

The CRISP method will first be discussed in general and after that it will be discussed in detail. To make the method better understandable, the Exact Online example will be used to show examples of deliverables. These examples will be included in the appendices. The final section of this chapter will discuss why this method is only suitable for the cloud computing part of a solution.

6.1

Model of the concepts in the CRISP method

In this section the concepts of the method and the connections between them will be discussed. These concepts are not the deliverables of the final method, but are the building blocks of which these deliverables will be constructed. As the meta-modeling method by (Van de Weerd & Brinkkemper, 2008) dictates, the concepts will be written in CAPITAL. To distinguish between the deliverables and the concepts, the deliverables will be written in GREEN and the concepts in BLUE. The concepts from the CSMM (Chapter 5) (CSSN) are closely related to the concepts used in the CRISP method, to distinguish them, they will be shown in RED. This color coding will be used throughout this entire chapter, including in the models that are shown. Figure 13 shows the concepts that are used in the method. The SOLUTION is the same concept as the solution in the conceptual model of this research (Figure 1) and consists of one or more ENTITIES. These ENTITIES can be ACTORs, RELATIONSHIPs, ECOSYSTEMs or PLATFORMs, the concepts from the Cloud Solution Supply Network that define a cloud solution. These ENTITIES have an ENTITY TYPE that corresponds to the ENITITY TYPES in Table 3 on p. 41 (Keystone, Niche Player, Ecosystem or Platform) and one or more PROPERTIES that define an ENTITY. An example of an ENITITY could be Microsoft, the PROPERTIES of this ENTITY are that it is a large company, is the keystone of multiple ECOSYSTEMs and has a good reputation. The ENITITY TYPE of Microsoft is keystone (see Table 3).
Chapter 6. CRISP Method - 6.1 Model of the concepts in the CRISP method Page 53 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

RISKs are applicable to one or more ENITTY TYPEs and consist of one or more INDICATOR VALUEs and one or more BUSINESS IMPACTs. If a RISK is actually applicable to an ENITITY that has the same ENITITY TYPE of which the RISK is applicable to, it is checked by comparing the INDICATOR VALUEs of a risk to the PROPERTIES of the ENITITY. If one of the RISKs associated with keystones has the INDICATOR VALUE that the ENITITY is a large company, this RISK is applicable to Microsoft. The BUSINESS IMPACT of a RISK that is applicable to an ENTITY is what really counts to the customer of the SOLUTION. The importance of the BUSINESS IMPACT of a RISK is the determinant of the presence of the PRECAUTION against a RISK in the final ADVICE. The ADVICE is the final deliverable of the CRISP method and contains the RISKs that potentially have the most BUSINESS IMPACT, combined with the PRECAUTIONS that can be taken to counteract these BUSINESS IMPACTS.

ACTOR SOLUTION RELATIONSHIP

ECOSYSTEM 1..* ENTITY PLATFORM

1 ENTITY TYPE 1..*

1..* PROPERTY

applicable to

1..* 1..*

1..* 0..* 1..*

INDICATOR VALUE

RISK 1..* BUSINESS IMPACT

0..* PRECAUTION 0..*

ADVICE

Figure 13 - Model of the concepts in the method's scope.

Chapter 6. CRISP Method - 6.1 Model of the concepts in the CRISP method Page 54 of 125

detects

has

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Concept SOLUTION ENTITY ACTOR (INTEGRATOR, RESELLER, VALUE ADDED RESELLER, SUPPLIER)

Description A SOLUTION is a combination of different RELATIONSHIPS and PLATFORMS between ACTORS (and their providers) that together solve the customers problem. An ENTITY is an ACTOR, a RELATIONSHIP, an ECOSYSTEM or a PLATFORM from the CSSN. An ACTOR is usually an organization. The role of this ACTOR can be: INTEGRATOR, or consultant, an organization that helps to create the solution. Help could be technical (INTEGRATOR) or business (consultant), both can of course be done by the same organization. RESELLER, an organization that sells services that are delivered by another organization. VALUE ADDED RESELLER, sells services from another organization, but adds something to this service. An example is the hosting provider who hosts a third party service. SUPPLIER, the actual vendor of the service.

RELATIONSHIP (MONEY-, DATA-, SERVICE-, PRODUCT FLOW)

In the CSSN there is also the CUSTOMER as an ACTOR. For this method however, the CUSTOMER is not relevant, as it is not an ACTOR in the SOLUTION, but the end-user of the SOLUTION A RELATIONSHIP is a connection between two ACTORS and exists in different forms: MONEY, the payments for DATA, SERVICE and/or PRODUCT flows. DATA, the flows of data and metadata, primarily present when two services share data through an API. SERVICE, the flow of a service from its provider to its customer. PRODUCT, the flow of a product from its provider to its customer. In a CSSN, this customer is typically not the CUSTOMER, but a SUPPLIER or a VALUE ADDED RESELLER, which turn that PRODUCT into a SERVICE or PLATFORM to deliver a SERVICE with.

ECOSYSTEM

The ECOSYSTEM to which certain ACTORS belong. An ACTOR can be part of multiple ECOSYSTEMS and only the relevant ECOSYSTEMS to this solution should be handled. PROPERTY A PROPERTY of an ENTITY, for an ACTOR a PROPERTY could be size of company or type of customers. ENTITY TYPE The type of an ENITITY determines whether a RISK is applicable to an ENITITY. Example, a RISK could describe the problems involved with Solution lacks solid availability; this is applicable to ENTITIES that are a RELATIONSHIP, or a PLATFORM, but not to ACTORS. PRECAUTION A PRECAUTION is something that a CUSTOMER could do to lower the BUSINESS IMPACT of a RISK, or to avert the RISK completely. INDICATOR VALUE An INDICATOR VALUE detects whether a PROPERTY of an ENITITY has a value that indicates that a RISK is applicable to this ENITTY. BUSINESS IMPACT The impact a RISK has on the business of a customer. RISK A situation or series of events that is applicable to an ENTITY and might negatively influence the customer. ADVICE The set of RISKS and the PERECAUTIONS that could be taken to mitigate those RISKS, presented in such a way that it shows which RISKS are most important to mitigate. Table 14 - Concept descriptions from the conceptual model of the method's scope (Figure 13).

6.2

High level overview of the CRISP method

A high level overview of the method is shown in Figure 14 and a high level description can be found in Table 15 and Table 16. The concepts that are named in the tables that are shown in blue refer to the concepts (Figure 13 and Table 14). The final method consists of three major steps. In the first step Identify risks, the SOLUTION is visualized. This is of vital importance to the rest of the method for a number of reasons.

Chapter 6. CRISP Method - 6.2 High level overview of the CRISP method Page 55 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

A visualized representation of the SOLUTION allows for the quick identification of missing ENTITIES and is easy to discuss with others. The visualization of the SOLUTION in the CSSN delivers the ENITITES that are used to determine the RISKS that are applicable. Visual representations can reveal connections that might otherwise not have been obvious.

Besides the visualization, the first step also distills all the ENTITIES from the SOLUTION and checks which RISKS might be applicable, based on the ENITITY TYPES of the ENTITIES. The second step looks at the BUSINESS IMPACTS of the RISKS and places the applicability of the RISKS in context. A RISK that is applicable is not necessarily problematic for the customer. In the third and last step, the RISKS that are applicable and have a BUSINESS IMPACT that cannot be neglected are gathered and PRECAUTIONS against them are found. This results in the final ADVICE that helps the customer to mitigate possible RISKS.

Identify risks

ENTITY-RISK MAPPING

Assess risks

LIST OF IMPACTS ON BUSINESS

Construct evaluation

ADVICE

Figure 14 - High level overview of the final method.

Activity Identify risks

Assess risks

Description In the first stage of this method, the SOLUTION is modeled and the RISKS involved in this solution are identified. The result of this activity is the ENTITY-RISK MAPPING, an overview of which RISKS might be applicable to which ENTITIES. The second stage is about the assessment of the RISKS. This is done by checking two things: Check whether the RISK is applicable to the ENTITY by comparing the VALUE of the PROPERTIES to the threshold VALUES of the RISKS. Determine the potential BUSINESS IMPACT of the RISK to the CUSTOMERS business.

Construct evaluation

The final ADVICE will be constructed in the final stage of this method. This is done by looking at the most important RISKS and the PRECAUTIONS that could be taken to mitigate them. Table 15 - Description of the activities from the high level overview of the final method.

Chapter 6. CRISP Method - 6.2 High level overview of the CRISP method Page 56 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

CONCEPT ENTITY-RISK MAPPING LIST OF IMPACTS ON BUSINESS ADVISE

Description These mappings show which RISKS might be applicable to which ENTITIES. A list of BUSINESS IMPACTS, ordered by how applicable it is to the ENTITY and how big the impact is. The final overview of what the most important RISKS are to tackle and what can be done to tackle those RISKS. Table 16 - Description of the concepts from the high level overview of the final method.

6.3

Detailed view of the final CRISP method

A detailed view of the method is shown in Figure 15 and a detailed description can be found in Table 17 and Table 18. This view of the CRISP method brings concepts from three different models together. The GREEN concepts are the actual deliverables of the method, for each deliverable, an example is based on the Exact Online SaaS. These examples can be found in Appendix H. to Appendix N. . RED concepts come from the CSMM to create the CSSN and the BLUE concepts are based on the model of the concepts (Figure 13)

Chapter 6. CRISP Method - 6.3 Detailed view of the final CRISP method Page 57 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Identify actors

COLLECTION OF ACTORS 1

2..*

ACTOR

1..*

RELATIONSHIP

Identify relationships

COLLECTION OF RELATIONSHIPS 1

1..*

ECOSYSTEM

0..*

PLATFORM

Create CSN
Model solution

CLOUD SOLUTION SUPPLY NETWORK 1


input for

0..*

SOLUTION

represents

1 List entities from modeled solution

1..* 4..* ENTITY

1..* 1

PROPERTY

LIST OF ENTITIES 1
input for

1..* applicable to

1 Map entities to risks


Identify risks

1..* 1..* RISK

1..*

INDICATOR VALUE

1..*

ENTITY-RISK MAPPING 1 1 input for

1..* 1..*

BUSINESS IMPACT

Assess if threshold metrics are met


input for

1 ENTITY-RISK MAPPING OF APPLYING RISKS 1 0..*

Assess business impact of risks

1 LIST OF IMPACTS ON BUSINESS 1

PRECAUTION 0..*

Order risks by business impact and applicability


Assess risks

Find appropriate precautions ADVICE

Color legend
Construct final conclusion
Construct evaluation

Method deliverables Method concepts CSN concepts

Figure 15 - Detailed view of the final CRISP method.

Chapter 6. CRISP Method - 6.3 Detailed view of the final CRISP method Page 58 of 125

detects

ENTITY TYPE

has

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Activity

Description Boucharas et al. do not give any precise steps to take in order to create a SSN. This is probably because a SSN is not a very complex model to create. The CSSN is of comparable complexity and does not need a lot of explanation either. There are however some insightful deliverables that are worth showing, therefore the modeling of the solutions is shown in three steps. Identify actors The first step of the modeling of the solution is the identification of the ACTORS. This activity has a COLLECTION OF ACTORS as deliverable. An example of this deliverable can be seen in Appendix H. . Identify relationships After the ACTORS are modeled, the RELATIONSHIPs between them can be identified. An example can be seen in Appendix H. . Create CSSN The ACTORS and the RELATIONSHIPS are the primary components of the CSSN. In this step the network is created and the ECOSYSTEMS and PLATFORMS are added to complete the CSSN. See Appendix I. . List entities from After the CSSN is made, a complete list of the ENTITIES in the solution can easily be modeled solution extracted from the model. See Appendix I. . Map entities to risks Using the entity and concept mapping (Table 3) and the mapping of risk on entity type (Table 11), a list of all the ENTITIES and the RISKS that could possibly be applicable to them is formed. See Appendix J. . Assess if threshold When the RISKS are identified that could possibly be applicable to a certain type of metrics are met ENTITY, the VALUES of the PROPERTY that could indicate the actual presence of a RISK is shown. See Appendix K. . Assess business impact The BUSINESS IMPACT of RISKS is closely related to what a specific ENTITY means of risks to the solution and to the business of the CUSTOMER. See Appendix L. for an example of how this could look like. Order risks by business There are different ways to order the risks by business impact. Which one is impact and appropriate depends on the customer and on the type of SOLUTION. Appendix M. applicability shows different ways of how the most important risks could be identified. Find appropriate Finding the appropriate precautions to mitigate the (most important) RISKS that precautions have been identified in the previous activity is something less straight forward than the rest of the CRISP method. To find suitable precautions, Risks-precautions mapping (Table 13), and of course the descriptions of the precautions (Appendix G. ) can be used. For an example of how the appropriate precautions are found, see Appendix N. Construct final The final conclusion is a textual description of how certain precautions should be conclusion applied to mitigate the most important risks in the solution. Table 17 - Description of the activities from the detailed view of the final CRISP method.

Sub activity

Table 18 shows only the descriptions of the deliverable concepts. For a descriptions of the BLUE and RED concepts, see Table 14 or Chapter 4.
CONCEPT COLLECTION OF ACTORS COLLECTION OF RELATIONSHIPS CLOUD SOLUTION SUPPLY NETWORK LIST OF ENTITIES ENTITY-RISK MAPPING ENTITY-RISK MAPPING OF APPLYING RISKS LIST OF IMPACTS ON BUSINESS ADVICE Description A list of all the ACTORS in the SOLUTION, made to be used in the CSSN. A list of all the RELATIONSHIPs between the ACTORS in the SOLUTION, made to be used in the CSSN. The visual representation of the SOLUTION in a CSSN (Appendix I. ). A list of all the ENTITIES in the CLOUD SOLUTION SUPPLY NETWORK The ENTITY-RISK MAPPING (Appendix J. ) shows which ENTITIES could be suffering from which RISKS. The ENTITY-RISK MAPPING OF APPLYING RISKS (Appendix K. ) shows the degree in which ENTITIES could be suffering from certain RISKS. A List of the most important BUSINESS IMPACTS (Appendix L. and Appendix M. )

The conclusion of the CRISP method, showing which PRECAUTIONS should be taken to mitigate what RISKS. Table 18 - Description of the deliverable concepts from the detailed view of the final method.

Construct evaluation

Assess risks

Identify risks

Model solution

Chapter 6. CRISP Method - 6.3 Detailed view of the final CRISP method Page 59 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

6.4

Usability of the method

A cloud computing solution is rarely an isolated system. It often connects with on-premise or outsourced solutions that are not cloud-based. It is therefore important to emphasize the usability of the method and the risks and precautions it contains. This section will explain why the method is only suitable for identifying risks in cloud computing solutions and should not be used for the evaluation of on-premise or outsourced applications Some of the reasons why an organization could use a solution that combines both cloud computing and regular on-premise solutions are: Legacy system - As cloud computing is relatively young; organizations often have on-premise solutions that perform as desired and are not (yet) moved to the cloud. It might be too large a transition to move these systems to the cloud or a suitable cloud solution is not yet available. Legislation about data location - There might be legislation in place that demands certain sensitive data about customers or patients to remain on premise. Lack of faith in cloud computing - Organizations often have little experience with cloud computing and like to keep systems that are vital to the business in house. Production environments - Some production processes that physically take place onpremise must be coordinated by systems that can best be placed on-premise.

A lot of the solutions that will be evaluated with the use of the CRISP method will consist of both onpremise and cloud computing parts. Although some of the risks might seem applicable to onpremise applications as well, there are some reasons to use the CRISP method only for the cloud computing parts of these solutions: Risks - The risks that are found in literature and through expert interviews are all focused at cloud computing solution. Some of the risks, like R1 (Solution lacks solid availability) and R2 (Solution is insecure) are applicable to on-premise systems as well, but the indicators that were identified around these risks are not. Using these risk-definitions to identify risks in onpremise solutions could result in false-positives or neglected risks. While it is thought that the eight risks that are found are rather complete for cloud computing, it is possible that onpremise systems run additional risks. This notion is strengthened by the first requirement that was set for the method. This states that all the risks have to be external, while onpremise solutions obviously also suffer from internal risks. Finally, the risks that apply to cloud computing might be farfetched for on-premise systems. From a risk perspective, these three reasons are given to leave out the on-premise solutions from this evaluation: o o o Not described to fit on-premise systems. On-premise systems might run other risks as well. Some risks might be farfetched for on-premise systems

Chapter 6. CRISP Method - 6.4 Usability of the method Page 60 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Precautions - Some of the precautions, P1 (Deploy escrow services) and P9 (Choose solutions on one shared platform) for example, have different ways of operating when used in on-premise situations. Escrow services for a SaaS solution are completely different from an escrow arrangement for the source code of an on-premise application. The mapping between the risks and precautions that is shown in Table 11 is constructed for cloud environments and where certain precautions could work for risks in cloud solutions, they might be worthless in on-premise situations. Seen from the other side, Cloud computing has some specific limitations that kept precautions that work in on-premise situations from entering the method. From a precautions perspective, these two reasons are given to leave out the on-premise solutions from this evaluation: o o The way precautions work differs from on-premise to cloud computing situations. Some precautions might be missed because of the limitations of cloud computing.

Method - The CRISP method is strongly based on the CSMM, described in Chapter 4. When on-premise solutions were to be modeled with the CSMM, it is possible that certain entities were missed or misrepresented and that therefore risks might not be identified.

The CRISP method is developed for identifying risks and suggesting precautions to these risks in cloud computing solutions. This makes it unfit for this task in on-premise solutions. When a combined solution is evaluated using this method, only the cloud computing ACTORS, RELATIONSHIPs, ECOSYSTEMs and PLATFORMs should be evaluated. A method suitable for the identification of risks in on-premise systems should be used to cover that part of the solution. What goes for on-premise entities, also goes for the entities (RELATIONSHIPs) that span the border between the cloud computing and the on-premise parts of the solution. The CRISP method is not suited to identify the risks in these entities. Further research should help to find how risks could be identified in these entities. After the creation of the method, in the next chapter the newly developed CRISP method will be validated by applying the method on a case at a client of Deloitte.

Chapter 6. CRISP Method - 6.4 Usability of the method Page 61 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 7

Validation of the CRISP method

The purpose of this validation is to check whether the results of the CRISP method (developed in the previous chapter) actually help in a real life situation. The method is executed and the results of the execution (the found risks and suggested precautions that should be taken) are discussed and evaluated with parties involved with the project. The validation of the CRISP method will be performed on a project of Deloitte at Canon. Due to the confidential nature of this project, the results of this validation are only partly available in this public version of this thesis.

7.1

Introduction to the solution

[Confidential parts removed]

7.2

Execution of the CRISP method

[Confidential parts removed]

7.2.1 Identify risks and Model solution


[Confidential parts removed]

7.2.2 Assess risks


[Confidential parts removed]

7.2.3 Construct evaluation


[Confidential parts removed] 7.2.3.1 Risk perspective [Confidential parts removed] 7.2.3.2 Precaution perspective [Confidential parts removed]

7.2.4 Findings during the execution of the CRISP method


[Confidential parts removed]

7.3

Validation of the results of the CRISP method

[Confidential parts removed]

7.3.1 Reactions to the results from the CRISP method


[Confidential parts removed]

Timing & results of the method


Chapter 7. Validation of the CRISP method - 7.1 Introduction to the solution Page 62 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

The CRISP method is usable when the design of a solution is complete. This means that it can be executed based on the solution and on the business needs that the solution will be satisfying. Where none of the interviewees had used a structured way of analyzing risks in multi-party cloud solutions before, they were convinced that the method would bring out risks earlier than without the method. However, this was not their primary interest in the method. The fact that the method uses a structured way to analyze risks in all the entities in the solution was regarded as the characteristic of the highest value. Performing a structured analysis minimizes the chance of missing entities or overlooking risks that might be applicable. Also the fact that the method is easy and quick to execute is regarded a good characteristic. [Confidential parts removed]

7.3.2 Results & limitations to validation


The interviewees regarded the results from the execution of the CRISP method on the project at Canon as highly useful, the risks that were found were relevant and although some precautions were hard to implement, most precautions made sense and were useful. Not only is the execution of the CRISP method useful for the risks around the cloud solution that is evaluated. The interviewees also pointed out that a structured look at the risks in one part of the solution, made them more aware of the risks in the on-premise part of the solution at Canon and even at other projects. Because the CRISP method takes relatively little time to execute and the risks are easy to understand, the results from the method are easily applicable in everyday consulting practice. Limitations to this validation lay in the fact that there was only one case used to validate the CRIPS method and that only three interviewees were asked for their opinion. To get a better basis for the validation of the method, more cases should be used, preferably in different industries and more interviews should be conducted, preferably with different parties involved with the solution. These results could then also be statistically analyzed. This limitation will be covered further in section 9.3. The CRISP method is built to identify risks and suggest precautions from an objective point of view. Scientific literature and experts who know both the vendors and the customers interests are combined to reach this goal. However, the method will not be used in objective situations, it will be used in situations where risk perception is often more important than the actual chance of a risk occurring. Therefore the next chapter shows how the risks and precautions that form the basis for the CRISP method are perceived by customers and vendors.

Chapter 7. Validation of the CRISP method - 7.3 Validation of the results of the CRISP method Page 63 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 8

Perception of risks and precautions

During the development of the CRISP method, it became more and more apparent that besides the objective side of the risks that this method is analyzing (validated in the previous chapter), there is an important factor that goes by the name of emotion. A lot of risks are perceived very different from their objective chance of happening. Decision making is dependent on these perceptions and a method that only focuses on objective parameters is therefore only worth so much. This chapter is added to give more insight in how the risks and precautions are perceived by customers and vendors. Results from these interviews can be used in applying the method and in managing customers expectations. The interviews revealed not only how the risks and precautions would work out in reality, but they also shed light on the differences between customers and vendors. These two dimensions (customer versus vendor, scientific approach versus real life) are strongly connected in these interviews. It must be stated that the differences found between customers and vendors, are based on their opinions on the risks and precautions. These risks and precautions are only part of the complete spectrum of customer and vendor interactions. This chapter consists of four sections. The first section handles the setup of the interview and which organizations were interviewed. The second section shows how the interviewees viewed the risks and precautions and focuses on the scientific approach versus the real life dimension. Section three strives to extract the customer versus vendor dimension and tries to show the differences between them. The fourth and last section contains a short summary of the two dimensions. It is emphatically stated that these interviews are not meant as a bearing as of how cloud computing is generally perceived. Although it was pursued to speak to as differentiated an audience as possible, the outcomes of these interviews are only to be used to put the objectively gathered, rather abstract, risks and precautions into the context of everyday business.

8.1

Interview setup

To be able to check how customers and vendors of cloud computing solutions feel about the risks of cloud computing, thirteen semi-structured interviews were held. The purpose of these interviews was to gain more insight in how the risks and precautions (that were found in literature and refined by experts) were experienced by customers and what vendors did to comfort customers. The interviews consisted of two parts, in the first part; a general discussion was conducted on what the interviewee felt were the most important risks of cloud computing. In the second part of the interview, the risks and precautions were discussed and the interviewee was asked to list the five risks they found most important.

8.1.1 Interviews with (potential) cloud computing customers


Six interviews were held with (potential) customers of cloud computing. Potential means that it did not matter whether they were actually using cloud computing or not, they belong to the group of potential users, as virtually all organizations are able to use cloud computing solutions. All the interviewees had basic knowledge about cloud computing and the interviews were about how they perceived risks, not how they dealt with them. The interviews took between one and two hours and
Chapter 8. Perception of risks and precautions - 8.1 Interview setup Page 64 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

were held at the headquarters of the organizations. Employees of the following organizations were kind enough to make time for the interviews:

Organization Type Artsen Zonder Grenzen Humanitarian medical aid organization De Brauw Blackstone Westbroek International law firm Coro Property investment company (listed) Huis van de Sport Combined housing for sports associations Rotterdamse Vereniging van Katholiek Onderwijs Association for the management of primary schools Ziekenhuis Rivierenland Tiel Regional hospital Table 19 - List of (potential) cloud computing customers

The six interviewees were all (some collectively, some solely) responsible for the IT of their organization. Some of them were actually CIO, but all of them had a major voice in how the organizational IT solutions were managed.

8.1.2 Interviews with cloud computing vendors


A total of seven cloud computing vendors or resellers were found prepared to participate in the interview around risks and precautions. From the start of the interviews on, it was thought of that vendors would see less risks than the customers would. This led to the decision that besides the questions regarding the risks and precautions, the vendors were also questioned about their strategy and why certain decisions were made in de past. As elaborately described in section 3.1, the strategy a vendor or supplier follows, greatly impacts the risks they and their customers run. Employees of the following organizations were kind enough to make time for the interviews:
Organization Type 42 Windmills Model driven development Eloqua Marketing Automation and Revenue Performance Management Inisi ICT infrastructure supplier Mendix Integrated and adaptable business applications platform Microsoft Software centipede Paston Local reseller of Process Maker Salesforce.com Large SaaS & PaaS company Table 20 - List of cloud computing vendors

In the vendor arena, it was attempted to find SaaS, PaaS and IaaS vendors/suppliers as well as some large and small organizations.

8.2

Interview results regarding risks and precautions

This section contains the combined results from the interviews of both vendors and customers. Some responses are quoted and contain a C1-C6 (customer) or V1-V7 (vendor) marker. This allows the different responses from the same organization to be identified. It is deliberately chosen not to connect the organizations to their specific quotes. When a general sentiment is observed and mentioned, it is not always attributed to all the organizations involved.

Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 65 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

To give more insight in how far specific organizations are with cloud computing and what the size is of their IT departments (without revealing their identity), Table 21 shows a short description of C1C6.
Description In-house IT department of about 10 people, in 2010 allegedly 100% uptime, business is used to high availability and quick solutions. See data security as the primary reason for keeping everything in-house, for this reason, suppliers have no remote access. Their own people know what is best for the business and are therefore better capable of servicing the business. C2 Availability is the most important part as downtime costs the company a lot of money. Organization has a lot of employees with little IT knowledge and tries to keep things as simple as possible. About 20 good all-rounders in IT department, specific knowledge is hired, often from like-sized companies. Besides availability, security is an important concern to keep information in house. C3 C3 has a lot of their IT outsourced due to a lack of knowledge in-house; outsourcing is preferably done to like-sized companies with a focus on service and now what is going on at the organization. Keeping data inside the Dutch borders is deemed of key importance. C4 C4 has much of the IT placed outside of the organization, because of lack of internal IT knowledge and because they believe that cloud computing providers are better at securing their data. They have experience with hosted exchange solutions and other cloud solutions, for example for their financial administration. More cloud solutions will probably follow in the future. C5 Outsourcing is difficult due to the nature of the organization and the data they handle, therefore everything is placed in-house. A lot of the needed knowledge is available in their internal IT department of around 20 people. Availability is by far the main concern. C6 This organization is only recently considering outsourcing or cloud computing, everything is hosted in-house and the IT department is understaffed at only two or three people, doing only the necessary work to keep the servers online and having little time for other projects. Table 21 - Description of the IT departments of the interviewed customers Customer C1

The vendors were not described in this way because it is impossible to give a proper description of a vendor while keeping its identity hidden.

8.2.1 Risks in cloud computing in context


In this section, the risks from Table 10 are all fitted with the remarks made by the customers and vendors. Each risk-section contains three items: Customers remarks - How customers feel about this risk. Vendors remarks - How vendors feel about this risk. Summary - What might be done to close the gap between customers and vendors?

8.2.1.1 R1 - Solution lacks solid availability Customers remarks - Unavailability is the most named risk by the customers. Some of them feel that the cloud vendor has a different standard then they do, as they strive for efficiency, they are bound to make compromises that their internal IT department does not C1, C2. C1, C2, C3 and C5 stated that external providers often do not show the sense of urgency that theyd like them to show. Only two confirmed their lack of knowledge on some of the complex IT topics and thought that external parties had more knowledge about the systems to be able to deliver greater up-times C3, C4. An often heard complaint was that in the case of unavailability, the service providers hide behind the best effort clause in their SLA. More about this in the Customers remarks on P3.
Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 66 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Vendors remarks - Without any exception, all cloud vendors state that they are better able to provide a stable environment than customers can provide for themselves in an onpremise situation. They have more resources and more in-depth knowledge about the service and therefore a better basis to provide a stable service. One of the vendors, V5, compared cloud computing vs. on-premise to flying a commercial airline vs. driving your own car. It is a fact that driving a car is riskier than flying a plane, but people feel that they have more control when they are driving themselves. Summary - The difference in points of view between customers and vendors are striking, but lack of knowledge on the side of the customer and lack of understanding of the customers lack of knowledge at the side of the vendor seems to be the primary factor in this difference.

8.2.1.2 R2 - Solution is insecure Customers remarks - From a customers point of view, the main security risk they see in cloud computing, is unnoticed, unauthorized access to their data C1, C2, C3, C5. They state that keeping data in house gives them a better sense of safety. As it comes to storing data abroad, especially in the US, a lot of uncertainty exists around the Safe Harbor program in combination with the Patriot Act. These two seem to be incompatible and it has yet to be found out which of the two has precedence. Vendors remarks - As with unavailability, most of the vendors say they are better at delivering secure solutions than the customers can do themselves. Another way to show how secure they are is pointing at big corporations and banks that are their customers, would these organizations use un-safe platforms and services? V4 and V5. Summary - Security is a difficult risk to manage, especially because you cannot prove that something is secure; you can only prove that something is not. Cloud vendors tend to be very serious about security and just as with banks, trust is an important factor in this. Trust is something that cannot be bought, it has to grow over time and as the market matures, vendors will do their very best to earn that trust.

8.2.1.3 R3 - Solution or ecosystem produces data lock-in Customers remarks - As data is no longer in-house, customers feel that the risk of lock-in becomes a bit bigger than it is at the moment, but they also admit that they are already locked in, so it is not much of a change from their current situation C2, C3, C4, C5. C6 noticed that although your data might not be locked in, once youve moved to the cloud and sold or abandoned most of your local hardware and IT knowledge, you are locked-in in de cloud. Moving everything back in-house again then requires a huge investment in staff and hardware. Vendors remarks - Most vendors support open formats for data export and have APIs to support these exports. They therefore feel that they are not locking-in customers more than their on-premise counterparts do. Vendors try to lock-in customers by creating features that customers can only get at that specific vendor and by being a constantly innovating platform V5, V7. Summary - Data lock-in has been around for decades and although it might feel as bigger when data is located off-site, in practice this is not really the case.

Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 67 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

8.2.1.4 R5 - Ecosystem contains power imbalance Customers remarks - C1, C2, C3, C4 all pointed out that cloud vendors are often large, much larger than their own organization. They prefer to do business with a like-sized organization, so that they at least have some influence on the other party, based on their share in the organizations revenue. Vendors remarks - Cloud computing is by design a one-size-fits-all solution. Vendors sell different versions with features enabled or disabled, but the single codebase makes it very inefficient to build custom features or deliver custom services. Some vendors, V5 and V7 have indicated that they have changed parts of their service because large organizations asked them to, but these changes then became enabled for the entire user base. Some vendors (C4, V5 and V7) however, actively support their customers and partners to come up with new features and alterations to the platform. Summary - Having little influence on what the cloud vendor is doing is something that the customers will have to learn to live with, the nature of cloud computing is that it tries to serve as many customers with the same feature set, so specific wishes are often hard to implement.

8.2.1.5 R6 - Entity lacks innovation Customers remarks - Customers did not really see this as a risk, but some did point out that innovation is not always about developing the next killer-features, keeping up with developments in the market is also important (C4, C5). C4 already had some experience with this; their hosted exchange provider is slow to upgrade their exchange solution to new versions, which leaves customers deprived of features that are already widely used. Vendors remarks - Vendors said this was not really a problem, as they all value innovation as highly important. Summary - As the market is still developing, innovation is essential and therefore at the top of the agendas of the vendors. When specific cloud computing services become more commoditized, lack of innovation might become a reality (as in the C4 example above).

8.2.1.6 R7 - Keystone neglects ecosystem needs Customers remarks - Customers generally like certified applications, but when critical features are only available on non-certified applications, this is often a reason to lower the standards and start using that application anyway. Vendors remarks - Dedication to the ecosystem is perceived very differently by different vendors. Some vendors see it as a strong point to have a platform and/or service that is as open as possible (V4, V7), while others believe that by certifying applications and by partnering with other organizations, they are creating more security for their platforms customers (V5).

Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 68 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Summary -What is best for the vendors business is hard to tell, but from a customers perspective, using applications that are certified by a keystone that is dedicated to keeping its platform healthy, definitely has its benefits.

8.2.1.7 R8 - Actor lacks size and stability Customers remarks - A few customer (C3, C5) were positive about small parties in general from the perspective that they are willing to work harder for their clients. When it comes down to cloud computing, big parties are the preferred species (C3, C4, C6). Vendors remarks - The small vendors that were interviewed all had a similar solution for the risks of doing business with a small party for their customers. They (V1, V2, V3) all had an on-premise version of their software as well, so the customers solutions would not stop functioning immediately when something unfortunately would happen to that vendor. Another aspect of small parties is that they are usually capable of reacting quickly to changing market demands and allow the users of their products to use cutting edge technologies. Summary - Small vendors can either make sure that they can deliver on-premise solutions as well, but they could also try to leverage a big players platform, for example by making use of the Azure or force.com platform.

8.2.1.8 R9 - Solution has an unpredictable lifecycle Customers remarks - Customers werent really bothered by the risk of uncertain releases, there was one customer (C1) that pointed out that new releases can also bring significant change in user experience. When you are an organization (as they are) that has a lot of users that have only little experience with IT, even small changes to the user interface or workflow of an application can have a big impact on user experience and helpdesk load. Vendors remarks - As could have been expected, vendors thought this wasnt really a risk. Summary - Apart from some specific situations, product lifecycles are not seen as a big risk.

8.2.2 Precautions in cloud computing in context


In this section, the precautions from Table 12 are all fitted with the remarks made by the customers and vendors. Each precaution-section contains three items: Customers remarks - How customers feel about this precaution. Vendors remarks - How vendors feel about this precaution. Summary -What might be done to close the gap between customers and vendors?

Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 69 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

8.2.2.1 P1 - Deploy escrow services Customers remarks - None of the customers appeared to be aware of the possibility of escrow agreements in cloud computing solutions. In general they prefer a bigger player with a good reputation to specific insurances against the risks involved with small parties. Vendors remarks - Some of the smaller players (V1) had such a solution in place, but customers that were interested in this escrow solution usually went for the on-premise version of the service. Summary - Escrow services are on paper a good way to mitigate risks, it seems however that many customers prefer to choose bigger parties that do not need such agreements.

8.2.2.2 P2 - Make regular backups Customers remarks - All customers named a separate backup as an important feature to ensure the safety of their data. Vendors remarks - Vendors were generally less inclined to point out the benefits of a backup in a third party environment. This is maybe because they see the advice to make a backup as an admission of weakness. Summary - It might be a good idea for vendors to be a bit more relaxed about backups, it could be both a boost in customer faith and in revenue when they started to resell third party backup facilities for their cloud solutions.

8.2.2.3 P3 - Sign contracts on required service levels Customers remarks - C1, C2, C4 and C5 all stated that SLAs are usually very unilateral. The customers can take-it-or-leave-it and with the best effort clause the responsibility for almost any type of accident is discarded. Vendors remarks - As mentioned before, cloud computing solutions are (among other reasons) efficient because they use a single codebase and treat every user equally. An effect of this situation is that it is very hard to differentiate between customers and give them different SLAs Summary - SLAs are standard because the services are, but it might be possible to change the perception of the customer by providing more specific SLAs within the boundaries of this generic offering. Just the fact that customers can choose could make the difference, but this would need further research before it is implemented.

8.2.2.4 P4 - Choose an ecosystem of significant size Customers remarks - Customers like the idea of being able to choose between different niche players, but in practice, once a choice has been made, it is often costly to move between those niche players. The fact that a larger ecosystem has a better chance of survival in the future wasnt pointed out by any of the interviewees.
Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 70 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Vendors remarks - Vendors like a big ecosystem because positive energy radiates from them, keystones like them because they bring in a lot of money, niche players like them because they attract a lot of customers. Summary - Customers and Vendors are both interested in large ecosystems, but for totally different reasons. Large ecosystems can help customers reduce their risks, as the ecosystem has a larger chance of surviving.

8.2.2.5 P5 - Choose solutions that support open data formats Customers remarks - Open data formats receive a positive valuation from each of the customers, as they make it easier to switch between solution providers. Formats do not have to be completely open, as long as they are widely used in the market. Two examples of these kinds of formats are PDF and XLS (C4). Vendors remarks - From a vendors perspective, open formats are less interesting, they like them because when they integrate with other services and support exporting in open formats where this is relevant (V5, V7), but it is not a key feature. Summary - Open formats is not really an issue in cloud computing, being able to export is nice and often this is supported in an open format, so little attention is paid to this precaution.

8.2.2.6 P7 - Choose Certified and Auditable partners Customers remarks - Especially the bigger organizations that were interviewed were serious about certifications, but it seemed that this was primarily a matter of habit. Using certified partners made them feel more secure. Vendors remarks - Larger vendors that take the security of their solution seriously are serious about certification (V4, V5, V7). Not only for their own product, but some also for the products of their add-on builders (V5). Some vendors do not yet have certifications for third party add-ons in place, also because they feel that they are better of growing their addon-base quickly, but will probably add certification programs in the future. Summary - Customers like the idea of certifications, especially bigger organizations can tick that box, certifications in order? Ok! . One certification, like the Safe Harbor program, have yet to prove its value in combination with the Patriot Act and another, SAS70 type II is actually not meant to be used on cloud computing. Having certifications in place is therefore not soul-saving.

8.2.2.7 P8 - Make use of an intermediary Customers remarks - Although intermediaries could help shield customers from certain risks, none of the customers got a warm feeling from the idea of an extra player in the supply chain. Not just based on the assumption that an intermediary would only add to the costs, but also because the intermediaries are another party in the solution that only adds to the complexity of it, usually without adding much value.
Chapter 8. Perception of risks and precautions - 8.2 Interview results regarding risks and precautions Page 71 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Vendors remarks - Vendors seem to be quite enthusiastic about intermediaries, especially because they sell products to customers that would probably otherwise not have bought their product (V7). Other vendors rely completely on other parties to make use of their platform to sell services (V4). Summary - Intermediaries that do not add significant value to the solutions will probably have a hard time surviving in the cloud computing arena. Customers only tend to by-pass them and go directly to the vendors and only a few vendors really rely on them to sell their services.

8.2.2.8 P9 - Choose solutions on one shared platform Customers remarks - In the on-premise solutions, all customers have a preference for a single platform, the Microsoft platform in particular. At the only organization that was already using cloud computing (C4), they used the same platform online as they used onpremise. This decision was made with the integration of cloud and on-premise in mind. The advantage of a shared cloud platform from the availability perspective and reduction of complexity was underwritten, but this was not based on experience. Vendors remarks - Vendors also agreed on the advantages of the use of a shared platform. The few vendors who own a platform (V4, V5) agreed with this precaution, but said that they did not want to see themselves as a sort of escrow-provider that keeps services running in case the vendor of that specific service goes out of business. Summary - The real benefits of a shared platform are only available when there are more than one cloud services working together, at the moment, a lot of companies havent reached that situation yet.

8.2.2.9 P10 - Choose a mature and reputable company There were no particular interesting remarks about this precaution; everyone seemed to agree that this is a good precaution.

8.3

Interview results regarding customers vs. vendors

Although the interviews were mainly focused around the risks and precautions, there were some interesting findings regarding the difference between customers and vendors. Lack of knowledge at potential customers - All the customers that did not yet use cloud computing showed a considerable lack of knowledge about the topic. They mainly used virtualization in their own datacenters and were acquainted with the possibility of running these virtual machines in the cloud at for example Amazon. However, the use of SaaS solutions was surrounded by misconceptions and lack of knowledge about what the precise benefits could be for their organization. IT managers protect their own realm - This finding is hard to confirm, but it appeared as if some IT managers were reluctant to place services outside of the organization, because they felt that with services leaving the organization their influence in and importance to the organization would decrease. For further research, it might be interesting to look into this, for example by comparing the views from the CIO and the CEO of the same organization.

Chapter 8. Perception of risks and precautions - 8.3 Interview results regarding customers vs. vendors Page 72 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Over-optimistic vendors - It was striking how self-assured the vendors were, a lot of risks were classified as irrelevant to their specific situation and precautions were deemed not necessary. As suggested before, it might be the case that vendors see it as a weakness to advice customers to take certain precautions. However, it might even be good for business and earn more trust when vendors are a bit more open about the risks.

Besides the qualitative analyses of the interviewees remarks, it was attempted to visualize the difference in risk-perception between the customers and the vendors (Figure 16). Because the production of Figure 16 was initially not the prime goal of these interviews, it must be stated that Figure 16 is unfortunately not statistically valid. The figure is included because it is thought that it provides an insightful representation of how vendors and customers view risks in different ways.

9,00

8,00

7,00

6,00

Vendors

5,00 R2 - Security 4,00 R7 - Lack of dedication to the ecosystem

3,00

R3 - Data Lock-in R8 - Small parties R1 - Unavailability

2,00

R5 - No influence on 3rd parties

1,00 R9 - Product lifecycles 0,00 0,00 1,00 2,00 3,00 4,00 5,00 6,00 7,00 8,00 9,00 Customers
Figure 16 - Visualization of difference in customers' and vendors' perception of risks.

R6 - Lack of innovation

Chapter 8. Perception of risks and precautions - 8.3 Interview results regarding customers vs. vendors Page 73 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Figure 16 shows the difference in customers and vendors perception of risks. As part of the interview, the interviewee was asked to point out the risks that they deemed most important. Together with the qualitative results from the interviews for each interviewee an ordering of risks was established. The average ordering of each risk for the customers and the vendors is shown in Figure 16. Example All the interviewees were asked to sort the risks in order of importance, placing the most important risk at 1. A lot of interviewees stopped at 3 or 4, saying the other risks are not important to me, because it was not thought of to create Figure 16 at the time, this was not regarded an issue. These not-mentioned risks were given a 9, as there are 8 risks. R1 - Solution lacks solid availability scores a 6.33 on the customer-scale and a 2.29 on the vendor-scale. These numbers are the averages of the ordering made by the interviewees. The customers gave R1: 2-1-2-1-1-9, making an average of 2.67. The vendors gave R1: 1-9-99-1-9-9, making an average of 6.71. These averages were subtracted from 9 to make the highest value the most important, instead of the highest value the lowest in priority. Customers: Vendors: 9 - 2.67 = 6.33 9 - 6.71 = 2.67

These figures show that the average customers finds R1 (Solution lacks solid availability) more important than the average vendor.

It must be stated that these results only cover the 13 parties that were interviewed, that the interviews were not setup per se to extract these orderings and that the results are not to be generalized over the entire field of cloud computing, it does show how customers risk perception is not met by how vendors rate these risks. The risks south of the blue line are found more important by customers, the risks north of the blue line are found more important by vendors. The number of risks below the blue line is striking.

Chapter 8. Perception of risks and precautions - 8.3 Interview results regarding customers vs. vendors Page 74 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

8.4

Summary of the interview results

The purpose of the interviews was to put the abstract risks and precautions into a real life business context. This business context was, as expected, quite different from the scientific basis of the risks and precautions and there were also some striking differences between customers and vendors. In this summary, an effort will be made to note the key takeaways from the interviews.

8.4.1 Scientific context versus real life situation


On the scientific context versus real life dimension, three interesting issues were found. Short-term focus of the business - Where a few risks indicate larger, long-term shifts in strategy of key players in the ecosystems on which the customer is dependent, these risks are not deemed very important by the interviewed customers. Customers are more interested in the possible urgency of a risk, while science tends to focus more on a risks importance. This focus on the more urgent risks is both supported by the qualitative results in section 8.2.1, as by the quantitative results in section 8.3. An attempt to visualize this difference is shown in Figure 17.

High Primary focus of science


Primary focus of business

Low -UrgenceLow High

Figure 17 - Visualization of possible explanation of differences between scientific context and real life.

Customers avoid precautions - What might sound as a contradictio in terminis, is that customers like to avoid precautions. When confronted with individual precautions that could be taken, customers prefer solutions that do not need these precautions at all. An example of this preference is found at P8 (Make use of an intermediary). Intermediaries could be a viable precaution to specific risks, but customers do not like intermediaries and prefer a solution that does not need intermediaries. Vendors are overconfident - Vendors tend to believe, or at least pretend to believe, that their solutions are infallible. This is unfortunately not the case, but this seems to be the basis for them to lower the impact of the risks found in this research and reject most of the precautions as unnecessary.

Chapter 8. Perception of risks and precautions - 8.4 Summary of the interview results Page 75 of 125

-Importance-

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

8.4.2 Customer versus Vendor


In the customer versus vendor dimension, these issues were found. Vendors are overconfident - The same issue plays an important part in both dimensions. Customers feel that vendors are underestimating the risks that apply to their services and do too little to prevent them. A good example is P2 (Make regular backups), vendors do not talk about backups. Their solution in perfect and will never fail or lose data. Customers are suspicious about this behavior and have a lack of trust in vendors. A solution to this issue might be that vendors integrate with a third party backup service, only the idea that it is possible to back up your cloud data might be enough to ease potential customers. Customers lack knowledge - A lot of customers have experiences with outsourcing that are not that positive. Because a lot of them feel that cloud computing is a step further, with a bigger player (who cares less about individual customers) and uncertain about where their data actually is, cloud computing is seen as a bridge too far. If customers had more knowledge about the actual working of cloud computing (or if vendors would be more open about how they manage their solution), they would be more willing to step in. Vendors have an internal focus - Vendors tell you that their software is infallible and that everything is looked after. Even if we assume this to be true, vendors often neglect the fact that their services are never used in isolation. Customers satisfaction is dependent on the combined workings of their services and other vendors services and vendors should be more aware of this. Especially keystones (the owners of platforms, p. 24) should be more concerned about the total experience their customers have.

The results of these interviews are not solid enough to adjust the CRISP method on, but they give a good insight in the environment the method is used in and could help the end-users of the CRISP method in putting things in perspective.

Chapter 8. Perception of risks and precautions - 8.4 Summary of the interview results Page 76 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 9

Conclusion and Discussion

This chapter contains five sections, the conclusion, results from the interviews, discussion, ideas for further research and a section on how the CRISP method can be used in everyday consulting practice.

9.1

Conclusion

The conclusion of this research consists of three parts. The first part will be a high level description about the general course of the research, the second part will provide the answers to the research questions and the third part contains the results from the interviews that were held with customers and vendors.

9.1.1 General conclusion


In this research, the Cloud solution Risk Identification and Selection of Precautions (CRISP) method is created that allows customers to identify risks early on in the process of selecting or adopting a cloud solution that is delivered by multiple parties. The CRISP method is based on scientific literature (Chapter 3) and fine-tuned with the help of experts in the field of cloud computing and outsourcing (Chapter 5). The method identifies eight different risks by systematically working through the solution and takes into account the applicability of risks as well as their impact on the business. Based on the risks found, the method suggests precautions that can be taken to mitigate the identified risks. To be able to visualize cloud computing solutions, the Cloud Solution Modeling Method is also developed. The CSMM produces a Cloud Solution Supply Network and is an extension of the Software Supply Network by Boucharas et al. (2009). More on the CSMM can be found in Chapter 4. The CRISP method, developed in Chapter 6, is validated by executing it on a project that Deloitte Consulting is currently carrying out at Canon (Chapter 7). The results from this execution, as well as the experiences during execution were discussed with people involved in the project. The validation led to the conclusion that the method is capable of identifying risks before cloud solutions are implemented and that the identified risks were relevant. However, not all of the proposed precautions were easy to implement in the situation at Canon. To further ground the developed method in everyday business, 13 interviews were held with (potential) customers and vendors of cloud computing solutions (Chapter 8). The interviews focused on the differences between the risks gathered in this research and the perception of the risks by the people who dealt with them on a daily basis. Results showed how science tends to emphasize importance while business is more preoccupied with the urgency of risks and that there is still a large difference in risks perception between customers and vendors.

9.1.2 Answers to the research questions


This section contains short answers to the research questions with references to the sections in the thesis where more can be read on the conclusion.

Chapter 9. Conclusion and Discussion - 9.1 Conclusion Page 77 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Main research question - How can a method be developed that identifies and assesses risks caused by service and company dependencies in cloud based solutions?

The CRISP method is developed based on three components. 1. A literature study in the fields of cloud computing and software ecosystems. 2. Expert interviews to put the results from this study in perspective. 3. The Cloud Solution Modeling Method, based on previous work by Boucharas et al (2009). The CRISP method is developed and extensively discussed in Chapter 9. The precise answer to this question is given by answering the sub questions in the next sections.

Sub question A - How can service and company dependencies be identified and modeled?

By expanding the Formal Software Ecosystem Modeling (FSEM) method by Boucharas et al. (2009), the Cloud Solution Modeling Method (CSMM) is created. The CSMM supports the creation of the cloud equivalent of the Software Supply Network (SSN), the Cloud Solution Supply Network (CSSN). A CSNN is a visual representation of the actors, products, services and ecosystems in a cloud solution. The Product Deployment Context (PDC) is not expanded to be usable with cloud solutions. This diagram has a purely technical focus and is not necessary to answer the research questions. To support the representation of cloud solutions, the following features were added to the SSN: Ecosystems, indication of whether an actor is a keystone, the Value Added Reseller, platforms, service flows and Composite flows (Chapter 4).

Sub question B - What kinds of risks exist in these dependencies?

The risks and precautions that can be present in compound cloud solutions are based on the literature studies that formed the theoretical background. In the beginning, a total of seven risks and seven precautions were identified in literature. By conducting eight expert interviews, the risks and precautions were further adjusted and finally eight risks and nine precautions were determined to be used as the basis for the further development of the method. The way the definitive risks (Table 22) relate to entities from the CSN, as well as how the precautions (Table 23) are able to counteract the risks can be found in section 0.

Risk Name R1 Solution lacks solid availability R2 Solution is insecure R3 Solution or ecosystem produces data lock-in R5 Ecosystem contains power imbalance R6 Entity lacks innovation R7 Keystone neglects ecosystem needs R8 Actor lacks size and stability R9 Solution has an unpredictable lifecycle Table 22 - Definitive risks in cloud solutions.

Chapter 9. Conclusion and Discussion - 9.1 Conclusion Page 78 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Precaution Name P1 Deploy escrow services P2 Make regular backups P3 Sign contracts on required service levels P4 Choose an ecosystem of significant size P5 Choose solutions that support open data formats P7 Choose Certified and Auditable partners P8 Make use of an intermediary P9 Choose solutions on one shared platform P10 Choose a mature and reputable company Table 23 - Definitive precautions to the risks in cloud solutions.

Sub question C - Can a method be developed that derives risk from these dependencies?

Each of the risks was provided with an extensive description of each of its characteristics (Appendix F. ). Metrics (indicators that show whether a risk is applicable or not) and Business impacts (to what extend a risk could have impact on a customers business) were the two characteristics that helped in the creation of the method. Using these characteristics, it is possible to identify applicable risks and to sort the risks that are applicable by how much impact they can have on the business. Based on how precautions can counteract certain risks, an advice can be formulated on how to counteract the most eminent risks in a cloud solution.

Sub question D - Are risks better identified with the method than without?

Whether the CRISP method adds value to the process of risk identification during the formation of a compound cloud solution, is possible by determining if more risks are found and/or if risks were identified in an earlier stage than without the use of the method. From discussions with experts, it was learned that the structured way of risk identification alone was already an improvement compared with the lack of risks identification method beforehand. The executions of the method at Canon confirmed these experts opinion. [Confidential parts removed]

9.2

Results from interviews

Although the interviews were not directly connected to the research questions, there are two results from the interviews with (potential) customers and vendors that are worth noting in the conclusion of this research. Scientific context versus real life situation - The most interesting difference between the risks based on literature and the way customers and vendors approach these risks, is the difference between importance and urgency. Science tends to emphasize importance while business is more preoccupied with urgency.

Customer versus Vendor - Based on the 13 interviews that have been conducted, it is impossible to make generic remarks on vendors and customer. What can be said, it that in
Chapter 9. Conclusion and Discussion - 9.2 Results from interviews Page 79 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

the group of interviewees, vendors are generally optimistic about problems and there is still a notable gap between the perception of risks by vendors and by customers.

9.3

Discussion and Limitations

This section handles the limitations to the research and to the method that is developed.

9.3.1 Limitations to the research


There are three limitations to the research that should be stated, these limitations concern the interviews, the CSMM and the validation of the method. Interviews - There are a few limitations to the results of the interviews with the customers and vendors. The interviews were set up to gain more insight in how risks around cloud computing were perceived by both parties. The interviews were not set up to indicate differences between customers and vendors or to be able to rank risks by importance. The results of the interviews confirmed these analyses to a certain extent, but the outcomes should be handled with restraint. CSMM - The Cloud Solution Modeling Method is developed to visualize cloud solutions. Although it appears that the CSMM is capable of depicting cloud solutions properly and it is based on a tested method, the FSEM method by Boucharas et al. (2009), the CSMM has not been validated yet. Its correctness and completeness have yet to be proved. Validation - The final validation of the CRISP method is performed by only one organization on only one project. This makes it impossible to generalize the results from this validation to all cloud solution projects.

9.3.2 Limitations to the CRISP method


There is no method that fully captures every detail of the subject it tries to analyze. The CRISP method is no exception and some aspects of the method have its limitations. Scope - The CRISP method is only capable of evaluating cloud solutions. As cloud solutions are rarely the only kind of solutions that an organization uses, there is always part of an organizations systems that the method cannot analyze. Section 6.4 discusses this subject indepth and section 7.1 shows how this focus on cloud works in practice. Risks - The risks in the CRISP method have two major limitations. The first comes from the way the method was developed, the second from the developing nature of the cloud computing field. 1. The risks are selected based on literature and were subsequently commented on by experts. As the basis of the risks lays in literature, some of the risks are rather abstract and these abstract risks are sometimes hard to apply in everyday situations. 2. The cloud computing field is fairly new and changes rapidly. This makes that risks that are relevant today, might be outdated tomorrow. The risks should be updated frequently to remain relevant. The requirements to the risks in section 5.1 show more (intentional) limitations to the risks.

Chapter 9. Conclusion and Discussion - 9.3 Discussion and Limitations Page 80 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

On-premise - Cloud solutions operate often in cooperation with on-premise applications. Identifying risks by only looking at the cloud part of the solution means neglecting risks in the on-premise part of the solution, as well as the risks in the entities that span across the cloud-on-premise border.

9.4

Further research

During this research, various interesting opportunities for further research came up. Some opportunities spring from the limitations of this research. Further research should be done on the creation of a single method that could identify risks in solutions that contain both cloud and onpremise systems. Having a single method that covers both types of systems reduces the risk of missing risks. The CRISP method has only been validated on one project. To gain more insight in how the method performs in different situations, the validation should be repeated on different projects. This also holds for the CSMM. The CSMM has so far only been used on two different solutions (Exact and Canon). More research is necessary to test this modeling method and perhaps improve it to suit different solutions. Ongoing research should be performed on the risks that can occur in cloud solutions. The cloud computing field is rapidly changing and the applicability of risks is likely to change over time. As the cloud solutions grow more mature, risks like availability and security might become less important, whereas data lock-in might become more important as more data enters the cloud. Interviews with customers and vendors and talks with experts were also a good source for further research opportunities. One of the interviewees (V3) suggested that to better support the assessment of the business impact of the risks, it might be possible to link the entities to different activities in Porters value chain (Porter, 2001). Supporting activities could indicate lower business impact, while a risk in a service that supports primary activities might instantly have a bigger impact. This might be combined with the division of the risks between urgent (primary activities) and less urgent (supporting activities) in nature. Several experts and vendors stated that CIOs often feel that moving to the cloud is not in their best interest. Cloud computing initially reduces the need for an internal IT department, because all the burdens of hosting and updating applications are covered by the hosting provider. This makes the CIO less important to the organization. Research should be done on whether cloud computing really makes IT departments (and their leaders) less important, or if it shifts responsibility from server maintenance to giving advice to the business and managing all the cloud services an organization needs.

Chapter 9. Conclusion and Discussion - 9.4 Further research Page 81 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

9.5

Application of the CRISP method in consulting practice

This section will describe how the CRISP method can help consultants in their daily work, when they should apply it and what the limitations of the method are. In short: The CRISP method is developed to identify risks and suggest precautions to these risks in cloud computing solutions that are delivered by multiple vendors. The most valuable results are obtained when the method is executed right after the solutions and the business processes that a solution will support are determine.

9.5.1 How the method works


The CRISP method contains three steps that are taken sequentially: Step 1 - In the first step, the cloud solution that is under evaluation is visualized using the Cloud Solution Modeling Method (Chapter 4) and the risks that could be applicable to the entities in the CSMM are listed. Step 2a - It is determined to what extent the risks that can be applicable actually are applicable (a service in general runs the risk of becoming unavailable, but whether that risk is applicable to a specific service needs to be determined). This is done by a consultant who knows the entities in the solution. Step 2b - The possible impact on the business of the risks is determined. This is done by (or with) someone who knows the business that the solution will be supporting. After the applicability and business impacts of the risks are determined, an ordering is made to select the risks with the highest possible impact on the business and the highest applicability. Step 3 - Precautions are selected that can counteract the risks that have the highest applicability and the highest possible impact on the business.

The following sections contain more in-depth information on how the method can be applied.

9.5.2 Users & intended audience


The CRISP method could be used by everyone who wants to evaluate a multi-party cloud solution. However, to use the method, at least some knowledge about cloud computing is required. The two parts of the method where cloud knowledge is a prerequisite are the determination of the applicability of the risks to the entities and the matching of the appropriate precautions to the identified risks. Therefore, the method is best used by consultants who have some experience in cloud computing. The results from the CRISP method are suitable for both consultants who work on the evaluated solution and their customers. Consultants will use the results from the CRISP method to propose suitable precautions to the risks that are involved in the solution that they are implementing for the customer. The precautions that the method offers are easy to understand by customers, but need the cloud computing knowledge of a consultant to be tailored to the customers specific needs.
Chapter 9. Conclusion and Discussion - 9.5 Application of the CRISP method in consulting practice Page 82 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

9.5.3 Timing of application


To execute the CRISP method, two pieces of information need to be clear. The first is the solution; the CRIPS Method is designed to evaluate a solution, so the scope of the solution needs to be clear. Second, it is necessary to know what the solution will be used for. The kind of business processes that are supported by the solution determine for a large part what the business impacts of specific risks will be. These two pieces of information are regularly available early on in the project that is set out to implement a solution. Right after the scope of the solution and the business processes that will be supported by this solution are determined, is the ideal moment to execute the method. The plans are still flexible and if precautions need to be taken, they can easily be added to the solution. When the CRISP method is executed after the implementation of the solution has already begun, there are a few problems that can arise: As the project is underway, costs that have been made might become superfluous, as the results from the CRISP method might alter the solution. Performing risk identification after approval has already been given by the project owners to implement a certain version of the solution might raise doubts about why a specific solution was chosen in the first place.

When the information is available, the execution of the method could be performed in about eight hours of work, dependent on the size of the solution of course.

9.5.4 Reusability of deliverables and how to keep the CRISP method relevant
Once it has been determined to what extent certain risks are applicable to an entity (for example a service), this information can be reused in the evaluation of other solutions. The business impact of the risk can be different, but the applicability remains the same as long as the entity remains the same. This speeds up the execution of the evaluation of solutions that also contain this entity. When a specific entity, that has highly applicable risks, is often used in places where it has high impact on the business when it fails, standard precautions could be included in solutions where that entity is used. To keep its value and remain relevant to the audience, the risks and precautions need to be reviewed regularly, new indicators to risks can be found and new ways to mitigate these risks will be developed. Also new risks might be added, as cloud computing becomes mature, old risks might not be applicable anymore and will possibly be replaced by new risks.

9.5.5 What the method cannot do


The CRISP method is developed to quickly evaluate cloud solutions. The results are therefore relatively high level and the precautions will need to be tailored to the specific needs of the customer. The method is also not able to indicate what costs certain risks will bring when actually occurring, nor will it tell you whether certain precautions are cost effective. Another thing that the method is not useful for is the comparison of solutions. The numbers that the CRISP method produces are only suitable for placing entities in order of importance and have not absolute meaning.
Chapter 9. Conclusion and Discussion - 9.5 Application of the CRISP method in consulting practice Page 83 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

9.5.6 Summary
Although the method has its limitations and is tested on only one project at only one organization, it is my profound belief that the method will contribute to the creation of well thought-out cloud computing solutions that incur less risks to the customers of the solution. The method may not cover every risks in cloud computing solutions, but gives IT professionals a guideline to examine the most eminent risks and helps in suggesting precautions to these risks.

Chapter 9. Conclusion and Discussion - 9.5 Application of the CRISP method in consulting practice Page 84 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 10

Works Cited

Abdat, N., Spruit, M., & Bos, M. (2010). Software as a Service and the Pricing Strategy for Vendors. In T. J. Strader, Digital Product Management, Technology and Practice: Interdisciplinary Perspectives (pp. 154-192). Hershey, New York, USA: Business science reference. Adner, R. (2006). Match Your Innovation Strategy to Your Innovation Ecosystem. Harvard Business Review, 1-10. Anderson, M. (2010, December 28). The value of trends watching... from all sides. Retrieved January 11, 2011, from Elm Streets Trends Blog: http://elmstreettrends.blogspot.com/2010/12/value-of-trend-watching-from-all-sides.html Antonsen, E. (2009, March 3). Data Standards for Web Applications. Retrieved January 11, 2011, from CloudAve: http://www.cloudave.com/2340/data-standards-for-web-applications/ Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., et al. (2010, April). A View of Cloud Computing. Communications of the ACM, 53(4), 50-58. Betcher, J. T. (2010). Cloud computing: Key IT-Related Risks and Mitigation Strategies for Consideration by IT Security Practitioners. Eugene: University of Oregon - Applied Information Managmeent Program. Bhardwaj, S., Jain, L., & Jain, S. (2010). Cloud Computing: a study of infrastructure as a service (IaaS). International Journal of Engineering and Information Technology, 60-63. Bisong, A., & Rahman, S. (. (2011, January). An overview of the Security Concerns in Enterprise Cloud Computing. International Journal of Network Security & Its Applications, 3(1), 31-45. Bodle, I. (2010, September 10). SaaS Agreements - Data Protection - Liability for Loss of Backup Tapes. Retrieved January 11, 2011, from Bodle Law - SaaS and the legal cloud made simple: http://www.bodlelaw.com/saas/saas-agreements-data-protection-%E2%80%93-liability-forloss-of-backup-tapes Bosch, J. (2009). From Software Product Linesto Software Ecosystems. 13th International Software Product Line Conference (SPLC 2009), (pp. 1-10). San Francisco, CA. Boucharas, V., Jansen, S., & Brinkkemper, S. (2009). Formalizing Software Ecosystem Modeling. IWOCE'09 (pp. 1-10). Amsterdam: AMC. Buyya, R., Yeo, C. S., & Venugopal, S. (2008). Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities. 10th IEEE International Conference on High Performance Computing and Communications, 2008 (pp. 5-13). IEEE Computing. Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emering IT platforms: vision, hype and reality for delivering computing as the 5th utility. Future Generations Computer Systems, 1-18.

Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 85 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chappel, D. (2010). Introducing the Windows Azure Platform. David Chappel & Associates Sponsored by Microsoft. CIO. (2010, Oktober 11). Virgin system crash cost up to $20m. Retrieved January 06, 2010, from CIO: http://www.cio.com.au/article/363843/virgin_system_crash_cost_up_20m/ Cloud Security Alliance. (2010). Top Threats to Cloud Computing V1.0. Cloud Security Alliance. Denson, W. D. (2002). The Source Code Escrow: A Worthwhile or Worthless Investment? Bankruptcy Law Journal, 1-16. Dillon, T., Wu, C., & Chang, E. (2010). Cloud computing: Issues and Challenges. 24th IEEE International Conference on Advanced Information Networking and Applications, (pp. 27-33). Perth. Dubey, A., & Wagle, D. (2007, May). Delivering software as a service. The McKinsey Quarterly - Web exclusive, 1-12. Durkee, D. (2010). Why Cloud Computing Will Never Be Free. ACM Distributed Computing, 1-10. Eisenmann, T. R., Parker, G., & Van Alstyne, M. (2008). Opening Platform: How, When and Why? Harvard Business School, 1-27. Emmerik, M. v. (2010, July 23). Strategien voor SaaS-backup. Retrieved 01 11, 2011, from Computable: http://www.computable.nl/artikel/ict_topics/cloud_computing/3442010/2333364/strategie n-voor-saasbackup.html Exact Salesdesk. (2010, December 17). Telephone call. (B. Verhoeven, Interviewer) Fonseca, B. (2008). SaaS Benefits Starting to Outweigh Risks. Computerworld. Google. (2010, 12). Run your web apps on Google's infrastructure. Retrieved 12 07, 2010, from Google App Engine: http://code.google.com/intl/nl-NL/appengine/ Hachman, M. (2010, Feburary 18). Google buys ReMail, shuts down app. Retrieved February 04, 2011, from PCMag.com: http://www.pcmag.com/article2/0,2817,2359993,00.asp Hagel III, J., Seely Brown, J., & Davidson, L. (2008). Shaping Strategy in a World of Constant Disprution. Harvard Business Review, 1-10. Hagel, J., & Seely Brown, J. (2008, July 23). How SAP Seeds Innovation. Retrieved November 12, 2010, from Bloomberg Businessweek: http://www.businessweek.com/innovate/content/jul2008/id20080723_353753.htm Hagel, J., Seely Brown, J., Arnowitz, B., Asnaani, J., Gillingham, I., & Sharma, S. (2010). Cloud Computing - Storms on the Horizon. Deloitte, Center for the Edge. Deloitte Center for the Edge.

Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 86 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Hartig, E. d., Tol, M., & Visscher, W. (2006). The Health Measurement of a Business Ecosystem. ECCON 2006 Annual Meeting - Organisations as Charordic Panarchies, (pp. 1-39). Heiser, J. (2010). Analyzing the Risk Dimensions of Cloud and SaaS Computing. Gartner. Heiser, J. (2010). Will Your Data Rain When the Cloud Bursts. Gartner Inc. Herbert, L., Ferrusi Ross, C., Rose, E., & Karcher, P. (2010). SaaS Valuations Criteria. Cambridge, MA: Forrester Research Inc. Herbert, L., Speyer, M., Gaynor, E., & Schule, I. (2006). SaaS Gathers Steam In Large Enterprises. Cambridge, MA: Forrester Research Inc. Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004, March). Design Science in Information Systems Research. MIS Quarterly, 28(1), 75-105. Hiles, A. N. (1994). Service Level Agreements - Panacea of Pain. The TQM Magazine, 14-16. Hofmann, P., & Woods, D. (2010). Cloud Computing: The Limits of Public Clouds for Business Applications. IEEE Internet Computing, 90-93. Huizing, M. (2008, Oktober 02). Vier regels om vendor lockin te vermijden. Retrieved January 06, 2011, from Computable: http://www.computable.nl/artikel/ict_topics/cloud_computing/2726361/2333364/vierregels-om-vendor-lockin-te-vermijden.html Hwang, K., Kulkarni, S., & Hu, Y. (2009). Cloud Security with Virtualized Defence and Reputationbased Trust Mangement. Eighth IEEE International Conference on Dependable, Autonomic and Secure Computing (pp. 717-722). Chengdu, China: IEEE Computer Society. Iansiti, M., & Levien, R. (2004). The Keystone Advantage - What the New Dynamics of Business Ecosystems Mean for Strategy, Innovation and Sustainability. Boston, Massachusetts: Harvard Business School Press. Iansiti, M., & Levien, R. (2004). The Keystone Advantage: Waht the New Dynamics of Buisness Ecosystems Mean for Strategy. Harvard Business School Press. Iansiti, M., & Richards, G. L. (2006, spring). The information technology ecosystem: Structure, health, and performance. The Antitrust Bullitin, 51(1), 77-110. Ingram, M. (2010, September 7). Value of Twitter Ecosystem Shrinks as Funding Falls. Retrieved December 23, 2010, from Gigaom: http://gigaom.com/2010/09/07/value-of-twitterecosystem-shrinks-as-funding-falls/ Ingthorsson, O. (2010, September 07). Cloud computing APIs who will be the winner? Retrieved January 06, 2011, from Cloud Computing Topics: http://cloudcomputingtopics.com/2010/09/cloud-computing-apis-who-will-be-the-winner/ Jansen, S., Brinkkemper, S., & Finkelstein, A. (2010). Business Network Management as a Survival Strategy: A Tale of Two Software Ecosystems. NTB.
Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 87 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Jansen, S., Finkelstein, A., & Brinkkemper, S. (2009). A Sense of Community: A Research Agenda for Software Ecosystems. 31st International Conference on Software Engineering, New and Emerging Research Track. Vancouver. Kane, Y. I., & Fowler, G. A. (2010, October 07). Amazon Amps Up Apps Rivalry. Retrieved January 07, 2011, from Wall Street Journal Online: http://online.wsj.com/article/SB10001424052748704696304575538273116222304.html Kaplan, J. M. (2007). SaaS: Friend Or Foe? Business Communications Review, 48-53. Kindness, A., Whiteley, R., & Hayes, N. M. (2010). The Data Center Network Evolution: Five Reasons This ISn't Your Dad's Network. Camebridge, USA: Forrester Research, Inc. Kontogiannis, K., Lewes, G. A., Smith, D. B., Litoiu, M., Mller, H., Schuster, S., et al. (2007). The Landscape of Service-Oriented Systems: A Research Persepective. International Workshop on Systems Development in SOA Environments (SDSOA'07) (pp. 1-6). IEEE Computer Society. Krs, M. (2010, December 15). Telephone call. (B. Verhoeven, Interviewer) Krigsman, M. (2010, September 27). Cloud-based IT failure halts Virgin flights. Retrieved January 06, 2011, from ZDnet: http://www.zdnet.com/blog/projectfailures/cloud-based-it-failure-haltsvirgin-flights/11061?tag=mantle_skin;content Lawton, G. (2008, june 24). Developing Software Online with Platform-as-a-Service Technology. Computer, 13-15. Layar. (2010, December). Download. Retrieved December 21, 2010, from Augmented Reality Browser: Layar: http://site.layar.com/download/ Levinson, M. (2007, may 15). Software as a Service (SaaS) Definitions and Solutions. Retrieved 11 09, 2010, from CIO.com: http://www.cio.com/article/print/109704 Lindner, M., Galan, F., Chapman, C., Clayman, S., Henriksson, D., & Elmroth, E. (2010). The cloud supply chain: A framework for information, monitoring, accounting and billing. 2nd International ICST Conference on Cloud Computing (CloudComp 2010) (pp. 1-22). Barcelona, Spain : University College London. Linthicum, D. (2009, September 15). Infoworld. Retrieved January 07, 2011, from Truth about lock-in and cloud computing: http://www.infoworld.com/d/cloud-computing/truth-about-lock-inand-cloud-computing-487 Marks, G. (2008, july 24). Beware the Hype for Software as a Service. Retrieved 11 09, 2010, from Bloomberg Businessweek: http://www.businessweek.com/technology/content/jul2008/tc20080723_506811.htm Mell, P., & Grance, T. (2010, 07 09). National Institutes of Standards and Technology. Retrieved 12 09, 2010, from NIST Cloud Computing Program: http://www.nist.gov/itl/cloud/upload/cloud-def-v15.pdf
Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 88 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Mietzner, R., Leymann, F., & Papazoglou, M. P. (2008). Defining Composite Configurable SaaS Application Packages Using SCA, Variability Descriptors and Multi-Tenancey Patterns. The Third International Conference on Internet and Web Applications and Services (pp. 156-161). Athens, Greece: IEEE Computer Society. Miller, R. (2009, July 27). Cloud Brokers: The Next Big Opportunity? Retrieved April 15, 2011, from Datacenter Knowledge: http://www.datacenterknowledge.com/archives/2009/07/27/cloudbrokers-the-next-big-opportunity/ Moore, J. F. (1993). Predators and Prey: A New Ecology of Competition. Harvard Business Review, 75-86. Mudumbai, V. (2010, 11 22). How I Learned to stop worrying and love EC2/Linode. Retrieved 12 17, 2010, from Artha42 Innovations Private Limited (BLOG): http://www.artha42.com/blog/how_i_learned_to_stop_worrying_and_love_iaas Musil, S. (2008, December 09). Cnet news. Retrieved January 06, 2011, from Sun looks for new direction for Network.com: http://news.cnet.com/8301-1001_3-10119969-92.html Niesen, A., & Slaats, T. (2010). Cloud Computing, de grote ontwrichter. Rotterdam: Deloitte. Olsen, L. (2009). Ten things to ask your SaaS vendor before entering the cloud. Bozeman, Montana: RightNow Technologies. Open Cloud Manifesto. (2009, Spring). Open Cloud Manifesto. Retrieved January 11, 2011, from Open Cloud Manifesto: http://opencloudmanifesto.org/Open%20Cloud%20Manifesto.pdf Parr, B. (2010, April 09). Twitter Acquires Tweetie. Retrieved January 05, 2011, from Mashable Business: http://mashable.com/2010/04/09/breaking-twitter-acquires-tweetie-iphone-app/ Penn, J. (2010). Security and the Cloud. Vendor Strategy Professionals. Forrester Research Inc. Perry, R., Hatcher, E., Mahowald, R. P., & Hendrick, S. D. (2009). Force.com Cloud Platform Drives Huge Time to Market and Cost Savings. Framingham: IDC - Sponsored by Salesforce.com. Pezzini, M., Driver, M., Feinberg, D., Natis, Y. V., Perkings, E., & Scott, D. (2010). Oracle Software Strategy After Sun: 'Software as Hardware'. Gartner Inc. Porter, M. E. (2001). Strategy and the Internet. Harvard Business Review, 1-20. Pring, B. (2010). Cloud Computing: The Next Generation of Outsourcing. Gartner. Gartner. SalesForce.com. (2008). The Force.com Multitenant Architecture. San Francisco: SalesForce.com. Spiotto, A. H., & Spiotto, J. E. (2003). The Ultimate Donwside of Outsourcing: Bankruptcy of the Service Provider. American Bankruptcy Institute Law Review, 47-92. Subramanian, K. (2008, November 19). SaaS Risk Reduction Open Formats. Retrieved January 10, 2011, from CloudAve: http://www.cloudave.com/2694/saas-risk-reduction-open-formats/
Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 89 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Takeda, H., Veerkamp, P., Tomiyama, T., & Yoshikawa, H. (1990). Modeling Design Processes. AI Magazine, 11(4), 37-48. ten Wolde, R. (2010, October 06). Reactie Yunoo op lek kasboek.nl. Retrieved January 07, 2011, from Yunoo.nl: http://www.yunoo.nl/blog/2010/10/reactie-yunoo-op-lek-kasboek-nl/ Van de Weerd, I., & Brinkkemper, S. (2008). Meta-Modeling for Situational Analysis and Design Methods. In M. Rahman Syed, & S. Nessa Syed, Handbook of Research on Modern Systems Analysis and Design Technologies and Applications (pp. 35-54). Hershey: Information Science Reference (an imprint of IGI Global). Van de Zande, T., & Jansen, S. (2010). Business Continuity with SaaS. weet ik niet, 1-19. Van der Beek, P. (2010, 09 02). Exact Online verwisselt KPN voor Rackspace - Cloud Computing / Nieuws. Retrieved 12 02, 2010, from Computable: http://www.computable.nl/artikel/ict_topics/cloud_computing/3493430/2333364/exactonline-verwisselt-kpn-voor-rackspace.html Van der Pluijm, U. (2010, October 14). Vendor lock-in grootste probleem cloudhosting. Retrieved January 07, 2011, from Webwereld: http://webwereld.nl/nieuws/67473/vendor-lock-ingrootste-probleem-cloudhosting.html Vaquero, L. M., Rodero-Merino, L., Caveres, J., & Lindner, M. (2009, January). A Break in the Clouds: Towards a Cloud Definition. ACM SIGCOMM Computer Communications Review, 39(1), 5055. Wei, Y., & Brian Blake, M. (2010). Service-Oriented Computing and Cloud Computing - Challenges and Opportunities. IEEE Internet Computing - Web-Scale Workflow, 72-75. Weinberger, M. (2010, August 2). SaaS: How To Back Up Google Apps. Retrieved January 11, 2011, from MSPmentor: http://www.mspmentor.net/2010/08/02/saas-how-to-back-up-googleapps/ Weissman, D. C., & Bobrowski, S. (2009). The Design of the Force.com Multitenant Internet Application Development Platform. The 35th SIGMOD international conference on Management of data (pp. 889-896). Providence: ACM New York, NY. Wikipedia. (2011, February 21). Dodo - Raphus cucullatus. Retrieved March 17, 2011, from Wikipedia: http://nl.wikipedia.org/wiki/Dodo Wokke, A. (2010, October 6). Kasboek.nl publiceert door foutje miljoen transacties van klanten. Retrieved January 07, 2011, from Tweakers.net: http://tweakers.net/nieuws/70068/kasboek-punt-nl-publiceert-door-foutje-miljoentransacties-van-klanten.html Wortham, J. (2010, february 22). Apple Bans Some Apps for Sex-Tinged Content. Retrieved January 07, 2011, from New York Times - Technology: http://www.nytimes.com/2010/02/23/technology/23apps.html
Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 90 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Yildiz, M., Abawajy, J., Ercan, T., & Bernoth, A. (2009). A Layered Security Approach for Cloud Computing Infrastructure. 10th International Symposium on Pervasive Systems, Algorithms, and Networks (pp. 763-767). Kaohsiung, Taiwan: IEEE Computer Society. Zeller, M., Grossman, R., Lingenfelder, C., Berthold, M. R., Marcad, E., Pechter, R., et al. (2009). Open Standards and Cloud Computing KDD-2009 Panel Report. KDD'09 (pp. 11-18). Paris, France: ACM.

Chapter 10. Works Cited - 9.5 Application of the CRISP method in consulting practice Page 91 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 11

Special thanks

This thesis could not have been written in such a short time without the help of a lot of people: At Utrecht University: Slinger Jansen, Inge van de Weerd. At Deloitte Consulting: Loan Nguyen, Niels Nuyens. Experts at Deloitte: Jacco Albers, Harald van Doorn, Victor Hoong, Dave Klingens, Melissa Raczak, Theo Slaats, Martin van Twisk, Joost Visse, Stephen Ward. External heads of IT-departments (including customer interviewees): Nard van Breemen (Coro), Martien van Deijzen (Huis van de Sport), Maurice Elema (Canon), Jan-Dennis de Graaff (Ziekenhuis Rivierenland Tiel), Taco Jansen (Artsen zonder grenzen), Andr Roling (Rotterdamse Vereniging voor Katholiek Onderwijs), Jeroen Winkel (de Brauw Blackstone Westbroek). Cloud computing supplier representatives (including vendor interviewees): Hans Brouwer (Salesforce), Bas Dijkhuizen and Bart Bossers (Inisi), Jochem de Graaf (Mendix), Herman Kui (Escrow4all), Michel Kis (Softcrow), Vincent van Nimwegen (42 Windmills), Alex Shootman (Eloqua), Bram Veenhof (Microsoft), Roel Veldhuizen (Paston). Not directly connected to my research but still of high value to my thesis: Jacq Cornelissen, Peter Dekkers, Douwe Egberts, Michel Farkas, Joost te Molder, Reinier Verhoeven, Floris Vlasveld, Martje de Vries Lentsch.

Appendix A. CSSN of Exact online (example to CSSN modeling) Page 92 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Chapter 12

Appendices

Appendix A. CSSN of Exact online (example to CSSN modeling) ....................................... 94 Appendix B. Risks based on literature. ............................................................................ 95 Appendix C. Precautions based on literature .................................................................. 99 Appendix D. Interview protocol for the first round of interviews ................................... 103 Appendix E. Raw expert interview results .................................................................... 105

Appendix F. Risks based on literature and expert interviews ........................................ 106


Appendix G. Precautions based on literature and expert interviews .............................. 111 Appendix H. Deliverables: entity collections for the CSSN ............................................. 115 Appendix I. Deliverables: CSSN of Exact online ............................................................ 116 Appendix J. Deliverable: ENTITY-RISK MAPPING .......................................................... 117 Appendix K. Deliverable: ENTITY-RISK MAPPING OF APPLYING RISKS............................ 118 Appendix L. Deliverable: Weighing BUSINESS IMPACTS ................................................ 119 Appendix M. Deliverable: Ordered List of BUSINESS IMPACTS.................................... 120

Appendix N. Deliverable: Precautions against risks ....................................................... 122 Appendix O. Validation: Complete LIST OF ENTITIES from CSSN ..................................... 123 Appendix P. Interview questions for validation of CRISP method .................................. 124 Appendix Q. Quantitative results from interviews with customers and vendors ............ 125

Appendix A. CSSN of Exact online (example to CSSN modeling) Page 93 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix A.

CSSN of Exact online (example to CSSN modeling)

E.2

AeroPlus
E.1

Microsoft

P.3 .3

PL.2

P.2 .4

D.1

3 S.

6 .

Supplier X

P.4 .7

Exact Software

.1 P.4
P.1

Rackspace
PL.1

S.4
S.1

Customer

Reseller

ID A1 A2 A3 A4 A5 A6 A7 P1 P2 P3 P4 D1 S1 S2 S3 S4 PL1 PL2 1 2 3 4 5 6 7 E1 E2 Table 24 - List of entities in CSSN.

Name/description AeroPlus Supplier X Exact Software Rackspace Microsoft Reseller Customer Exact Online Windows Server SQL Server Add-on for P1 Dataflow between S3 and S1 Exact Online Support AeroPlus Flight Planning P4 as a service Rackspace Hosting Platform AeroPlus platform Fee for hosting P1 Fee for S1 Fee for P3 Fee for P2 Fee for S1 and S2 Fee for S3 Fee for P4 Exact Microsoft

Appendix A. CSSN of Exact online (example to CSSN modeling) Page 94 of 125

S . .2 5

2 .

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix B.

Risks based on literature.

This section contains a structured description for each of the seven risks. These risks were based on literature only and were the input for the expert interviews. Each table contains the following fields: Risk - The ID of the risk for future reference in this research. Name - Name of the risk. Applicable to - The entity type this risk is related to. Type - The type of risk, business, legal or both. Description - General description of the risk. Metrics - Ways to measure if the risk is present in a certain situation. Impact on business - The impact a risk can have on a business that is reliant on a service. Example - A short example of the risk in a real-life situation. Sources - Sources that underpin the risk as described.
Risk Name Applicable to Type Description R1 Solution lacks solid availability Service Business and legal Services can become unavailable through a number of different reasons. The risk that is involved with a service becoming unavailable depends on how important this service is to the business.

If a HRM system goes off-line for a day, the HR-department might be hindered for that particular day, but there is probably little impact. However, if the central booking system of an airline fails, the consequences might be huge, both in direct costs, but also in indirect costs like damage to the Airlines image (see example). Metrics The availability of a service is usually measured in a percentage of uptime. 99.9% availability means that a service is unavailable for about eight hours per year. A services track record gives an idea of how the service has performed in the past, this is of course no guarantee for the future. Impact on The impact of services being unavailable is of course dependent on what the service is used for. The business more important a service is to the core business, the higher the impact when it becomes unavailable. Legal issues may arise when third parties are dependent on a product or service that the uses the service that became unavailable. The impact on business may however be even greater when a supplier goes out of business. Example A catastrophic systems failure at cloud-based software provider, Navitaire, *+, disrupted travel for 50,000 customers of Virgin Blue airlines in Australia (Krigsman, 2010). Due to a system failure at the service provider that took care of the reservations and check-ins of Virgin Blue airlines, the airlines business was disrupted for about eight hours, costing them between $15 and $20 million in lost revenue. (CIO, 2010) Sources (Armbrust, et al., 2010), (Dubey & Wagle, 2007), (Durkee, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Kaplan, 2007), (Kontogiannis, et al., 2007), (Olsen, 2009), (Wei & Brian Blake, 2010), (Van de Zande & Jansen, 2010) Table 25 - V.1: Detailed description of the risk Solution lacks solid availability. Risk Name Applicable to Type Description R2 Solution is insecure Service Business and legal Security threats come in all sorts. A data leak or unauthorized access to a system could be the result of a badly designed system, two services that do not collaborate properly or an employee that is dissatisfied. This makes security probably one of the hardest parts of a service to control. Security is hard to measure, one could use the services track record, but that there never was a security breach, could also mean that the security-holes are not yet identified. There are some security certificates that show that a system has passed a certain security audit, like SAS 70.

Metrics

Appendix B. Risks based on literature. Page 95 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Impact on business

The impact of a security breach on a business can be huge, for example when confidential company or customer data has leaked to competitors, or when hackers have managed to take down or deface the companys website. Legal issues arise when the data that has been lost also puts clients at a disadvantage. Example Kasboek.nl is a Dutch SaaS that allows users to keep track of their personal finances. In October of 2010, more than a million of the uploaded transactions were compromised through a leak in their system. These kinds of leaks to not only hurt the supplier of that specific solution (Wokke, 2010). It also hits other SaaS vendors that deal with sensitive information (ten Wolde, 2010). Sources (Armbrust, et al., 2010), (Cloud Security Alliance, 2010), (Dillon, Wu, & Chang, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Hwang, Kulkarni, & Hu, 2009), (Levinson, 2007), (Penn, 2010), (Yildiz, Abawajy, Ercan, & Bernoth, 2009) Table 26 - V.1: Detailed description of the risk Solution is insecure. Risk Name Applicable to Type Description R3 Solution or ecosystem produces lock-in Service, ecosystem, keystone, niche player Business Being locked-in is the situation when leaving your current supplier or ecosystem is either too expensive, or impossible because of a lack of a suitable alternative. Too expensive could also mean that retraining thousands of employees to work with a new system does not outweigh the benefits of that new system. Metrics A lock-in situation can be identified by the transfer costs involved in switching suppliers. Impact on The impact on the business of being locked-in is not always apparent in the short term. In the long run business however, being locked-in to one vendor or platform could seriously hinder the business, especially when alternative platforms offer solutions that better suit the business needs. Example A good example of lock-in is the Windows OS. Ubuntu offers a comparable OS for a fraction of the cost, but retraining employees and finding alternative applications makes it too cumbersome and expensive to switch. Sources (Armbrust, et al., 2010), (Bosch, 2009), (Huizing, 2008), (Kindness, Whiteley, & Hayes, 2010), (Linthicum, 2009), (Mietzner, Leymann, & Papazoglou, 2008), (Mudumbai, 2010), (Van der Pluijm, 2010), (Pezzini, Driver, Feinberg, Natis, Perkings, & Scott, 2010) Table 27 - V.1: Detailed description of the risk Solution or ecosystem produces lock-in. Risk Name Applicable to Type Description R4 Entity becomes unpopular Service, ecosystem, keystone, niche player Business Services, companies or ecosystems which become less popular, for example because there is a better alternative available, pose a risk to the customer. As services become less popular, suppliers might try different things to make the service more attractive or keep revenue levels steady. Lowering prices (and thus lowering service), limiting communications (to try to lock-in current users) or even abandoning the service completely are all options that could negatively impact the end user. Metrics Market size and size, number and type of customers are metrics that could help determine the popularity of a platform or service relative to alternatives. Impact on Whatever the supplier is going to do (increase lock-in, lower prices or sell the service to another business organization), a service or ecosystem that becomes less popular poses a threat to anyone who is reliant on it for its core business. The impact is also determined by the degree of dependence that a player has on an ecosystem. Example In the course of 2008, Suns Network.com cloud computing platform was abandoned by developers, among others because Developers did not have access to APIs for system configuration and controlling *+ (Ingthorsson, 2010). The consequence was that Sun stopped the development and stopped accepting new customers. Current customers were still being served, but new innovations were not implemented and the future of the service became uncertain (Musil, 2008). Sources (Adner, 2006), (Bosch, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel & Seely Brown, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Ingthorsson, 2010), (Moore, 1993), (Musil, 2008) Table 28 - V.1: Detailed description of the risk Service or ecosystem becomes less popular Risk Name R5 Ecosystem contains power imbalance

Appendix B. Risks based on literature. Page 96 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Applicable to Type Description

Service, keystone, niche player Business Having influence on your supplier means that you could persuade him to change designs or move release dates to suit your situation better. Also the supplier could discuss new features with you. It might seem as if keystones are always able to influence the ways in which the ecosystem is evolving. However, careless keystones who try to impose their will on an ecosystem could end up making their ecosystem less attractive, regardless of what the change they advocate implies. Metrics The amount of influence an customer has on a supplier depends primarily on how big the customer is relative to other customers, and how much influence the customer has on other customers. Impact on Having no influence on which way your supplier is going with the service you are using, has no business immediate impact on your business. That is for as long as you do not need anything other than your supplier delivers. Impact on your business could be far-reaching when suppliers change their services. Example Developers who develop for Apples iOS ecosystem generally have low influence on what the course of the ecosystem will be. In February of 2010, Apple banned all apps that feature sexually suggestive material, including photos of women in bikinis and lingerie, a move that came as an abrupt surprise to developers who had been profiting from such programs. (Wortham, 2010) Sources (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Jansen, Finkelstein, & Brinkkemper, 2009) Table 29 - V.1: Detailed description of the risk Little or no influence on third parties. Risk Name Applicable to Type Description R6 Entity lacks innovation Service, ecosystem, keystone, niche player Business Cloud computing is a relatively new field with new innovations coming up about every month. When a company stops preparing for the future, by neglecting these innovations, this is the beginning of the end. Being future proof is technology-wise and business-wise. Adopting new technologies and keeping services up to standard with new developments is one of the important factors to the attractiveness of these services. From a business perspective, being future proof means that the service is able to keep its current developers and customers, as well as it is able to attract new ones. Whether a service is able to do this depends mostly on the strategy of the vendor and the ecosystem it is in. The way in which the keystone follows a strategy that allows niche players to join their ecosystem, explore new niches and make the ecosystem more valuable for new groups of customers, is important to the innovative strength of the ecosystem. Metrics Measuring if a company lacks innovation is only possible by comparing the companys rate of innovation to competitors. Impact on When a service is not able to support new developments it will be less appealing to potential business customers and will be less useful to existing customers willing to expand. Impacts could be the same as with R4 Example The fall of the American automobile industry during the recent financial crises is a good example of how a continual lack of innovation can disrupt huge companies at the blink of an eye. Cars that did more miles to the gallon became more interesting and the petrol guzzlers that GM was making and that meant the end of success. Sources (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel & Seely Brown, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Iansiti & Levien, 2004), (Iansiti & Richards, 2006) Table 30 - V.1: Detailed description of the risk Entity lacks innovation. Risk Name Applicable to Type Description R7 Keystone neglects ecosystem needs Keystone, niche player Business The dedication a keystone or a niche player shows towards an ecosystem is the degree in which the player is dedicated to improve the ecosystem. It is important to note that being dedicated to an ecosystem is not the same as following the Disciple-strategy. Dedication can be seen as a voluntary effort, which does not benefit the player, but benefits the ecosystems as a whole. Customers of a players that are not dedicated to their ecosystem, are not directly at risk. However, non-dedicatedbehavior could point towards a negative attitude towards the ecosystem of might show that the ecosystems itself is not in such a good shape. When a keystone is showing non-dedicated-behavior, this could indicate that the keystone is moving to another ecosystem, or that the keystone is moving to a Dominator-strategy

Appendix B. Risks based on literature. Page 97 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Metrics

Dedication towards an ecosystem shows from indicators that are hard to measure precisely. Examples of questions that determine dedication are: Is the player actively involved in ecosystem discussion groups? Has the keystone made a useful knowledge base available to the niche players? Is the niche player actively communicating about new found solutions to ecosystem problems?

Impact on business

The impact on business of a partner that lacks dedication to the ecosystem should not be exaggerated; however, it is important to find out why an ecosystem inhabitant is not supporting the ecosystem it benefits from. Example As Google put less energy in the Android market, leaving it with some loose ends and a less-thanperfect user experience, Amazon saw an opening and announced that they might start their own market. Although Amazon might build a better market, which of course benefits customers and developers, different markets that might have different offerings also cloud the ecosystem and make it less user friendly (Kane & Fowler, 2010). Sources (Bosch, 2009), (Hagel & Seely Brown, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Jansen, Brinkkemper, & Finkelstein, 2010) Table 31 - V.1: Detailed description of the risk Lack of dedication to the ecosystem.

Appendix B. Risks based on literature. Page 98 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix C.

Precautions based on literature

These precautions were based on literature only and were the input for the expert interviews. Each table contains the following fields: Precaution - The ID of the precaution for future reference in this research. Name - Name of the precaution. Counteracts - Which risks could be counteracted with this precaution in place. Type - The type of precaution, business, legal or both. Description - General description of the precaution. Example - A short example of the precaution in a real-life situation. Explanation of counteracting - A short description of how this precaution counteracts a risk that was mentioned under Counteracts. Sources - Sources that underpin the precaution as described.
Precaution Name Counteracts Type Description P1 Deploy escrow services R1 (Solution lacks solid availability), R3 (Solution or ecosystem produces lock-in), R5 (Ecosystem contains power imbalance) Business and legal A source-code escrow arrangement is an arrangement between a software vendor, the buyer and a trusted third party. (Van de Zande & Jansen, 2010). The trusted third party (also: escrow agent) places the source code of the software vendor in a secure holding place. The escrow arrangement contains the condition under which the source code will be released to the buyer. Bankruptcy of the vendor is an example of a condition that could trigger the release of the source code. For onpremise software this is a good protection for the buyer, against possible bankruptcy of the vendor. With the source code, the buyer will be able to (have other developers) update his systems and facilitate transition to another software solution. Cloud customers on the other hand, have not only their software, but also their data in hands of the service supplier. Escrow solutions in their situation need not only cover the source code of the software, but should also cover the (daily) backups of their data. Exact Software has such an agreement for their Exact Online financial accounting SaaS. This makes sure that in the event of Exact Software going bankrupt, the service as well as the data will remain available for at least one month (Exact Salesdesk, 2010). Traditional escrow agents are also making the move towards escrow for SaaS. Amsterdam based escrow agent Softcrow is developing an escrow product that not only contains the source code and daily backups, but also the complete virtual machine images of the service. In case of an emergency, the entire service can be revived from the contents of the escrow. R1 - Escrow services cannot prevent short disruptions in services, but as a service provider goes out of business, the escrow could help recover the service as fast as possible. Escrow terms could be negotiated to make this recovery time as short as possible. R3 - Having a daily data back-up to an escrow-provider could indicate that your service supplier is willing to participate in data exports. Although escrow is not a direct way to counteract lock-in, it could be an indication of how free your data is. R5 - When you have little or no influence on your service provider(s), having an escrow in place makes you a little less vulnerable to bad business decisions those parties can make. Sources (Exact Salesdesk, 2010), (Krs, 2010), (Denson, 2002), (Herbert, Ferrusi Ross, Rose, & Karcher, 2010), (Spiotto & Spiotto, 2003), (Van de Zande & Jansen, 2010) Table 32 - V.1: Detailed description of the precaution "Deploy escrow services". Precaution Name P2 Make regular backups

Example

Explanation of counteracting

Appendix C. Precautions based on literature Page 99 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Counteracts Type Description

Example Explanation of counteracting

R1 (Solution lacks solid availability), R2 (Solution is insecure), R3 (Solution or ecosystem produces lock-in) Business That whenever your data is in the cloud, backups are unnecessary and superfluous, is a myth (Marks, 2008). Customers always run the risks of losing their data, whether it be through a fire in the service providers datacenter, or through a human error, one cannot rely solely on the service providers data backup measures. As with the security issue, backups become more complex when the service is provided by multiple parties (Heiser, 2010). R1 & R2 - Having recent backups do not make your service more stable or secure, however, when malicious hackers or a fire have destroys your data, a backup helps you to recover from such an event quickly.

R3 - Having off-site backups of your data (assuming the data is in an open format) makes it easier to switch service suppliers. Sources (Bodle, 2010), (Emmerik, 2010), (Heiser, 2010), (Marks, 2008), (Weinberger, 2010) Table 33 - V.1: Detailed description of the precaution "Make regular backups". Precaution Name Counteracts Type Description P3 Sign contracts on required service levels R1 (Solution lacks solid availability), R2 (Solution is insecure) Legal A service level agreement (SLA) is an agreement between the provider of a service and its customer which quantifies the minimum quality of service which meets the business need (Hiles, 1994). A SLA is a legal contract that covers areas like availability, security, monitoring and performance. The example below shows not only how SLAs work, but also shows that SLAs are not the one solution to all your potential problems.

Example

*+ a well-known cloud provider guarantees an availability level of 99.999% uptime, or five minutes a year, with a 10% discount on their charges for any month in which it is not achieved. However, since their infrastructure is not designed to reach five-nines of uptime, they are effectively offering a 10% discount on their services in exchange for the benefit of claiming that level of reliability. If a customer really needs five-nines of uptime, a 10% discount is not going to even come close to the cost of lost revenue, breach of customer service levels, or loss of market share due to credibility issues. (Durkee, 2010) and (Hofmann & Woods, 2010). Explanation of R1 & R2 - Just like having a back-up in place, a SLA is not actively keeping your services safe and counteracting secure, it merely determines who is to blame when problems arise and who should be fixing those problems. Sources (Buyya, Yeo, & Venugopal, 2008), (Durkee, 2010), (Herbert, Speyer, Gaynor, & Schule, 2006), (Hiles, 1994), (Hofmann & Woods, 2010), (Vaquero, Rodero-Merino, Caveres, & Lindner, 2009) Table 34 - V.1: Detailed description of the precaution " Sign contracts on required service levels ". Precaution Name Counteracts Type Description P4 Choose an ecosystem of significant size R3 (Solution or ecosystem produces lock-in), R4 (Entity becomes unpopular), R5 (Ecosystem contains power imbalance), R6 (Entity lacks innovation) Business When an ecosystem inhibits more players that are able to deliver more or less the same solution, this lowers the power of the service provider and places the customer more in control of his own situation When an customer needs a CRM-solution with a marketing calendar, the SalesForce CRM platform offers different vendors of such calendars, making it possible to switch vendors if necessary.

Example

Appendix C. Precautions based on literature Page 100 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Explanation of counteracting

R3 - When there are multiple players per niche in an ecosystem, chances are that they will try to make the transition from another service to their service as smooth as possible. R4 - A service that becomes less popular in an ecosystem with multiple players per niche is a service that probably loses a lot of customers to other players in the same niche. When an customer needs to replace the service of the tormented supplier, there are probably other customers that have made the transition before, making it possible to learn from their mistakes. R5 - When there are multiple players in the same niche, each player will be more fanatic to please its customers and take their wishes into account.

R6 - Niche players in a crowded niche, are stimulated more to keep up with the latest technologies available in the ecosystem, because they cannot take the risk of letting a competitor be ahead of them. Sources (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Richards, 2006) Table 35 - V.1: Detailed description of the precaution " Choose an ecosystem of significant size". Precaution Name Counteracts Type Description P5 Choose solutions that support open data formats R3 (Solution or ecosystem produces lock-in) Legal Open formats are those file formats whose specifications are published openly and are not restricted to a single vendor or software or a web application (Subramanian, 2008).

One of the things to keep in mind is that an open format is not the same as easily usable. If a CRM SaaS allows users to export the entire contents of the service to a single .csv file, it might be an open format but it will be hard to retain data integrity (Antonsen, 2009). Example Google Spreadsheet allows users to export data in various formats that are easily usable for other developers like CSV, XLS and PDF. Explanation of R3 - When data is easily exported in a format that is open to anyone (XML for example), it is easier counteracting to switch to another service supplier. Sources (Antonsen, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), (Subramanian, 2008), (Open Cloud Manifesto, 2009), (Zeller, et al., 2009) Table 36 - V.1: Detailed description of the precaution " Choose solution that support open data formats ". Precaution Name Counteracts P6 Beware of trends in the market and your ecosystem R4 (Entity becomes unpopular), R6 (Entity lacks innovation), R7 (Keystone neglects ecosystem needs) Type Business Description Being aware of what is going on in your suppliers ecosystems or in their markets can give a lot of insight in what kinds of challenges they are facing and puts their behavior in perspective. This will allow the customer to act on new developments as soon as possible, instead of being overtaken by them. Following such developments is not a direct solution to any risks, but allows for early discovery and acting. Example It is hard to find an example, because insight often comes gradually, rather than at once Explanation of R4, R6 & R7 - Being aware of what is going on does not directly mitigate the risks, but allows for counteracting early discovery, so the problems can be tackled as early as possible. Sources (Anderson, 2010) No other sources are found that explicitly state this precaution. Table 37 - V.1: Detailed description of the precaution "Beware of trends in the market and your ecosystem". Precaution Name Counteracts Applicable to Type Description Example P7 Choose Certified and Auditable partners R1 (Solution lacks solid availability), R2 (Solution is insecure) Service, keystone, niche player Legal Companies that are certified or audited are believed to have less chance of failing. IT is hard to give an example of a company that chose for a certified partner and encountered no problems,

Appendix C. Precautions based on literature Page 101 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Explanation of Using certified and auditable partners does not directly counteract unavailability or a solutions being counteracting not-secure, however, using certified partners reduces the change of these risks becoming reality. Sources (Armbrust, et al., 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Heiser, 2010) Table 38 - V.1: Detailed description of the precaution "Choose Certified and Auditable partners".

Appendix C. Precautions based on literature Page 102 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix D. Interview protocol for the first round of interviews


This interview protocol was used in the first round of expert interviews, in which the experts were asked to comment on the initial set of risks and precautions and on the method that should allow these risks and precautions to be identified and evaluated.

Interview protocol Goals


The expert interviews are held for a number of reasons: To have experts comment on: o The contents of the designed method for identification and assessment of risks in multi-party SaaS solutions (the risks and precautions that were determined based on the literature study.) The steps of the method that was designed and if and how these steps could be incorporated in consultants every day work.

To gain insight in what risks are encountered in real projects and how these risks align with the risks found in (scientific) literature. To discover projects and solutions that might be suitable for validating the completed method at the end of this research.

Steps of the interview


This section will describe the steps of the interview, which kind of questions will be asked, what information will be given, etc. This is to make sure that all the interviews will happen in a similar way, making sure that the answers will be comparable between interviews.

Intro
I will start by telling the same introduction to every interviewee: The purpose of my research is to develop a method that allows the identification and assessment of risks in multi-party SaaS solutions. Multi-party SaaS solutions are SaaS solutions that are delivered by more than one party. For example: a SalesForce CRM solution with a Zuora Z-Force app that allows SalesForce to handle payments. My method should be able to identify the risks related to the (direct and indirect) dependencies that services and companies have. Direct dependencies are dependencies between firms that are partners. Indirect dependencies are dependencies that spring from the fact that a company is part of an ecosystem. Of course the intro will also contain a brief overview of what the different sections of the interview will be and I will show them my conceptual model of a SaaS solution. This makes sure that there is no confusion about the terms used in this interview.

Appendix D. Interview protocol for the first round of interviews Page 103 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Questions about the risks and precautions


Once the stage has been set and the interviewee is aware of what the purpose of the method is, I will first ask the interviewee to come up with risks and precautions him/herself. This way I can find out what their ideas about risks and precautions are, without polluting their minds with the risks I found. Q1a: Q1b: Q2: Could you tell me what the most common risks are in multi-party SaaS solutions? Are there any risks specifically related to the dependencies the services have? Could you tell me what precautions are generally taken to prevent the risks you mentioned before from happening?

After these questions Ill show them the risks and precautions I derived from the literature Ive studied. Q3: The risks you mentioned are different from the risks I found in literature, lets walk through those risks and see why your risks differ from the ones I found. The precautions you mentioned are different from the precautions I found in literature, lets walk through those precautions and see why your precautions differ from the ones I found.

Q4:

The most important differences will be discussed and will be the input for next method version.

The method for identifying and assessing risks


This section of the interview will focus on the method for identifying and assessing the risks. The steps of the method will be explained and discussed. Q5: What is the current method that you use for identifying and assessing the risks of a customers solutions?

The method is shown and explained Q6: What additions do you have on the method that is used for identifying and assessing the risks of a multi-party SaaS solution do you have? How would this method fit into the work of a consultant when evaluating a possible solution? What would need to change to make this method (more) valuable?

Q7:

Other questions
Apart from the questions directly related to the method, I would also like to use the opportunity to ask some other things about their experience. Q8: Do you have experience with failing SaaS solutions? What were the causes, what would you have done to solve this problem or prevent it in the future? Do you have any other remarks that could make this method more effective in evaluating multi-party SaaS solutions?
Appendix D. Interview protocol for the first round of interviews Page 104 of 125

Q9:

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix E.

Raw expert interview results

Since the table with the raw expert interview results is too big to fit into a text editor file, it will be added at the end of this thesis on separate A3 sheets.

Figure 18 - Example of raw expert interview results.

Appendix E. Raw expert interview results Page 105 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix F.

Risks based on literature and expert interviews

This section contains a structured description for each of the risks. These risks were based on literature and the expert interviews. Each table contains the following fields: Risk - The ID of the risk for future reference in this research. Name - Name of the risk. Applicable to - The entity type this risk is related to. Type - The type of risk, business, legal or both. Description - General description of the risk. Metrics - Ways to measure if the risk is present in a certain situation. Impact on business - The impact a risk can have on a business that is reliant on a service. Example - A short example of the risk in a real-life situation. Sources - Sources that underpin the risk as described.
Risk Name Applicable to Type Description R1 Solution lacks solid availability Service Business and legal Services can become unavailable through a number of different reasons. The risk that is involved with a service becoming unavailable depends on how important this service is to the business.

If a HRM system goes off-line for a day, the HR-department might be hindered for that particular day, but there is probably little impact. However, if the central booking system of an airline fails, the consequences might be huge, both in direct costs, but also in indirect costs like damage to the Airlines image (see example). Metrics The availability of a service is usually measured in a percentage of uptime. 99.9% availability means that a service is unavailable for about eight hours per year. A services track record gives an idea of how the service has performed in the past, this is of course no guarantee for the future. Impact on The impact of services being unavailable is of course dependent on what the service is used for. The business more important a service is to the core business, the higher the impact when it becomes unavailable. Legal issues may arise when third parties are dependent on a product or service that the uses the service that became unavailable. The impact on business may however be even greater when a supplier goes out of business. Example A catastrophic systems failure at cloud-based software provider, Navitaire, *+, disrupted travel for 50,000 customers of Virgin Blue airlines in Australia (Krigsman, 2010). Due to a system failure at the service provider that took care of the reservations and check-ins of Virgin Blue airlines, the airlines business was disrupted for about eight hours, costing them between $15 and $20 million in lost revenue. (CIO, 2010) Sources (Armbrust, et al., 2010), (Dubey & Wagle, 2007), (Durkee, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Kaplan, 2007), (Kontogiannis, et al., 2007), (Olsen, 2009), (Wei & Brian Blake, 2010), (Van de Zande & Jansen, 2010) Table 39 - V.2: Detailed description of the risk Solution lacks solid availability Risk Name Applicable to Type Description R2 Solution is insecure Service Business and legal Security threats come in all sorts. A data leak or unauthorized access to a system could be the result of a badly designed system, two services that do not collaborate properly or an employee that is dissatisfied. This makes security probably one of the hardest parts of a service to control. Closely related to security is the compliance with legislation and the protection of privacy.

Appendix F. Risks based on literature and expert interviews Page 106 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Metrics

Security is hard to measure, one could use the services track record, but that there never was a security breach, could also mean that the security-holes are not yet identified. There are some security certificates that show that a system has passed a certain security audit, like SAS 70. The presence of relevant certifications would be the best indication of how a company is dealing with its security. Certifications are no guarantee, but they indicate that it is taken seriously. It is beyond the scope of this method to specify how a solution complies with legislation and the privacy guidelines that are applicable in a specific situation. Impact on The impact of a security breach on a business can be huge, for example when confidential company or business customer data has leaked to competitors, or when hackers have managed to take down or deface the companys website. Legal issues arise when the data that has been lost also puts clients at a disadvantage. Example Kasboek.nl is a Dutch SaaS that allows users to keep track of their personal finances. In October of 2010, more than a million of the uploaded transactions were compromised through a leak in their system. These kinds of leaks to not only hurt the supplier of that specific solution (Wokke, 2010). It also hits other SaaS vendors that deal with sensitive information (ten Wolde, 2010). Sources (Armbrust, et al., 2010), (Cloud Security Alliance, 2010), (Dillon, Wu, & Chang, 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Hwang, Kulkarni, & Hu, 2009), (Levinson, 2007), (Penn, 2010), (Yildiz, Abawajy, Ercan, & Bernoth, 2009) Table 40 - V.2: Detailed description of the risk Solution is insecure Risk Name Applicable to Type Description R3 Solution or ecosystem produces data lock-in Service, ecosystem Business Lock-in is a complex situation that arises when leaving your current supplier or ecosystem is either too expensive, or impossible. There are a lot of complex reasons why a company could be locked in by its supplier, but they are too specific to handle in this research. The simpler part about lock-in is data lock-in. Data is locked-in when you are unable to export your data in a generic format, are unable to export it with all the metadata and relations or are unable to export your data at all. Metrics Data is locked-in if there is no export function available that can export all the data, with metadata and relationships to a generic format like XML. Impact on The impact on the business of being locked-in is not always apparent in the short term. In the long run business however, being locked-in to one vendor or platform could seriously hinder the business, especially when alternative platforms offer solutions that better suit the business needs. Example The docx-format is a format which true potential can only be used by Microsoft Word. Other applications can read this format, but cannot use all the features it offers. Sources (Armbrust, et al., 2010), (Bosch, 2009), (Huizing, 2008), (Kindness, Whiteley, & Hayes, 2010), (Linthicum, 2009), (Mietzner, Leymann, & Papazoglou, 2008), (Mudumbai, 2010), (Van der Pluijm, 2010), (Pezzini, Driver, Feinberg, Natis, Perkings, & Scott, 2010) Table 41 - V.2: Detailed description of the risk Solution or ecosystem produces data lock-in

Risk Name Applicable to Type Description

Metrics Impact on business

R5 Ecosystem contains power imbalance Service, keystone, niche player Business Having influence on your supplier means that you could persuade him to change designs or move release dates to suit your situation better. Also the supplier could discuss new feature with you. It might seem as if keystones are always able to influence the ways in which the ecosystem is evolving. However, careless keystones who try to impose their will on an ecosystem could end up making their ecosystem less attractive, regardless of what the change they advocate implies. The amount of influence an customer has on a supplier depends primarily on how big the customer is relative to other customers, and how much influence the customer has on other customers. Having no influence on which way your supplier is going with the service you are using, has no immediate impact on your business. That is for as long as you do not need anything other than your supplier delivers. Impact on your business could be far-reaching when suppliers change their services.

Appendix F. Risks based on literature and expert interviews Page 107 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Example

Developers who develop for Apples iOS ecosystem generally have low influence on what the c ourse of the ecosystem will be. In February of 2010, Apple banned all apps that feature sexually suggestive material, including photos of women in bikinis and lingerie, a move that came as an abrupt surprise to developers who had been profiting from such programs. (Wortham, 2010) Sources (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Jansen, Finkelstein, & Brinkkemper, 2009) Table 42 - V.2: Detailed description of the risk Little or no influence on third parties Risk Name Applicable to Type Description R6 Entity lacks innovation keystone, niche player Business Cloud computing is a relatively new field with new innovations coming up about every month. When a company stops preparing for the future, by neglecting these innovations, this is the beginning of the end. Being future proof is technology-wise and business-wise a necessity. Adopting new technologies and keeping services up to standard with new developments is one of the important factors to the attractiveness of these services. From a business perspective, being future proof means that the service is able to keep its current developers and customers, as well as it is able to attract new ones. Whether a service is able to do this depends mostly on the strategy of the vendor and the ecosystem it is in. The way in which the keystone follows a strategy that allows niche players to join their ecosystem, explore new niches and make the ecosystem more valuable for new groups of customers, is important to the innovative strength of the ecosystem. Metrics Measuring if a company lacks innovation is possible by comparing the companys rate of innovation to competitors, one can look at the internal processes around innovation or the percentage of their turnover is being spent on innovation. Impact on When a service is not able to support new developments it will be less appealing to potential business customers and will be less useful to existing customers willing to expand. Impacts could be the same as with R4 Example In the course of 2008, Suns Network.com cloud computing platform was abandoned by developers, among others because Developers did not have access to APIs for system configuration and controlling *+ (Ingthorsson, 2010). The consequence was that Sun stopped the development and stopped accepting new customers. Current customers were still being served, but new innovations were not implemented and the future of the service became uncertain (Musil, 2008). Sources (Adner, 2006), (Bosch, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel & Seely Brown, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Levien, 2004), (Iansiti & Richards, 2006), (Ingthorsson, 2010), (Moore, 1993), (Musil, 2008) Table 43 - V.2: Detailed description of the risk Entity lacks innovation Risk Name Applicable to Type Description R7 Keystone neglects ecosystem needs Keystone, ecosystem Business What is meant by the dedication a keystone shows to his ecosystem, is the way in which the keystone stands for his ecosystem. Does the keystone show that the apps on his platform are worth using and will serve the customers needs? There are a couple of positive sides on the situation in which the keystone certifies that (certain) apps are suitable for his platform. Niche players that meet the keystones demands, have a proof to show that their solution is fit for the platform. Metrics The keystone assures potential customers that add-ons on his platform can safely be used. Certifications programs generally increase the trust in a platform, which in turn can have a positive influence on the popularity of a platform. Are there certain demands or levels of quality that an add-on should meet to be allowed on the platform? Is the platform owner actively certifying third party add-ons? Is action taken when an accepted add-on fails to meet the required levels?

Appendix F. Risks based on literature and expert interviews Page 108 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Impact on business

The impact on business of a keystone that lacks dedication to his ecosystem should not be exaggerated; however, it is important to find out why a keystone is not supporting or managing the ecosystem it benefits from. The ultimate goal of dedication and regulation to the ecosystem is to raise the overall quality of the ecosystem, which is positive for the keystone. Example Apples iOS app-policy is not only highly criticized; it is also a good example of a keystone that wants to make sure that his ecosystem meets a high quality level. By making sure that only tested applications can enter the app store, Apple makes sure that whatever you install, your iOS-device will still be functioning properly. Customers know this and have no reticence to try out new add-ons. Sources Expert interviews Table 44 - V.2: Detailed description of the risk Lack of dedication to the ecosystem Risk Name Applicable to Type Description R8 Actor lacks size and stability Niche player Business Small parties impose a risk on customers in the current cloud computing market. The main problem is the uncertainty that small companies bring to the table, both from a business as from a technical point of view. From a business perspective, small companies (with innovative products or services) are an interesting prey for large companies. When a large company takes over a small one, chances are that they are bought for their technique and smart employees, not for their services. More than once a takeover marks the end of the smaller companies services. Technically, smaller companies often do not have the ability to make their solution as stable as larger companies can. Smaller companies tend to be less risk averse and their services have often yet to become mature. Metrics It is hard to say when a company is large or small, as an indication the relative size of the companies could be used, or the relative size in the market. Impact on The impact a small player can have on the customer is hard to point out and is largely dependent on business how vital the service of that company is. If a small player services an important niche in the solution, which cannot easily be taken over by another player, this is potentially a big problem. Example In February 2010, Google bought ReMail, an iOS email reader app. Directly after they bought this company they removed ReMail from the appstore and shutdown further development. The employees of ReMail were the main focus of the takeover and their knowledge was put to work elsewhere within Google (Hachman, 2010). Sources Expert interviews Table 45 - V.2: Detailed description of the risk Small parties Risk Name Applicable to Type Description R9 Product lifecycle Service Business New releases of software product always bring a certain degree of uncertainty. Will all the functions I use frequently remain supported? Will the quality of the new release be as good as the current version? Will the integration with other products still work as before? With SaaS, these questions are even more important, because of the shared codebase there is no choice whether or when you are going to update; everybody moves to the new version at the same time. This makes the product lifecycle of a service very important, when a vendor is known to release important changes on short notice; this could be a risk to the customer. The following indicators could indicate that the product lifecycle management of a vendor could pose a risk to the customer. Little or no information sharing about upcoming releases Impact on business No regular release moments Not enough time for third party developers to adjust their products to the new features.

Metrics

Impact on the business is potentially big. Unanticipated changes in one of the services could bring down the entire solution and custom built connections to other services could take days or even weeks to be re-established.

Appendix F. Risks based on literature and expert interviews Page 109 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Example

An example of a product lifecycle that is properly managed is that of SalesForce and the Force.com platform. Via a special website (http://developer.force.com/releases/), developers can keep track of what is going to change and there are three fixed releases every year. Sources Expert interviews and (Durkee, 2010), (Lindner, Galan, Chapman, Clayman, Henriksson, & Elmroth, 2010) Table 46 - V.2: Detailed description of the risk Product lifecycle

Appendix F. Risks based on literature and expert interviews Page 110 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix G. Precautions based on literature and expert interviews


These precautions were based on literature only and were the input for the expert interviews. Each table contains the following fields: Precaution - The ID of the precaution for future reference in this research. Name - Name of the precaution. Counteracts - Which risks could be counteracted with this precaution in place. Type - The type of precaution, business, legal or both. Description - General description of the precaution. Example - A short example of the precaution in a real-life situation. Explanation of counteracting - A short description of how this precaution counteracts a risk that was mentioned under Counteracts. Sources - Sources that underpin the precaution as described.
Precaution Name Counteracts Type Description P1 Deploy escrow services R1 (Solution lacks solid availability), R3 (Lock-in), R5 (Ecosystem contains power imbalance) Business and legal A source-code escrow arrangement is an arrangement between a software vendor, the buyer and a trusted third party. (Van de Zande & Jansen, 2010). The trusted third party (also: escrow agent) places the source code of the software vendor in a secure holding place. The escrow arrangement contains the condition under which the source code will be released to the buyer. Bankruptcy of the vendor is an example of a condition that could trigger the release of the source code. For onpremise software this is a good protection for the buyer, against possible bankruptcy of the vendor. With the source code, the buyer will be able to (have other developers) update his systems and facilitate transition to another software solution. Cloud customers on the other hand, have not only their software, but also their data in hands of the service supplier. Escrow solutions in their situation need not only cover the source code of the software, but should also cover the (daily) backups of their data. There are also escrow services that besides de above measures, also offer an added service that takes over the payment of the hosting when a vendor goes out of business. This gives customers some time to think about what the next best step would be, while being able to continue using the service for some months. Exact Software has such an agreement for their Exact Online financial accounting SaaS. This makes sure that in the event of Exact Software going bankrupt, the service as well as the data will remain available for at least one month (Exact Salesdesk, 2010). Traditional escrow agents are also making the move towards escrow for SaaS. Amsterdam based escrow agent Softcrow is developing an escrow product that not only contains the source code and daily backups, but also the complete virtual machine images of the service. In case of an emergency, the entire service can be revived from the contents of the escrow. R1 - Escrow services cannot prevent short disruptions in services, but as a service provider goes out of business, the escrow could help recover the service as fast as possible. Escrow terms could be negotiated to make this recovery time as short as possible. R3 - Having a daily data back-up to an escrow-provider could indicate that your service supplier is willing to participate in data exports. Although escrow is not a direct way to counteract lock-in, it could be an indication of how free your data is. R5 - When you have little or no influence on your service provider(s), having an escrow in place makes you a little less vulnerable to bad business decisions those parties can make. Sources (Exact Salesdesk, 2010), (Krs, 2010), (Denson, 2002), (Herbert, Ferrusi Ross, Rose, & Karcher, 2010), (Spiotto & Spiotto, 2003), (Van de Zande & Jansen, 2010) Table 47 - V.2: Detailed description of the precaution "Deploy escrow services"

Example

Explanation of counteracting

Appendix G. Precautions based on literature and expert interviews Page 111 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Precaution Name Counteracts Type Description

Example Explanation of counteracting

P2 Make regular backups R1 (Solution lacks solid availability), R2 (Solution is insecure), R3 (Lock-in) Business That whenever your data is in the cloud, backups are unnecessary and superfluous, is a myth (Marks, 2008). Customers always run the risks of losing their data, whether it be through a fire in the service providers datacenter, or through a human error, one cannot rely solely on the service providers data backup measures. As with the security issue, backups become more complex when the service is provided by multiple parties (Heiser, 2010). R1 & R2 - Having recent backups do not make your service more stable or secure, however, when malicious hackers or a fire have destroys your data, a backup helps you to recover from such an event quickly.

R3 - Having off-site backups of your data (assuming the data is in an open format) makes it easier to switch service suppliers. Sources (Bodle, 2010), (Emmerik, 2010), (Heiser, 2010), (Marks, 2008), (Weinberger, 2010) Table 48 - V.2: Detailed description of the precaution "Make regular backups" Precaution Name Counteracts Type Description P3 Sign contracts on required service levels R1 (Solution lacks solid availability), R2 (Solution is insecure), R9 (Product lifecycle) Legal A service level agreement (SLA) is an agreement between the provider of a service and its customer which quantifies the minimum quality of service which meets the business need (Hiles, 1994). A SLA is a legal contract that covers areas like availability, security, monitoring and performance. What makes SLAs less useful as a precaution, is that (most) SaaS vendors are not willing (mostly because they are not able) to provide a custom SLA. This makes the SLA only useful for finding out what the arrangement is, rather than for changing it. The example below shows not only how SLAs work, but also shows that SLAs are not the one solution to all your potential problems.

Example

*] a well-known cloud provider guarantees an availability level of 99.999% uptime, or five minutes a year, with a 10% discount on their charges for any month in which it is not achieved. However, since their infrastructure is not designed to reach five-nines of uptime, they are effectively offering a 10% discount on their services in exchange for the benefit of claiming that level of reliability. If a customer really needs five-nines of uptime, a 10% discount is not going to even come close to the cost of lost revenue, breach of customer service levels, or loss of market share due to credibility issues. (Durkee, 2010) and (Hofmann & Woods, 2010). Explanation of R1 & R2 - Just like having a back-up in place, a SLA is not actively keeping your services safe and counteracting secure, it merely determines who is to blame when problems arise and who should be fixing those problems. R9 - SLAs will not solve your product lifecycle problems, but it is possible to use a SLA to specify what and when the vendor must communicate about the releases, so the developers could minimize the impact of the release on current affairs. Sources (Buyya, Yeo, & Venugopal, 2008), (Durkee, 2010), (Herbert, Speyer, Gaynor, & Schule, 2006), (Hiles, 1994), (Hofmann & Woods, 2010), (Vaquero, Rodero-Merino, Caveres, & Lindner, 2009) Table 49 - V.2: Detailed description of the precaution " Sign contracts on required service levels " Precaution Name Counteracts Type Description P4 Choose an ecosystem of significant size R3 (lock-in), R5 (Ecosystem contains power imbalance), R6 (Entity lacks innovation) Business When an ecosystem inhibits more players that are able to deliver more or less the same solution, this lowers the power of the service provider and places the customer more in control of his own situation When a customer needs a CRM-solution with a marketing calendar, the SalesForce CRM platform offers different vendors of such calendars, making it possible to switch vendors if necessary.

Example

Appendix G. Precautions based on literature and expert interviews Page 112 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Explanation of counteracting

R3 - When there are multiple players per niche in an ecosystem, chances are that they will try to make the transition from another service to their service as smooth as possible. R5 - When there are multiple players in the same niche, each player will be more fanatic to please its customers and take their wishes into account.

R6 - Niche players in a crowded niche, are stimulated more to keep up with the latest technologies available in the ecosystem, because they cannot take the risk of letting a competitor be ahead of them. Sources (Eisenmann, Parker, & Van Alstyne, 2008), (Hagel III, Seely Brown, & Davidson, 2008), (Hartig, Tol, & Visscher, 2006), (Iansiti & Richards, 2006) Table 50 - V.2: Detailed description of the precaution " Choose an ecosystem of significant size Precaution Name Counteracts Type Description P5 Choose solutions that support open data formats R3 (Solution or ecosystem produces data lock-in), R8 (Actor lacks size and stability) Legal Open formats are those file formats whose specifications are published openly and are not restricted to a single vendor or software or a web application (Subramanian, 2008). One of the things to keep in mind is that an open format is not the same as easily usable. If a CRM SaaS allows users to export the entire contents of the service to a single .csv file, it might be an open format but it will be hard to retain data integrity (Antonsen, 2009). Google Spreadsheet allows users to export data in various formats that are easily usable for other developers like CSV, XLS and PDF. R3 - When data is easily exported in a format that is open to anyone (XML for example), it is easier to switch to another service supplier.

Example Explanation of counteracting

R8 - Because the future of small parties is often unsure, it is best to be able to export your data in an open format that makes it easy to switch. Sources (Antonsen, 2009), (Eisenmann, Parker, & Van Alstyne, 2008), (Subramanian, 2008), (Open Cloud Manifesto, 2009), (Zeller, et al., 2009) Table 51 - V.2: Detailed description of the precaution "Choose Certified and Auditable partners" Precaution Name Counteracts Type Description P7 Choose Certified and Auditable partners R1 (Solution lacks solid availability), R2 (Solution is insecure) Legal Although P7 does not give any guarantees (E2), it does show that an organization is paying attention to the aspects that are certified. Another aspect is that certifications do add to the reputation of the company. There are however still no certifications that are purely designed to certify cloud solutions (E7) Example There are a lot of certifications that (partly) relate to the field of cloud computing and SaaS, to name a few: VeriSign Secured, TRUSTe, SysTrust, SAS70 type II, SafeHarbor, ISO 27001. Explanation of R1 & R2 - As written above, a certificate does not solve these problems, but it indicates that an counteracting organization is taking the subjects of the certifications seriously. Sources (Armbrust, et al., 2010), (Heiser, 2010), (Hofmann & Woods, 2010), (Heiser, 2010) Table 52 - V.2: Detailed description of the precaution "Certificates and auditability" Precaution Name Counteracts Type P8 Make use of an intermediary R1 (Solution lacks solid availability), R8 (Actor lacks size and stability) Technical, business

Appendix G. Precautions based on literature and expert interviews Page 113 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Description

An intermediary could catch a lot of issues before they affect the customer (E2, E3, E4, E5, E7). This means that there is only one party to deal with and as the intermediary is experienced with the task of handling multiple parties, there are fewer problems to expect. An intermediary could act on (at least) two different levels and could come in three forms.

The intermediary could act on a technical level, hosting a shared platform or a service that ties all the other services together, or on a business level, where it is primarily involved with managing the contracts between the different parties. The intermediary could be a third party, like a consultancy company, or it could be the platform vendor on which the solutions run. In some cases the platform vendor might also guarantee the working of external services. Example In some cases, especially for larger clients, SalesForce is known to guarantee applications that run natively on the Force.com platform, for some time after a vendor went out of business. Another example is middleware that can be used to link different systems. Explanation of R1 - See example. counteracting R8 - As a platform owner guarantees the working of native applications, this would lower the business impact from a small company going out of business. Sources Expert interviews and (Miller, 2009) Table 53 - V.2: Detailed description of the precaution "Make use of an intermediary" Precaution Name Counteracts Type Description P9 Choose solutions on one shared platform R1 (Solution lacks solid availability), R8 (Actor lacks size and stability), R9 (product lifecycle) Technical A shared platform on which all the add-ons run forms a solid basis that makes integration easier (E6) and eventually allows the platform vendor to guarantee the workings of the entire solution through certification (see P7 and NP1) E4, E5. As shared platform could also provide a standard to connect services, limiting the finger-pointing when things go wrong E8. Example The SalesForce platform offers ways for applications to connect that would not be possible when the applications would run on separate platforms. Explanation of R1 - Although a shared platform might seem a single point of failure, it is in most cases a solid counteracting foundation that allows applications to communicate more easily and more robust. Also, if 1 platform has 99% uptime, 4 different platforms have 99%^4 =96%. R8 - Small parties have a reputation of having more downtime, because their platform is often less fail safe. A shared platform with a good reputation on which all applications run, lowers the risk of one of the smaller applications failing, making the entire system unusable. R9 - When all the applications that from the solution run on the same platform, the releases from the platform will be a direct reason for all vendors to update their code. If theyd run on another platform, adapting to other platforms updates would maybe have less priority. Sources Expert interviews and (SalesForce.com, 2008), (Perry, Hatcher, Mahowald, & Hendrick, 2009), both funded by SalesForce. Table 54 - V.2: Detailed description of the precaution "Choose solutions on one shared platform Precaution Name Counteracts P10 Choose a mature and reputable company Mature companies with a good reputation R1 (Solution lacks solid availability), R2 (Solution is insecure), R8 (Actor lacks size and stability), R9 (product lifecycle) Type Legal Description All experts have pointed out how larger companies with a good reputation and proven success in the market are to be preferred. It is hard to point out when a company is mature, or when it is large enough, but the tenor is clear that experience and reputation are considered important factors. Example Compare the way in which SalesForce shows what the changes in the upcoming release for the Force.com platform will be (http://developer.force.com/releases/). And compare this to how a small start-up like 42Windmills (http://www.42windmills.com) shows their platform developments. Explanation of R1, R2, R8 and R9 - It is not that P10 directly solves these problems, but in general, mature counteracting companies have these risks better under control than small companies who just started. Sources Expert interviews and (Betcher, 2010), (Bisong & Rahman, 2011) Table 55 - V.2: Detailed description of the precaution "Choose a mature and reputable company"

Appendix G. Precautions based on literature and expert interviews Page 114 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix H. Deliverables: entity collections for the CSSN


Example COLLECTION OF ACTORS from the activity Identify actors
ID Name/description A1 AeroPlus A2 Supplier X A3 Exact Software A4 Rackspace A5 Microsoft A6 Reseller A7 Customer Table 56 - Example COLLECTION OF ACTORS. Type Actor - Supplier Actor - Supplier Actor - Supplier - Keystone Actor - Value Added Reseller Actor - Supplier - Keystone Actor - Reseller Actor - Customer

Example COLLECTION OF FLOWS from the activity Identify actors


Name/description Exact Online Windows Server SQL Server Add-on for Exact Online Dataflow between Exact Online at Rackspace and AeroPlus S1 Exact Online S2 Support S3 AeroPlus Flight Planning S4 Add-on for Exact Online as a service delivered by Rackspace 1 Fee for hosting P1 2 Fee for S1 3 Fee for P3 4 Fee for P2 5 Fee for S1 and S2 6 Fee for S3 7 Fee for P4 Table 57 - Example COLLECTION OF FLOWS. ID P1 P2 P3 P4 D1 Type Product Product Product Product Dataflow Service Service Service Service Fee Fee Fee Fee Fee Fee Fee

Appendix H. Deliverables: entity collections for the CSSN Page 115 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix I.

Deliverables: CSSN of Exact online


E.2

AeroPlus
E.1
P.3 .3

Microsoft

PL.2
P.2 .4
D.1

3 S.

6 .

Supplier X

P.4 .7

Exact Software

.1 P.4
P.1

Rackspace
PL.1

S.4
S.1

Customer

Reseller

ID A1 A2 A3 A4 A5 A6 A7 P1 P2 P3 P4 D1 S1 S2 S3 S4 PL1 PL2 1 2 3 4 5 6 7 E1 E2

Name/description AeroPlus Supplier X Exact Software Rackspace Microsoft Reseller Customer Exact Online Windows Server SQL Server Add-on for P1 Dataflow between S3 and S1 Exact Online Support AeroPlus Flight Planning P4 as a service Rackspace Hosting Platform AeroPlus platform Fee for hosting P1 Fee for S1 Fee for P3 Fee for P2 Fee for S1 and S2 Fee for S3 Fee for P4 Exact Microsoft

Type Actor - Supplier Actor - Supplier Actor - Supplier - Keystone Actor - Value Added Reseller Actor - Supplier - Keystone Actor - Reseller Actor - Customer Product Product Product Product Dataflow Service Service Service Service Platform Platform Fee Fee Fee Fee Fee Fee Fee Ecosystem Ecosystem

S . .2 5

2 .

Added in the third step of the method.

Table 58 - List of entities in CSSN and their type.

Appendix I. Deliverables: CSSN of Exact online Page 116 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix J.

Deliverable: ENTITY-RISK MAPPING


R1 R2 R5 R6 R7 R8 V V V V V V V V V V V V V V V V V V Not included in this risk evaluation V V V V V V V V V V V V V V V V V V V V V V R3 R9

ID Name/description Type A1 AeroPlus Actor - Supplier A2 Supplier X Actor - Supplier A3 Exact Software Actor - Supplier - Keystone A4 Rackspace Actor - Value Added Reseller A5 Microsoft Actor - Supplier - Keystone A6 Reseller Actor - Reseller A7 Customer Actor - Customer P1 Exact Online Product P2 Windows Server Product P3 SQL Server Product P4 Add-on for P1 Product D1 Dataflow between S3 and S1 Dataflow S1 Exact Online Service S2 Support Service S3 AeroPlus Flight Planning Service S4 P4 as a service Service PL1 Rackspace Hosting Platform Platform PL2 AeroPlus platform Platform 1 Fee for hosting P1 Fee 2 Fee for S1 Fee 3 Fee for P3 Fee 4 Fee for P2 Fee 5 Fee for S1 and S2 Fee 6 Fee for S3 Fee 7 Fee for P4 Fee E1 Exact Ecosystem E2 Microsoft Ecosystem Table 59 - ENTITY-RISK MAPPING.

V V V V V V V V V V V

V V V V V V V V V V V

V V V V V V V V V V V

Not included in this risk evaluation

V V

V V

Appendix J. Deliverable: ENTITY-RISK MAPPING Page 117 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix K.
0 0.25 0.5 0.75 1

Deliverable: ENTITY-RISK MAPPING OF APPLYING RISKS


None Low Medium High Full

There are three levels of applicability:

Note that the applicability in Table 60 is only an example and does not necessarily reflect the real situation that entity is in. R4 is not present in this mapping because it was removed from the method based on expert interviews.
ID Name R1 R2 A1 AeroPlus A2 Supplier X A3 Exact Software A4 Rackspace A5 Microsoft A6 Reseller P1 Exact Online 0 0 P2 Windows Server 0 0,5 P3 SQL Server 0 0 P4 Add-on for P1 0,5 0 D1 Dataflow between S3 and S1 0,5 0 S1 Exact Online 0,25 0 S2 Support 0 0 S3 AeroPlus Flight Planning 0,5 0 S4 P4 as a service 0 0 PL1 Rackspace Hosting Platform 0 0,25 PL2 AeroPlus platform 0,5 0 E1 Exact E2 Microsoft Table 60 - ENTITY-RISK MAPPING OF APPLYING RISKS. R3 R5 0,5 1 1 0 1 0,25 0 0 0 0 0 0,5 0,5 1 0,5 0 0,5 0,5 0,75 R6 0 0 0 0,5 0,5 0 0 0,5 0,5 0 0 0 0,5 0 0 0,25 0 R7 R8 1 1 0 0 0,25 1 0 0 0 0 0 0 0 0,75 0 0 0 0 0,25 R9

Appendix K. Deliverable: ENTITY-RISK MAPPING OF APPLYING RISKS Page 118 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix L.
0 0.25 0.5 0.75 1

Deliverable: Weighing BUSINESS IMPACTS


None Low Medium High Full

There are four levels of BUSINESS IMPACT on the CUSTOMERs business:

Note that the risks weights in Table 61 are only an example and do not necessarily reflect the real situation that entity is in. R4 is not present in this mapping because it was removed from the method based on expert interviews.
ID Name R1 R2 R3 R5 A1 AeroPlus 0,25 A2 Supplier X 0,5 A3 Exact Software 0,5 A4 Rackspace 0,5 A5 Microsoft 0,25 A6 Reseller 0,5 P1 Exact Online 0,75 1 0 P2 Windows Server 0,5 0,75 0 P3 SQL Server 0,5 0,75 0 P4 Add-on for P1 0,5 1 0 D1 Dataflow between S3 and S1 0,75 0,5 0 S1 Exact Online 1 1 0,25 S2 Support 0,5 0,25 0,5 S3 AeroPlus Flight Planning 1 0,25 0,75 S4 P4 as a service 0,25 0,25 0,25 PL1 Rackspace Hosting Platform 1 1 0,25 PL2 AeroPlus platform 1 0,25 0,75 E1 Exact 0,25 E2 Microsoft 0,25 Table 61 - Weighing the BUSINESS IMPACTS of APPLICABLE RISKS R6 1 0,25 0,5 0,25 0,5 0,25 0,75 0,25 0,25 0,25 0,25 0,75 0,5 1 0,25 0,25 1 R7 R8 1 0,25 1 0,5 0,25 0,5 1 0,25 0,25 0,25 0,25 0,75 0,25 1 0,25 1 1 1 0,25 R9

Appendix L. Deliverable: Weighing BUSINESS IMPACTS Page 119 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix M. Deliverable: Ordered List of BUSINESS IMPACTS


Table 62 is (Table 60 +Table 61) / 2 and shows how much BUSINESS IMPACT a RISK on an ENTITY has times the applicability of the RISK. R4 is not present in this mapping because it was removed from the method based on expert interviews.
ID Name R1 R2 R3 R5 A1 AeroPlus 0,38 A2 Supplier X 0,75 A3 Exact Software 0,75 A4 Rackspace 0,25 A5 Microsoft 0,63 A6 Reseller 0,38 P1 Exact Online 0,38 0,50 0,00 P2 Windows Server 0,25 0,63 0,00 P3 SQL Server 0,25 0,38 0,00 P4 Add-on for P1 0,50 0,50 0,00 D1 Dataflow between S3 and S1 0,63 0,25 0,00 S1 Exact Online 0,63 0,50 0,38 S2 Support 0,25 0,13 0,50 S3 AeroPlus Flight Planning 0,75 0,13 0,88 S4 P4 as a service 0,13 0,13 0,38 PL1 Rackspace Hosting Platform 0,50 0,63 0,13 PL2 AeroPlus platform 0,75 0,13 0,63 E1 Exact 0,38 E2 Microsoft 0,50 Table 62 - Weighed ENTITY-BUSINESS IMPACT of RISK -mapping. R6 0,50 0,13 0,25 0,38 0,50 0,13 0,38 0,38 0,38 0,13 0,13 0,38 0,50 0,50 0,13 0,25 0,50 R7 R8 1,00 0,63 0,25 0,25 0,75 0,50 0,13 0,13 0,13 0,13 0,38 0,13 0,88 0,13 0,50 0,50 0,50 0,25 R9

0,50

Ordering method 1: RISK-BUSINESS IMPACT measure.


The first and probably the easiest way to derive the most important risks to the customer, is to look at the figures in Table 62. The most important risks to tackle are then: 1,00 0,88 0,88 0,75 0,75 0,75 0,75 0,75 AeroPlus AeroPlus Flight Planning AeroPlus Flight Planning Supplier X Exact Software Reseller AeroPlus Flight Planning AeroPlus platform R8 (Actor lacks size and stability) R3 (Solution or ecosystem produces data lock-in) R9 (Solution has an unpredictable lifecycle) R5 (Ecosystem contains power imbalance) R5 (No influence) R8 (Ecosystem contains power imbalance) R1 (Availability) R1 (Availability)

Table 63 - Most important risk using ordering method 1.

Ordering method 2: Actor comparison


Another way to look at the risks, is to look at which ACTOR is responsible for creating the most risks.
Actor Aeroplus Rackspace Microsoft Exact Supplier X Combined weight of risks 8,06 6,56 4,63 4,13 2,75

Appendix M. Deliverable: Ordered List of BUSINESS IMPACTS Page 120 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Reseller Table 64 - Actors ordered by weight of risks.

2,75

This ordering method shows that AeroPlus is definitely the ACTOR that causes the most risks.

Ordering method 3: Risk comparison


Ordering the weight of the risks gives insight in what risks the most business impact has and this might be useful to find a single solution to mitigate the entire risk at once.
Risk R1 (Solution lacks solid availability) R2 (Solution is insecure) R3 (Solution or ecosystem produces data lock-in) R5 (Ecosystem contains power imbalance) R6 (Entity lacks innovation) R7 (Keystone neglects ecosystem needs) R8 (Actor lacks size and stability) R9 (Solution has an unpredictable lifecycle) Table 65 - Risks ordered by weight of entities. Combined weight of entities 5,00 3,88 3,75 3,13 5,50 1,50 2,63 3,50

Appendix M. Deliverable: Ordered List of BUSINESS IMPACTS Page 121 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix N. Deliverable: Precautions against risks


In this example, the risk-ordering was used from ordering method 3. The goal is to find precautions to mitigate the entire risks, regardless of which entity it caused. Finding precautions against risks can be done by taking the risk-precaution mapping (Table 13) and find out how certain precautions tackle the risk in this particular situation. The top three risks are taken
Risk R1 (Solution lacks solid availability) R2 (Solution is insecure) R3 (Solution or ecosystem produces data lock-in) Table 66 - Top three risks from ordering method 3. Combined weight of entities 5,00 3,88 3,75

and the risk-precaution mapping shows which precautions might be suitable to mitigate the risks. Precautions
P1 P2 P3 P4 P5 P7 P8 P9 P10 V V

R1 V V V V V V R2 V V V R3 V V V V Table 67 - Relevant part of the risk-precaution mapping (Table 13) for use with this example.

From this example comes that at least P2, P3, P7 and P10 should be reviewed, as they all (try to) mitigate two risks at once.

Risk s

Appendix N. Deliverable: Precautions against risks Page 122 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix O. Validation: Complete LIST OF ENTITIES from CSSN


[Confidential parts removed]

Appendix O. Validation: Complete LIST OF ENTITIES from CSSN Page 123 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix P.

Interview questions for validation of CRISP method

These questions were asked during the semi-structured interviews for the validation of the CRISP method. Introduction 1. Short introduction to the method. 2. Presentation of the risks that were found. 3. Presentation of the precautions that are advised.

Questions about the results from the CRISP method 4. What do you think about the risks that were identified? a. Are they relevant to look into? b. Are they the most important external risks to this solution? 5. What do you think about the precautions that were suggested? a. Do these precautions counteract the identified risks? b. Are the precautions feasible? 6. Are these risks new to you? 7. Would the development of the solution have been different if these risks were known earlier on?

Question in general about the CRISP method 8. Do you see any problems in using the results from the CRISP method in general? 9. If, how and when would you find it useful to employ the CRISP method?

- Room for any other remarks regarding the method, etc.-

Appendix P. Interview questions for validation of CRISP method Page 124 of 125

Evaluating compound cloud based solutions in an ecosystem context - master thesis - Bernard Verhoeven

Appendix Q. Quantitative results from interviews with customers and vendors


This appendix shows the data gathered from the interviews with customers and vendors. Although the data and the figures derived from it are not statistically valid, provide an insightful view on how customers and vendors view the risks around cloud computing.
Ordered by importance (Customers) R1 C1 C2 C3 C4 C5 C6 AVG 2 1 2 1 1 9 2,67 R2 1 2 1 9 2 1 2,67 R3 9 9 3 5 3 2 5,17 R5 9 9 4 4 9 9 7,33 R6 9 4 9 3 4 9 6,33 R7 9 9 9 9 9 9 9,00 R8 9 5 5 2 9 3 5,50 R9 9 3 9 9 9 9 8,00

Ordered by importance (Vendors) R1 V1 V2 V3 V4 V5 V6 V7 AVG 1 9 9 9 1 9 9 6,71 R2 2 9 9 9 2 1 2 4,86 R3 9 9 9 1 9 3 3 6,14 R5 9 9 9 9 3 4 9 7,43 R6 9 9 9 9 9 9 9 9,00 R7 3 9 1 2 9 9 9 6,00 R8 9 9 9 9 9 2 1 6,86 R9 9 9 9 9 9 9 9 9,00

Reversed order (High value = High importance) Customers 6,33 6,33 3,83 1,67 2,67 0,00 3,50 2,14 1,00 0,00 Vendors 2,29 4,14 2,86 1,57 0,00 3,00 Table 68 - Quantitative results from interviews with vendors and customers

Appendix Q. Quantitative results from interviews with customers and vendors Page 125 of 125