2012

ResearchEvaluationPaper
NextstepinCRM
Course Lecturer Student Studentno. Date Wordcount :ProfessionalResearchOrientation :Dr.ElsvanderSchoot :MarcelloIlmer :6382398 :26February2012 :2236(excludingreferences)

[COMPARINGMETHODS]

Contents
Abstract.....................................................................................................................................2 Introduction..............................................................................................................................3 Literaturereview ......................................................................................................................4 . Article1QuestionnairestudybyFranke&Sjah(2003)......................................................4 Article2ExperimentbyFranke&Piller(2004)...................................................................5 Article3CasestudybyJeppesen&Frederiksen(2006).....................................................7 Researchmethodcomparison..................................................................................................9 Conclusion...............................................................................................................................11 Discussion................................................................................................................................11 References..............................................................................................................................12

Page|1

Abstract
Whencompanieswanttoenternewmarketslackofknowledgecouldbeanentrybarrier, oftenraisedthroughthestrategicR&Dassetsofcompaniesalreadyinthatmarket.However thisbarrierseemstodisappearaccordingtoChesbrough(2003).Headvocatesanewmodel ofOpenInnovation;theboundarybetweenafirmanditssurroundingenvironmentismore porous,enablinginnovationtomoveeasilybetweenthe. In the light of this model this paper discusses three articles about innovation between companiesandtheircustomers/users.Thispapercriticallyreviewstheirresearchmethods totrainthewritersskillstoassessacademicliterature.

Page|2

Introduction
If companies do not innovate they eventually cease to exist; innovation is critical for companies operating in todays markets (Chesbrough, 2003). Chesbrough (2003) suggests that companies should adapt to a new way of innovating: Open Innovation. The key assumption of this model is that companies should be more open in their innovation processtowardstheexternalenvironment.Thisiscontrarywithmostoftodaysinnovation; muchoftheinnovationdonereliesontheStructuralInnovationParadigm.Itisfocusedon delivering a better product or services to customers faster and cheaper than competitors (Simanis&Hart,2009).ManyofthesesolutionsarefrommoneyconsumingcompanyR&D facilities;thedirectinteractionwithexternalsourcesislimited. Basic marketing knowledge tells us that companies want to fulfill customers needs and wantsthroughinnovationsontheirproductsandservices.Howevercompaniesfirstneeds to translate those needs into a product or services. For this translation there is access required to user information; sticky (also called tacit) information. The transfer of this information is involved with high costs for the manufacturer (Von Hippel 1994). For this reasoncompaniestrytococreateproductsorproductinnovationswiththeircustomersas awaytoreducecosts,andtoprovideasolutionwhichfitstheneedofthecustomer. This paper discusses three empirical studies about user innovation. The paper critically reviewsandcomparestheacademicarticleswithspecialfocusontheresearchmethods;all three papers have a different method. Finally this papers ends with conclusion about the valuesofthestudies,againspecificallyonthewaythestudiesgathereddata.

Page|3

Literaturereview
Inthischapterthethreearticleswillcriticallyreviewed.First,aquestionnairestudyabout supportinginnovativeactivitiesbycommunities.Thesecondpaperisanexperimentfocused user innovation through toolkits. The last paper uses a case study design and addresses innovationinfirmhostedusercommunities. Allthreearticlesquiteoftencited;accordingtoGoogleScholarDatabase(2012)thearticles have been 586, 229 and respectively 273 times cited. Keywords that the articles have in commonare:innovation,userinnovation,communities.Thenextchapterwillcompare theresearchmethodsandapproachesusedbythestudies.

Article1QuestionnairestudybyFranke&Sjah(2003)
Published in 2003 by Elsevier, Franke & Sjah conducted a questionnaire study prior. This exploratory study aims at a better understanding of the innovation process of (lead) end userinusercommunities. Firsttheauthorssetup twobasiccriteriaforchoosingcommunities.First,thecommunity should be involved in innovative activities. The second criterion is the differencebetween the communities; the selected communities should differ in structure and user characteristicsformoregeneralizablefindings. After selecting four communities (sailplaning, canyoning, boardercross and handicapped cycling) theauthors questioned thecommunityleaders thebest way to contact individual members. As a result they distributed paper questionnaires to sailplaning, canyoning, and handicapped cycling communities. The members of the boardercross received an email withalinktoadigitalversionofthequestionnaire.

Page|4

The questionnaire had four parts. The first and final parts contained questions regarding personal characteristics and community behavior and attitude. The second part was for innovatorsonly,thethirdformemberswhoassistedintheinnovationdevelopment.These twopartsfocusedontheinnovationprocessandhowtheindividualswereinvolvedinthe process.Aftertwoweekstherespondentsreceivedareminderbyemail,mail,telephoneor viapersonalcontact.Thesurveywasanonymousandhadanoverallresponserateof37,8%. The first part of the questionnaire did not deal with innovation at first, to prevent non innovators for not responding. The explicit difference between innovators and assists mightenhancethepossibilityforselfselectioninfavorforproudinnovators,howeverthe authorsperformedatwotailedttestandthedifferencewasnotsignificant. The authors believe that their findings are quite generalizable. The finds may be representativeinthesportingcommunitiesbutoutsidetheexternalvalidityisexpectedto be far lower. Internal validity appears to be OK but is quite focused on the innovators. Howevertheyarguethatthereisnobiasinfavoroftheinnovatorsonemightdoubtifthat is truly the case. Respondents may exaggerate their contributions or roles within the innovationprocess.

Article2ExperimentbyFranke&Piller(2004)
In this study Franke & Piller performed an experiment to analyze the value created by toolkits for user innovation. The outcomes of this study were published in 2004 in The JournalofProductInnovationManagement. Theauthorsconductedatotaloffourexperimentstoquantifytoolkitgeneratorvalue.They designed a relative simple toolkit for a high heterogeneity product area: Swatchtype

Page|5

watches. Inthe toolkit users engage in designactivities ofthe visual aspects of the watch design.ThetoolkitisoperatedbythecompanyIdtown. Inthefirstexperiment(A)300graduatelevelmanagementstudentwereasked:Wouldyou beinterestedintakingpartinashortresearchexperiment?Youwillgetawatchinreturn. (Franke & Piller, 2004). The high acceptance rate (55%) likely is explained by the high incentive. After the participants designed their watches they were asked to indicate how much they wanted to pay for the selfdesigned watches compared to a standard watch (willingnesstopay;WTP).Inasecondexperiment(B)anothersamplewasaskedabouttheir WTP, but now for the selfdesign watches of experimentA and for the standard watches. Both of these experiments were repeated, but now the WTP was measured by Vickrey auctions.In aVickreyauctionparticipantsplacesealedbidsinwhichtheitemgoestothe highestbidder.TheresponseratesofexperimentB(n=248),C(n=102)andD(n=202)were respectively50,45and50percent.Duetheexploratorynatureofthestudyacompletely random sample was not necessary, however the participant represented the major target groupofIdtown. Asacknowledgedbytheauthorsthedataisbiasedinfavorofyoungpersonswhocanfind theirwayintheInternet.Theinternalvalidityofthisstudyishigh,butitveryrestrictedinits generalizability.Withregardtotheinternalvalidityonemightarguethatthehighincentive for experiment A and B may be of influence in the results. Another incentive, or at least moreequallyspreadbetweentheexperiments,couldhavepositiveinfluenceintheinternal validity. Another point against the incentive is the reproducibility of the study, in other productsettingstheincentivemaybetooexpensiveorimpossible.

Page|6

Article3CasestudybyJeppesen&Frederiksen(2006)
In 2006 the journal Organization Science published a study performed by Jeppesen & Frederiksen. They studied the personal attributes of innovative users in firmhosted communitiesinordertoexplainvaluecreationinorganizationalcontext. For this case study they studied the user community of Propellerhead Software ( a manufacturer of computercontrolled music instruments). Propellerhead benefits from complementaryinnovationsmadebytheir(digital)communityusers. Inordertoenhancetheprecisionoftheirconclusiontheyusedmultipledatasources.Inline with the exploratory nature of their research, they started with a netnographic approach. Thisgavetheminsightofthelocallanguage,inordertoavoidlatermisunderstanding.They didtheseobservationsforathreemonthperiod,onehouraday.Theseconddatasource wasaWeblogwhichcontainedaboutquantitativeaspectsofthecommunity;activityofthe users, visit frequency etc. Third, they conducted 15 interviews in total with the CEO, developers, managers, and lead users from the online community. These interviews were semistructered. Nine of them were conducted before the questionnaire (fourth data source), and six after the questionnaire. The fourth and final data source was an online survey to the Propellerhead community. The purpose of the survey was to collect data aboutthepersonalattributesofthecommunityusers,especiallytheinnovativeleadusers. Thequestionnairehadatotalof442responses,whichresultsina67,2%responserate.The authorshadtodrop97responsesduemissingvalues.Theyacknowledgepossiblebiasesin webbased surveys, therefore they compared the last 10% respondents with earliest 10% respondentstotestitforhigherfrequencyofinnovativeusers.

Page|7

The use of triangulation enhances the construct validity (Gibbert & Ruigrok, 2010) of the study and is a strength of research done by the authors. Because they understood the language used in the community they were able to use this language in the surveys and interview.Togetherwiththeuseofmultiplemeasurementmethodstheintervalidityofthe paper appears to be high. The external validity may be addressed as low. The community theychoseisquitespecificandputsrestrictionsonpossiblegeneralizations.

Page|8

Researchmethodcomparison
Thischapterwilldiscussandcomparethethreeacademicpaperswitheachother. All three papers aim for a better understanding of the userinnovation process and try to findgeneralizablefindings.Howevertheyacknowledgethattheirresearchisofexploratory nature,whichsoundsquitecontradicting. ThefirstarticlediscussedinthisevaluationpaperisthatofFrankeandShah(2003). They used questionnaires in an exploratory study, according to Saunders (2009, pp.362) this methodisnotparticulargoodtouse.Itsmoreappropriatetouseindeductiveresearch.Its a very popular method since it enables you to gather a lot of data and allows you to generalizefindings.Prerequisiteisarepresentativesample.Thisisnotthecaseofthefirst article. It might be a representative sample for the sporting communities, but outside the sportingcommunitytheexternalvalidityislikelynottobehigh.Theauthorshowevertryto excludeanybiaswithextrastatisticaltesttocheckifthesampleisfreefrombias.Another disadvantagebesidesarepresentativesampleispossiblebiasinrespondentsanswers.They mightgivesociallydesirableanswers(Saunders,2009,pp.365)ortheclosedquestionsjust limittherespondentsanswers. Thesamecanbesaidforthesecondarticle,inwhichtheauthorsconductedanexperiment. Anexperimentisthemotherofallresearchdesignsandoffersagreatdealofcontrolof theresearchvariables.Inotherwords;theinternalvalidityishigh,thedownsideisthatthe externalvalidityishardertoestablish(Saunders,2009,pp.142142).Anotherpointofcritic inthisstudyistheuseofstudents.Studentsmightbenotthemostrepresentativegroupof participantsforthepopulation.Thechoiceforstudentonlyfurtherrestrictstheauthorsin generalizingtheirfindings.
Page|9

The third and last paper uses a case study research design. As Saunders (2009, pp.146) describes the case study is convenient for exploratory studies. The case study allows a deeperunderstandingofthecontextandprocess(Morris&Wood,1991).Thatswhatthe authorsweretryingtodo.Anotheradvantageofthecasestudyistheusageofmultipledata sources.Thetriangulationofmultipledatacollectiontechniquesenhancesthecredibilityof findings. A consequence of the case study is the limited external validity. The authors encounterthisproblemandacknowledgeit,theysuggestfurtherresearchincommunities inmorephysicalsettings.

Page|10

Conclusion
Asthearticleshavecriticallybeenreviewedsomeconclusionscanbedrawn.Thedifferent researchdesignsdonothavetobelabeledgoodorbad,butsomemaybemoreappropriate incertainsettings. Basedonthecomparisonamultiplecasestudywhichusestriangulationwouldbeagreat researchdesigninuserinnovationresearch.Thethirdstudyhasagoodinternalvalidity,but it lacks external validity. By selecting multiple cases findings probably result in better generalizable findings. The survey provides a lot of data, but the sample is in this matter quiterestricting,whichisaproblemforallthethreestudies. Apointofinterestismotivationtoparticipateinresearch.Respondentsorparticipantmight bedrivenbytheincentiveorbyshowingof(innovatorsinthefirststudy).

Discussion
Reviewingtheresearchmethodsofseveralarticlesinafieldgivesnewinsights.Throughthe lens of a critical research methodologist new aspects of academic research came to my attention.Itbecomesmoreevidentthattheresearchquestionshouldthefittheresearch strategyandmethod. An interesting field to explore is the use of social media communities to innovate on products. There might be huge potential in the billions of people who participate in networksasFacebookandLinkedin.Inparticularspecializedgroupswithinthesenetworks such as car hobby groups where car are being review; it could be an opportunity for automotiveindustrytoanticipateonexperiencefromusers.

Page|11

References
Chesbrough,H.W.2003,Openinnovation:Thenewimperativeforcreatingandprofiting fromtechnology,HarvardBusinessPress. Franke,N.&Piller,F.2004,"Valuecreationbytoolkitsforuserinnovationanddesign:The caseofthewatchmarket",JournalofProductInnovationManagement,vol.21,no.6, pp.401415. Franke,N.&Shah,S.2003,"Howcommunitiessupportinnovativeactivities:anexploration ofassistanceandsharingamongendusers",Researchpolicy,vol.32,no.1,pp.157 178. Gibbert,M.&Ruigrok,W.2010,"TheWhatandHowofcasestudyrigor:threestrategies basedonpublishedwork",OrganizationalResearchMethods,vol.13,no.4,pp.710 737. Jeppesen,L.B.&Frederiksen,L.2006,"Whydouserscontributetofirmhosteduser communities?Thecaseofcomputercontrolledmusicinstruments",Organization Science,,pp.4563. Morris,T.andWood,S.1991Testingthesurveymethod:continuityandchangeinBritish industrialrelations,WorkEmploymentandSociety,Vol.5,No.2,pp.259282. Saunders,M.,Lewis,P.,&Thornhill,A.(2009)Researchmethodsforbusinessstudents,5th edition.Harlow,England:PearsonEducationLimited. VonHippel,E.1994,""Stickyinformation"andthelocusofproblemsolving:Implicationsfor innovation",Managementscience,,pp.429439. Othersources GoogleScholarDatabase(scholar.google.com)

Page|12