Mystery Shopping Guide November 2006

/ sa/0ov/u1 All ri0hts reserved . !ondon. Canary Whar .© Financial Services Authority 2006 25 The North Colonnade. ($$ )0*20 +066 #0-Website' . "#$ 5%S Tele&hone' ($$ )0*20 +066 #000 Fa...

een retros&ective consumer research 2 e/0/ amon0 those .tensively in many industry sectors.ith our rules amon0 irms/ We ty&ically conduct mystery sho&&in0 because it ills the 0a& bet. to measure the 8uality o service &rovided/ We are not alone in underta1in0 inancial services mystery sho&&in0/ 3any lar0e inancial services &roviders use mystery sho&&in0 to research their com&etitors and their o.ay inancial &roducts are sold to consumers/ We use a ran0e o tools to measure &olicy im&acts and assess levels o com&liance .&eriences 2 and ormal su&ervisory visits to re0ulated irms to assess their advice and sales documentation/ .info#FSA#ht l# handbook#S$%#2#4 What is mystery shopping? The 3ar1et 4esearch Society )34S* de ines mystery sho&&in0 or 5mystery customer research6 as' &The use of individuals trained to e'perience and easure an! custo er service process( b! acting as potential custo ers and in so e )a! reporting back on their e'periences in a detailed and ob*ective )a!. such as retail and the motor trade.n sales sta . and . to im&rove customer services and to hel& ensure they are treatin0 customers airly/ Why does the FSA use mystery shopping? 3ystery sho&&in0 0ives us a detailed insi0ht into the .+ 3ystery sho&&in0 is a lon07established research techni8ue and is used e.e 2 the Financial Services Authority )the FSA* 2 conduct mystery sho&&in0 and use the indin0s/ The Supervision Manual in the FSA Handbook (section 2.4) gives further background on !ster! shopping" http"##fsahandbook.ho have recently bou0ht a &articular &roduct or used a s&eci ic sales &rocess to discover their e.hy .&lains ho.A Guide to Mystery Shopping in Retail Financial Services This short 0uide e.

hat . assess levels o irms6 com&liance .hether retail irms treat their customers airly/ 9n &articular. or to hel& evaluate the e ectiveness o our &olicies and assess the e.n an evolving regulator! environ ent( there is an increasing need for us to gather evidence on )hether our retail polic! initiatives have had the intended outco es and )hether retail fir s treat their custo ers fairl!.hether our retail &olicy initiatives have had the intended outcomes and .ho have recently bou0ht &roducts 2 consumers don6t al.e already have concerns about com&liance and .here .hich our statutory ob.or1/ We . .e try to minimise the number o times a 0iven irm is 5sho&& .&erience consumers have o the retail mar1et/ We re0ard mystery sho&&in0 as a necessary means o 0atherin0 such in ormation/ This is because o the &roblems inherent to surveyin0 those .e o ten use mystery sho&&in0 to com&lement our &olicy and thematic .ish to con irm and identi y the 1ey issues so .amine the e. but they do not &rovide the unedited version o .ay in . includin0 direct sales and inde&endents/ There may be other instances .e need to measure and evaluate the im&act o FSA &olicy. there is an increasin0 need or us to 0ather evidence on .ays accurately recall all the details and it is im&ractical to 0ather the re0ulated disclosure material irms 0ive them/ :ur su&ervision visits to irms ca&ture ormally documented in ormation about sales &rocesses. or .ish to use retail mar1et intelli0ence to sam&le &articular ty&es o irms that are more li1ely to e. .o reasons 2 either to carry out research in an area that has been identi ied as &otentially hi0h ris1.ay that is &ro&ortionate to . but in more s&ecialist mar1ets and or very lar0e irms.hat irms actually say in an ostensibly 5real sales6 situation/ What might prompt the FSA to undertake a mystery shopping exercise? We ty&ically use mystery sho&&in0 or t.ercise. . .ill need to ca&ture a 0ood cross7section o irms &rovidin0 advice and &roducts in the mar1et under scrutiny/ This means 0ettin0 a mi.or &layers in the mar1et/ We mi0ht also .ould not choose to conduct mystery sho&&in0 s&eculatively.e are investi0atin0/ Sometimes . o small and lar0e irms.tent to .or1..e only need to ocus on the ma.ectives are bein0 met/ We ensure there is a clear business need or mystery sho&&in0 and consider alternative a&&roaches i these are more li1ely to achieve the intended &ur&ose/ How do we select irms to !e mystery shopped? Where .hich individual irms are selected de&ends on the nature o the mar1et .e .ith our rules and e. re&eat sho&&in0 is sometimes unavoidable/ .e are tryin0 to achieve/ The .e mi0ht discover &roblems/ 9nstead.hibit non7com&liant behaviour/ 9deally.e ta1e ste&s to ensure it is conducted in a .e decide it is necessary to carry out a mystery sho&&in0 e. on the o 7chance that .e can conduct more ocused su&ervisory . 9n an evolvin0 re0ulatory environment.

the sho&&ers .hen a &articular mystery sho& too1 &lace or the name o the individual adviser concerned/ We .or1 to establish .ished.or1 ocusin0 on a &articular irm/ For e.in0 due &rocess/ That is the issue o a Warnin0 <ecision and.e . as this hel&s them 1ee& their story consistent and reduces the amount o s&eci ic in ormation they have to remember/ .The ke! to successful !ster! shopping is having si ple( believable scenarios for the shoppers to follo).ect by matchin0 their &ersonal characteristics and inancial circumstances .hy these ailin0s occurred and the &otential ris1s to consumers/ =ltimately.ith their irms/ :ur su&ervisors may have a discussion .hich .ithout ollo.een ailin0s that are attributable to a small number o under7&er ormin0 individuals and those that are more systemic/ How does mystery shopping work? The 1ey to success ul mystery sho&&in0 is havin0 sim&le.n &ersonal circumstances.ithin a irm. . .ould tri00er a need or a &articular &roduct or service. a Final Notice/ FS3A &rohibits us rom &ublicisin0 any Notice other than the Final Notice/ "an mystery shopping results !e used in any re erral to en orcement against a irm? Where mystery sho&&in0 indicates com&liance &roblems .ere to conduct a lar0e scale mystery sho&&in0 e. .ith invaluable eedbac1 to share . this may lead to en orcement action bein0 ta1en.ith our e.ant to 8uanti y the e.e could.ill do ollo.ercise/ %o.ill &resent this anonymously. to0ether .e mi0ht . althou0h or lar0er irms it may be &ossible to mention the branch)es* concerned/ We need to &rotect the identity o the mystery sho&&er so they can carry out mystery sho&&in0 in the uture/ So.7u& . believable scenarios or the sho&&ers to ollo. thou0h it is unli1ely in &ractice that a decision to investi0ate .e o 0ood and bad &ractice that the mystery sho&&in0 identi ied/ We . i .in0 any re erence to the Financial Service and 3ar1ets Tribunal. .am&le.ith the re8uirements o the scenarios to be used/ Wherever &ossible. How does the FSA use mystery shopping indings? 3ystery sho&&in0 evidence does not intend to 0ive us re&resentative data on an individual irm.e 0ive the sho&&er a set o circumstances .e .ith the irms they re0ulate. ollo.&ectation o .ever.tent to .e tend not to 0ive &recise details o . sharin0 the the adviser should do as a result/ We identi y the mystery sho&&ers or a &ro. carry out more in7 de&th mystery sho&&in0 . it can &rovide our su&ervisors . unless . .hich certain sellin0 &ractices &revail or to hel& distin0uish bet.&ect .or1 i the issue to be researched is not clearly de ined/ Scenarios are not scri&ts./ The scenario is made as realistic as &ossible so it mimics natural consumer behaviour in the mar1et concerned and can be enacted convincin0ly by the mystery sho&&er/ 3ystery sho&&in0 is unli1ely to .ill not ma1e &ublic details o the irms concerned in any mystery sho&&in0 as it is a basic &ro&osition in FS3A that the FSA should not ma1e a &ublic indin0 o ault on the &art o a re0ulated irm .ill be based solely on such results/ =nder very rare circumstances.e .ill use their o. but rather a brie in0 or the sho&&er that &rovides a &lausible reason or their en8uiry/ For e.

n being proportionate( )e have to achieve the right balance bet)een the cost to ourselves( the cost to the industr! and the level of coverage re/uired to generate useful conclusions.ill 0enerally include a &ilot sta0e to chec1 that the scenarios ma1e sense in the real .een the cost to ourselves.ects have recorded intervie.e cannot necessarily assume that all irms behave in a similar . many o our mystery sho&&in0 &ro.e ma1e any necessary chan0es and the main ield.ith irms/ .ay to the research indin0s/ #o we record the shop? Ces.e &roduce t.ithout over7burdenin0 irms and .ects.hole/ That is.s .e have conducted so ar has been more 8ualitative than 8uantitative in nature/ Bualitative research is used to 0et the ran0e o behaviour or attitudes &resent in a &articular &o&ulation and N:T to measure the &revalence o these behaviours or attitudes across the mar1et &lace as a . most o the mystery sho&&in0 . .ays to ma1e the research it or &ur&ose/ 9n bein0 &ro&ortionate.e .ectives dictated it/ 9n our more recent &ro.in0 their visit@tele&hone call/ :ur mystery sho&&in0 &ro.ectives.ould only use it i our research ob.&ensive and time7consumin0 than 8ualitative research and .ect/ :ur aim is al.orld and to test .or1 be0ins/ How !ig are mystery shopping samples? There is no sin0le &rescribed sam&le siAe or a mystery sho&&in0 &ro.hich 0ive the sho&&er the bac10round and rationale or each scenario? and the eedbac1 8uestionnaire or the sho&&er to com&lete ollo. .o main documents' > > the scenario brie in0 notes . the cost to the industry and the level o covera0e re8uired to 0enerate use ul conclusions/ <ue to the nature o our research ob. 8uantitative research is more e. 8uantitative research is used to measure the &revalence o behaviours or attitudes and there ore involves lar0e sam&les that are re&resentative o the tar0et &o&ulation/ <ue to the much lar0er sam&les re8uired. they are desi0ned to re lect the tar0et &o&ulation by com&risin0 a s&read o the di erent ty&es .ith 0ood 8uality research7based evidence .hole/ So.ur ai is al)a!s to ake the research fit for purpose. .ithin the &o&ulation/ 9n contrast. 8ualitative sam&les are not )and are not meant to be* statistically re&resentative o the &o&ulation bein0 researched/ 9nstead.ects . . .hich themselves re lect the mar1et rather than &rovide statistically robust indin0s or the mar1et as a .hether the brie in0 notes and eedbac1 8uestionnaire cover all the issues ade8uately/ A ter com&letin0 the &ilot. . . a sam&le o around +07#00 mystery sho&s has &rovided us .There is no single prescribed sa ple si-e for a !ster! shopping pro*ect. To conduct the mystery sho&&in0.ithout bein0 too costly or us/ These sho&s are allocated across a ran0e o di erent irms chosen to re lect the characteristics o the mar1et in 8uestion/ These 8ualitative sam&les &rovide an indication o current &ractices amon0st the irms sam&led.e have to achieve the ri0ht balance bet.

e are best &laced to ma1e .e are im&osin0/ Eet.e can 0ive irms clear and actionable eedbac1. .as said/ The recordin0 can also &rovide clear evidence or su&ervision &ur&oses so .hat a comment on the sho&&er6s eedbac1 8uestionnaire re ers to/ :r . Fact$ We .hen ans. e. .e do not share the recordin0s .ther 1 publications#2onsu er#inde'.e consider .ard ha&&ened. Fact$ 4ecordin0 is the most accurate and air .ers &ermit us to record sales situations/ .ercise . in order to dischar0e our statutory unctions .e .#00 sho&s across a ran0e o mar1ets/ This re&resents only a tiny raction o the time s&ent by advisers meetin0 the needs o 0enuine customers/ Myth$ It is rong for the FSA to record these . the sho&&er may thin1 that a sho& has been com&leted. ul/ Ey recordin0.e 0et the actual .ords s&o1en by the adviser &lus the tone and em&hasis .ectives/ :ur main reasons or underta1in0 mystery sho&&in0 are concerns over &otential com&liance &roblems in a 0iven mar1et or sector.ell ne.hom re8uirements are &laced under the Financial Services and 3ar1ets Act )FS3A*.ercise/ 9t is an im&ortant &art o the 8uality assurance &rocess and 0ives our su&ervisors unambi0uous and actionable in ormation .e need to 1no.hether it is &ro&ortionate.ith re0ulated irms &rovidin0 it is not other.ith irms/ :ur re0ulatory &o.ith irms/ How do we analyse and report on the results? A undamental sta0e in the analytical &rocess is determinin0 ho.nce the anal!sis is co pleted( )e usuall! publish a )ritten su ar! report of the research findings on the consu er research publications page of our )ebsite" )))..actly .ers to the 8uestionnaire su00est somethin0 unto.e may ind a di erent outcome .ud0ements o this 1ind/ Some popular myths de!unked Myth$ The FSA likes to use mystery shopping as a “fishing expedition”.hich .e conduct a mystery sho&&in0 e. Fact$ Where . .uk#%ages#0ibrar!#.e can use . rules or rule chan0es have bedded in/ Myth$ Mystery shopping poses an unnecessary burden on firms. .hich they are said/ This can be very use ul or 8uality assurance &ur&oses/ For e.sht l We are re8uired to maintain arran0ements to determine . .clude any incom&lete sho&s rom the inal re&orted sam&le/ We do not dele0ate res&onsibility or this as&ect o 8uality control to the research a0ency as .e may need to discover . but on closer ins&ection . a sho&&er may believe that investment advice .ei0h u& .hether &ersons on .ith them/ 3ystery sho&&in0 is one such method mentioned in the Su&ervision 3anual 2 see sections 2/$/D and 2/$/$/ There ore. i/e/ .e are tryin0 to achieve a0ainst any burden .e can record . or the need to test ho.hat .as &rovided .ects/ 9n not/ We e.e are &ermitted to record mystery sho&&in0 conversations .hat .hen it .een Fanuary 2005 and Au0ust 2006.hat too1 &lace durin0 a mystery sho&&in0 e.e carried out around achieved/ For e. .ise unla. many eli0ible sho&s actually too1 &lace/ For instance.fsa.e undertoo1 ten mystery sho&&in0 &ro.ill only use mystery sho&&in0 i other methods o in ormation 0atherin0 .ith . althou0h . are com&lyin0 .am& not enable us to achieve our ob.

Sign up to vote on this title
UsefulNot useful