A MATURITY ASSESSMENT MODEL TO BENCHMARK VIRTUAL

COMMUNITIES OF PRACTICE: A STUDY IN THE OIL AND GAS INDUSTRY

Alejandro Germán Frank

Federal University of Rio Grande do Sul, Brazil

Luca Gastaldi, Emanuele Madini, Mariano Corso

Politecnico di Milano, Italy

luca.gastaldi@polimi.it

ABSTRACT
This paper proposes a model to assess the maturity level of companies’ organ-
isational, managerial and technological levers for Virtual communities of
practices (VCoP). The proposed model serves as a benchmarking tool in order
to analyse the performance of firms in the management of VCoP. The model
has been applied in four of the major Oil & Gas global companies focusing on
their Exploration & Production practices.

Keywords: Communities of practice, Maturity assessment model, Oil and gas

industry

1. INTRODUCTION
The concept of Communities of Practice (CoP) was introduced by Wenger and Leave
(1991), who referred to a group of people sharing knowledge, problems, solutions, in-
formation and news about a specific issue, and who extend group learning through re-
ciprocal interaction (Wenger et al., 2002). At the beginning, CoP used to be built by
means of a personnel interaction among members, but this changed with the advance of
technologies. Today, there are also Virtual CoP (VCoP), which is referring to a group of
people who interact, learn together and build relationships through specific social media,
potentially crossing geographical and political boundaries in order to pursue mutual in-
terests or goals, developing a sense of membership and reciprocal commitment (Leave
and Wenger, 1991; Kowch and Schwier, 1997; Wenger et al., 2002; Jeon et al., 2011).
VCoP is the community approach that is attracting more interest in the business envi-
ronment, mainly in large companies. This is because it helps to bring near experts from
different sectors and regions, enabling them to build a common base of knowledge
among people dispersed in different places (Song et al., 2007; Montoya et al., 2009).
Following such approach, VCoP have become a common knowledge management prac-
tice in Oil & Gas companies. In this sector, VCoP are seen as one of the most powerful
knowledge management tool (Corso et al., 2009), since teams and experts are frequently
dispersed in different facilities and different countries. Thus, VCoP are a way to reduce
displacement costs of experts that traditionally should be physically present in order to

ISBN 978-90-77360-17-0 © CINet 2014 287

help solving a specific problem. Moreover, VCoP are considered a way to share best
practices adopted by some experts that achieved a successful result, aiming to standard-
ize the organisational processes based on the most successful practices and tools used.
In the Oil & Gas industry there are different approaches to VCoP. Some companies have
adopted models oriented toward individual initiatives, without a centrality of the VCoP
management functions. Other companies have adopted structured models for VCoP, in
which member activities are coordinated by a central management team. Consequently,
there is not a unique strategy to address VCoP (Corso et al., 2012). Prior research has
studied the peculiarities of the VCoP in this sector (e.g. Edmonds, 2002; Scarso et al.,
2009) and it is noted that there is a lack of standardization and systematization of the
best practices used for VCoP. Moreover, such a lack of systematization of best practices
drives to another issue, which consists in measuring the maturity level of VCoP man-
agement in a specific company of this sector when compared with its competitors.
The aim of this paper is to propose a model to measure the maturity level of companies’
organisational, managerial and technological levers for virtual communities of practices
(CoP), based on best practices proposed for VCoP management. The proposed model
serves as a benchmarking tool in order to analyse an industry sector performance in the
management activities of VCoP. The main contribution of this paper is that an assess-
ment carried out by the proposed model provides an overview of the strengths and
weaknesses of the VCoP management system, and the opportunity for comparison with
other organisations. Moreover, the assessment also enables the identification of a series
of potential actions that can be undertaken to improve the current status of the VCoP.

2. MODELS TO EVALUATE VCOP

There are several perspectives in which the CoP are evaluated. The academic literature
has proposed some models, frameworks and domains that should be considered for a
community evaluation. Basically, Wenger et al. (2002) identify three common charac-
teristics of CoP that should be considered:
• Domain: the area of interest;
• Community: relationships and the development of a sense of reciprocal commitment;
• Practice: the shared repertory of competences and resources developed by members.
Based on these principles, some authors have developed different models for CoP eval-
uation. For instance, Kim et al. (2012) developed a diagnosis framework that help iden-
tifying the current knowledge sharing activity status in a CoP. Lee et al. (2010) pro-
posed an evaluation of CoP based in an evolutionary process. Bertone et al. (2013) pre-
sented a conceptual framework for CoP assessment in health policy; while Loyarte and
Rivera (2007) proposed a model for CoP cultivation. Chu and Khosla (2007) proposed
index evaluations for CoP. Probst and Brozillo (2008) studied the success and fails of
CoP, aiming at identifying best practices for CoP management.
These works help in identifying several factors and characteristics that should be con-
sidered in order to provide a clear understanding of weaknesses and strengths of a CoP.
These models present different perspectives, since some of them look at CoP assess-
ment from a perspective of growing, and others from a perspective of index that should
be implemented. Both point of views help us to build an assessment framework that
help for a benchmarking in the Oil & Gas industry, aiming at evaluating companies’
best practices and performances in the use of VCoPs.
ISBN 978-90-77360-17-0 © CINet 2014 288
3. RESEARCH METHOD
The methodological approach was divided into two main stages:
• The development of the VCoP Maturity Assessment Model, and
• An application of such model to different realities (benchmarking).
First, the maturity model was developed based on the study of international literature
and the analysis of implementations of knowledge management systems and communi-
ties. Several CoP characteristics and best practices have been analysed from Kim et al.
(2012), Lee et al. (2010), Bertone et al. (2013), Loyarte and Rivera (2007), Chu and
Khosla (2007) and Probst and Brozillo (2008), Corso et al. (2009).
Based on these characteristics for CoP assessment, we developed a model based on the
main common issues mentioned in such works. Complementary practices and character-
istics cited only by few works were also added. The model proposes the assessment of
eight key elements to measure the company’s organisational, managerial and technolog-
ical maturity in order to find a correlation with the performance and business impact of
the VCoPs. These eight VCoPs’ key elements are: culture, sponsorship, architecture
alignment, development, management, policy, technology and community assessment.
On the other hand, the performance of the VCoP is evaluated based on four main crite-
ria: utility, trust, contribution and sense of belonging.
After the development of the model, we carried out an application of it in a bench-
marking study of six of the major Oil & Gas global companies focusing on the Explora-
tion & Production sector of such companies. In order to meet the benchmarking objec-
tives, the collection and analysis of data were conducted using different methodologies:
• An online survey of 22 multiple-choice questions addressed to 14 KM managers with
the aim of assessing the VCoP maturity level according to the aforementioned elements;
• A set of in-depth interview to validate the data collected and to identify the most
relevant initiatives supporting the VCoPs management;
• A collection of documents and presentations describing the Knowledge & Commu-
nity Management System.
The benchmarking panel includes some of the major global companies, comparable in
terms of size (number of employees) and internationalization level. In particular, the
analysis is focused on the use of Knowledge & Community management in the Explo-
ration & Production Unit of Oil & Gas companies

4. RESULTS

4.1 A MODEL TO ASSESS THE MATURITY OF VCOP

Our model proposes the assessment of eight key elements to measure the company’s or-
ganisational, managerial and technological maturity in order to find a correlation with
the performance and business impact of the VCoPs. These eight VCoPs’ key elements
are: culture, sponsorship, architecture alignment, development, management, policy,
technology and community assessment. On the other hand, the performance of the
VCoP is evaluated based on four main criteria: utility, trust, contribution and sense of
belonging. Following, we describe these elements showed in Figure 1.

ISBN 978-90-77360-17-0 © CINet 2014 289

 Figure 1. VCoP assessment model

4.1.1 ASSESSMENT OF VCOP PERFORMANCE

The performance of the VCoP is evaluated through 4 variables:
• Utility: users believe in the utility and effectiveness of the communities for their
own activities;
• Trust: there is strong sense of trust between the members of a community;
• Sense of belonging: members have offline relationships and connections among
themselves, and a strong sense of belonging to the community;
• Contribution: members participate in the community in a continuous and systematic
manner.

4.1.2 ASSESSMENT OF VCOP LEVERS

The KM Maturity Assessment Model considers 8 variables in order to evaluate the
company’s organisational, managerial and technological maturity.

4.1.2.1. CULTURE
The analysis of the “culture” evaluates the attitude of top management, middle man-
agement and the employee towards the following organisational behaviours supporting
Knowledge & Community Management:
• Emerging collaboration (cross-functional and independent of predefined hierarchies)
between people and internal experts;
• Openness to knowledge-sharing with actors in the company’s extended network
(customers, partners, suppliers, consultants, etc.);
• Co-creation, creation of broad participatory content and knowledge by people to
foster widespread innovation;
• Sociality, enhancement of interpersonal relationships and stimulation of the creation
and management of extended networks of contacts;
• Transparency and opportunity to communicate freely in a climate of trust and be-
longing to the organisation;
• Flexibility to change, capacity and speed of process and activity changes in response
to shifts in business needs and context.

ISBN 978-90-77360-17-0 © CINet 2014 290

4.1.2.2. SPONSORSHIP
Management, countries and business units support in development and use of the com-
munities. Regarding the management’s sponsorship of the VCoP, the analysis considers
two different dimensions:
• Top and Middle Management Involvement, i.e. sponsorship toward the development
and use of communities;
• Top and Middle Management Commitment in Communities, i.e. whether there is or
not a widespread participation by managers in the launch, promotion and change
management activities; and in contributing to the communities.

4.1.2.3. ARCHITECTURE ALIGNMENT

Definition of objectives and long-term development plans for the communities, and
analysis of the target needs and methods of interaction. The analysis in this category
considers two different dimensions:
• Target-needs alignment, i.e. domain of the VCoP (common themes and problems,
expertise of potential members, tools and language to be used, etc.), the needs of the
target users, the methods of interaction and the informal networks of the target users
(e.g. organisational network analysis), the company objectives and the needs of
community members, identifying shared goals;
• Business alignment, i.e. long term development plans with objectives that are
aligned with corporate strategy; a clear mission for each community defined and
communicated (e.g. technical improvement, process innovation, training, etc.); top
and middle management involvement in the community concept definition (goals,
scope, domain, etc.).

4.1.2.4. DEVELOPMENT
Approach for the development of the communities in terms of design, activities, roles,
people and country involvement, technology tools and change management. Regarding
this construct, the analysis considers two different dimensions:
• Open and cross-organisational participation, including people with expertise in
communications, organisational and business process, in addition to technical roles;
identification and involvement of internal “champions” in the development phase of
the community (i.e. users with some experience in the use of community tools and
promoters of change); collection and acceptance of independent initiatives by em-
ployees for the creation of new communities; and various countries involved in the
definition and design of the communities;
• Structured roles and activities defined with a team dedicated to the technical devel-
opment of community tools and a launch and change management plan for each
com-munity established.

4.1.2.5. COMMUNITY MANAGEMENT

Regarding the management of the communities, the analysis considers two different
dimensions:
• Animation and incentive activities, like strategies to promote the community (pro-
motional videos, communication campaigns), activities to involve the community
members and to stimulate contributions (editorial plans, contests), specific commu-
nication and engagement activities defined for the various countries, offline Meet-

ISBN 978-90-77360-17-0 © CINet 2014 291

 ings and events organised for community members (work-shops, conventions, etc.),
open and free policies of use are established (free access to community, contribu-
tions without moderation, etc.), activities for training and support on community
tools; economic incentives tied to user participation and contribution in the commu-
nities (e.g. bonuses, benefits, MBO); formal recognition tied to participation and
contribution in the communities (e.g. awards, contests, visibility in the community,
etc.), organisational policies and procedures to increase user participation (ex. eval-
uation work-flow of business application, request;
• Structured roles and activities, like community manager and/or a central team (KM
team) that coordinates the various initiatives, the roles and the coordination mecha-
nisms regulating community management (e.g. community leaders, facilitators, pro-
cess owners, KM champions and area experts), adequate training and enough time
dedicated to VCoP management activities, use of tools and initiatives for sharing
best practices among the people who manage the communities, and monitoring of
the evolution of the community over time.

4.1.2.6. POLICY
Regarding community use policies, the analysis considers two different dimensions:
• Accessibility, i.e. all employees can obtain access to any community through a sim-
ple registration process;
• Transparency, i.e. users may freely express their own opinions within the bounda-
ries of corporate etiquette.

4.1.2.7. TECHNOLOGY
The technological assessment considers 3 main categories of IT tools:
• Social network and community, which supports in managing and creating relation-
ships between individuals through tools promoting discussion, the exchange of ideas
and involvement in networks of extended acquaintances, including those beyond
company borders (blogs, forums, social network tools, expert research, advanced
user profiles, etc.);
• Unified communication and collaboration, which supports in managing each type of
communication and collaboration both within and outside the company, uniformly
and independently of the media adopted to transmit contents (web, landlines, mobile,
TV) through specific infrastructures and tools audio/web/videoconferencing, instant
messaging, VoIP, etc.);
• Enterprise content management, which provides support in managing contents and
documents within and outside an organisation through tools that improve accuracy,
accessibility and integrity.

4.1.2.8. ASSESSMENT
Regarding the monitoring of community activities, the analysis considers two different
dimensions:
• Monitoring, which considers the use of a measure of community access and use in a
systematic way; an appropriate KPIs and success metrics; executive reporting car-
ried out in a systematic manner; and an analysis of utilisation levels for the redefini-
tion of the strategy for the development of the communities;

ISBN 978-90-77360-17-0 © CINet 2014 292

• Analysis of the benefits and business impact, which considers standard organisation-
al tools and mechanisms established to identify the benefits and the business impact
of the community system; benefits and business impact of the community system
identified in qualitative terms (e.g. qualitative descriptions, indicators, etc.); benefits
and business impact of the community system identified in quantitative/economic
terms (time/cost reduction, in-creased quality, increased production); and evidence
of the benefits and success stories provided to community stakeholders.

4.2 APPLYING THE VCOP MATURITY ASSESSMENT MODEL IN THE OIL & GAS INDUSTRY
Our proposed model was applied in four leading Oil & Gas companies in order to eval-
uate their VCoP management performance. First, Figure 2 presents the results of VCoP
performance, which was measured by utility, trust, contribution and sense of belonging
thorough the answer provided to an electronic survey confirmed by a set of case studies.

Figure 2. Assessment of VCoP performance

After evaluating VCoP performance, the eight VCoP management levers were also
evaluated, based on the items presented in previous section. Figure 3 shows the main re-
sult about such levers.
As expected by our maturity assessment model, the company that presented the best
performance in results showed in Figure 2 (Company B) was the same that showed the
best condition in terms of levers maturity in Figure 3. The same happens with the com-
pany with the worst VCoP performance in Figure 2 (Company C), which is the compa-
ny that presented the lowest maturity in the evaluated levers (Figure 3), especially in
policy, technology and assessment monitoring.
It is also worth to note that all companies presented higher scores to the utility and sense
of belonging of VCoP, and that the most important problems associated to VCoP per-
formance are related to the lack of contribution of the members and the lack of trust
among communities members (Figure 2). On the other hand, when considering the lev-
ers (Figure 3), technology appears as the most critical lever in which VCoP are still in a
very low maturity stage. In this sense, we collected information about the most used
technologies in the companies analysed. Figure 4 presents the relative results.

ISBN 978-90-77360-17-0 © CINet 2014 293

 Figure 3. Assessment of VCoP levers of maturity

Based on the in-depth interviews we a carried out with KM managers, we classified

VCoP technologies using two axis of scores (Figure 4). The first axis shows the level of
presence or use of a specific technology in the VCoP studied. The second axis shows
the level of contribution or impact of such a technology for VCoP in the Oil & Gas sec-
tor. We classified these technologies based on the two axes in four quadrant: marginal,
commodities, differentiating and killer applications.

Figure 4. Technological tools used in VCoP of the Oil & Gas companies

ISBN 978-90-77360-17-0 © CINet 2014 294

The first two quadrants have a low contribution for VCoP, according to the interviewees.
On the other hand the differentiating and killer applications are the most powerful tech-
nological tools for VCoP in this industrial sector. Technologies in blue colour are those
related to enterprise content management, in orange are those related to social network,
and in green colour the related to unified communication and collaboration.

5. CONCLUSIONS
This paper presented a model to measure the maturity level of companies’ organisation-
al, managerial and technological levers for virtual communities of practices (CoP). The
proposed model serves as a benchmarking tool in order to analyse an industry sector
performance in the management activities of VCoP. The main contribution of this paper
is that an assessment carried out by the proposed model provides an overview of the
strengths and weaknesses of the VCoP management system, and the opportunity for
comparison with other organisations. Moreover, the assessment also enables the identi-
fication of a series of potential actions that can be undertaken to improve the current sta-
tus of the virtual communities.
Our benchmarking study put in evidence the usefulness of the proposed model. Compa-
nies were compared and ranked in terms of the performance of VCoP management.
Such results helped companies to have a clear vision of their own state in management
practices. Moreover, the assessment helped in identifying the best practices to be shared
in order to improve the overall usage of VCoPs within the Oil and Gas industry

REFERENCES

Bertone M.P., Meessen B., Clarysse G., Hercot D., Kelley A., Kafando Y., Witter S. (2013) Assessing
Communities of Practice in Health Policy: A Conceptual Framework as a First Step Towards Empir-
ical Research, Health Research Policy and Systems, Vol. 11, No. 1, p. 39.
Mei-Tai C., Khosla R. (2009) Index Evaluations and Business Strategies on Communities of Practice,
Expert Systems with Applications, Vol. 36, No. 2, pp. 1549–1558.
Corso M., Giacobbe A., Martini A. (2009) Designing and Managing Business Communities of Practice,
Journal of Knowledge Management, Vol. 13, No. 3, pp. 73–89.
Edmonds R (2002) Lessons in eLearning from de Oil and Gas Industry: Learning on Demand, SRI Busi-
ness Inteligence.
Jeon S-H., Kim Y-G., Koh J. (2011) Individual, Social, and Organizational Contexts for Active
Knowledge Sharing in Communities of Practice, Expert Systems with Applications, Vol. 38, No. 10,
pp. 12423–12431.
Kim S.J., Hong J.Y., Suh E.H. (2012) A Diagnosis Framework for Identifying the Current Knowledge
Sharing Activity Status in a Community of Practice, Expert Systems with Applications, Vol. 39, No.
18, pp. 13093–13107.
Kowch E., Schwier R. (1997) Considerations in the Construction of Technology-based Virtual Learning
Communities, Canadian Journal of Educational Communication, Vol. 26, No. 1, pp. 1–12.
Lave J., Wenger E. (1991) Situated Learning, Cambridge University Press, Cambridge.
Lee J., Suh E.H., Hong J. (2010) A Maturity Model Based CoP Evaluation Framework: A Case Study of
Strategic CoPs in a Korean Company, Expert Systems with Applications, Vol. 37, No. 3, pp. 2670–2681.
Loyarte E., Rivera O. (2007) Communities of Practice: A Model for their Cultivation, Journal of
Knowledge Management, Vol. 11, No. 3, pp. 67–77.
Probst G., Borzillo S. (2008) Why Communities of Practice Succeed and Why they Fail, European Man-
agement Journal, Vol. 26, No. 5, pp. 335–347.
Scarso E., Bolisani E., Salvador L. (2009) A Systematic Framework for Analysing the Critical Success
Factors of Communities of Practice, Journal of Knowledge Management, Vol. 13, No. 6, pp. 431–447.
Wenger E., Mc Dermott R., Snyder W. (2002) Cultivating Communities of Practice: Guide to Managing
Knowledge, Harvard Business School of Press, Boston.1

ISBN 978-90-77360-17-0 © CINet 2014 295