Open Source Science Journal

Vol. 2, No. 1, 2010

Auditing Open Source Applications by Using COBIT 4.1

Assist. Cristian AMANCEI, PhD candidate Academy of Economic Studies, Bucharest, Romania Department of Computer Science in Economics cristian.amancei@ie.ase.ro

Abstract: Open source application becomes more and more a viable solution for organizations. The access to the source code enables organizations to adapt the capabilities of the applications to the business processes that are supported by the application. The cost constraints and the opportunity to improve the application in order to respond to the changes on the economic environment, requires the auditor to identify the associated risks and the controls that mitigate those risks. In this paper we present a selection of controls from the COBIT framework that we considered mandatory for open source applications. Keywords: IT audit, open source applications, COBIT.

1. COBIT The Control Objectives for Information and related Technology (COBIT) were first released in 1996 by ISACA and have since been revised several times, the current version being 4.1. The COBIT framework [3] does not specify which technologies an organization should or shouldn't use. It is a high-level framework that can be used to evaluate an organization's existing or planned controls: be they policies, processes or technologies. Each of the sections of COBIT consists of four subsections: description, control objectives, management guidelines and maturity model. The description is a high-level overview of the section that follows. The control objectives are a high-level list of requirements. In the management guidelines, the control objectives are mapped according to which members of an organization should be responsible, accountable, consulted or informed (RACI) on how each control objective is actually implemented. Also it takes the control objectives and breaks them down into goals and high-level metrics to measure how close the enterprise is to achieving those goals. The maturity model, gives auditors a framework for determining how mature an organization is in regard to a specific section [5].

2. Key Issues in Using OSA The goal of the audit mission is to obtain a reasonable assurance concerning the deployment and the operating of the open source software, in accordance with the appropriate rules and settlements (regulations), and with specific security standards [1]. The audit plan for open source application is driven by the following issues: - external support [4] For OSA that is downloaded over the Internet, no official support is available. There are organizations that provide consultancy and external support for the adoption and implementation of OSA, enforcing this solution on the market.

87

Open Source Science Journal

Vol. 2, No. 1, 2010

Even if user assistance is provided through online channels such as mailing lists, forums or wikis, corporate users have no assurance that they will be provided with accurate and timely assistance for any problems they experience. It the case of business-critical systems, the support offered by the OSA community is insufficient and professional support is required. Based on this requirement, several vendors offer commercial versions of OSA that include professional support services. - cost consideration [4] Even if OSA can be downloaded from the Internet free of charge, the real cost of using OSA are difficult to assess, due to the fact that the implementation is not free of charge, and additional costs are incurred for training, migration and maintenance. Based on this it is difficult to evaluate whether the total cost of ownership of OSA is lower than the total cost of ownership for proprietary software. - the business processes supported by the application The impact on the organization mainly depends on the type of software that is adopted and the business processed that will be supported by the application. Training users may represent a considerable investment, as users may have a long experience working with other proprietary software solutions. In order to have good results during training, it is important to create a favorable attitude with end users toward the OSA solution, and to help users to see the differences of the new solution and who it will adjust their working habits by reducing the work load. - the source code security on the production environment When implementing an OSA solution the organizations has direct access to the source code, as it is free, and can adapt and improve the solution to properly fit the organizations needs. Even if we have these possibilities, the source code must not go into production directories. The complied class files are all that is required in most cases. All source code should be removed and only the executables should remain. The source code should be access only in secured areas and a versioning system must be implemented by the organization. Also we must remember that no development tools should be present on a production environment. All these issues can be addressed by implementing a good change management policy and a versioning system. - the application controls defined for business purposes The responsibility for application controls is an end-to-end joint responsibility between business and IT, but the nature of the responsibilities changes as follows: The business is responsible to properly: define functional and control requirements, and the use of automated services; IT is responsible to: automate and implement business functional and control requirements, and establish controls to maintain the integrity of applications controls.

3. Application of COBIT to OSA Based on the five key issues noted for using OSA, relevant COBIT processes and the subprocesses that can be especially useful in this case have been identified. The application of COBIT to OSA audit should help in addressing the issues mentioned above and, allow for the use of OSA in a more controlled manner.

88

Open Source Science Journal

Vol. 2, No. 1, 2010

COBIT describes a set of good practices for management, control and security of information technology, and organizes them around a logical framework of IT processes. COBIT 4.1 contains several new important concepts, such as the alignment of business and IT goals, their relationship with supporting IT processes, roles and responsibilities within IT processes, and the interrelationship between IT processes [6]. Figure 1 provides an overview of the mapping of COBIT with the five identified key issues of OSA. Only the most relevant control objectives have been retained and discussed here. The COBIT control objectives are divided into six domains: Process Control (PC), Plan and Organize (PO), Acquire and Implement (AI), Deliver and Support (DS), Monitor and Evaluate (ME) and Application Control (AC) [2]. The following breaks down each COBIT process and its relation to OSA operation. External Support PO 1.4 IT strategic plan - Organizations may consider it a strategic choice to implement OSA. For example, organizations may choose OSA in an effort to reduce constraints of the vendor. The future developments should be presented in the IT strategic plan. PO 3.1 Technological direction planning: - A strengths, weaknesses, opportunities and threats (SWOT) analysis for OSA should be made, based on the specific environment of the organization. A SWOT analysis helps to get insight into the potential benefits of using OSA for the organization. - OSA should fit within the enterprise architecture and should be compatible with the overall IT infrastructure. If not, the migration would require too much time and financial resources, and reduce the benefits of adopting OSA. When making the analysis and analyzing the options, the type of support needed should be already established and included in the scenarios. PO 3.3 Monitor future trends and regulations - Since OSA is distributed under licenses that are fundamentally different from proprietary software, the legal counsel should investigate the implications for the organization. PO 4.15 Relationships - The organization must develop good relationships with those vendors that provide external knowledge on OSA software. Relevant OSA activities in the organization should be coordinated with these external parties. AI 5.3 Supplier selection - Potential providers that can meet the required level of support identified. The organization needs to determine if support contracts offered by commercial OSA vendors are acceptable or whether additional support services from a third-party consultant are desirable. An investigation will be performed to identify the list of officially recognized support vendors, if available, or the persons involved in the project will prepare one. The organization has to know the support vendors that have a thorough knowledge of the OSA product and where it can find easy access to the OSA developers and at what price. DS 1.1 Service level management framework - Decision makers must first determine the requested level of support that is required for OSA. Factors that may

89

Open Source Science Journal

Vol. 2, No. 1, 2010

influence this choice are the lack of in-house OSA expertise and the importance of the business process that is supported by the OSA installed. DS 1.3 Service level agreements - It is important to ensure that service level agreements (SLAs) are agreed upon with the external party providing support for OSS. The SLA management process should ensure that the SLA objectives are measured, so that they can be reported back to the stakeholders. SLAs should be revised when requirements change. DS 2.3 Supplier risk management - Organizations need to consider the potential risks involved in relying on small, local firms that offer OSA related services. It is possible that these organizations may not be able to fulfill their contractual agreements, due to their limited size. DS 2.4 Supplier performance monitoring - The performance of external parties should be monitored and evaluated. Since new OSA support vendors continue to appear on the market, the organization must regularly benchmark the offering of the current suppliers to the market. Cost Considerations PO 5.1 Financial management framework - A financial framework for the assessment of the costs and benefits should be in place. This is helpful in comparing the cost of an OSA solution with a proprietary solution. It is important that costs be calculated over the whole life cycle of the project, including migration costs, implementation consultants, training cost and the support fees. DS 6.3 Cost modeling and charging - The costs of the IT infrastructure should be charged back to business process that uses the OSA software. These costs are likely to differ between OSA and proprietary software, and the difference will probably impact the adoption decision. The business processes supported by the application PC 2 Process Ownership - Each process supported by the OSA software must have an owner assigned. The organization will clearly define the role and the responsibilities of the process owner, such as design, interaction with other processes and performance measurement. The process owner must have sufficient authority to implement, drive and improve the process. PC 5 Policy, Plans and Procedures - All the policies, plans and procedures will be documented, reviewed, maintained, approved, communicated and used for training. This will help in decreasing the number of incidents and will increase the staff awareness. PO 7.2 Personnel competencies - The personnel involved in the process sustained by the application should possess or develop the required OSA competencies. Therefore, organizations should encourage the staff to obtain the necessary knowledge on OSA and acquire the necessary certifications if needed.

90

Open Source Science Journal

Vol. 2, No. 1, 2010

PO 7.4 Personnel training - Staff members should receive appropriate training on OSA in order to improve their job performance. Lately, an increasing number of training institutions are offering courses on OSA products. AI 2.6 Major upgrades to existing systems - If the introduction of OSA constitutes a major change in the organization, the impact of this change should be properly assessed, by the users that will work in the new environment. AI 4.3 Knowledge transfer to end users - Users should have access to documentation on the OSA product. For some OSA products, limited documentation is available. Commercial vendors of OSA products generally provide high-quality documentation. Firms that offer training courses may also provide documentation for OSA. It is essential that the availability of good documentation from suppliers be evaluated. AI 7.1 Training - A training approach should be developed to assist users in making the transition. All affected users should have the opportunity to attend the training sessions. The training should be structured in the phases: the first one focusing on generic skills, and a second phase that will focus on specific and more advanced tasks. DS 7.1 Identification of education and training needs - Following the adoption of OSA, the training plan for affected employees should be revised to include the necessary staff related skills. DS 7.2 Delivery of training and education - Sufficient training sessions on OSA should be organized shortly before or after the migration. All users must attend the training session, in order to decrease the level of incidents after the migration. DS 7.3 Evaluation of training received - The effectiveness of the training sessions should be assessed by testing the users knowledge. Possible gaps in the knowledge required for performing tasks should lead to a revision of the training approach or result in additional training sessions. The source code security on the production environment AI 3.3 Infrastructure maintenance - New versions of OSA products can be released more frequently than proprietary software. Therefore, maintenance procedures should state which types of updates and upgrades are applied. Organizations may also prefer to implement new OSA versions that come with important changes needed in the business process supported by the software. ME 2.1 Monitoring of internal control framework - A policy with the accepted frameworks and practices for internal control monitoring and evaluation activities has to be defined by the organization. The organization should take into consideration an independent evaluation of the internal control system for proactive detection and resolution of control deviations. Promptly report, follow up and analysis of the exception should be a priority for the organization.

91

Open Source Science Journal

Vol. 2, No. 1, 2010

ME 2.4 Control self-assessment Defining and identifying evaluation criteria for conducting self-assessments will increase the ability of the organization to implement preventive measures for recurring exceptions by applying corrective measures. By using the control self-assessment, the organization will have a proactive approach in improving the quality of service that drives the client relationship. ME 4.5 Risk management - The implementation of a new OSA product will have an impact on the organization risk assessment. The risk assessment changes have to be indentified and compared with the board appetite for risk exposures. Approval must be received for levels that are above approved previous approved residual risk. A clear defined approach for managing risk must be defined in order to achieve a desired level for the control environment. ME 4.6 Performance measurement - A performance measurement system for the defined objectives must be put in place in order to assess the management performance in the execution and achievement of the business strategies. This system will highlight the objectives that have not been achieved and an action plan will be prepared for future compliance. The application controls defined for business purposes AC 2 Source data collection and entry - The business processes that are supported by OSA product should propose controls that will ensure the data input in a timely manner and by authorized and qualified staff. Clear access rights matrix must be defined, in order to secure the access to input, edit, authorize, accept and reject transactions, and override errors. Segregation of duties for data collection and entry must be defined and accepted by the business owners. AC 3 Accuracy, completeness and authentic checks - Based on business reasons, controls that will check for accuracy, completeness and validity will be defined. Where it is possible, these controls should be automated. All the transactions that fail the validation rules will be posted in special file for proper review. AC 4 Process integrity and validity - During the processing cycle, the detection of erroneous transactions must not disrupt the processing of valid transactions. The review of adjustments, overrides and high-value transactions must be performed promptly and in detail by appropriate personnel who does not perform data entry. These controls should be defined by the business key users, during implementation phase. AC 5 Output review, reconciliation and error handling - Procedures should be defined and implemented, to ensure that the business owners review the final output for reasonableness, accuracy and completeness, and that output is handled in line with the applicable confidentiality classification. Report potential errors, log them in an automated, centralized logging facility, and address errors in a timely manner.

92

Open Source Science Journal

External Cost support consideration Business processes supported by application x x x x x x x x x x

Vol. 2, No. 1, 2010

Source code security on the production environment Application controls defined for business purposes

PC 2 Process Ownership PC 5 Policy, Plans and Procedures PO 1.4 IT Strategic plan PO 3.1 Technological direction planning PO 3.3 Monitor future trends and regulations PO 4.15 Relationships PO 5.1 Financial management framework PO 7.2 Personnel competencies PO 7.4 Personnel training AI 2.6 Major upgrades to existing systems AI 3.3 Infrastructure maintenance AI 4.3 Knowledge transfer to end users AI 5.3 Supplier selection AI 7.1 Training DS 1.1 Service level management framework DS 1.3 Service level agreements DS 2.3 Supplier risk management DS 2.4 Supplier performance monitoring DS 6.3 Cost modeling and charging DS 7.1 Identification of education and training needs DS 7.2 Delivery of training and education DS 7.3 Evaluation of training received ME 2.1 Monitoring of internal control framework ME 2.4 Control self-assessment ME 4.5 Risk management ME 4.6 Performance measurement AC 2 Source data collection and entry AC 3 Accuracy, completeness and authentic checks AC 4 Process integrity and validity AC 5 Output review, reconciliation and error handling

x x x x x x x x x x x x x x x x x x x x

Fig. 1. Mapping of COBIT with OSA Key Issues

4. Conclusions This article has described some key issues that required attention during the audit of a OSA product. This selection of control objectives from COBIT 4.1 addresses only the key issues introduced in this paper and addresses only the minimal requirements from the audit point of view. Other control objective can be selected from COBIT in order to provide assurance over management practices. The provided set of control objectives can be leveraged

93

Open Source Science Journal

Vol. 2, No. 1, 2010

as a quick scan to verify if current management practices in using OSA are complete and sufficient for the organization.

References [1] I. Ivan, G. Noca and S. Capisizu, Auditul sistemelor informatice, ASE Printing House, Bucharest, 2005 [2] IT Governance Institute, COBIT 4.1, 2007 [3] D. Mortman, How to use COBIT for compliance, in Information Security Magazine, March 2010. [4] K. Ven, S. De Haes, W. Van Grembergen and J. Verelst, Using COBIT 4.1 to Guide the Adoption and Implementation of Open Source Software, in Information System Control Journal, Vol 3, 2008. [5] C. Lahti, S. Lanza and R. Peterson, Sarbanes-Oxley IT Compliance Using COBIT and Open Source Tools, Syngress Printing House, Bucharest, 2005. [6] T. Surcel and C. Amancei, ERP System Audit a Control Support for Knowledge Management, in Economic Informatics Journal, Vol XII, No. 4(48), 2008, Inforec Publishing Huouse, Bucharest. [7] M. Popa and F. Alecu, ERP Informatics System Audit, in Informatica Economic 2nd supplement Knowledge Management Projects, Systems and Technologies: Reinforcement and Extension of Universities & Business Community Partnerships in the Knowledge Era, vol. 10, pp. 109 116, November 2006.

Author Cristian AMANCEI is University Assistant at Academy of Economics Studies Bucharest, Faculty of Economic Cybernetics, Statistics and Informatics. He is a PhD candidate from October 2007 at Economic Informatics Department from Academy of Economic Studies. He holds a Master in Science Computerized Project Management from Academy of Economic Studies, Bucharest. He is Certified Information Systems Auditor (CISA). He graduated in Economic Informatics at Faculty of Economic Cybernetics, Statistics and Informatics in 2006. His main research areas are: information system audit, data structures, metrics in information systems and object oriented programming.

94