Report to/Rapport au :

 

Audit Sub-Committee

Sous-comité de la vérification

 

and / et

 

Finance and Economic Development Committee

Comité des finances et du développement économique

 

and Council / et au Conseil

 

May 16, 2012

16 mai 2012

 

Submitted by/Soumis par : Alain Lalonde, Auditor General/Vérificateur général

 

Contact Person / Personne ressource:  Alain Lalonde, Auditor General/Vérificateur général

613-580-9602, oag@ottawa.

 

CITY WIDE / À L’ÉCHELLE DE LA VILLE

Ref N°: ACS2012-OAG BVG-0001

 

SUBJECT:

 

Office of the auditor general (OAG) – Response to the quality assurance review

 

OBJET :

 

bureau du vÉrificateur gÉnÉral (bvg) – rÉponse à l’examen d’assurance qualitÉ

 

 

REPORT RECOMMENDATION

That the Audit Sub-Committee recommend that the Finance and Economic Development Committee recommend Council approve the OAG’s response to the Quality Assurance Review, including the adoption of City of Ottawa OAG Audit Standards and amendments to the By-law governing the OAG, as contained in this report and its attachments.

 

 

Recommandation du rapport

Que le Sous-comité de la vérification recommande que le Comité des finances et du développement économique recommande à son tour au Conseil municipal d'approuver la réponse BVG à l'Examen d’assurance qualité, y compris l’adoption de normes de vérification du BVG de la Ville d’Ottawa et les amendements au règlement qui régit le BVG, tel que contenue dans ce rapport et ses pièces jointes.

 

 

Executive Summary

In keeping with the Council direction of November 24, 2010, an external quality assurance review (QAR) of the OAG has now been completed.  The results of the QAR are to be presented to Committee by PricewaterhouseCoopers (PwC).

 

In response to the review, the AG is pleased to note that the QAR confirms the City’s OAG has operated in full conformance with all aspects of the framework established by Council in creating the Office of the Auditor General. 

 

The AG agrees with all recommendations contained in the QAR and as such has implemented the following:

 

·         The adoption of a set of modified Institute of Internal Auditor (IIA) standards, to be referred to as the City of Ottawa OAG Audit Standards.

·         For internal quality assurance purposes, the rationale for any changes to draft audit reports will be more fully documented in the future. 

·         As confirmed in discussions with the Chair and Vice Chair, the current practice of reporting on all audit objectives established by the Audit Plan for each project will continue in the future. 

·         A formalized quality assurance program, including periodic internal file reviews, is now in place to ensure conformance to the proposed OAG auditing standards.  A second external QAR is scheduled within five years of this report. 

·         Based on the results of the consideration of this report by the Audit Sub-Committee, the Finance and Economic Development Committee and Council, the OAG will proceed with any required further amendments to the By-law governing the operation of the OAG

 

 

SOMMAIRE EXÉCUTIF

Conformément aux directives émises par le Conseil le 24 novembre 2010, un examen externe de l'assurance de la qualité (EAQ) du BVG a été réalisé.  Les résultats de cet examen seront présentés au Comité par Price.

 

En réponse à l'examen, le VG a constaté avec satisfaction que l'EAQ attestait la conformité entière du BVG à l'égard de tous les aspects du cadre de travail établi par le Conseil lors de la mise sur pied du Bureau du vérificateur général.

 

Le VG accepte toutes les recommandations contenues dans l'EAQ et, à ce titre, a mis en œuvre les éléments suivants :

·         Adoption d'un ensemble de normes modifiées de l'Institute of Internal Auditors (IIA), désignées sous le nom de normes de vérification du BVG de la Ville d'Ottawa.

·         Aux fins de l'assurance de la qualité à l'interne, la justification de toute modification aux rapports de vérification provisoires devra être mieux documentée à l'avenir.

·         Tel que confirmé lors de discussions avec le président et le vice-président, la pratique actuelle qui consiste à faire rapport de tous les objectifs de vérification établis par le plan de vérification pour chacun des projets se poursuivra.

·         Un programme d'assurance de la qualité officiel comprenant des études régulières des dossiers internes est désormais en place pour vérifier la conformité avec les normes de vérification proposées pour le BVG. Un second EAQ est prévu dans les cinq années suivant la publication de ce rapport.

·         Selon les résultats de l'examen effectué par le sous-comité de vérification, le Comité des finances et du développement économique et le Conseil municipal, au sujet de ce rapport, le BVG apportera toutes les modifications supplémentaires requises au règlement régissant ses activités

 

 

Background

On November 24, 2010, as part of the 2010-2014 Council Governance Review report, Council directed the Audit Sub-Committee to undertake an external quality assurance review (QAR) of the OAG.  The following discussion presents the Auditor General’s (AG) response to each of the opportunities identified by PricewaterhouseCoopers (PwC) in the QAR and the AG’s intended course of action for each area.

 

 

DISCUSSION

PwC Recommendation from the Executive Summary

 

As a next step following the creation of the ASC, PwC recommends that the OAG’s functional mandate and operating model be updated and clarified.  This would include adoption of the IIA standards, as modified to ensure the continued independence of the AG.  We recommend that modifications to the standards serve to confirm that:

 

OAG Response

 

The AG agrees with this recommendation.  The AG further agrees to the adoption of a set of modified Institute of Internal Auditor (IIA) standards, to be referred to as the City of Ottawa OAG Audit Standards.   The AG has prepared a complete set of proposed City of Ottawa OAG Audit Standards which are based on IIA standards, and which reflect the modifications recommended in Appendix A of the PwC report. These standards are included as Document 1 to this report. 

 

The AG has also attached revisions to the By-law governing the OAG to identify a statement of mandate.  Changes to the By-law and mandate of the OAG are subject to review by the City Solicitor and the direction of the ASC and Council as a result of consideration of this report.

 

Based on discussions with the Chair and Vice-Chair of the ASC, any future changes to the approved mandate and related protocols will be subject to deliberation and approval by the ASC.

 

Pending approval by Council, the standards attached to this report will be implemented, including:

 

2110.A1 – The audit activity must evaluate the design, implementation, and effectiveness of the City’s ethics-related objectives, programs, and activities; and,

 

2110.A2 – The audit activity must assess whether the information technology governance of the City supports the City’s strategies and objectives.

 

Accordingly, audits in these two areas will be proposed as part of the 2013 annual work plan.

 

QAR Objective 1:  Assess the OAG against the duties and other requirements established within the framework (Statute, Mandate and By-Law) which originally created the position of Auditor General in 2004.

 

PwC Conclusion

 

The OAG was found to Conform with the eight (8) key criteria as established within the Bylaw, Statute and Mandate.

 

OAG Response

 

The AG is pleased to note that the QAR confirms the City’s OAG has operated in full conformance with all aspects of the framework established by Council in creating the Office of the Auditor General.  The framework under which the OAG was established includes the Report of Mr. Desautels on the City’s Audit Function approved by Council on September 3, 2003; the amended By-Law No. 2009-323 establishing the position and duties of the AG and the Municipal Act section 223. 

 

The QAR demonstrates the successful transformation of the previous internal audit function at the City to the current Auditor General model.  The AG believes that, through the efforts of OAG staff, the office is effectively achieving its fundamental mandate of assisting City Council.

 

QAR Objective 2:  Identify opportunities, offer recommendations for improvement and provide counsel to the Auditor General and his Staff for improving the effectiveness and efficiency of the OAG.

 

a)    Annual Audit Planning

 

Opportunities Identified by PwC in the QAR Report:

 

When developing the annual audit work plan, we suggest the OAG:

·         Look for opportunities to enhance collaboration with key stakeholders, including Councillors, senior City Staff, and relevant functions/groups (e.g. the compliance group, ERM function) within the City;

·         Review recent risk assessment results and related risk management policies/procedures;

·         Enhance description of proposed audit objectives and scope and communicate in sufficient detail to allow stakeholders to better understand the nature of the proposed work.

 

OAG Response

 

The AG agrees with the recommendations.  As part of developing the 2013 work plan, the OAG will enhance its practice of soliciting input from the Mayor, Councillors and senior staff and will confirm its consideration of relevant assurance activities conducted by, or on, the City (e.g. Provincial regulators, City Compliance function) in the preparation and presentation of the Annual Work Plan. 

 

The City is currently in the process of developing its risk assessment procedures.  The OAG will monitor this development and incorporate it into the annual planning process once operational.

 

For the 2011 work plan, the OAG provided greater detail regarding proposed objectives and scope for the planned audits.  This will be incorporated as standard practice for future work plans.  In addition, discussions with the Chair and Vice-Chair of the Audit Sub-Committee will be held prior to presenting the final work plan to the Committee.  The rationale for any changes from the draft to the final annual work plan will be outlined in presenting the plan to the Chair and Vice-Chair of the Audit Sub-Committee.

 

b)    Fact Validation and Management Responses

 

Opportunities Identified by PwC in the QAR Report:

 

The AG should consider opportunities to strengthen the initial fact review process by:

·         Reviewing the extent to which City Staff are engaged during the development of key audit findings and recommendations;

·         Clearly and concisely identifying the evidence supporting preliminary findings either within the reporting itself or within factsheets remitted to auditee; and

·         Prioritizing and grouping findings for fact review as well as clearly identifying root causes or systematic issues.

 

OAG Response

 

The AG agrees with the recommendations.  The current fact validation protocol allows 8-10 weeks for management to respond to an audit report.  While the current process allows for clarifications and revisions, the AG will review the existing protocol with the ASC and senior management in time for the 2013 audit cycle and incorporate any opportunities for improving the efficiency and address any gaps in the process.  The revised protocol will be presented as part of the OAG’s 2011 Annual Report.

 

As part of the 2012 audits, the OAG  is ensuring  all evidence supporting the findings is clearly documented in each audit.  As suggested in the QAR, for internal quality assurance purposes, the rationale for any changes to draft audit reports will be documented in the future.  Finally, for all 2012 audits, the OAG will prioritize findings for fact review by clearly indicating that the summary of key findings in each report are priority-based.

 

c)    Audit Reporting

 

Opportunities Identified by PwC in the QAR Report:

 

 

OAG Response

 

The AG agrees with the recommendations.  In discussions with the Chair and Vice-Chair of the Audit Sub-Committee it was confirmed that the OAG should report to Council on all the findings of each audit.  As such, current practice has been to report on all audit objectives established by the Audit Plan for each project.  This practice will continue in the future.  In addition, it will be clearly stated that the summary of key findings in each report is considered to be priority-based.  Executive summaries and press releases for each audit will be provided to offer a more concise discussion of audit findings.

 

QAR Objective 3:  Identify and map those additional activities or other changes, if any, that would be required to support the OAG’s conformance with Institute of Internal Auditors (IIA) Auditing Standards.

 

OAG Response

 

The AG agrees with the recommendations.  It is important to note that the City’s OAG was not created with the intent of complying with IIA standards.  IIA standards were created to govern an Internal Audit model and, as such, some are not necessarily appropriate or relevant to the Auditor General model that the City has implemented (for example, standards related to providing consulting services).  Each model has merit and can be effective in providing management and Council with information on efficiency and effectiveness however the governance and independence features are, in the AG’s opinion, different.  A full set of proposed audit standards are contained in Attachment 1 of this report.  These Standards are based on the October 2010 version of the IIA International Standards for the Professional Practice of Internal Auditing and were modified by, and are for the use of, the City of Ottawa’s OAG.

 

a)    Purpose, Authority, and Responsibility

 

Opportunities Identified by PwC in the QAR Report:

 

Amend OAG’s Mandate (or “charter”) to have it better reflect:

 

OAG Response

 

The AG agrees with the recommendations.  This report and its attachments present proposed changes to the OAG mandate.  Attached are revisions to the By-law governing the operation of the OAG and include a clear statement of mandate.  Based on the results of the Audit Sub-Committee’s consideration of this report, the OAG will proceed with any required further amendments.

 

b)    Quality Assurance and Improvement Program

 

Opportunities Identified by PwC in the QAR Report:

 

Formalize the Quality Assurance and Improvement Program for ASC approval. This program would include formal requirements related to:

·         Performance management – including performance metrics/indicators and performance reporting requirements;

·         Periodic internal file reviews (e.g. independent reviews of a sample of audit files) and internal assessments of the OAG’s compliance status against IIA Standards; and

·         External assessments (at least every five years).

 

OAG Response

 

The AG agrees with the recommendations.  The OAG’s current performance metrics include “completion of the annual work plan”, and “feedback from Council”.  A formalized quality assurance program, including periodic internal file reviews, is now in place to ensure conformance to the proposed OAG auditing standards.  A second external QAR is scheduled within five years of this report.  The standards and modified AG by-law are contained in the attachments to this report. As was stated earlier, modifications were required to better reflect an AG versus an internal audit model.  In addition, the modifications better reflect the municipal environment as well as the current relationship between management and the OAG and do not include references to consulting services.

 

c)    Annual Audit Planning Process

 

Opportunities Identified by PwC in the QAR Report:

 

 

OAG Response

 

The AG agrees with the recommendations.  The Annual Work Plan is structured to be completed within the resources provided by Council.  It will be communicated to both the ASC and senior city staff and resource requirements will be communicated via the budget process.

 

As stated earlier, the City is currently in the process of developing its risk assessment procedures. As such, there has been no opportunity to align the OAG annual work plans to the City’s risk assessments to-date.  The OAG will monitor this development and incorporate it into the annual planning process once operational. Consideration of City goals and relevant assurance activities conducted by, or on, the City (e.g. Provincial regulators, City Compliance function) will be confirmed in the future.  Discussions with the Chair and Vice-Chair of the Audit Sub-Committee will be held prior to presenting the final work plan to the Committee.  The rationale for any changes from the draft to the final annual work plan will be outlined in presenting the plan to the Chair and Vice-Chair of the Audit Sub-Committee.

 

d)    Engagement Planning

 

Opportunities Identified by PwC in the QAR Report:

 

 

OAG Response

 

The AG agrees with the recommendations.  A detailed audit plan is provided for each project documenting objectives, scope, timing, criteria and the preliminary assessment of the risks.  Opportunities to enhance the documentation of this assessment have been actioned for the 2012 audits and will continue to be pursued.  Detailed audit programs are now required in each case.

 

e)    Quality Control Practices

 

Opportunities Identified by PwC in the QAR Report:

 

Ensure there is consistent documentation evidencing supervision and review for each audit. This would include maintaining evidence demonstrating supervisory review and approval on the following documents:

·         Engagement work program (prior to its implementation);

·         Summary/conclusion working papers (prior to sending draft reports for fact validation); and

·         Draft audit reports.

 

OAG Response

 

The AG agrees with the recommendations.  The OAG will ensure that the supervision occurring throughout each audit consistently documented in each case.  Audit/Engagement programs will be required for each audit.  Opportunities to enhance the documentation of summary working papers and draft reports will be pursued.  As suggested, for internal quality assurance purposes, the rationale for any changes to draft audit reports will be documented in the future

 

In addition, the AG wishes to reiterate that an ongoing internal quality assurance program is now in place which will include annual reporting on performance as part of the AG’s Annual Report.  A second external QAR is scheduled for 2017.

 

f)     Communicating Results

 

Opportunities Identified by PwC in the QAR Report:

 

 

OAG Response

 

The AG agrees with the recommendations.  The amount of detail in many audits reports is significant, however, as confirmed in discussions with the Chair and Vice-Chair of the Audit Sub-Committee, this approach has been taken in response to Council’s and management’s desire for this level of detail in order to fully describe and substantiate the observations.  Furthermore, the AG has a duty to report on all findings of each audit and this practice will continue in the future.  As was done for the 2010 Annual Report, detailed reports will only be available on-line.

 

All reports will contain a Conclusion presenting an overall opinion on the audit.  In addition, the preparation of executive summaries for each audit will provide a concise summary of findings.

 

Many reports have in the past acknowledged satisfactory performance.  This practice will continue where deemed appropriate.  The standard in this regard does not require that this be included.  As such, no changes to the current process are intended.

 

 

RURAL IMPLICATIONS

There are no rural implications associated with this report.

 

 

PUBLIC CONSULTATION

As this is considered an internal administrative matter, no public consultation was undertaken.

 

 

Comments by the Ward Councillor(s)

This is a city-wide issue.

 

 

LEGAL IMPLICATIONS

As noted in the “Discussion” section of this report, changes to the By-law and mandate of the OAG are subject to review by the City Solicitor and the direction of the ASC and Council as a result of consideration of this report.

 

 

RISK MANAGEMENT IMPLICATIONS

There are no risk management implications associated with this report.

 

 

FINANCIAL IMPLICATIONS

There are no financial implications associated with approval of the report recommendation.

 

 

ACCESSIBILITY IMPACTS

There are no accessibility impacts associated with this report.

 

 

Environmental Implications

There are no environmental implications associated with this report.

 

 

TECHNOLOGY IMPLICATIONS

There are no technology implications associated with this report.

 

 

CITY STRATEGIC PLAN

Impacts on Term of Council priorities are outlined in the associated report titledQuality Assurance Review – Office of the Auditor General” (ACS2012-CMR-OCM-0002).

 

 

SUPPORTING DOCUMENTATION

DOCUMENT 1:         City of Ottawa Audit Standards

DOCUMENT 2:         Modified AG By-law

 

 

disposiTION

The OAG will proceed according to the direction of the Sub-Committee in considering this report.