Corresponding author: Tjibbe Bosman ( t.bosman@uva.nl ) Academic editor: Anna Gold
© 2021 Tjibbe Bosman.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits to copy and distribute the article for non-commercial purposes, provided that the article is not altered or modified and the original author and source are credited.
Citation:
Bosman T (2021) The measurement of audit quality in the Netherlands: a practical note. Maandblad Voor Accountancy en Bedrijfseconomie 95(1/2): 17-31. https://doi.org/10.5117/mab.95.56820
|
Several committees, institutions, and practitioners are currently working on defining appropriate, and reliable Audit Quality Indicators (AQIs). The experiences the Foundation for Auditing Research (FAR) made with collecting audit quality data may inform the search for appropriate and reliable AQIs. In this paper I discuss different types of audit (quality) measures and their availability in the Netherlands. Furthermore, I discuss the (potential) information value, limitations, and recommendations for a wide range of audit quality measures and sources.
Audit Quality data, Audit Quality Indicators
This paper includes several recommendations and lessons learned from the data gathering efforts of the FAR to inform on the availability of audit quality data and measurement of potential AQIs in practice. This paper is also of interest to those audit practitioners managing and designing the quality control (monitoring) systems of audit firms, those involved in preparing the transparency reports, and policymakers and regulators in their considerations of AQI’s.
To develop appropriate and reliable Audit Quality Indicators (AQI’s), an understanding of Audit Quality measures is a prerequisite. Audit Quality can be defined as: “…greater assurance that the financial statements faithfully reflect the firms underlying economics, conditioned on its financial reporting system and innate characteristics” (
The Dutch auditing profession (ten largest audit firms) and academia founded the Foundation for Auditing Research (FAR) to conduct research on audit quality. The FAR started to commission research projects to international research groups in 2016, is currently involved in 34 (international) research projects and has co-funded 17 PhD positions. In the context of these research projects, audit firms and researchers started to collect Dutch (audit quality) data from public and proprietary data-sources for purposes of academic research. Based on the experiences made with the FAR projects, this paper discusses both potential audit quality measures based on publicly available data in comparison to existing international research as well as proprietary (audit) data.
The structure of this paper is as follows. First, the
Audit Quality can be defined as: “…greater assurance that the financial statements faithfully reflect the firms underlying economics [greater assurance of financial reporting quality], conditioned on its financial reporting system and innate characteristics” (
Audit services can be defined as a credence good. A credence good is a good or service where (1) the seller is an expert who recommends and provides a level of service to a buyer, (2) buyers cannot assess how a service is delivered and must rely on seller’s recommendation and (3) buyers cannot assess how well the service was performed (
Given the credent nature of auditing,
Academics have defined several publicly observable proxies to analyze the level of assurance that auditors provide. These have been the subject of academic debates ever since.
The most direct observable of the level of assurance is the lack of assurance
When audited financial statements need to be restated, this is a relatively hard indication of a lack of audit quality of the restated audit. Article 362 sub 6 Book 2 Title 9 of the Dutch civic code requires that, if filed financial statements are seriously deficient (“in ernstige mate tekort schieten”) in providing the insight needed to form an informed opinion about the financial situation of the company, the board of the company should file a statement (restatement) at the trade register and inform its shareholders immediately. However, as
Since October 2020 Audit Analytics Europe included the restatements for European companies who have an equity-listing at a regulated market in their database. Audit Analytics hand-collected this data starting from January 2018. On November 12, 2020 Audit Analytics identified 672 European restatements of which 18 relate to financial statements audited by Dutch Public Interest Entity (PIE) auditors. Between January 2018 and November 12, 2020 Audit Analytics analyzed 251 audit opinions of Dutch PIE auditors issued in the Netherlands. The restatement rate for annual financial statements of Dutch equity listed PIEs therefore seems to be 7.2% (18 divided by 251). By comparison, in the US almost 15% of all filed (interim) financial statements by listed companies are restated, which gives researchers interesting variation to analyze (
For non-equity listed organizations, there is no central database of filed restatements in the Netherlands. According to the 2019 Transparency report of PwC Netherlands (
It is expected that accounting scandals lead to lawsuits against the audit firm and the audit partners involved. Lawsuits against audit firms and audit partners are only partly publicly observable in the Netherlands, as most identifying information is stripped from the published court proceedings. Court proceedings by the discipline court for auditors (“Accountantskamer”) are published and kept in an accountant.nl database for ten years, but they are anonymized and therefore cannot be linked to individual companies, their financial statements and auditor. Over the last ten years more than 633 auditors (NBA Register as of January 2020) were convicted by the disciplinary court for auditors. However, while we can identify who is convicted and what the sentence was, it remains almost impossible to identify which audit engagement the conviction is related to. The professional register of the NBA is hard to access and there is no central list or download option available to obtain the full register (
Accounting and Auditing Enforcement Releases (AAERs) are published by the US SEC since 1982 (
The auditor should reflect a material uncertainty about the going concern of the auditee in his audit opinion (Nadere Voorschriften Controle- en Overige Standaarden (NV COS) 570). A concern for financial statement users is when a company files for bankruptcy within 12 months from the last balance sheet date and the auditor failed to issue a paragraph pointing to the material uncertainty relating to going concern (type II ‘error’). The opposite situation, where the auditor recognizes a material uncertainty related to going concern and reports this in the audit opinion and the company does subsequently not file for bankruptcy (type I ‘error’) could make going concern opinions less informative, which is also an issue. The likelihood of type I and II GCO ‘errors’ can be used as audit quality measure and there is much research on this topic (
There is currently no central database with going concern (or other modified) opinions of Dutch auditors. A researcher needs to go through the original filings of the auditee to be able to identify the going concern and/or modified audit opinions. Another data challenge is the fact that the information in the Dutch insolvency register is erased six months after court proceedings. Therefore, it is hard for researchers to estimate the actual going concern (type II) error rate in the Netherlands. Fortunately, the information service provider Company.info does collect, keep, and provide most of the data on going concern statements in the PDFs they store and is therefore suitable to analyze this data and to identify Type II errors. In other EU countries such as Spain (
The outcomes of audit file inspections of internal and external inspectors provide a direct feedback on audit (file) quality. If work is not documented in the audit file it is not considered to be done (adequately). The AFM, like the PCAOB, does not disclose individual audit file inspection outcomes and are exempted from the freedom of information act (other than for example the SEC
Another limitation of audit inspection outcome as an audit quality measure, is that the sampling procedure for the inspection process is not always clear and rarely random. Recent research on 3.864 US firm internal inspections finds that the internal inspection risk of the audit firm is predictable and audit teams strategically respond to inspection risk by increasing effort of the engagement with inspection risk and reducing effort on other engagements (
Academics use the Earnings Response Coefficient (ERC) to proxy for how intensely the financial market responds to the publication of the audited financial statements or the publication of accounting restatements (
Similar assessments of audit quality by stakeholders such as the auditee’s audit committee, auditee’s (financial) management, investor interest groups and the financial analysts following the audited company are rare and usually not structurally collected by the audit firms and professional organizations for every audit, even though this could give interesting insights in the quality of the service provided (
An understanding of the values, ethics and attitudes within an audit firm are important, as it tells something about what is considered acceptable behavior in the audit firm (
A specific application of values, ethics and attitudes is the acceptability of quality threatening behaviors (QTB). QTBs are any behavior by auditors which have the potential to “threaten the outcome of the engagement and the validity of the audit opinion” (
Audit firms typically survey all their employees at least once a year. These surveys contain various standardized questions, and the PIE audit firms disclose some highlights of these survey outcomes in their transparency reports together with their code of conduct. Additionally, the NBA has developed a culture survey with several metrics of audit firm culture which audit firms can take. A common risk in interpreting the survey results is that the surveys rely on self-reporting data (
The financial statements are the object of the audit. Therefore, variation in financial reporting quality gives an indication about the quality of the audit. The overall assumption is that when an auditor accepts the financial statements, this is an indication of the extent of aggressive accounting that he or she is willing to accept for this client. A goal of (accounting) researchers examining audit quality, is to analyze the variation in the financial reporting quality of audited companies and link this to certain audit characteristics. It is important to note that financial reporting quality and audit quality are distinct concepts, but they often cannot be disentangled to measure audit quality (
To measure financial reporting quality, a set of variables from financial statements need to be collected. In most cases this data is available through sources such as the KVK (trade register), investor relations websites, jaarverantwoordingzorg.nl, findo.nl, and database providers such as Company.info, Bureau van Dijk, Compustat, and Audit Analytics. In analyzing the variation in financial reporting quality, researchers use accrual and other earnings quality models that are not that precise, vary in consensus over how to apply them (
Most of the current accounting literature is focused on companies with listed equity instruments on regulated markets. These companies produce the most detailed publicly observable accounting information, have the highest degree of regulation and a high rate of compliance with these regulations as they are actively enforced by the financial markets regulator and the stock exchange. Furthermore, the accounting information is prepared under common GAAP and is therefore comparable.
Other than in the US or Belgium, the information format of financial statements is not uniform in the Netherlands. In the Netherlands, the regulations leave significant discretion to the company and its auditor
Private companies are not registered at the AFM, but their financial statements are available at the KVK. Other than in for example the UK, Belgium, or Germany, the Dutch KVK requires a fee of EUR 3.05 to be able to read and download a single set of financial statements from a company. This is a relatively low fee, but a significant burden for a researcher who wants to statistically analyze the variation in thousands of financial statements. A common solution to bypass this fee is a Company.info license. Company.info provides users with a platform to share the PDF’s of KVK information among its users. Bureau van Dijk also offers financial reporting databases for private companies in their databases Amadeus, Orbis and Reach, however the coverage is much smaller than that of listed companies.
When a financial statement user or researcher has the PDFs documents, he/she needs to go through every PDF to extract more detailed information from the financial statements. The filed PDF’s are often non-machine readable, which require an application of Optical Character Recognition (OCR) technique first to make them readable for the computer. An alternative to using OCR is to apply hand-collection
The ideal situation for an information user or researcher would be to have a database of the approximated 20.000 yearly statutory audits (
A common challenge in collecting financial reporting information for non-listed entities is that the compliance with filing regulations in the Netherlands is not actively enforced.
Part of the consequence of the Dutch filing and reporting policies is pointed out in
The question occurs whether an auditor may accept an audit engagement when prior years financial statements where not (completely) filed at the trade register, especially given the Non-Compliance with Laws and Regulations (NOCLAR) regulation of the auditing profession (NBA 2018). On the political side, a regulatory change to a system where non-, late- or incomplete-filing companies get automatic fines from the courts when they do not (timely) file (audited) financial statements seems to have value. Germany implemented such a system in 2006 after pressure from the European Commission and it significantly improved the compliance rate with financial reporting regulation (
This section discusses several financial reporting systems and innate characteristics from audit clients and audit engagements and the information sources associated with them. Research access to these sources of proprietary data (via FAR) is relatively new and therefore requires some context of the source. I will therefore also point out some opportunities for future research with these sources. Finally, I will discuss how audit firms could use this data and research to improve their quality management systems. In this section I will comment on the topics of client acceptance and risk assessment, internal control deficiencies, followed by audit effort, specialist involvement and consultations and identified audit misstatements.
Before an audit firm can accept or continue an (audit) engagement, it needs to follow an acceptance procedure. The auditors involved need to evaluate the risk of the engagement, identify the client to be compliant with anti-money laundering regulation, address potential independence and reputation concerns, before an engagement is accepted. When an auditor does not accept or dismisses high-risk clients, this is considered a sign of conservatism which is usually interpreted as a sign of audit quality. There is a risk that the biggest audit firms only accept easy to manage and well behaving clients and so pass the problem on to smaller audit firms. The non-acceptance of audit clients can only be observed if they fail to find any auditor. The non-continuance of an audit engagement can be observed but it is hard to disentangle whether this is due to the client, the auditor or some (independence) regulation.
The client acceptance data is usually centrally managed by the audit firm and monitored. A high-risk score or potential independence concerns could result in the assignment of an engagement quality review partner (“opdrachtgerichte kwaliteitsbeoordelaar”) to the engagement. The total number of engagement quality reviews and the number of clients refused by the audit firm is usually disclosed in the transparency report of the audit firm. The auditor’s risk assessment of the engagement risk, internal control risk but also the fraud risk are interesting variables for audit firms’ management but are proprietary data, which underly auditor’s confidentiality obligations. For these risk assessment variables each audit firm uses their own format and procedures which are currently not necessarily comparable. Where the formats differ between audit firms, the data structure is relatively comparable within audit firms, making it a very interesting data source for financial statement users and potentially for future research with proprietary data. Audit firms could learn from collaborating with researchers on this topic to optimize their risk assessments and decisions.
If an auditor identifies internal control deficiencies and has recommendations to improve the companies’ internal control system, this is interpreted as an indication of audit quality. The management board members of companies who are subject to the Dutch Corporate Governance Code
In the absence of an audit of the in-control statement of management, research of the financial reporting systems and company’s innate characteristics is hard with only publicly observable data. Researchers have been using corporate governance indicators (
The identified internal control deficiencies by the auditor are not publicly observable but are reported to the supervisory board (or management) in the auditor’s report (“accountantsverslag”) and are therefore also unobservable for the company’s shareholders. In some cases, auditors issue a disclaimer of opinion (“oordeelonthouding”) because the audited company does not have enough segregation of duties (internal controls) and the auditor can therefore not rely on the completeness of the underlying accounting system. This disclaimer of opinion is publicly observable but is relatively rare for larger organizations.
In obtaining reasonable assurance over whether the company’s financial statements give a true and fair view, the auditor has the choice over how to obtain assurance. At a minimum, the auditor should evaluate the appropriate design of the internal controls relating to the significant risks (ISA 315.12) and the accounting system (ISA 315.18). Additionally, Dutch law requires that auditors report on the reliability and continuity of automated data processing in the non-publicly observable auditor’s report. Despite these basic principles, it is up to the auditor and his/her professional judgement to decide what is the most effective and efficient audit strategy to be able to issue an audit opinion. The issuance of a management letter where the auditor reports his/her observations relating to the internal control system is not legally required but generally considered best practice. Therefore, the information relating to the quality of the financial reporting system could be incomplete, when auditors conclude that the (documentation of the) internal control system is insufficient to be relied upon and chose to primarily rely on substantive testing at an early stage in the audit. The quality of the financial reporting system is not the auditor’s primary audit object in the Netherlands, this is and remains the true and fair view of the financial statements as a whole. On the other hand, if the auditor identifies a misstatement, he/she needs to reevaluate the audit strategy (ISA 450.6). Identified misstatements can be an indication of internal controls that are not appropriately designed, implemented, or operating effectively (ISA 450.A7).
Information relating to the identified control deficiencies is of great interest to financial statement users and researchers, as it provides insight in the quality of the audited company’s financial reporting system. The information relating to internal control deficiencies originate from the audit file’s working papers and the non-publicly observable auditor’s report issued to management / supervisors. The internal control deficiencies are currently not always documented at a central place in the working papers, which makes this information hard to collect and centrally manage / monitor. Another complexity in collecting this information, is that control deficiency descriptions are specific to the audited organization and its personnel and sometimes hard to understand for those not involved in the audit and who do not understand the complete context. This requires several steps of (manual) cleaning and anonymizing the data to give the relevant context and ensure client confidentiality, as the deficiency descriptions usually contain names of key personnel, legal entities, products and reporting segments. Given the proprietary nature of internal control deficiencies under ISA audits, I believe that there is still much to be learned from this data for audit firms and academics alike.
When the audit firm exercises a high level of effort by experienced professionals, this is considered an important input factor for delivering audit quality. The number of hours incurred by the audit team and the level, experience, and education of those who make these hours are a proxy for audit effort (
Recently, the NBA introduced the policy (recommendation) to disclose the number of audit hours incurred in the audit report to the oversight body (or management) of the company. The audit report is usually the most reliable source for observing the audit hours incurred. Audit firms’ internal systems and billing codes are not necessarily linked directly to one (statutory) audit hour code only. This applies especially in situations with multiple audit opinions in a group audit and some additional assurance procedures, which were all negotiated and managed as one contract. Furthermore, the presence of work and billing arrangements with the international network of the audit firm can make these hours and work-in-progress reporting systems even more complex. The disclosure of the actual audit hours incurred in the auditor’s report makes it possible for auditees to observe audit effort. This is therefore a potentially important intervention to help to reduce the credence problem in auditing. To my knowledge the effect of the policy to disclose the planned and actual audit hours incurred to the client (as announced by the NBA) has yet to be studied.
Another interesting variable in the auditing process is the number of specialists involved (
The specialist and consultation information is usually available in the audit files and in central databases of the audit firms and can therefore be collected and analyzed relatively quickly. More details relating to the nature of the consultation contain proprietary information that needs to be manually cleaned. Furthermore, the consultation policies differ between audit firms. Where certain audit firms require specific complex or high-risk subjects to be consulted, this is a free choice of the audit partner in other firms. On an audit firm level this information is disclosed in the transparency reports, which are only prepared by the six audit firms with a license to audit PIE’s. The practical question for audit firm managers might be how this information could help them to ensure that the right questions are being asked at the right time in the audit process. Furthermore, from discussions with professional practice managers and specialists I understand that the completeness and timelines of consultations are a regular concern of the audit firm. Central monitoring of specialist involvement, consultations and client complexity might help to identify engagement that tend to under- or over-utilize specialist resources and help prevent accidents from happening.
If an auditor finds more material misstatements, this is considered an important output factor of audit quality. The correction of material misstatements in the financial reporting process is an important, perhaps the most important, added value of the audit. The misstatements that an auditor identified during the audit are therefore of great interest to the regulator, audit firms and the research community. The misstatements, both corrected and uncorrected, are not publicly observable and form a unique insight into the black box of auditing. The corrected misstatements always need to come from the audit file that the audit team had prepared to support their audit opinion. A general issue with collecting this data is that a clear first version of financial statements is not always documented and explicitly identified in the audit files. From a behavioral and client-relationship perspective it is attractive for an auditor that the client itself concludes that a certain accounting treatment is incorrect and corrects it. Therefore, there is little incentive for the auditor to document all the identified and corrected misstatements in the process. On the other hand, structural documentation of first unaudited versions of financial statements can provide unique insights into the added value of the audit. Central (electronic) monitoring and active discussions over documented corrected misstatements might help here.
As part of that Letter of Representation (LoR) the audit client needs to confirm and sign-off that it is the opinion of its management that the uncorrected misstatements are immaterial for the financial statements as a whole (ISA 450.14). In analyzing this data, the assessment of the client makes it hard to attribute uncorrected misstatements solely to the auditor, as the client also needs to make a final assessment. Structural analyses of the differences between the first and final audited version of financial statements, could give stakeholders to the auditing profession some important insights.
There is room for improvement of the audit quality data sources discussed in this paper. Stakeholders of the audit profession would benefit from a cost-free complete central publicly assessable register with standardized filings like this is available in for example the UK, Germany, Belgium and for the SEC registrants. There is currently no central database with all Dutch statutory audits, the corresponding audit opinions, including going concern and restatement information, the responsible audit firms and partner. Having such a central register would make it easier for audit clients to select an audit firm and audit partner based on audit quality metrices and not (only) on the audit fee.
The profession and stakeholders could learn from publicly available AAERs in a central European or AFM database. I would also advocate that the bankruptcy register for insolvent medium sized and large entities is not erased after only six months but remains assessable for a long period. Audit firms and stakeholders could also benefit from a more structural stakeholder feedback cycle relating to assurance services. Where audit firms already survey their employees on various aspects, it might be insightful for them to learn from survey research (techniques) on various topics.
A serious concern is the timeliness and completeness of trade register filings in the Netherlands, including the non- or incomplete filing or of financial statements (
In relation to the data sources of identified internal control deficiencies and audit misstatements, I recommend that audit firms centrally monitor, analyze and act on this data as part of their quality system. In addition, central monitoring of specialist involvement, consultations and client complexity might help to identify engagement that tend to under- or over-utilize specialist or consultation resources and help prevent accidents from happening.
With this discussion I hope to provide insights for the interested reader in what audit quality data (sources) in the Netherlands are available, under which limitations and the corresponding information value.
WP Tjibbe Bosman RA MSc – PhD researcher in auditing at the Amsterdam Business School of the University of Amsterdam and research program manager at the Foundation for Auditing Research.
The author thanks Chris Knoops (editor), Anna Gold and an anonymous reviewer for their valuable comments and recommendations, they significantly improved the quality of this article. The author furthermore thanks Wendy Groot, Olof Bik, Jan Bouwens, Willem Buijink, and Joris Roosen for their valuable input and encouragement, all remaining errors are my own. The author thanks the FAR for its research grant 2019F02. The views expressed in this document are those of the author and not necessarily those of the FAR.
For a discussion see also
See the special MAB issue on this topic https://mab-online.nl/issue/1620/. For a literature study that maps international archival audit research to the IAASB Framework for audit quality, refer to
As
For a discussion of the legal context refer to Van Geffen (2019) and the special issue of the Tijdschrift voor Jaarrekeningrecht on this topic and the guidance in Dutch Accounting Standard RJ 150.
For example, the AFM only published the detailed court decision relating to this. In the “Deponeringen” view at the KVK, one can only see the date of the Art 2:362 sub 6 BW filing, not the year it relates to. Discussions with senior employees at the KVK, learned that one needs to buy all financial statements and manually through the several financial statements to try to identify which year and to which financial statement line items the restatement relates to. I believe this is a sub-optimal situation for such important filings for financial statement users.
The author thanks Ton Meershoek for explaining the information that is publicly observable.
See
Questions such as: “From your impression of colleagues in your direct working environment in general, to what extent do you think your colleagues…”
The Dutch Ministry of Finance approved several models for the design of the financial statements (Besluit Modellen Jaarrekeningen) however this means that there is variation in the model that companies chose. Furthermore, companies are free to and regularly do rename or add financial statement line items in the approved models. This makes it harder for the researcher to structure the data. The same applies for financial statements prepared under IFRS (IAS 1). Recent developments such as the Primary Financial Statements Project of the
For example, the Dutch audit market of client fiscal year 2018, only 852 of the 19.870 yearly statutory audits (
FAR manually collected financial statement data for more than 6.000 company years for her research projects.
Observation made by the author from practical experience, several discussions with senior employees at the KVK and the observation that 18% of firms do not file within the maximum legal filing period (
A company qualifies as medium-sized as two of three criteria are met on two subsequent years: assets > EUR 6 m, revenues > EUR 12 m and/or more than 50 average employees. The criteria used to be assets >EUR 4.4 m, revenues >8.8 m and/or more than 50 average employees for fiscal years starting before January 1, 2016.
The responsible “Bureau Economische Handhaving” of the Dutch tax authorities does not even have an own website or public (e)mail address. The author was only able to contact the “Bureau Economische Handhaving” by information that was provided from a different authority. There seems to be no central place where financial statement users can report missing information in the trade register.
For a discussion, see some blogs by law firms such as https://thuispartners.nl/nieuws/artikelen/de-risicos-van-het-te-laat-publiceren-van-de-jaarrekening-verwaarloosbaar-of-re%C3%ABel, https://www.rechtnet.nl/jaarrekening-niet-deponeren-en-dan/ and https://frankfortsluis.nl/de-gevolgen-van-niet-tijdige-deponering-en-de-mogelijkheden-om-deze-af-te-wenden/.
Companies with equity instruments listed on regulated markets or unregulated markets when total assets exceed EUR 500 m.
Discussed Audit Quality topics and measurements, their information value, limitations, and availability in the Netherlands.
Topic / Measure | Rationale / Information-value | Limitations | Availability in the Netherlands |
---|---|---|---|
3.1) Restatements | If audited financial statements need to be restated, this is a relatively hard indication of a lack of audit quality of the restated audit. | Extreme relatively seldom occurring events. | Most misstatements are corrected in the next years financial statements, with a correction of the comparable figures and a corresponding disclosure. A researcher needs to go through several PDFs of financial statements an organization to be able to identify such misstatements. The restatements in accordance with Art 2:362 sub 6 BW are very rare and are hard to identify in the KVK. Several PIE audit firms report on this topic in their transparency report. For equity listed PIEs available from January 2018 onward in Audit Analytics Europe. |
3.2) Lawsuits against or convictions of the responsible audit partner | A professional disciplinary conviction of an auditor is an indication that he/she has not delivered audit quality by complying with the regulations. | Depends on the size and risk of the audit (firm/partner) portfolio, which is not directly observable. Furthermore, these are relatively extreme events. It is very hard for outsiders to link the convictions to the audits they relate to, due to the anonymization of information. | The names and sentences of convicted auditors are available in the NBA register for a period of 10 years after court proceedings. This register is hard to access and has no download option. |
3.3) Accounting and Auditing Enforcement Releases (AAERs) | If the regulator issues a regulatory action against an audited company and/or its auditor, this is a sign of a lack of audit quality. | Relatively seldom and extreme event in the Netherlands. Most of the interactions between the financial market regulator and the audited organization from the comment letters process is unobservable. | Only the restatements resulting from this process are publicly observable (“Aanbevelingsbericht”). Where these are relatively rare, and the last instance was in 2014 (as of November 2020). |
3.4) Going Concern Opinions (GCOs) and type II error rates | When an organization files for bankruptcy and an auditor did not issue a GCO, this is a sign of a lack of audit quality. | Relatively seldom and extreme events. The information the auditor had at the time of the issuance of a GCO might not indicate any bankruptcy risk for the next 12 months. In addition, a too conservative stance on GCOs (type I errors) might cause the CGO to lose its informational value for stakeholders. | Available in the PDFs of filed financial statements. There is no central database with the Dutch audit opinions. Furthermore, the Dutch bankruptcy register is erased six months after court proceedings. For equity listed PIE’s this information is included in Audit Analytics Europe from 2010 onwards. |
3.5) Inspection Outcomes | The outcomes of audit file inspections of internal and external inspectors provide a direct feedback on audit (file) quality. If work is not documented in the audit file it is not considered to be done (properly). | The sampling of audit files subject to inspection is seldomly pure random (mainly risk-based) and therefore subject to selection bias. Furthermore, the inspection outcome usually only tells something about the quality of the audit file and not the entire audit process. | Disclosed for audit firms with a PIE license in their transparency report. Individual audit file inspection outcomes or the audit partners they relate to are not disclosed. Where the AFM now can inform the auditee of adverse audit file inspection outcomes. |
3.6.a) Earnings Response Coefficient (ERC) | When investors respond to audited financial statements intensely, this is an indication of the level of confidence they have in the audit. | Relatively noisy measure that is only available for equity listed entities (limited population). Furthermore, it is hard to disentangle which part is due to the auditor and which part is due to the company / credible management. | Is available, but the metric only applies to listed companies, which is a relatively small sample in the Netherlands. |
3.6.b) Stakeholder feedback / ratings | When stakeholders value the audit, this is an indication of audit quality. Furthermore, service quality is also a factor in the auditing landscape. | Important to understand the sampling and survey techniques applied. There is a potential risk of only surveying friendly clients or stakeholders (sampling bias) and receiving socially desirable answers. | Not always structurally collected by audit firms after each audit. Collecting is hard and subject to several GDPR concerns and complexities. |
3.7) Values, Ethics and Attitudes | Provide information about what is considered acceptable behavior in the audit firm, how the firm is perceived to deal with important issues and how the error and learning climate is. | General risks of survey research such as providing socially desirable answers and dependency on self-reporting. | Highlights are disclosed in the transparency reports of PIE audit firms. The surveys and survey techniques differ between audit firms. |
4) Financial reporting quality | The financial statements are the object of the audit. Therefore, variation in financial reporting quality gives an indication about the quality of the audit. | Relatively noisy (imprecise) measures, there is not always academic consensus over how to measure them, are susceptible to measurement error, potentially sensitive for design choices and possibly subject to selection biases. Furthermore, it is hard to attribute the quality of the financial statements to the auditor or other factors. | Proxies include small earnings (surprises), earnings discontinuities, (discretionary) accruals (the level of earnings management) and disclosure quality proxies. These can all be collected in the Netherlands. Practical concerns exist relating to the enforcement of / compliance with filing requirements for private companies and the role of auditors therein. |
5.1) Client Acceptance and Risk Assessment | When an auditor does not accept or dismisses high-risk clients, this is considered a sign of conservatism which is usually interpreted as a sign of audit quality. | There is a risk that the biggest audit firms only accept easy to manage and well behaving clients and so pass the problem on to smaller audit firms. The non-acceptance of audit clients can only be observed if they fail to find any auditor. The non-continuance of an audit engagement can be observed but is hard to disentangle whether this is due to the client, the auditor or some (independence) regulation. | Information on an audit firm level disclosed in the transparency reports of PIE audit firms. Overarching database with all Dutch statutory audits is not available. |
5.2) Internal Control Deficiencies | If an auditor identifies internal control deficiencies and has recommendations to improve the companies’ internal control system, this is interpreted as an indication of audit quality. | Not all audits rely on the internal controls of a company and the internal controls are not the audit object of the auditor. There is anecdotal evidence that auditors are using internal controls less and less to obtain their audit evidence. Furthermore, the base rate of internal control deficiencies varies from organization to organization and cannot be observed, where auditors are more inclined to rely, evaluate and report over internal controls for relatively better organized organizations. | Available in the audit files, management letters and auditor reports to the clients’ oversight body. But not publicly observable as it is confidential information. |
5.3) Audit Effort | When the audit firm exercises a high level of effort by experienced professionals, this is considered an important input factor for delivering audit quality. | High effort might also be a sign of inefficiency. | Information on an audit firm level disclosed in the transparency reports of PIE audit firms and on an individual level disclosed to the auditees in the (non-publicly observable) auditors report. |
5.4) Specialists Involved and Consultations | When an auditor engages specialists and consults technical issues, this is considered an important input factor for delivering audit quality. | A high number of consultations might also be an indication of an auditor that is not willing to take ownership of important decisions in the audit. | Information disclosed on the audit firm level in the transparency reports of PIE audit firms. Not publicly observable on an individual engagement level. |
5.5) Identified Audit Misstatements | If an auditor finds more important misstatements, this is considered an important output factor of audit quality. The correction of material misstatements in the financial reporting process is an important, perhaps the most important, added value of the audit. | There is seldomly a clear first and documented version of the financial statements prepared by the client. Therefore, this measure might be incomplete. Furthermore, not all auditors comply with the internal regulations to document all misstatements in a certain field in the documentation system and include an attachment to the LoR with the uncorrected misstatements. Finally, the base rate of misstatements in the first version of financial statements is unobservable and varies between organizations and years. It is unobservable how many material misstatements the auditor did not find, other than subsequent restatements. | Documented in the audit files and as an appendix to the Letter of Representation (LoR) signed by the client. Not disclosed in the transparency reports or elsewhere. |