Print
From performance measurement to performance management in the impact investment industry
expand article infoJeroen van Raak, Amber Raaphorst§
‡ Universiteit van Amsterdam, Amsterdam, Netherlands
§ KPMG, Den Haag, Netherlands
Open Access

Abstract

Impact investments have the potential to play an important role in solving social and environmental problems. Although the sector is growing rapidly, it does face a number of challenges, in particular related to impact measurement. Measuring the impact of such investments, which aim to achieve social and/or environmental impact while simultaneously generating financial returns, has proven difficult.

This study examines the design and application of measurement systems related to impact investments. To investigate this, the seven impact measurement guidelines of the IMWG are used as a framework. We study to which degree impact investors set concrete investment objectives, how they measure and collect data related to the generated impact of the investments, and how they use such data to evaluate investment opportunities. We rely on a qualitative research methodology, including 13 semi-structured interviews among Dutch institutional investors.

We find that impact investors typically set general, but not specific impact objectives. Furthermore, we note that impact investors are still searching for and experimenting with performance measures, and that they would value the development of standardized measures. Such standardized measures may assist in reducing the cost of obtaining investment data, while simultaneously increasing data reliability. Although the obtained impact data is currently hardly used for external reporting and impact data driven investment decisions, the institutional investors expect this to happen in the near future as the process of impact measurement matures. This would enable institutional investors to transition from performance measurement to performance management in the impact investment industry.

Keywords

Impact investing, impact management, impact measurement, institutional investors, impact driven investments

Relevance to practice

This paper outlines the design and application of measurement systems in the impact investment industry by institutional investors, and suggests the need for better impact measurement in order to support management decisions related to impact investments. This will support the growth of the impact investment industry.

1. Introduction

Investors experience an increasing pressure to integrate environmental and social performance into the selection of their investments (Dyck et al. 2019). In fact, insurance companies and banks have already committed billions of dollars to impact investments.1 Such investments are aimed at generating both social and/or environmental goals as well as financial returns (Hebb 2013). In particular, the capital from institutional investors is key for the development of the impact investment industry (Wood et al. 2013) and can eventually contribute to solving problems related to food scarcity, climate change, inequality, clean water, and human rights (Brown 2009; Hummels and Fracassi 2016).

Although promising, the impact investment industry still faces many challenges. The impact investment market is surrounded by inefficiencies through poor coordination, duplicative activity, and ambiguities in language (Olsen and Galimidi 2008). Despite progress in the development of social impact metrics (Jackson 2013b), various studies emphasize that impact measurement is one of the biggest challenges to address (e.g., Grabenwarter and Liechtenstein 2011; Jackson 2013b; Mudaliar et al. 2016).

Performance measurement and reward systems are widely used by companies to align interests and increase economic performance (Lingle and Schiemann 1996). Indeed, Ittner et al. (2003) show that more extensive use of financial and nonfinancial performance metrics is related to higher stock market returns in the financial industry. Whereas generally accepted accounting principles (e.g., US GAAP and IFRS) are established to measure and report on the economic impact at an organizational level, quantifying the impact on society for a wide range of impact investments is a difficult task. This is further complicated by the reliance on typically sensitive social and economic data as inputs for the performance measurement systems (Jackson 2013a). Consequently, current practice in performance measurement tends to focus on measuring only part of the total impact that organizations have on society (Maas and Liket 2011).

To improve impact measurement practices, the Social Impact Taskforce (as established by the G8) created the Impact Measurement Working Group (IMWG). Their aim is to provide a common foundation in impact measurement practices, which could be accustomed to investments and operations of the individual impact investor (IMWG 2014).2 Improved measurement facilitates performance evaluation (Jackson 2013b), which helps to better understand and improve the impact performance of future impact investments (Mudaliar et al. 2016). This is crucial for the growth of the impact investment industry (IMWG 2014). Through thirteen interviews with Dutch impact investors, we examine how impact investors design and implement impact measurement systems.

This study contributes to the literature on the impact investment industry by analyzing the design and application of measurement systems related to impact investments by institutional investors. We provide insight into the drivers underlying the adoption of performance measurement systems for the evaluation of past investments and the screening of new investment opportunities. We also contribute to the literature on managerial accounting by examining the use of performance measurement in a setting where non-financial information is arguably of greater importance than financial information.

Our findings show that most of the institutional investors do not yet set specific performance objectives, but that they believe that setting impact objectives will become increasingly important in the future. Subsequently, impact investors signal a need for standardization in the way impact is measured (i.e., through standardized measures), and that there is a need to create learning platforms to achieve this. Collecting and validating such performance measures is however costly and time consuming, making it important for investors to be selective in terms of the data which they ask investees to provide. External agencies, such as assurance providers that assess and verify operational, social and environmental performance, could play a role in validating the data and increasing the reliability and consistency of the data. While a lot of data is currently gathered, we note that the data is hardly used to benefit future investment opportunities or to report the generated impact to stakeholders. The interviewees, however, expect data driven investment management and the external reporting of impact data to become more important in the near future.

The remainder of this paper consists of the following sections. Section 2 provides an overview of the prior literature and discusses the context of impact investments and the seven impact measurement guidelines as developed by the IMWG. Section 3 outlines the research design. Subsequently, section 4 presents the main findings of the conducted interviews. The final section provides the conclusion and implications of this study.

2. Literature review

This section first discusses impact investing in general in order to provide a clear context for this study. Subsequently, we look at the various types of impact investors. Finally, we describe the seven impact measurement guidelines as developed by the Impact Measurement Working Group (2014) and discuss related prior research.

2.1 Impact investing

Prior research identifies different types of investments. Traditional investments are investments with a core focus on financial returns, with limited regard for environmental, social or governance factors. In the 1980s, Socially Responsible Investments (SRI) came to existence. These are investments which integrate various non-financial targets, such as ethical, environmental or social targets (Sandberg et al. 2009).3 These investments are typically identified by excluding specific organizations or industries.4 Sustainable investments occur when organizations adopt progressive environmental, social and governance practices which might enhance the financial return of the investments in combination with a greener focus than responsible investments. Impact investing, instead, relates to a proactive approach aimed at achieving a direct positive impact on social and environmental practices, while maintaining a targeted financial return (Duiker et al. 2016). In contrast to traditional investments, impact investments typically take place in sectors that experience difficulties in obtaining financing (Simon and Barmeier 2010). The investments mostly target domains with market frictions, such as: imperfect information, skepticism about the ability to achieve a financial return as well as a social impact, inflexible institutional practices, smaller deals, markets with limited exit strategies, and governance problems (Brest and Born 2013).5

2.2 Impact investors

Jackson (2013b) distinguishes four types of actors in the impact investment industry: asset owners, asset managers, demand-side actors and service providers. The asset owners and asset managers can be classified as impact investors and consist among others of: high net worth individuals, corporations, governments, retail investors, foundations, fund managers, commercial banks, impact investment funds, retirement funds and development finance institutions (Jackson 2013b).6

In general, impact investors can be categorized in two types: Impact First investors and Finance First investors. Both types of investors aim to achieve social or environmental goals in addition to financial returns, but Impact First investors prioritize the social objectives, whereas Finance First investors prioritize the financial objectives. Impact First investors are willing to accept returns below market rate and are typically foundations, family offices and social investment funds (Ragin and Palandjian 2013). Although Finance First investors are committed to seek market returns, they are also crucial for the impact investment market, since they can scale the investments through which eventually economies of scale can be achieved (Ragin and Palandjian 2013). Finance First investors are usually institutions like retirement funds and commercial banks.7 Overall, both types of impact investors are engaged in different stages of the development of the impact investment industry, since Impact First investors use their market position to test and provide a proof-of-concept, and Finance First investors step in at a later stage when the Impact First investors have proven viability.

2.3 The guidelines for impact measurement

This subsection discusses the seven impact measurement guidelines as developed by the IMWG: 1) impact objectives, 2) impact framework and metrics, 3) collect and store data, 4) validate data, 5) impact data analysis, 6) reporting, and 7) data driven investment management decisions. We follow this framework, since it fits the research question by giving an exhaustive overview of the design and application of measurement systems related to impact investments. We discuss each individual guideline and related prior research below.

2.3.1 Impact objectives

Setting an impact goal is the first step towards measuring and managing the performance of impact investments (Mudaliar et al. 2016). An impact goal relates to the social and/or environmental impact that an investor tries to achieve through its investment.8 Like philanthropists, impact investors may aim for a wide range of social and/or environmental goals, which are subjective in nature (Brest and Born 2013). Impact investors may formulate objectives such as supporting clean cooking in developing countries, providing bed nets to prevent malaria, or the reduction of greenhouse gas emissions.

Simon and Barmeier (2010) state that organizations should initially focus on one or two more specific social or environmental goals which are in line with the organization’s mission, and measure the progress towards these specific goals. Concrete impact expectations and objectives should be formulated prior to the investment. A good understanding of the relevant goals is required for high-quality decision-making and helps to gain insight into the extent to which the objectives were achieved and how future investments may be improved (Maas et al. 2016).

2.3.2 Impact framework and metrics

There is a vast literature on management accounting and accountability that shows that performance measurement systems are used by companies in production and service industries (e.g., Ittner et al. 2003). Companies use financial metrics and forward looking, non-financial, measures to create economic value (Ittner and Larcker 1998; Ittner et al. 2003). Furthermore, Adams and Frost (2009) show the importance of key performance metrics for decision making, planning and performance measurement. Perego and Hartmann (2009) show that the quantification of environmental measures can improve the alignment with the environmental strategy.

Although the identification of successful investment strategies and opportunities is crucial for the success and growth of the impact investment industry (e.g., Jackson 2013b; IMWG 2014; Mudaliar et al. 2016), measuring the outcomes or impacts of investments is often a difficult task (IMWG 2014). In particular, it may be difficult to identify which outcomes can be directly attributed to the impact investments. Moreover, impact measurement is complex and constrained through the intangible nature of outcomes, and the required resources to quantify and describe these outcomes. Furthermore, the industry struggles with sensitive social and economic data, as well as limited data access (Jackson 2013a). Hence, these factors explain why impact investors are slow in adopting performance measurement systems despite the clear benefits.9

Many different social impact measurement models have emerged over the past few years (Maas and Liket 2011).10 These models help organizations to measure, manage and report the impact they generated and to take responsibility to those whom they aim to serve (IMWG 2014). The impact data should eventually provide insight in the extent to which value was created for the stakeholders. Standardization, analysis and reporting of such data can subsequently help impact investors to more efficiently raise and allocate capital in the future. This, in turn, should enable impact investors to generate more impact, while increasing transparency and accountability (IMWG 2014).

2.3.3 Collect and store data

For investors to take impact into account, they first need to identify the impact itself. The role of the stakeholders is crucial in the impact measurement process, since the impact an investor creates is ultimately dependent on the investee (Brest and Born 2013). Furthermore, a potential investee should find a suitable way to create, measure and communicate impact (e.g., by filling out a survey that was sent out by the investor). Hence, investors should examine the degree to which an investee can meet the objectives of the organization, and it is important that the interests of the impact investor and investee are aligned (Epstein and Yuthas 2014).

While it is important to track key outcomes in order to help investors make more informed decisions based on relevant data (Golden et al. 2010; Brest and Born 2013), it is essential that data requests from the impact investor are in line with the daily business of the investee and the investees information needs (IMWG 2014). An overload of indicator reporting is an unnecessary burden for the investee, and may distract the investee from achieving their impact objectives and enhancing their organization (Simon and Barmeier 2010; Reeder and Colantonio 2013). Moreover, impact measurement requires financial resources, which can lead to higher transaction costs for institutional investors and investees, thereby making impact investments less attractive from a financial perspective. Hence, impact investors need to examine which data is worth collecting.

2.3.4 Validate data

The impact investment industry is a relatively young market, hence limited data about impact investment performance is available (Ragin and Palandjian 2013). This implies that existing data often has to be evaluated in an isolated manner, due to the lack of benchmarks (Saltuk et al. 2015).

There are several actors in the field of impact measurement which offer a direct, measurement-related service to impact investors. These are actors that are specialized in impact measurement advice, benchmarking and reporting specialists. Examples of well-known organizations are Sustainalytics, Trucost, B Analytics and MSCI. These intermediaries can facilitate the selection of investments which have a high likelihood to achieve a considerable impact. Although such initiatives are a good start, the practices of these organizations are currently not universal (Ragin and Palandjian 2013).

2.3.5 Impact data analysis

An impact investor needs to analyze data to assess the progress toward the goals of the investment. Where relevant and feasible, this analysis needs to incorporate the achieved contribution or attribution. To monitor goals, key indicators and targets are essential. A good impact analysis typically compares the key performance indicator against a historical baseline, a forecast, or industry peers (Behn 2003).

When the impact data is analyzed, it is important to look at the indicators which did not contribute to decision-making and reporting (IMWG 2014), since requesting unnecessary KPIs creates a burden for the investee (Simon and Barmeier 2010). It is however unclear if impact investors have obtained sufficient expertise to evaluate the outcomes (Duiker et al. 2016).

2.3.6 Reporting

There is an increasing interest in non-financial information (Cecil 2010; Eccles and Krzus 2010; Eccles and Saltzman 2011). Reasons for organizations to report nonfinancial performance are to manage the perception of stakeholders (Skouloudis et al. 2010; Ragin and Palandjian 2013), to show the organization’s value to the world, and to prove that the activities of the organization are in line with social norms (Simnett et al. 2009; Cecil 2010). In addition, communicating and reporting of the achieved impact keeps organizations accountable to their stakeholders (Skouloudis et al. 2010; Ragin and Palandjian 2013). Simon and Barmeier (2010) and Ragin and Palandjian (2013) state that impact investors should be transparent about the social impact which the investments have generated.

2.3.7 Data driven investment management decisions

Maas et al. (2016) state that sustainable performance measurement, reporting and management can be used to create transparency and facilitate decision support to improve performance. Similarly, Jackson (2013b) states that information about the performance of investments can lead to improved selection of new investments. Furthermore, good outcome performances can enable organizations to attract new investment capital (Ragin and Palandjian 2013). The ability of companies to attract capital and to direct it to quality impact investments will determine the growth of the impact investment industry. In fact, Hummels and Fracassi (2016) find that there are ample investment opportunities, but that most do not comply with the size and risk-return impact profiles of institutional investors, making it harder to use previous impact data for new investment decisions. According to Jackson (2013b), the impact investment industry could become more robust, resilient and self-sustaining through impact evaluation.

2.3.8 Summary

From the discussion of prior research and the seven guidelines on impact measurement it has become apparent that performance measurement is a crucial aid to evaluate prior investments and to identify and select good future investment opportunities. However, it is unclear to which degree impact investors have adopted performance measurement systems since social and environmental impact is typically difficult to quantify and because collection of impact data is time consuming and puts a burden on the investees. In the next section we discuss the research methodology which is used to examine to which degree impact investors measure social and/or environmental impact and to which extent they currently incorporate and expect to incorporate this information for future investments and related reporting.

3. Methodology

This section discusses the methodology chosen to conduct this research. In the first section the research method is outlined. Hereafter, the data collection is discussed. Finally, the data analysis is presented, in which is explained how the collected data is analyzed in order to draw findings from it.

3.1 Research method

Given that the application of measurement systems by impact investors is a relatively new phenomenon, this study aims to explore how impact investors design and apply measurement systems. We use a qualitative approach to study this research question, as this approach is suitable when limited quantitative data is available and when ongoing events are studied (Yin 2015). For this research a semi-structured interview method was used. With the semi-structured method, mostly open-ended questions are asked, and an interview guide is formulated beforehand. This method enables the interviewer to probe for more information and clarification (Qu and Dumay 2011).

3.2 Sample characteristics

Hubbard (2009) states that many organizations try to emulate the strategies of successful companies. In this sense, it is important to analyze the design and application of measurement systems by top institutional investors which are engaged in the impact investment industry. Therefore, institutional investors were selected based on total assets under management, current performance on impact investing, or the impact investments they have planned for the near future.

The total sample includes thirteen Dutch institutional investors, consisting of three banks, two pension funds, one insurance company, five asset managers, one development finance institution (DFI), and one private family office. Currently, the Dutch institutional investment market has 1.7 percent of the total investments engaged in impact investments, which is equivalent to 24 billion Euros in impact investments (Duiker et al. 2016). The majority of the sample consists of Finance First investors. There is only one Impact First investor and one institutional investor which falls under both categories.

In total thirteen interviews were conducted with seventeen interviewees, of which nine male and eight female interviewees. In four of the thirteen conversations there were two interlocutors, in the others there was one interlocutor. The interviewees had a variety of different functions, among others: fund managers, impact investing analysts, impact investment managers, impact evaluation officers, and directors responsible for the impact investing practices. Table 1 provides an overview of the institutional investors and the interviewees.

Table 1.

Overview of investors and participants. The table shows the type of investors and the job functions of the interviewees.

Investor Type of investor Participant Job function(s)
1 Pension fund 1 Senior advisor responsible investments
2 Bank 2 Manager institutional relations
3 Asset management company 3 Senior sustainability specialist
4 Bank 4 Manager social impact fund
5 Asset management company 5 Director Netherlands
6 Senior relationship manager
6 Insurance company 7 Strategy consultant sustainability
7 Development finance institution 8 Senior impact evaluation officer
8 Asset management company 9 Co-founder
9 Private family office 10 Impact investment manager
11 Head of operations
10 Asset management company 12 Managing partner
11 Bank 13 Commercial director
14 Manager corporate communications
12 Asset management company 15 Impact investing analyst
16 Impact investing analyst
13 Pension fund 17 Advisor responsible investments & governance

3.3 Data collection

The semi-structured interviews were conducted while keeping a pre-determined guideline in mind. The guideline was divided into eight parts, where the first questions were related to impact investing in general, followed by questions related to the seven impact measurement guidelines of the IMWG. The guideline consisted of 30 pre-determined interview questions, without a chronological set-up. The interview guide is shown in Appendix 1. Out of the interviews, eleven interviews were held face-to-face and two interviews were done by telephone.

Both types of interviews, face-to-face and telephone, were recorded with permission of the respondents. In addition, field notes were taken which have been used as a supplement for the transcripts. The average time per interview, counted from the first until the last interview question, is 56 minutes. The recorded tapes were listened and transcribed immediately after the interview. Transcribing interviews directly after they are conducted is advantageous, since it makes transcribing easier when the interview is fresh in mind (Longhurst 2010). In addition, this also ensured that the field notes could be added to the right parts in the transcripts.11

4. Results

In the following sections the main findings of the interviews are discussed. A number of institutional investors from the sample indicated that they prefer to stay anonymous, therefore we opt to anonymize their responses. In the first section, general results with respect to impact investing are presented. The second section deals with results related to the design and application of measurement systems by institutional investors.

4.1 Impact investing

4.1.1 Impact definition and characteristics

When asked to define impact investing, the interviewees mentioned that numerous different definitions of impact investing are used both in the Netherlands and internationally, suggesting that no definition of impact investing is used consistently among investors. This is in line with studies that indicate regional differences, as well as differences between institutions, and individuals (Jackson 2013b; Calderini et al. 2018). Despite the fact that the participants mentioned different definitions and characteristics of impact investments, five out of thirteen of the investors referred to the impact investing definition of the Global Impact Investing Network (GIIN).

Core characteristics of impact investing that were referred to are intentionality, impact measurement, investments with a social return as well as a financial return, and positive change. However, none of the impact investors referred to the attribution characteristic of impact measurement when asked to define impact investing. While it is feasible for investors to identify whether an investment contributed to the realization of results (i.e., the contribution approach), it is undoubtedly more complicated to accurately identify an investment’s impact while accounting for simultaneous investments by other investors and other factors (such as changing social and economic conditions) that may have contributed the achievement of results (i.e., the attribution approach). When asked further, three interviewees indeed indicated that measuring attribution is really difficult, and sometimes impossible. Participant 4 describes the issue as follows: “Attribution is very difficult to measure. There are certain assumptions that arise from research. On the basis of such assumptions we can report data at an outcome level”. One of the interviewees mentioned that the measurement of attribution is not so important for the organization, as they find it sufficient when the invested money leads to solutions for social problems within the specified themes of the organization. This interviewee claimed that attribution is very difficult to measure, especially in the case of liquid markets, because the organization then only plays a small role in the total investments in the investee. Only in the private market, infrastructure, and private equity, attribution can sometimes be expressed (Participant 1). Therefore, it may not be surprising that only one participant tries to measure attribution. Another interviewee stated to look at contribution instead: “We let go of attribution. We look at contribution instead” (Participant 10).

4.1.2 Impact investment market

The investment developments in the social field contribute to new business models. With respect to the business models the following was indicated: “First we had finance driven business models, then finance and risk business models and currently finance, risk and sustainable driven business models” (Participant 7). Several interviewees indicated the need for a growing impact investment market, and one of the interviewees stated that “Conventional financing is not unlimited” (Participant 1). In addition, impact investors search for beneficial solutions to address problems in the economy (Participants 1, 2 and 5). However, currently the impact investment industry is not yet seen as successful. Only one third of the interviewees stated that the impact investment industry is a success already, while the majority was more skeptical. One of the interviewees argued that impact investing is successful when using existing vehicles, like green bonds, but that it is not successful when you talk about the best-in-class approach (Participant 7).12 This interviewee argued that this is not the way impact investments are supposed to be carried out, because: “In that case you claim to make impact, although your contribution as an organization is limited” (Participant 7). In contrast, another interviewee stated: “Often impact investing comes in combination with a best-in-class approach. This is in particular an appropriate solution for institutional investors” (Participant 5). Another interviewee stated that the impact investment industry is not successful since it is not common practice. Moreover, some interviewed institutional investors indicated that too little scalable impact investment opportunities exist. This is in line with Hummels and Fracassi (2016), who indicate that the largest problem for investors is finding suitable investment opportunities.

4.1.3 Impact investment developments

To gain more insight into the expected developments, the interviewees were asked about their view on future developments. Eight institutional investors indicated that impact investments will be a success in the future, while five institutional investors were more skeptical. An increasing demand for impact investment opportunities combined with growing prosperity could lead to a bright future for impact investments. One of the interviewees argued: “Forecasts and trends indicate that more money will be invested in the future, due to growing prosperity. From this perspective, you also can expect that there will be more attention to Triodos-like organizations” (Participant 11).13 Moreover, most interviewees emphasized the importance of retirement funds for impact investing in the future.

In contrast, one of the more skeptical interviewees argued: “There is a reputation risk. If one thing goes wrong it will be all over the news, and then impact investing will fall apart” (Participant 15). Another interviewee also indicated the possibility of reputation risk, and illustrated this with an example of an investment made in a company that contributes to food security through genetic modification. Such an investment might result in negative publicity as the growth hormones could potentially be carcinogenic (Participant 1).

When asked for challenges in the impact investment industry, a couple of points were indicated. Some argued that they fear that the definition of impact investing will diminish (Participants 1, 4, 5 and 8). The lack of clearness of the impact investing definition is also recognized by Olsen and Galimidi (2008) and Calderini et al. (2018). Other impact investors indicated challenges in classifying impact investments into different asset classes (Participant 5). However, it was stressed that impact investing itself does not belong to a separate asset class, as claimed by O’Donohoe et al. (2010). Other interviewees indicated challenges with regard to impact measurement (i.e. Participant 3), which will be extensively discussed in the next section.

4.2 The guidelines for impact measurement

4.2.1 Impact objectives

Most of the interviewed impact investors have a general impact goal, but indicate that they are also exploring new impact areas. Only three of the thirteen institutional investors formulated specific impact goals. Nine investors linked their impact goals, which are either general or specific, to the Sustainable Development Goals (SDGs), and eleven indicated willingness to (continue to) do this in the future. Several advantages of the SDGs which were mentioned are: it is a universal tool, it makes communicating easier, it provides a broad overview of the development area, and it gives a framework to structure investments. Some disadvantages that were mentioned are: there are too many goals which makes it daunting, the goals are too high-level, and that the system could end up being used as a marketing tool.

A majority of the interviewees indicated that they expect that navigating companies towards impact goals becomes extremely important in the future. Participant 3 stated: “It all starts with you. What do you want to achieve?”. In particular, it is suggested that companies need to think about the way in which they are distinctive, and that they need to be transparent about it. The impact goals should be integrated within the whole organization, and aligned with the mission according to several interviewed institutional investors (Participants 1, 3, and 15). This is also indicated by Simon and Barmeier (2010), who state that organizations should start focusing on one or two social or environmental goals that are in line with the organization’s mission.

The Impact First investor in the sample pointed out that they search for new investment opportunities to provide a proof-of-concept for other impact investors. This Impact First investor stated that the organization is able to do this from the viewpoint that achieving impact is priority. The interviewee gave an example of such an investment, which is an investment that supports clean cooking in rural areas in Africa. With regard to this investment, the interviewee stated: “The financial risk is very high, but the potential impact and added value are high as well” (Participant 10).

4.2.2 Impact measurement frameworks and metrics

During the interviews it became apparent that the approach to impact measurement depends on the impact commitment of the organization and its role in society. Most impact measurement models used by the interviewed institutional investors have the objective of measuring the estimated impact, but the institutional investors in the sample still search for a suitable model. From the thirteen institutional investors in the sample, twelve indicated that they recently tried or are soon going to try a new impact measurement model, suggesting that they strive to improve the way of measurement. However, there is some variation in the quality of measurement as illustrated by the following quote: “We are not much further than telling what our outreach is” (Participant 10). Despite this, the interviewed institutional investors indicated that there is no need for new impact measurement models. Instead, they indicate a need to outline the best practices of impact measurement models for each sector. According to one of the interviewees, a lot has changed in the last six years in impact measurement. The interviewee claimed that while the research of Maas and Liket (2011) indicates 30 different quantitative impact measurement models, there is now a tendency towards a universal model (Participant 4). Many institutional investors currently use impact measurement models from existing initiatives, which are then adapted to their organizations. Within the sample the most widely used models are the models of Bridges Ventures and B Lab. One interviewee, who uses both models, stated the following “The method of Bridges Ventures, that we have aligned with our organization, is actually a kind of roadmap. It starts with a radar and eventually leads to a scorecard with KPIs” (Participant 10). When asked why this model was chosen, the interviewee responded: “Primarily because the model takes risks into account, and secondly because the model focuses on actual outcomes. Besides this, we are under the impression that, since they use the model for their own investment team, the model is quite advanced.” In addition to the models of Bridges Ventures and B lab, interviewees indicated the promising expectations for the new update of SPI4 from the Social Performance Task Force, which can be used for the inclusive finance sector.

All interviewees indicated the need for standardized impact metrics. One of the interviewees stated: “We firmly believe in the standardization of metrics” (Participant 1). It is indicated that standard impact metrics may lead to less burden for the investees, who then need to fill in one survey instead of multiple surveys. Currently, the investees get an excessive amount of surveys, therefore it is important that organizations only measure the necessary metrics (Participant 5, Participant 10).

In general, standardization in the impact investment industry is important because it enables the market to become more mature. Thereby impact investments will become more accessible for investors, leading to an additional inflow of money to the industry. However, there is also a risk associated with standardization. One of the interviewees referred to the risk of: “hitting the target, but completely missing the point” (Participant 13). This may happen when institutional investors make standardization an objective in itself. The interviewee argued: “We all know the history with the high credit ratings. If you want to assess risk, you need to look at the underlying investments!”. Furthermore, this interviewee stated: “Each sector that matures, will find a fixed terminology, methodology, and best practices. Then there is uniformity”. This indicates that patience is required, as one of the interviewees mentioned: “The financial world is built in 300 years and that is not the case with this industry” (Participant 4).

The interviewees were skeptical about the possibility of reaching one general comprehensive model. One of the interviewed institutional investors argued: “Standardizing metrics is important to us. But the weighting of certain playing fields is something that cannot be standardized”. Later in the interview this interviewee referred back to this question and argued: “A procedure can be standardized, but assigning weights is full of judgments, which is something you cannot standardize” (Participant 1). Another interviewee argued about standardization: “I think it is an illusion, that through standardization, everything can be added up and becomes comparable” (Participant 8).

One interviewee argued that it would be ideal to have one model in which all indicators are available, so that each impact investor can select its own indicators within a specific sector (Participant 15). Another investor pointed out that some organizations, like banks, would be able to standardize certain methodologies, but are not able to do so since they depend on the needs and requirements of the investors (e.g., retirement funds). Therefore, this interviewee mentioned: “In the end, the large investors will set the standard” (Participant 12). Overall, according to most interviewees, impact models should be made specific for each different industry. One comprehensive model with all different sectors is not beneficial according to the interviewees. It is hard or even impossible to create systems that can accommodate all different intentions in the world. Eventually, the management of impact data should be developed in the same way as the management of financial data (Participant 1, Participant 15).

One of the participants argued “There is a need for a platform in which companies can share their methodologies” (Participant 7). This was supported by another interviewee who mentioned that they are working with another organization to collect and aggregate the data of investees, so they can increase knowledge about certain impact areas. This information can subsequently be shared with others to lower the cost of measurement (Participant 9).

4.2.3 Collect and store data

The majority of the interviewees collect data through investee surveys. During the interviews several interviewees indicated that good cooperation with the investees is extremely important, and that in general this cooperation is improving. One of the interviewees stated: “We do not only invest in the investee, but we are working quite close with the entrepreneurs. It is not unusual for one of our employees to assist at the investee’s location at the beginning of a deal” (Participant 9). However, sometimes the collection of data is difficult: “Sometimes you expect a series of data, but you receive nothing because there was a flood. That is the reality with impact investing. With all these ifs and buts, you should be happy with whatever data you get. Why should you only interpret numbers? We never interpret these numbers without context” (Participant 13).

Some of the interviewees pointed out that the investors find metrics alone insufficient, and that they feel the need for more background information on their impact investments. This information could include but is not limited to the gender of the supported group, or characteristics of the geographical investment area (e.g., rural/urban area) (Participant 15).

Impact measurement requires a lot of resources, as substantiated during the interviews. One of the interviewees argued: “Impact measurement is time-consuming, specifically for your workforce, and is not easily done. Besides this, it is often costly to let other organizations do the research” (Participant 12).14

4.2.4 Validate data

When asked about the manner in which they guarantee data accuracy, the answer of one of the interviewees was: “Our ambition is to be able to do impact audits, but we still have a long way to go. We have no solutions for data accuracy. You just trust what they are doing” (Participant 10). Another interviewee argued the following: “Internally, we have an annually assessment of the focus sectors and look at several projects that we adopted five years ago. We subsequently look back at the approval documentation and examine which effects were expected” (Participant 8).

According to Ragin and Palandjian (2013), data agencies can play an important role in evidence-based practice. However, data providers do not always have comparable data. “It becomes interesting when they have many comparable datasets which I can compare with the investment” (Participant 12). One of the interviewees stressed: “Not only do measurement methods need to be standardized, but data agencies need to strive to a certain quality label and specific standards. There are currently too many different data agencies” (Participant 5). Some interviewees indicated that data providers need to merge in order to provide a large accessible database. Sometimes the different data providers present totally different outcomes as well: “Two years ago we made an agreement between three parties that could deliver carbon data. All three parties had received the same benchmark from us. After which they all presented three completely different outcomes. And that was on carbon, which is one of the easiest things to measure” (Participant 7).

One of the interviewees stated that the data which are available for evidence-based measurement are not accurate at the moment. This is deemed very problematic, since “Garbage in equals garbage out” (Participant 6). In one of the interviews it was stressed that the international research agencies should be linked to universities in order to have an immediate feedback loop (Participant 7).

4.2.5 Impact data analysis

Given that most of the interviewed investors lack specific impact goals, they did not set specific desired results either. Only two institutional investors have set impact targets, one is currently working on setting impact targets, and the other ten did not set any targets at all. One of the interviewees, active in the inclusive finance sector, mentioned that they did not have a target yet, but do analyze if the investment achieved the goal. When asked how they are able to do that, the interviewee argued: “Even though you don’t set a fixed number, you can try to see to how many people you are reaching, and if this number increases over the years”. Hereafter the interviewee stressed: “We don’t have the targets, but we still look at which developments we want to see and if we are making progress in that direction” (Participant 15). In general, the interviewees indicate that they analyze impact data more frequently than in the past.

When impact data is analyzed, it is important to look at the indicators which do not contribute to decision-making and reporting. Such data is irrelevant, which should be taken into account when adjusting collection methods (IMWG 2014). It was striking that one of the institutional investors pointed out the following: “We have measured a lot of indicators over the years, but we never reported any of them” (Participant 15), which may have led to an unnecessary burden for the investee.

One of the interviewees stressed “Currently, impact investors are not much further than saying that they are committed to, or have caused a difference to some topic. That is ultimately what you are looking for with impact investing. Making a difference. We need to verify this with an impact evaluation. But in the end, you are not able to do that for everything” (Participant 8). Only one of the interviewed investors performed an impact evaluation and mentioned: “I think I did a good job, but I have also seen that the administrative costs for micro finance institutions (MFIs) are enormous when information is requested. And most MFIs don’t do anything with this data” (Participant 2). Other interviewees argued that it is not yet feasible for them, and that they don’t see it happening any time soon due to the required resources (Participant 1, Participant 3). Some institutional investors pointed out that they have attracted more employees for impact measurement and analysis over the last two years (Participant 7, Participant 10, Participant 17). When asked if evaluation on impact will gain more importance in the future, one of the interviewees answered: “Yes, for sure. If we are only able to say that we received the money back properly, but we do not know who truly benefited from the investment, we are not doing a good job! We cannot afford this” (Participant 8).

4.2.6 Reporting

Until recently, social impact was mostly shown through non-financial and qualitative information, like a statement about pollution prevention and corporate social responsibility, or a sustainable report (e.g., Medley 1997; O’Dwyer and Owen 2005). Some interviewees indicated that they are searching for other ways to report their social impact. One of the interviewees is working towards integrated reporting (Participant 4), while another indicated that there may come reporting requirements from the government: “Organizations can expect reporting requirements from the government in the near future. They will not demand that you engage in impact investing, but that you report about it. Even if you make no impact” (Participant 1). In fact, institutional investors perceive that they are required to be increasingly transparent about their impact investments, and for the future they expect more reporting requirements. As one of the interviewees stated: “We do not only report our impact data to show what we did and to fulfill the reporting needs, gradually our whole business model asks for it” (Participant 8).

4.2.7 Data driven investment management

One of the interviewees argued that impact measurement is just a component of impact investing. Ultimately it comes down to how the data may be informative for your decisions and strategies. The interviewee argued that in recent years the sector was focused on the kind of impact they wanted to measure, how they were going to measure it, and how the data would emerge from the market. However, little thought went into what to do with the impact data. The interviewee argued: “We need to go from impact measurement to impact management” (Participant 10).

It was remarkable that none of the interviewed institutional investors uses the impact data for new investment decisions. However, they recognized the importance of it. A variety of reasons were given. One interviewee pointed out a rapidly changing world makes it difficult to use past impact data as new input (Participant 15). This is particularly an issue because of the typically long horizon of impact investments, which makes it difficult to base decisions on past impact data. Furthermore, it was stated that basing investment decisions on impact data is only possible when the measurement practices become more mature. Another reason is that there is a lack of evidence that financial returns can be combined with social returns. Nonetheless, of the total sample, six investors think that they will use impact data for new investments, five have doubts, and two are convinced that impact data will not be used for new investment decisions.

5. Conclusion

This study considers the design and application of measurement systems by institutional investors related to impact investments. A qualitative research methodology is used, including thirteen semi-structured interviews among Dutch institutional investors. The study contributes to the understanding of how institutional investors design and apply measurement systems related to impact investments.

In summary, it can be concluded that impact measurement within the impact investment industry is still in an early stage. While there is room for improvement for some of the activities within the process of impact measurement, other activities are still in an early development stage. Our findings show that the majority of the institutional investors do not set specific impact goals. Furthermore, we note that the extent of impact measurement depends on the impact commitment of the organization and its role in society. While there are many measurement models, impact investors are still struggling to select appropriate KPIs and to gather the respective data. The interviewees indicate that they could benefit from standardized impact metrics. Regarding data collection, the impact investors indicated that collaboration with investees is getting more intensive in order to collect accurate impact data and that the collection and storage of data is very resource intensive. To accommodate and validate the data, institutional investors use data agencies. However, the practices of these data agencies are not universal. The majority of institutional investors analyze the impact through storytelling and internal data analysis, while there is a move towards external data analysis. We also find that institutional investors are searching for different ways of reporting the impact which the investments have generated. Although no impact data driven investments take place at the moment, the institutional investors indicate that it is likely that impact data will be used for future investment decisions, enabling the impact investment industry to transition from impact measurement to impact management.

Based on our study, we note the following opportunities for future research. First of all, it would be interesting to distinguish between different asset classes (e.g., high yield, private equity, and infrastructure), and to see to what extent impact investing is feasible for each of these classes. Second, it is interesting to conduct an inquiry in the future, when the market is more mature, to see which impact measurement models have emerged. Third, it will be worthwhile to examine whether specification of impact goals and the number of KPIs requested from investees influences the financial return and generated impact of impact investments.

Dr. J.J.F. van Raak is assistant professor of accounting at the Amsterdam Business School, University of Amsterdam.

A. Raaphorst MSc. is senior consultant Finance & Business services at KPMG.

Notes

1

The Rockefeller Foundation, a leading philanthropic organization, first introduced the definition impact investing in 2007 as investments made with the intention of generating both financial return and social and/or environmental impact.

2

The IMWG developed seven guidelines and a vision for impact measurement for the subsequent years. These guidelines are a set of generally accepted activities in impact measurement.

3

Whereas SRI has a bias towards corporate governance (Sandberg et al. 2009), this is not the case for impact investments (Höchstädter and Scheck 2015).

4

An example of socially responsible investments is a fund with a best-in-class approach in which investors select organizations, based on their expected sustainability performance level relative to their peers.

5

Examples of sectors having a hard time finding financing are the renewable energy, rural development, and health sectors (Simon and Barmeier 2010).

6

These impact investors vary widely in character, motivation, impact objectives, types of assets, risk expectations, and return expectations.

7

There are also investors with capital in both Impact First and Finance First investments, such as Development Finance Institutions.

8

An impact goal could either be general or specific. An example of a general goal is doing good for the environment, while a specific goal is offering residents in a region of Africa anti-malaria bed nets.

9

Competitive industries tend to be quicker at adopting (non-financial) performance metrics (Hussain and Hoque 2002), which may provide an additional reason why the impact investing industry, in which there is arguable limited rivalry between the investors and in which economic performance is only one of the goals, is relatively slow in implementing performance measurement systems.

10

The impact measurement models vary based on the objectives, activities and social impact information that companies would like to measure (Maas and Liket 2011).

11

For this research a combination of inductive and deductive approaches is used. In this research the deductive approach is used in the sense of using the seven impact measurement guidelines as a framework to analyze the data, while overall the process of interviewing has an inductive nature. Ali and Birley (1999) indicate that using existing theory to do an inductive research can help the researcher identify where it should narrow its focus on. However, the combination of an inductive and deductive approach can compromise the researcher’s ability to understand the view of the respondents. Nonetheless, they state that the benefits are higher than the drawbacks, when using this combining approach.

12

When using a best-in-class approach, investors aim to identify and select organizations that will outperform their peers in terms of sustainability performance.

13

Triodos bank is a leader in sustainable banking and won the Financial Times Sustainable Bank of the Year Award in 2009.

14

This guideline refers to both data collection and storage. However, given the professional backgrounds of our interviewees, we have chosen to not ask questions about data storage as this refers to operational IT decisions that are less relevant to interviewees.

References

  • Ali H, Birley S (1999) Integrating deductive and inductive approaches in a study of new ventures and customer perceived risk. Qualitative Market Research: An International Journal 2(2): 103–110. https://doi.org/10.1108/13522759910270016
  • Atkinson AA, Waterhouse JH, Wells RB (1997) A stakeholder approach to strategic performance measurement. Sloan Management Review 38(3): 25–37.
  • Dyck A, Lins KV, Roth L, Wagner HF (2019) Do institutional investors drive corporate social responsibility? International evidence. Journal of Financial Economics 131(3): 693–714. https://doi.org/10.1016/j.jfineco.2018.08.013
  • Eccles RG, Krzus MP (2010) Integrated reporting for a sustainable strategy. Financial Executive 26(2): 28–33.
  • Epstein MJ, Yuthas K (2014) Measuring and improving social impacts: A guide for nonprofits, companies, and impact investors. Berrett-Koehler Publishers.
  • Grabenwarter U, Liechtenstein H (2011) In search of gamma-an unconventional perspective on impact investing. Working paper, IESE Business School. https://doi.org/10.2139/ssrn.2120040
  • Höchstädter AK, Scheck B (2015) What’s in a name: An analysis of impact investing understandings by academics and practitioners. Journal of Business Ethics 132(2): 449–475. https://doi.org/10.1007/s10551-014-2327-0
  • Hubbard G (2009) Measuring organizational performance: beyond the triple bottom line. Business Strategy and the Environment 18(3): 177–191. https://doi.org/10.1002/bse.564
  • Hussain M, Hoque Z (2002) Understanding non-financial performance measurement practices in Japanese banks: a new institutional sociology perspective. Accounting, Auditing & Accountability Journal 15(2): 162–183. https://doi.org/10.1108/09513570210425583
  • Ittner CD, Larcker DF (1998) Are nonfinancial measures leading indicators of financial performance? An analysis of customer satisfaction. Journal of Accounting Research 36: 1–35. https://doi.org/10.2307/2491304
  • Ittner CD, Larcker DF, Randall T (2003) Performance implications of strategic performance measurement in financial services firms. Accounting, Organizations and Society 28(7/8): 715–741. https://doi.org/10.1016/S0361-3682(03)00033-3
  • Jackson ET (2013a) Evaluating social impact bonds: questions, challenges, innovations, and possibilities in measuring outcomes in impact investing. Community Development 44(5): 608–616. https://doi.org/10.1080/15575330.2013.854258
  • Lingle JH, Schiemann WA (1996) From balanced scorecard to strategic gauges: is measurement worth it? Management Review 85(3): 56–61.
  • Longhurst R (2010) Semi-structured interviews and focus groups. In: Clifford NJ, Valentine G (Eds) Key Methods in Geography. London: Sage, 103–116.
  • Maas K, Liket K (2011) Social impact measurement: Classification of methods. In: Burritt R, Schaltegger S, Bennett M, Pohjola T, Csutora M (Eds) Environmental management accounting and supply chain management. Eco-efficiency in industry and science, volume 27. Springer (Dordrecht), 171–202. https://doi.org/10.1007/978-94-007-1390-1_8
  • Maas K, Schaltegger S, Crutzen N (2016) Integrating corporate sustainability assessment, management accounting, control, and reporting. Journal of Cleaner Production 136, part A: 237–248. https://doi.org/10.1016/j.jclepro.2016.05.008
  • O’Dwyer B, Owen DL (2005) Assurance statement practice in environmental, social and sustainability reporting: A critical evaluation. The British Accounting Review 37(2): 205–229. https://doi.org/10.1016/j.bar.2005.01.005
  • Simon J, Barmeier J (2010) Impact investing for development. Center for Global Development (Chicago, IL).
  • Skouloudis A, Evangelinos K, Kourmousis F (2010) Assessing non-financial reports according to the global reporting initiative guidelines: evidence from Greece. Journal of Cleaner Production 18(5): 426–438. https://doi.org/10.1016/j.jclepro.2009.11.015
  • Yin RK (2015) Qualitative research from start to finish. Guilford Publications.

Appendix 1

Interview guide

General questions

  1. 1. How do you define impact investing?
  2. 2. Why is your organization engaged in impact investing?
  3. 3. For how long has your organization been engaged in impact investing and measurement?
  4. 4. What are, in your opinion, the fundamental characteristics of impact investing?
  5. 5. What percentage of the total investment portfolio do you qualify as impact investments?
  6. 6. In which asset classes do you invest for impact?
  7. 7. Would you describe impact investing as successful?
  8. 8. Do you think impact investing becomes successful in the future?

Impact objective

  1. 9. Do you have impact objectives? If so, what are those objectives?
  2. 10. Are you familiar with the 17 Sustainable Development Goals (SDGs)? If so, do you use the SDGs in relation to your impact investments (e.g., as impact areas)?
  3. 11. Do you think you will adjust/focus your impact investing strategy on some of the SDGs, in the (near) future?

Framework and metrics

  1. 12. Do you measure the impact of your investments? And, if so, how?
  2. 13. Do you measure attribution?
  3. 14. How does your organization measure risk in impact investing?
  4. 15. Does your organization use a specific impact measurement framework? If yes, which one? Why did you choose this one?
  5. 16. What do the model looks like?
  6. 17. What do you find the most important characteristics of that model? And why?
  7. 18. Did you change the way of measurement recently?
  8. 19. How many performance measures (e.g., KPIs) do you have aligned with impact objectives?

Collect and store data

  1. 20. What is the role of the investee in impact measurement and evaluation?
  2. 21. What are, in your opinion, the main opportunities and barriers related to standardized impact measurement?
  3. 22. Do you think that standardization of impact measurement is beneficial?

Validate data

  1. 23. How do you guarantee data accuracy?
  2. 24. Do you use a benchmark or do you only make use of your own data?

Analyze impact data

  1. 25. Do you analyze the performance of the impact investments? And, if so, how?
  2. 26. How often does your organization compare performance with its objectives for impact investments?

Report impact data

27. How do you report on impact investments?

Data driven investments

  1. 28. Is the impact data used as input for new investments decisions?
  2. 29. What role does this evaluation of the performance of impact investments have in the organization?
  3. 30. Who is responsible for performance measurement and evaluation?