Start Submission Become a Reviewer

Reading: Benchmarking energy performance: indicators and models for Dutch housing associations

Download

A- A+
Alt. Display

Research

Benchmarking energy performance: indicators and models for Dutch housing associations

Authors:

Herman S. van der Bent ,

Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, NL
X close

Henk J. Visscher,

Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, NL
X close

Arjen Meijer,

Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, NL
X close

Niek Mouter

Faculty of Technology, Policy and Management, Delft University of Technology, Delft, NL
X close

Abstract

Benchmarking is a method that can be used to measure progress and create awareness about the performance of organisations. Benchmarking the housing stock energy performance of Dutch housing associations can be used to measure and assess progress towards the decarbonisation of the housing stock. A new national climate agreement was signed in 2019, and in 2021 a new method to determine the theoretical energy performance of dwellings came into force in the Netherlands. To benchmark energy performance, a set of indicators is created that adequately represents the performance of Dutch housing associations according to the changed policies. A process involving key stakeholders is presented here to identify, assess and combine possible indicators. These were then integrated into four integrated models, which led to a final benchmark model. A model was chosen that consists of three indicators covering the energy performance of Dutch housing associations. The process and arguments that led to this final model are presented. While applicable within the Dutch context, the method and research results provide generalisable insights for the creation of energy performance benchmarks for building stocks.

 

Practice relevance

This paper provides both researchers and policymakers with a practical approach to monitor and benchmark the energy performance of dwellings owned by organisations. An analysis of the Dutch policy context is presented. Examples of possible benchmark indicators are described and evaluated. A method is created to assess indicators and it is shown how to integrate indicators in different benchmark models. The final model consists of three indicators: (1) the average theoretical primary fossil energy consumption (energy label value); (2) the difference between the theoretical heating demand (quality building envelope) and the theoretical maximum heating demand of dwellings; and (3) the average actual CO2 emissions from gas consumption. Researchers and policymakers from other countries can adapt both the process and the final benchmark model to create similar benchmark models across housing stocks.

How to Cite: van der Bent, H. S., Visscher, H. J., Meijer, A., & Mouter, N. (2022). Benchmarking energy performance: indicators and models for Dutch housing associations. Buildings and Cities, 3(1), 417–432. DOI: http://doi.org/10.5334/bc.207
93
Views
26
Downloads
  Published on 21 Jun 2022
 Accepted on 13 May 2022            Submitted on 21 Feb 2022

1 Introduction

The United Nations agreed to keep global warming below 2°C and continue efforts to limit it to 1.5°C (UNFCCC 2015). Subsequent United Nations conferences of the parties (COP) agreed to limit the effects of climate change, up to the Glasgow Climate Pact (UNFCCC 2021). With worldwide CO2 emissions of > 10 Gt/yr (IEA 2020), the built environment is an important factor. Decarbonising the energy system in the built environment is a major challenge. This challenge is adopted at a European level, among others through the Energy Performance of Buildings Directive (EPBD) (European Commission 2018) and adopted at national levels as national policies.

In 2021, Dutch national policy on the sustainable development of the Dutch built environment entered a new phase. A new national climate agreement was signed in 2019 (National Climate Agreement 2019) and a new method to determine the theoretical energy performance of dwellings came into force in 2021 (NEN 2020). These changes (among others) influence how Dutch non-profit housing associations (hereafter ‘housing associations’) measure the energy performance of their building stock.

Dutch housing associations own 2.4 million dwellings, organised in 286 housing associations in 2021. This means one-third of the Dutch housing stock is owned by social housing associations (Autoriteit Woningcorporaties 2020). Other European countries with a large share of social housing are Austria at 24%, Denmark at 21%, Sweden at 17%, UK at 17%, France at 16%, Norway at 14% and Finland at 11% (Housing Europe 2021). Therefore, the non-profit housing stock plays an important role in helping to fulfil the Paris Agreement.

Benchmarking is a method to measure progress and create awareness about the performance of organisations in relation to goals. Benchmarking can be defined as:

a continuous analysis of strategies, functions, processes, products or services, performances, etc. compared within or between best-in-class organisations by obtaining information through appropriate data collection method, with the intention of assessing an organisation’s current standards and thereby carry out self-improvement by implementing changes to scale or exceed those standards.

(Anand & Kodali 2008: 259)

The present paper examines the process of making a model to benchmark the energy performance of the housing stock of Dutch housing associations.

1.1 Benchmarking the energy performance of dwellings

The Paris Agreement requires actions to reduce global warming worldwide, including in the built environment. The sustainable development of the housing stock is a challenge for many countries and is the subject of several papers that discuss the sustainable development of the housing stock, e.g. Bulgaria, Serbia, Hungary, the Czech Republic (Csoknyai et al. 2016), Sweden (Hjortling et al. 2017), Switzerland (Streicher et al. 2018), Ireland (Ahern & Norton 2019) and the Netherlands (van der Bent et al. 2021b). These papers describe changes of (parts of) the housing stock using many different descriptive parameters, without the aim to benchmark the progress.

Several papers seek to find appropriate conditions for establishing accurate models to benchmark the energy performance. Jiang et al. (2014) highlight the importance of establishing accurate and efficient databases to have a sound basis to assess and monitor the improvement of the energy performance of several types of Chinese buildings. They stress the need for clear definitions and discuss the differences in indicators to be used in benchmarks. They argue for indicators that include both technical and non-technical measures. Moreover, besides the energy performance unit of kWh/m2/yr used for assessing the overall energy performance, the unit kg CO2/m2/yr could be used for assessing the CO2 emissions of buildings.

However, Jiang et al. (2014) do not extrapolate their research to an organisational level or address the possibility of a performance gap between theoretical and actual energy consumption. This performance gap can be defined as the difference between the theoretical modelling of energy consumption through the EPBD and the actual energy consumption of dwellings, and is acknowledged throughout several papers covering multiple countries (Sunikka-Blank & Galvin 2012; Saunders 2015; Summerfield et al. 2019; Cozza et al. 2020). Reasons for the performance gap are an inadequate assumption of the actual indoor temperature, the influence of user behaviour on actual consumption, and differences between the theoretical calculation and the actual building quality of dwellings. Aranda et al. (2018) found that the energy performance gap for social housing is even larger than the energy performance gaps found for other dwellings.

Duvier et al. (2018) argue that it can be difficult to build and use high-quality datasets. They considered the UK social housing sector and argued that using the potential is difficult due to the constant changes of regulations which makes it difficult to develop long-term strategies. However, Steadman et al. (2020) give an example of large-scale available data within the UK to monitor and benchmark the energy performance of buildings. Ding & Liu (2020) compared several data-driven methods to benchmark the energy performance of individual dwellings. They highlight the need for both high-quality data and robust benchmark models. They address the concern that different indicators could lead to a different ranking of subjects, and recommend that policymakers consider multiple benchmarking methods or select a method, while being aware of what they actually measure. However, they do not apply this to housing associations, but at the level of individual dwellings.

Roth et al. (2020) examined the possibilities of using open data to benchmark building energy consumption in cities. They concluded that benchmarking by itself will not lead to energy savings, but benchmarking the energy consumption of dwellings can help to develop effective policies to lower actual energy consumption. Benchmarking the average energy performance of the housing stock of housing organisations could lead to similar results. Laaroussi et al. (2020) realised that the large amount of energy used by residential buildings means that the renovation of the building stock is one of the key strategies to reach energy performance targets. As a part of a European H2020 project, they analysed the situation in Spain, France, Italy, Slovenia and Austria and used a weighted method as an analytical approach to evaluate the main indicators of collected data. Bordass (2020) analysed different metrics for benchmarking the energy performance of buildings. He argued that benchmarks should not only focus on a single most important indicator, but a more diverse range of indicators could help users of benchmarks to take the correct actions. Too few indicators may lead to wrong decision-making. Furthermore, too many indicators lead to fogginess and also to wrong decision-making. Although applied to individual dwellings, this is applicable for benchmarking the energy performance of large sets of dwellings as well.

Several lessons arise from this literature:

1.2 The Dutch context: benchmarking and housing associations

The definition of benchmarking by Anand & Kodali (2008) indicates that benchmarking the energy performance of dwellings could be useful at an organisational level as well. Aedes, the Dutch umbrella organisation of housing associations located in The Hague, has organised a benchmark between housing associations since 2014, covering topics such as overhead costs, maintenance, tenant appreciation, availability and affordability, and energy performance (Aedes 2020). The benchmark helps to enhance the knowledge and factual basis of the development of the Dutch built environment owned by housing associations. Until 2020, the benchmark for energy performance consisted of two indicators: (1) the energy label value: the average Energy Index Nader Voorschrift (EI NV); and (2) the average actual CO2 emissions from heating demand through gas consumption and district heating. Due to changes in policy, these indicators need to be adjusted.

Social housing in the Netherlands has its origin in the Industrial Revolution when the Vereeniging ten behoeve der Arbeidersklasse te Amsterdam (Association for the Benefit of the Working Class in Amsterdam) was established in 1852 (Boissevain 1865). The Housing Act of 1901 provided the means for state funding for social housing organisations, which as a result grew in number and size. A policy change in 1995, the so-called ‘bruttering’ (Netherlands & Schorer 2004), made social housing organisations financially independent and focused their role as social entrepreneurs. In 2020, one-third of the Netherlands’ housing stock was owned by social housing associations. (Autoriteit Woningcorporaties 2020). The social housing stock of 2.4 million dwellings is organised into 286 housing associations. Housing associations are able to make decisions on how to manage their housing stock, but are regulated by strong central law (Overheid.nl 2018). For example, laws govern the allocation of tenants, maximum rents and sustainable transformation of dwellings. Housing associations in the Netherlands cover a broad range of organisations. Some are small (fewer than 1000 dwellings), while others are large (more than 80,000 dwellings). Some operate mainly in urban environments, while other housing associations own assets in more rural areas. Also, differences occur in the financial position of Dutch housing associations. Housing associations also have different types of dwellings. Some have relatively new dwellings, e.g. in the province of Flevoland, which is largely a polder that was developed in the second half of the 20th century. Other housing associations own old dwellings in historical city centres. Also, differences occur in the quality of the dwellings regarding energy performance. Some housing associations improved the energy performance of their stock extensively, while others have a stock with a lower energy performance, as benchmarked in the Dutch context up to 2020 (Aedes 2020).

Policies aimed at increasing the energy performance of dwellings of housing associations originate from the Energy Agreement (Sociaal Economische Raad 2013). Until 2020, the aim was to achieve an average energy label B in 2020 as directed from the EPBD, as analysed by van der Bent et al. (2021b).

In 2021, national policy on the sustainable development of the Dutch built environment entered a new phase. A new national climate agreement was signed in 2019 (National Climate Agreement 2019), and a new method to determine the theoretical energy performance of dwellings came into force in 2021, the NTA 8800 (NEN 2020). These changing policies affect the measuring of energy performance, and consequently the benchmarking of the energy performance of the housing stock of Dutch housing associations. These policies are further discussed in phase 1 of the results section.

1.3 Research question

No scientific research was found regarding the process of creating a benchmark for the energy performance of the housings stock of housing organisations. Creating a benchmark model is relevant both in the Dutch context and for the international community, helping to move to a sustainable built environment. Researchers and policymakers from other countries can benefit both from the following process as well from the final benchmark model to create similar benchmark models across their housing stock. Dutch housing associations aim to benchmark the energy performance of the non-profit housing stock, given the changes in policy derived from the National Climate Agreement 2019 (National Climate Agreement 2019) and the enforcement of the NTA 8800 (NEN 2020) in 2021.

The aim of the present research is to find a set of indicators that adequately represents the performance of Dutch housing associations according to the changed policies. The research question is therefore: Which set of indicators can be used to benchmark the energy performance of Dutch housing associations and what can we learn from the process to find these indicators?

2 Research method

The research method used is a combination of desk research with action research where the principal researcher participated in group sessions with experts from Dutch housing associations.

The expert group consisted of 18 employees from housing associations and two employees of Aedes, the umbrella organisation of housing associations. The expert group guided the existing benchmark until 2020 and now guides the adjustment of the benchmark model beyond 2021. The group was a mix representing both small and large housing associations, from different areas of the Netherlands, operating in different parts of the non-profit housing stock, both rural and urban. Staff members in this group typically have several to many years (10 or more) of working experience in the field of sustainable development of housing associations and typically have job descriptions related to the strategic advisory of sustainable development or the actual planning of sustainable projects at housing associations. Based on availability they attended sessions following phases 2–4, as described below, usually having 12–15 attendees. Decisions were made on shared agreement after discussion sessions following the phases.

In addition to the expert group, two different advisory groups of Aedes comprising directors of housing associations and the general board of Aedes (also comprising directors of housing associations) were involved in the selection of the final model. The advisory groups and general board are part of the decision-making structure of Aedes.

Seven phases are distinguished in this research: policy review, identifying the available data, assessment of the indicators, integration in the benchmark models, selection of the benchmark model, data collection and benchmark results. These phases were not predetermined but suggested throughout the process by the principal researcher and acknowledged by the expert group. The phases took place between September 2020 and December 2021, as specified in Table 1.

Table 1

Research phases and participation.


PHASE GROUP PERIOD

1: Policy review Expert group September 2020

2: Identify the available data Expert group September 2020

3: Assessment of the indicators Expert group October 2020

4: Integration in the benchmark models Expert group November 2020

5: Selection of the benchmark model Two directors groups and general board Aedes January and April 2021

6: Data collection Principal researcher June–August 2021

7: Benchmark results Principal researcher September–December 2021

Phase 1 consists of a policy review, to identify relevant incentives for housing associations. Phase 2 consists of desk research where available benchmark indicators were identified from the available data sources. After this phase, a group session with expert staff members from housing associations was organised, where they discussed the policy review and confirmed the list of available indicators. Phase 3 consists of an assessment of the available indicators using a weighted multicriteria approach with five assessment criteria to narrow down the identified options in phase 2 to a smaller list of viable options. The assessment of the available indicators ended with a group session of the expert group confirming the assessment of the available indicators. Phase 4 consists of a desk assessment and expert group discussion about four integrated models with indicators to measure the sustainable development of Dutch housing associations. The expert group confirmed the four selected models and proposed a final model. Phase 5 consists of a group discussion with two different advisory groups of directors of housing associations. They analysed and judged the four selected models. The general board of Aedes, also comprising directors of housing associations, affirmed the selected final model. Phase 6 consists of the data collection used for benchmarking. Phase 7 describes shortly the results of the benchmark.

3 Results

The results of this research are described according to the phases in the research method section.

3.1 Phase 1: policy review

The first phase consisted of a policy review. The aim was to understand the policy context relevant for benchmarking the energy performance of housing associations.

In the summer of 2019, a new Climate Agreement (National Climate Agreement 2019) was agreed between the Dutch government, Dutch companies and organisations to mitigate climate change in agreement with European goals. The Climate Agreement is enforced with legislation and further specified agreements with businesses and sector organisations. For the Dutch non-profit housing sector, several policies apply. A generic goal was formulated to lower CO2 emissions from the built environment. This is framed as the fossil fuel (natural gas) consumption used by the built environment (primarily for space heating and water heating). This is enforced with several policies discussed below. These policies are different from those resulting from the Climate Agreement in 2013 (Sociaal Economische Raad 2013), where it was agreed that the housing stock of housing associations should obtain an average energy label B in 2020, based on the theoretical energy performance of buildings calculation. Newly proposed policies from the Climate Agreement 2019 do not enforce a goal expressed as an energy label. These proposed policies relevant to housing associations are the neighborhood-oriented approach, subsidies to accelerate the rate of the renovation, and the proposed introduction of a theoretical maximum heating demand for dwellings.

The neighborhood-oriented approach is a policy that is driven by municipalities. It aims to enhance a sustainable energy system by coordinating actions of (local) authorities, energy infrastructure companies, local companies, inhabitants and also housing associations. Regional and local development plans are written to combine the availability of heat, the quality of the energy infrastructure, and the energy demand from buildings and companies. With one-third of the Dutch housing stock owned by housing associations, they are an important stakeholder in this neighborhood-oriented approach.

Policy aiming to accelerate the renovation pace of housing associations consists of two main subsidies: one for district heating solutions, and another to increase renovations by bundling demand and coupling it to innovative steady supplies.

The new policy introducing a theoretical maximum heating demand for dwellings in 2050 aims to lower the energy demand of dwellings by improving the quality of the building envelope.

These policies are successors of the old policy where housing associations aimed at improving the energy performance of dwellings to an average energy label B. These new policies form a wider approach with different incentives to improve the energetic quality of the non-profit housing stock. These incentives are mainly aimed at the energy performance of dwellings, with an aim to decrease energy consumption during the operation phase of dwellings.

Following European regulations from the EPBD, an improved calculation method for the theoretical energy performance of buildings has been enforced in the Netherlands from 2021, via the NTA 8800 (NEN 2020). This improved calculation method has some major changes as opposed to its predecessor the NEN 7120 NV, as examined by van der Bent et al. (2021b). Previously, the energy label had a dimensionless value: the theoretical energy index (a calculation of the theoretical energy use divided by a combination of floor area and building envelope area). The new energy label value is still based on a theoretical energy consumption, but divided only by the floor area. Therefore, it can be expressed in the dimension of kWh/m2/yr. Also, the calculation of the theoretical energy consumption is improved and updated regarding the characteristics of building installations and building physics.

3.2 Phase 2: data available for benchmarking

In the second phase, the available indicators for benchmarking the sustainable development of Dutch housing associations were identified by the principal researcher in consultation with the expert group. High-quality datasets are recommended as a basis for measuring and benchmarking (Jiang et al. 2014; Duvier et al. 2018; Steadman et al. 2020). Different metrics should be examined (Bordass 2020), where the units kWh/m2/yr and kg CO2/m2/yr are specifically mentioned by Jiang et al. (2014). A combination of desk research, expert knowledge from the principal researcher and expert knowledge from the expert group led to the identification of three main data sources which are available to benchmark the sustainable development of Dutch housing associations: the SHAERE database, the dVi database and data from the Central Bureau of Statistics (CBS). Creating new data sources was not considered feasible due to high overhead costs to collect new data.

  • SHAERE database
    The SHAERE database is maintained by Aedes. Data on building characteristics and energy performance of the individual dwellings of voluntarily participating housing associations are collected annually as a basis for the existing benchmark. In 2021, the structure of the database was adapted for the implementation of the NTA 8800 (NEN 2020). The database is filled with data exported from software to administrate the energy performance of dwellings. This software is called VABI Assets Energy. Every housing association with this software can export their dwellings’ data with descriptive parameters of each of their dwellings. Over 90% of housing associations use this software. However, some mostly smaller housing associations do not, mainly due to the cost, hence they are not able to deliver these data. The exports are gathered in a central data management environment. the main indicators are the theoretical primary fossil energy consumption, energy label, heating demand, maximum theoretical heating demand, type of heating systems, type of ventilation systems, insulation components of the outer shell and installed solar systems.
  • dVi database
    The dVi is a different central database among others managed by the Dutch Ministry of Internal Affairs. It collects many indicators about housing associations, one being the energy index (energy label value up to 2020) of every dwelling of social housing associations. This is only a single indicator, without other clarifying indicators about the quality of the dwelling. Every housing association is obliged by law to deliver these data.
  • CBS
    The Central Bureau of Statistics (CBS) collects the actual energy consumption of individual dwellings, consisting of both gas and electricity consumption, which are available under an anonymisation procedure. The anonymisation assures no ethical issues arise during the collection and handling of data. Energy consumption for district heating is not available at the CBS, but some housing associations are able to provide these data separately.

From these sources, a list of potential indicators was extracted to measure the development of the energy performance of the housing stock of Dutch housing associations. A proposal was made by the principal researcher to the expert group which discussed and agreed that these 12 indicators are options to consider measuring the development of Dutch housing associations. The potential indicators are listed in Table 2.

Table 2

Possible indicators to benchmark the energy performance of dwellings.


POSSIBLE INDICATOR UNIT SOURCESa DESCRIPTION

1. Average theoretical primary fossil energy consumption kWh/m2/yr SHAERE Energy label value after 2020 under the NTA 8800. Based on a theoretical energy consumption divided by floor area

2. Average theoretical energy index n dVi Energy label value up to 2020

3. Average number of label steps in the energy label n SHAERE Average number of energy label steps after 2020 under the NTA 8800

4. Average theoretical heating demand kWh/m2/yr SHAERE Theoretical unit of measurement of the heating demand of dwellings

5. Average difference heating demand and maximum heating demand kWh/m2/yr SHAERE Average difference between the heating demand of dwellings (quality building envelope) and the maximum heating demand based on the layout and topology

6. Percentage dwellings complying with the maximum heating demand % SHAERE Percentage of dwellings below the maximum heating demand of the dwelling based on its layout and topology

7. Percentage of dwellings that are gas-free % SHAERE Percentage of dwellings without a gas-fired heating system

8. Percentage of dwellings with photovoltaic panels % SHAERE Percentage of dwellings with photovoltaic (PV) panels, and therewith contributing to the production of clean electricity

9. Indicator mix of building characteristics Undefined SHAERE Undefined combination of building characteristics; however, not yet operationalised

10. Average actual energy consumption (gas + electricity and district heating) kWh/m2/yr CBS Combined average actual energy consumption on a dwelling level per m2 of the three main energy carriers in the Netherlands: gas, electricity and district heating

11. Average actual CO2 emission (gas + electricity and district heating) kgCO2/m2/yr CBS Combined average actual CO2 emissions on a dwelling level per m2 of the three main energy carriers in the Netherlands: gas, electricity and district heating

12. Average actual CO2 emission (gas) kgCO2/m2/yr CBS Average actual CO2 emissions on a dwelling level per m2 of gas consumption. CO2 emission within the building

Note: Indicators are listed without any order of preference.

3.3 Phase 3: indicators most suitable for benchmarking

In the third phase, the indicators from the list were assessed on suitability by scoring the indicators on five assessment criteria by the principal researcher in consultation with the expert group (see below). Using a weighted method as an analytical approach can contribute to the selection of main indicators (Laaroussi et al. 2020). The aim of this phase was to narrow down the identified options in phase 2 to a smaller list of viable options. The criteria to assess the identified options were determined by the principal researcher and validated by the expert group. The five criteria aim to ensure indicators are effectual (available and comparable), are communicable (recognisable), and relate to policy (both at the national level and as perceived by housing associations). The five criteria are:

  • Availability of the data
  • Comparability of the data
  • Recognisability of the indicator
  • Relation to national policy
  • Relation to housing association policy

Each possible indicator was scored from –2 (totally non-compliant) to +2 (totally compliant). A weighting factor of 1 was given to the criteria availability, comparability and recognisability, and a weighting factor of 2 was given to the criteria the relation to national policy and the relation to housing association policy, because these last two are regarded as more important by the expert group. This led to the scoring table shown in Table 3.

Table 3

Assessment of indicators with a scoring table.


WEIGHTING FACTOR ASSESSMENT CRITERIA: 1 1 1 2 2

INDICATOR NO. INDICATOR DESCRIPTION AVAILABILITY COMPARABILITY RECOGNISABILITY NATIONAL POLICY HOUSING ASSOCIATION POLICY TOTAL SELECTED FOR THE NEXT PHASE

Building indicator

1 Average theoretical primary fossil energy consumption 1 1 1 2 2 11 ×

2 Average theoretical energy index –2 –2 1 –1 0 –5

3 Average number of label steps in the energy label 1 1 1 –1 –1 –1

Insulation indicators

4 Average theoretical heating demand 1 0 1 –1 –1 –2

5 Average difference between heating demand and maximum heating demand 1 1 1 2 1 9 ×

6 Percentage of dwellings complying with the maximum heating demand 1 1 1 1 1 7 ×

Installation indicators

7 Percentage of dwellings gas-free 1 2 2 1 1 9 ×

8 Percentage of dwellings with photovoltaic (PV) panels 1 2 2 1 1 9 ×

9 Indicator mix of building characteristics 1 0 0 0 0 1

Effect indicator

10 Average actual energy consumption (gas plus electricity plus district heat) –1 0 1 0 1 2

11 Average actual CO2 emissions (gas plus electricity plus district heat) –1 0 1 1 2 6 ×

12 Average actual CO2 emissions (gas) 0 0 1 2 1 7 ×

It was then decided in consultation with the expert group to consider seven indicators scoring > 5 (Table 3). Five indicators scoring < 5 were not considered in the next phase. The most salient trade-offs from the assessment of the indicators are described as follows:

  • Indicator 1: The average primary fossil energy consumption (value of the energy label) scores highest in the assessment. This indicator is available and comparable for housing associations that can generate SHAERE exports and is in line with national and housing association policy. Having an energy label is mandatory for every dwelling.
  • Indicator 2: The average theoretical energy index from the dVi scores low. In 2021, the energy label value in the dVi is still the energy index based on the old policy up to 2020 (NEN 2014). The indicator that will be included from 2022 is not yet known, nor is it available for benchmarking in 2021.
  • Indicator 3: The average number of label steps does not score well because this does not play an important role in policy.
  • Indicators 4–6: The average difference between heating demand (quality building envelope) and the maximum heating demand scores positive on all assessment criteria. It is available, comparable, recognisable, and relates to both national and housing association policies. This can be expressed as indicator 5, the average difference between heating demand (quality building envelope) and maximum heating demand (kWh/m2), or indicator 6 in the percentage of dwellings that meet the maximum heating demand on a dwelling level.
  • Indicator 7: The percentage of gas-free dwellings that score well on comparability and recognisability, and also indirectly linked to policy.
  • Indicator 8: The percentage of dwellings with photovoltaic (PV) panels scores well on comparability and recognisability, and is in line with policy in a broad social sense.
  • Indicator 9: A mix of housing indicators that score low because they are not in line with comparability and recognisability, and have no direct link with policy.
  • Indicator 10: The actual consumption (kWh) of gas, electricity and district heating is not directly in line with policy. The availability of actual district heating consumption is problematic because these are not available by the CBS, but must be collected separately from housing associations.
  • Indicator 11: The actual CO2 emission from gas, electricity and district heating is less in line with national policy where gas is attributed to the built environment, and CO2 emissions from electricity and district heating are attributed to the industrial sector. The availability of actual heat consumption is less good because these are not available from the CBS, but must be collected separately from housing associations.
  • Indicator 12: The actual CO2 emission from gas consumption alone is more in line with the national objectives from the Climate Agreement for the built environment. This indicator scores better on availability because the actual gas consumption is available from the CBS.

3.4 Phase 4: integration in benchmark models

In the fourth phase, the available and ranked indicators were integrated by the principal researcher in consultation with the expert group in different benchmark models consisting of one or several indicators. Examining multiple benchmark models to select a robust model is recommended by Ding & Liu (2020). This was done to be able to assess relations between indicators. The integrated models all have a different bandwidth of benchmarking, ranging from a small model (housing associations are benchmarked on a single indicator) to a wide model (housing associations are benchmarked on six indicators). The different combinations of indicators were determined by the principal researcher and validated by the expert group. The combinations are shown in Table 4. We gave the four benchmark models a name with a description, explaining the nature of the combination of indicators.

Table 4

Schematisation of the four proposed benchmark models.


INDICATOR NO. INDICATOR DESCRIPTION BASIC MODEL REAL ESTATE MODEL POLICY PERFORMANCE MODEL WIDE MODEL

1 Average theoretical primary fossil energy consumption × × × ×

5 Percentage of dwellings complying with maximum heating demand × /

6 Average difference heating demand and maximum heating demand × /

7 Percentage of dwellings gas-free × ×

8 Percentage of dwellings with photovoltaic (PV) panels × ×

11 Average actual CO2 emissions (gas + electricity + district heat) ×

12 Average actual CO2 emissions (gas) × ×

Note: The symbol ‘/’ means that one of these two indicators should be chosen.

  • The basic model: focusing on a single indicator
  • The real estate model: focusing on real estate
  • The policy performance model: focusing on policy
  • The wide model: focusing on a broad spectrum

In the expert group, the advantages and disadvantages of these benchmark models were discussed. These are described as:

  1. The basic model
    • Advantages: Having one indicator makes benchmarking the energy performance transparent. The average value of the energy label scores highest in the assessment, lies within the direct sphere of influence of housing associations, and is also recognisable and communicable.
    • Disadvantages: In the Climate Agreement 2019 improving the energy label value is not a single dominant policy and therefore leads to a non-optimal focus. Also, an assessment of the performance based on a single indicator can be interpreted as limited (Bordass 2020).
  2. The real estate model
    • Advantages: It gives housing associations space to make their own real estate strategy visible in the energy performance assessment. This then applies to the commitment to making dwellings gas-free, the installation of PV panels and the quality of the building envelope of dwellings.
    • Disadvantages: The indicators gas-free, share of PV panels and quality of the building envelope can be seen as inputs for the primary fossil energy demand (the energy label), and from a benchmark perspective it is better to benchmark double indicators only in the top indicator. There is also a policy goal for the maximum heating demand, but not for the share of PV panels, and not for the share of natural gas-free homes.
  3. The policy performance model
    • Advantages: This combination of indicators is best in line with policy objectives: having an energy label, improving dwellings to the maximum heat demand, and the Climate Agreement target to reduce CO2 emissions from the built environment by reducing CO2 emissions from gas consumption.
    • Disadvantages: The actual CO2 emissions from gas consumption are an outcome of policy, but do not lie within the entire sphere of influence of a corporation. For example, the behaviour of the resident and the price of natural gas also influence gas consumption.
  4. The wide model
    • Advantages: Makes many nuances of the sustainability performance made transparent by the various indicators.
    • Disadvantages: From a benchmark perspective, a large number of indicators is less desirable and ambiguous (Bordass 2020). As in the real estate variant, this applies to the indicator share of natural gas-free and share of PV panels. Finally, as in the policy variant, the actual CO2 emissions from natural gas consumption do not lie within the entire sphere of influence of the corporation, but this applies even less to the actual CO2 emissions from electricity and heat. Actual electricity use has an even stronger resident component (household consumption) and the responsibility for CO2 emissions from electricity generation is a responsibility of the industrial sector in the Dutch Climate Agreement.

3.5 Phase 5: selecting the final model

The fifth phase consists of a group discussion of two advisory groups of housing association directors. They analysed and judged the four proposed models and agreed with the proposal of the expert group to select the policy performance model as the final model. The combination of three indicators in the policy performance model most accurately reflects the efforts of housing associations to improve the average energy performance of their housings stocks in accordance with the current policies. The general board of Aedes, the umbrella organisation of housing associations, affirmed the selected policy performance model. Therefore, the selected benchmark model consists of three indicators: (1) the average theoretical primary fossil energy consumption; (2) the average difference between the theoretical heating demand (quality building envelope) and the theoretical maximum heating demand; and (3) the average actual CO2 emissions from gas consumption.

3.6 Phase 6: data collection

Data were collected in June and July 2021, commissioned by Aedes and executed by the principal researcher. Housing associations voluntarily delivered a standardised data export with indicators of the energetic quality of their dwellings, together forming the SHAERE database. A total of 246 housing associations participated with over 2 million dwellings, covering 95% of the Dutch non-profit housing sector. This data source delivered data for indicators 1 and 2. In August 2021, the dwellings were anonymously connected to actual energy consumption data in an analysis environment at the CBS to be able to deliver data for indicator 3. The anonymisation ensured that no ethical issues arose during the collection and handling of data.

3.7 Phase 7: benchmark results

Figures 1, 2, 3 show the ranking of the housing associations according to the three indicators of the selected benchmark model. A more extensive description of the benchmark results was published by Aedes as part of a wider benchmark of the Dutch non-profit housing sector (Aedes 2021). No absolute benchmark values are stated for the different indicators. In this wider benchmark, housing associations are scored with an A, B or C, respectively, one-third of the population per indicator, accompanied by a more detailed dataset with secondary indicators to enhance the learning opportunities from the benchmark results.

Housing associations ranked by the average theoretical primary fossil energy consumption
Figure 1 

Housing associations ranked by the average theoretical primary fossil energy consumption.

Housing associations ranked by the average difference between theoretical heating demand and maximum theoretical heating demand
Figure 2 

Housing associations ranked by the average difference between theoretical heating demand and maximum theoretical heating demand.

Housing associations ranked by the average actual CO2 emissions from gas
Figure 3 

Housing associations ranked by the average actual CO2 emissions from gas.

4 Discussion

This research addressed the question: Which set of indicators can be used to benchmark the energy performance of Dutch housing associations and what can we learn from the process to find these indicators? After an extensive process, three indicators were selected that cover the most important policy aspects for the sustainable change of the housing stock of housing associations. The process was divided into several phases, which helped to arrive at a well-founded benchmark model. During the process several aspects from literature were applied. Different metrics (Bordass 2020) were examined and the metrics kWh/m2/yr and kg CO2/m2/yr were applied in the benchmark, complying with Jiang et al. (2014). High-quality datasets were found and used as a basis for measuring and benchmarking (Jiang et al. 2014; Duvier et al. 2018; Steadman et al. 2020). A weighted method as an analytical approach contributed to the selection of main indicators (Laaroussi et al. 2020), and we found that exploring multiple benchmark models could support decision-making to a final model (Ding & Liu 2020).

Several limitations arose. The analysis is country specific, which means it is only applicable for Dutch housing associations within the current Dutch policy context. However, the process used to create a benchmark model is generic. A policy analysis, selections of indicators, assessment of indicators, integration into models, selection of the final model, data collection and delivering results are applicable to a wide variety of similar questions in other countries that seek energy performance measurement and benchmarking of the built environment.

A second limitation is the Dutch specific policy context where the built environment focus is on the quality of dwellings and the emissions of CO2 related to the energy consumption within the dwelling. For example, no clear goals or targets are formulated regarding the use of materials in retrofitting dwellings (although these exist for new construction). Moreover, no data are available describing the sustainable use of materials for every housing association. This limits the measurement of the sustainable performance to energy consumption. Other countries may have a different policy context with a wider interpretation of sustainable performance.

A third limitation within the Dutch policy context is the presence of the energy performance gap, widely researched for both the Dutch (Majcen et al. 2013; Filippidou et al. 2019; van der Bent et al. 2021a) and European (Laurent et al. 2013; Summerfield et al. 2019) contexts. Theoretical indicators measuring the energy performance of dwellings, derived from the EPBD, all have a performance gap: the theoretical energy consumption deviates strongly from actual energy consumption. In the Dutch benchmark model, this is covered to some extent by having three indicators. Two are theoretical, but the third is based on the actual energy consumption of fossil fuel–natural gas (translated to CO2 emissions). This ensures housing associations also have an incentive to lower actual gas consumption and related CO2 emissions. Other countries measuring and benchmarking sustainable performance should be aware of this performance gap as well.

Finally, Anand & Kodali (2008) state that benchmarks should be understood as a repetitive process. The results of benchmarking should lead to improvement in organisations, so the effectiveness of the benchmark model needs to be reviewed and updated periodically. During our research we were not able to close this loop for time reasons, but a suggestion for improvement would be to include this in future research.

5 Conclusions

A process was created for formulating an energy performance benchmark for Dutch housing associations. A similar process can be used by other researchers aiming at benchmarking the energy performance between organisations within their policy context. The final policy performance model to measure and benchmark the sustainable performance of Dutch housing associations consists of three indicators closely related to governing policies regarding the sustainable improvement of the Dutch non-profit housing sector: The average theoretical primary fossil energy consumption, the average difference between the theoretical heating demand (quality building envelope) and the maximum theoretical heating demand, and the average actual CO2 emissions from gas consumption. The first indicator is related to the current policy regarding the energy labelling of dwellings, derived from the Energy Performance of Buildings Directive (EPBD), the NTA 8800. The second indicator relates to the policy to decrease the average theoretical heat demand of dwellings. The third indicator is related to the goal for the Dutch built environment to lower actual CO2 emissions. The model was then used to collect data and benchmark the energy performance of dwellings of housing associations. This research contributes to the wider literature by creating a model for benchmarking the energy performance of dwellings within the relevant policy context. This will be increasingly relevant for policymakers and landlords who need to respond to the United Nations Paris Agreement by reducing greenhouse gas emissions.

Acknowledgements

The creation of the benchmark model was commissioned by Aedes, the Dutch association of housing associations. Aedes is acknowledged for its cooperation in the development and execution of the benchmark among Dutch non-profit housing associations.

Author contributions

HSvdB was responsible for conceptualisation, data collection, methodology, analysis and writing. HJV was responsible for funding, review and editing. AM and NM contributed the review and editing.

Competing interests

The authors have no competing interests to declare.

Data availability

Data used in the research project are not publicly available due to restrictions of ownership.

Funding

Aedes, the Dutch association of housing associations, is a sponsor of the research project. This project was executed with the support of the MMIP 3 & 4 grant from the Netherlands Ministry of Economic Affairs & Climate Policy, as well as the Ministry of the Interior and Kingdom Relations.

References

  1. Aedes. (2020). Meer tevreden huurders ondanks moeilijke tijden. https://aedes.nl/aedes-benchmark/benchmarkresultaten-en-publicaties 

  2. Aedes. (2021). Huurders tevreden, corporaties overbelast. https://aedes.nl/aedes-benchmark/benchmarkresultaten-en-publicaties 

  3. Ahern, C., & Norton, B. (2019). Thermal energy refurbishment status of the Irish housing stock. Energy and Buildings, 202, 109348. DOI: https://doi.org/10.1016/j.enbuild.2019.109348 

  4. Anand, G., & Kodali, R. (2008). Benchmarking the benchmarking models. Benchmarking: An International Journal, 15(3), 257–291. DOI: https://doi.org/10.1108/14635770810876593 

  5. Aranda, J., Zabalza, I., Llera-Sastresa, E., Scarpellini, S., & Alcalde, A. (2018). Building energy assessment and computer simulation applied to social housing in Spain. Buildings, 8(1). DOI: https://doi.org/10.3390/buildings8010011 

  6. Autoriteit Woningcorporaties. (2020). Staat van de corporatiesector 2020. https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/rapporten/2021/02/05/staat-van-de-corporatiesector-2020/bijlage-1-staat-van-de-corporatiesector-2020.pdf 

  7. Boissevain, J. (1865). Betere woningen voor de arbeidersklasse. In Staatkundig en Staathuishoudkundig Jaarboekje 1865 (pp. 387–411). De Vereeniging voor de statistiek in Nederland. 

  8. Bordass, B. (2020). Metrics for energy performance in operation: The fallacy of single indicators. Buildings & Cities, 1(1), 260–276. DOI: https://doi.org/10.5334/bc.35 

  9. Cozza, S., Chambers, J., Deb, C., Scartezzini, J.-L., Schlüter, A., & Patel, M. K. (2020). Do energy performance certificates allow reliable predictions of actual energy consumption and savings? Learning from the Swiss national database. Energy and Buildings, 224, 110235. DOI: https://doi.org/10.1016/j.enbuild.2020.110235 

  10. Csoknyai, T., Hrabovszky-Horvath, S., Gerogiev, Z., Jovanovic-Popovic, M., Stankovic, B., Villatoro, O., & Szendro, G. (2016). Building stock characteristics and energy performance of residential buildings in Eastern-European countries. Energy and Buildings, 132, 39–52. DOI: https://doi.org/10.1016/j.enbuild.2016.06.062 

  11. Ding, Y., & Liu, X. (2020). A comparative analysis of data-driven methods in building energy benchmarking. Energy and Buildings, 209, 109711. DOI: https://doi.org/10.1016/j.enbuild.2019.109711 

  12. Duvier, C., Neagu, D., Oltean-Dumbrava, C., & Dickens, D. (2018). Data quality challenges in the UK social housing sector. International Journal of Information Management, 38(1), 196–200. DOI: https://doi.org/10.1016/j.ijinfomgt.2017.09.008 

  13. European Commission. (2018). Energy Performance of Buildings Directive. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2018.156.01.0075.01.ENG 

  14. Filippidou, F., Nieboer, N., & Visscher, H. (2019). Effectiveness of energy renovations: A reassessment based on actual consumption savings. Energy Efficiency, 12(1), 19–35. DOI: https://doi.org/10.1007/s12053-018-9634-8 

  15. Hjortling, C., Björk, F., Berg, M., & Klintberg, T. A. (2017). Energy mapping of existing building stock in Sweden—Analysis of data from Energy Performance Certificates. Energy and Buildings, 153, 341–355. DOI: https://doi.org/10.1016/j.enbuild.2017.06.073 

  16. Housing Europe. (2021). The state of housing in Europe 2021. https://www.stateofhousing.eu/#p=1 

  17. IEA. (2020). Tracking buildings 2020. International Energy Agency (IEA). https://www.iea.org/reports/tracking-buildings-2020 

  18. Jiang, P., Chen, Y., Dong, W., & Huang, B. (2014). Promoting low carbon sustainability through benchmarking the energy performance in public buildings in China. Urban Climate, 10, 92–104. DOI: https://doi.org/10.1016/j.uclim.2014.10.004 

  19. Laaroussi, Y., Bahrar, M., Zavrl, E., El Mankibi, M., & Stritih, U. (2020). New qualitative approach based on data analysis of European building stock and retrofit market. Sustainable Cities and Society, 63, 102452. DOI: https://doi.org/10.1016/j.scs.2020.102452 

  20. Laurent, M. H., Allibe, B., Galvin, R., Hamilton, I., Oreszczyn, T., & Tigchelaar, C. (2013). Back to reality: How domestic energy efficiency policies in four European countries can be improved by using empirical data instead of normative calculation. ECEEE Summer Study Proceedings 2013, 2057–2070. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2013/7-monitoring-and-evaluation/back-to-reality-how-domestic-energy-efficiency-policies-in-four-european-countries-can-be-improved-by-using-empirical-data-instead-of-normative-calculation/ 

  21. Majcen, D., Itard, L., & Visscher, H. (2013). Actual and theoretical gas consumption in Dutch dwellings: What causes the differences? Energy Policy, 61, 460–471. DOI: https://doi.org/10.1016/j.enpol.2013.06.018 

  22. National Climate Agreement. (2019). National climate agreement. https://www.klimaatakkoord.nl/binaries/klimaatakkoord/documenten/publicaties/2019/06/28/klimaatakkoord/klimaatakkoord.pdf 

  23. NEN. (2014). NEN 7120—Energieprestatie van gebouwen: Bepalingsmethode inclusief Nader Voorschrift (NV). Nederlands Normalisatie-Instituut (NEN). https://www.nen.nl/nen-7120-2012-c5-2014-nl-197669 

  24. NEN. (2020). NTA 8800: Energieprestatie van gebouwen: Bepalingsmethode. Nederlands Normalisatie-Instituut (NEN). https://www.nen.nl/nta-8800-2020-a1-2020-nl-278296 

  25. Netherlands & Schorer, J. R. (2004). Woningwet: Wet van 29 augusts 1991 …; Wet balansverkorting geldelijke steun volkshuisvesting: Wet van 31 mei 1995. Kluwer. 

  26. Overheid.nl. (2018). Woningwet (13 June). Overheid.nl. https://wetten.overheid.nl/BWBR0005181/2018-06-13 

  27. Roth, J., Lim, B., Jain, R. K., & Grueneich, D. (2020). Examining the feasibility of using open data to benchmark building energy usage in cities: A data science and policy perspective. Energy Policy, 139, 111327. DOI: https://doi.org/10.1016/j.enpol.2020.111327 

  28. Saunders, H. (2015). Recent evidence for large rebound: Elucidating the drivers and their implications for climate change models. Energy Journal, 36, 23–48. DOI: https://doi.org/10.5547/01956574.36.1.2 

  29. Sociaal Economische Raad. (2013). Energieakkoord voor duurzame groei. https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/convenanten/2013/09/06/energieakkoord-voor-duurzame-groei/energieakkoord-voor-duurzame-groei.pdf 

  30. Steadman, P., Evans, S., Liddiard, R., Godoy-Shimizu, D., Ruyssevelt, P., & Humphrey, D. (2020). Building stock energy modelling in the UK: The 3DStock method and the London Building Stock Model. Buildings & Cities, 1(1), 100–119. DOI: https://doi.org/10.5334/bc.52 

  31. Streicher, K. N., Padey, P., Parra, D., Bürer, M. C., & Patel, M. K. (2018). Assessment of the current thermal performance level of the Swiss residential building stock: Statistical analysis of energy performance certificates. Energy and Buildings, 178, 360–378. DOI: https://doi.org/10.1016/j.enbuild.2018.08.032 

  32. Summerfield, A. J., Oreszczyn, T., Palmer, J., Hamilton, I. G., Li, F. G. N., Crawley, J., & Lowe, R. J. (2019). What do empirical findings reveal about modelled energy demand and energy ratings? Comparisons of gas consumption across the English residential sector. Energy Policy, 129, 997–1007. DOI: https://doi.org/10.1016/j.enpol.2019.02.033 

  33. Sunikka-Blank, M., & Galvin, R. (2012). Introducing the prebound effect: The gap between performance and actual energy consumption. Building Research & Information, 40(3), 260–273. DOI: https://doi.org/10.1080/09613218.2012.690952 

  34. UNFCCC. (2015). The Paris agreement. United Nations Framework Convention on Climate Change (UNFCCC). https://unfccc.int/sites/default/files/english_paris_agreement.pdf 

  35. UNFCCC. (2021). The Glasgow climate pact. United Nations Framework Convention on Climate Change (UNFCCC). https://unfccc.int/sites/default/files/resource/cop26_auv_2f_cover_decision.pdf 

  36. van der Bent, H. S., van den Brom, P. I., Visscher, H. J., Meijer, A., & Mouter, N. (2021a). The energy performance of dwellings of Dutch non-profit housing associations: Modelling actual energy consumption. Energy and Buildings, 253, 111486. DOI: https://doi.org/10.1016/j.enbuild.2021.111486 

  37. van der Bent, H. S., Visscher, H. J., Meijer, A., & Mouter, N. (2021b). Monitoring energy performance improvement: Insights from Dutch housing association dwellings. Buildings & Cities, 2(1), 779–796. DOI: https://doi.org/10.5334/bc.139 

comments powered by Disqus