Accepted on 13 May 2022
Submitted on 21 Feb 2022
The United Nations agreed to keep global warming below 2°C and continue efforts to limit it to 1.5°C (UNFCCC 2015). Subsequent United Nations conferences of the parties (COP) agreed to limit the effects of climate change, up to the Glasgow Climate Pact (UNFCCC 2021). With worldwide CO2 emissions of > 10 Gt/yr (IEA 2020), the built environment is an important factor. Decarbonising the energy system in the built environment is a major challenge. This challenge is adopted at a European level, among others through the Energy Performance of Buildings Directive (EPBD) (European Commission 2018) and adopted at national levels as national policies.
In 2021, Dutch national policy on the sustainable development of the Dutch built environment entered a new phase. A new national climate agreement was signed in 2019 (National Climate Agreement 2019) and a new method to determine the theoretical energy performance of dwellings came into force in 2021 (NEN 2020). These changes (among others) influence how Dutch non-profit housing associations (hereafter ‘housing associations’) measure the energy performance of their building stock.
Dutch housing associations own 2.4 million dwellings, organised in 286 housing associations in 2021. This means one-third of the Dutch housing stock is owned by social housing associations (Autoriteit Woningcorporaties 2020). Other European countries with a large share of social housing are Austria at 24%, Denmark at 21%, Sweden at 17%, UK at 17%, France at 16%, Norway at 14% and Finland at 11% (Housing Europe 2021). Therefore, the non-profit housing stock plays an important role in helping to fulfil the Paris Agreement.
Benchmarking is a method to measure progress and create awareness about the performance of organisations in relation to goals. Benchmarking can be defined as:
a continuous analysis of strategies, functions, processes, products or services, performances, etc. compared within or between best-in-class organisations by obtaining information through appropriate data collection method, with the intention of assessing an organisation’s current standards and thereby carry out self-improvement by implementing changes to scale or exceed those standards.
The present paper examines the process of making a model to benchmark the energy performance of the housing stock of Dutch housing associations.
The Paris Agreement requires actions to reduce global warming worldwide, including in the built environment. The sustainable development of the housing stock is a challenge for many countries and is the subject of several papers that discuss the sustainable development of the housing stock, e.g. Bulgaria, Serbia, Hungary, the Czech Republic (Csoknyai et al. 2016), Sweden (Hjortling et al. 2017), Switzerland (Streicher et al. 2018), Ireland (Ahern & Norton 2019) and the Netherlands (van der Bent et al. 2021b). These papers describe changes of (parts of) the housing stock using many different descriptive parameters, without the aim to benchmark the progress.
Several papers seek to find appropriate conditions for establishing accurate models to benchmark the energy performance. Jiang et al. (2014) highlight the importance of establishing accurate and efficient databases to have a sound basis to assess and monitor the improvement of the energy performance of several types of Chinese buildings. They stress the need for clear definitions and discuss the differences in indicators to be used in benchmarks. They argue for indicators that include both technical and non-technical measures. Moreover, besides the energy performance unit of kWh/m2/yr used for assessing the overall energy performance, the unit kg CO2/m2/yr could be used for assessing the CO2 emissions of buildings.
However, Jiang et al. (2014) do not extrapolate their research to an organisational level or address the possibility of a performance gap between theoretical and actual energy consumption. This performance gap can be defined as the difference between the theoretical modelling of energy consumption through the EPBD and the actual energy consumption of dwellings, and is acknowledged throughout several papers covering multiple countries (Sunikka-Blank & Galvin 2012; Saunders 2015; Summerfield et al. 2019; Cozza et al. 2020). Reasons for the performance gap are an inadequate assumption of the actual indoor temperature, the influence of user behaviour on actual consumption, and differences between the theoretical calculation and the actual building quality of dwellings. Aranda et al. (2018) found that the energy performance gap for social housing is even larger than the energy performance gaps found for other dwellings.
Duvier et al. (2018) argue that it can be difficult to build and use high-quality datasets. They considered the UK social housing sector and argued that using the potential is difficult due to the constant changes of regulations which makes it difficult to develop long-term strategies. However, Steadman et al. (2020) give an example of large-scale available data within the UK to monitor and benchmark the energy performance of buildings. Ding & Liu (2020) compared several data-driven methods to benchmark the energy performance of individual dwellings. They highlight the need for both high-quality data and robust benchmark models. They address the concern that different indicators could lead to a different ranking of subjects, and recommend that policymakers consider multiple benchmarking methods or select a method, while being aware of what they actually measure. However, they do not apply this to housing associations, but at the level of individual dwellings.
Roth et al. (2020) examined the possibilities of using open data to benchmark building energy consumption in cities. They concluded that benchmarking by itself will not lead to energy savings, but benchmarking the energy consumption of dwellings can help to develop effective policies to lower actual energy consumption. Benchmarking the average energy performance of the housing stock of housing organisations could lead to similar results. Laaroussi et al. (2020) realised that the large amount of energy used by residential buildings means that the renovation of the building stock is one of the key strategies to reach energy performance targets. As a part of a European H2020 project, they analysed the situation in Spain, France, Italy, Slovenia and Austria and used a weighted method as an analytical approach to evaluate the main indicators of collected data. Bordass (2020) analysed different metrics for benchmarking the energy performance of buildings. He argued that benchmarks should not only focus on a single most important indicator, but a more diverse range of indicators could help users of benchmarks to take the correct actions. Too few indicators may lead to wrong decision-making. Furthermore, too many indicators lead to fogginess and also to wrong decision-making. Although applied to individual dwellings, this is applicable for benchmarking the energy performance of large sets of dwellings as well.
Several lessons arise from this literature:
The definition of benchmarking by Anand & Kodali (2008) indicates that benchmarking the energy performance of dwellings could be useful at an organisational level as well. Aedes, the Dutch umbrella organisation of housing associations located in The Hague, has organised a benchmark between housing associations since 2014, covering topics such as overhead costs, maintenance, tenant appreciation, availability and affordability, and energy performance (Aedes 2020). The benchmark helps to enhance the knowledge and factual basis of the development of the Dutch built environment owned by housing associations. Until 2020, the benchmark for energy performance consisted of two indicators: (1) the energy label value: the average Energy Index Nader Voorschrift (EI NV); and (2) the average actual CO2 emissions from heating demand through gas consumption and district heating. Due to changes in policy, these indicators need to be adjusted.
Social housing in the Netherlands has its origin in the Industrial Revolution when the Vereeniging ten behoeve der Arbeidersklasse te Amsterdam (Association for the Benefit of the Working Class in Amsterdam) was established in 1852 (Boissevain 1865). The Housing Act of 1901 provided the means for state funding for social housing organisations, which as a result grew in number and size. A policy change in 1995, the so-called ‘bruttering’ (Netherlands & Schorer 2004), made social housing organisations financially independent and focused their role as social entrepreneurs. In 2020, one-third of the Netherlands’ housing stock was owned by social housing associations. (Autoriteit Woningcorporaties 2020). The social housing stock of 2.4 million dwellings is organised into 286 housing associations. Housing associations are able to make decisions on how to manage their housing stock, but are regulated by strong central law (Overheid.nl 2018). For example, laws govern the allocation of tenants, maximum rents and sustainable transformation of dwellings. Housing associations in the Netherlands cover a broad range of organisations. Some are small (fewer than 1000 dwellings), while others are large (more than 80,000 dwellings). Some operate mainly in urban environments, while other housing associations own assets in more rural areas. Also, differences occur in the financial position of Dutch housing associations. Housing associations also have different types of dwellings. Some have relatively new dwellings, e.g. in the province of Flevoland, which is largely a polder that was developed in the second half of the 20th century. Other housing associations own old dwellings in historical city centres. Also, differences occur in the quality of the dwellings regarding energy performance. Some housing associations improved the energy performance of their stock extensively, while others have a stock with a lower energy performance, as benchmarked in the Dutch context up to 2020 (Aedes 2020).
Policies aimed at increasing the energy performance of dwellings of housing associations originate from the Energy Agreement (Sociaal Economische Raad 2013). Until 2020, the aim was to achieve an average energy label B in 2020 as directed from the EPBD, as analysed by van der Bent et al. (2021b).
In 2021, national policy on the sustainable development of the Dutch built environment entered a new phase. A new national climate agreement was signed in 2019 (National Climate Agreement 2019), and a new method to determine the theoretical energy performance of dwellings came into force in 2021, the NTA 8800 (NEN 2020). These changing policies affect the measuring of energy performance, and consequently the benchmarking of the energy performance of the housing stock of Dutch housing associations. These policies are further discussed in phase 1 of the results section.
No scientific research was found regarding the process of creating a benchmark for the energy performance of the housings stock of housing organisations. Creating a benchmark model is relevant both in the Dutch context and for the international community, helping to move to a sustainable built environment. Researchers and policymakers from other countries can benefit both from the following process as well from the final benchmark model to create similar benchmark models across their housing stock. Dutch housing associations aim to benchmark the energy performance of the non-profit housing stock, given the changes in policy derived from the National Climate Agreement 2019 (National Climate Agreement 2019) and the enforcement of the NTA 8800 (NEN 2020) in 2021.
The aim of the present research is to find a set of indicators that adequately represents the performance of Dutch housing associations according to the changed policies. The research question is therefore: Which set of indicators can be used to benchmark the energy performance of Dutch housing associations and what can we learn from the process to find these indicators?
The research method used is a combination of desk research with action research where the principal researcher participated in group sessions with experts from Dutch housing associations.
The expert group consisted of 18 employees from housing associations and two employees of Aedes, the umbrella organisation of housing associations. The expert group guided the existing benchmark until 2020 and now guides the adjustment of the benchmark model beyond 2021. The group was a mix representing both small and large housing associations, from different areas of the Netherlands, operating in different parts of the non-profit housing stock, both rural and urban. Staff members in this group typically have several to many years (10 or more) of working experience in the field of sustainable development of housing associations and typically have job descriptions related to the strategic advisory of sustainable development or the actual planning of sustainable projects at housing associations. Based on availability they attended sessions following phases 2–4, as described below, usually having 12–15 attendees. Decisions were made on shared agreement after discussion sessions following the phases.
In addition to the expert group, two different advisory groups of Aedes comprising directors of housing associations and the general board of Aedes (also comprising directors of housing associations) were involved in the selection of the final model. The advisory groups and general board are part of the decision-making structure of Aedes.
Seven phases are distinguished in this research: policy review, identifying the available data, assessment of the indicators, integration in the benchmark models, selection of the benchmark model, data collection and benchmark results. These phases were not predetermined but suggested throughout the process by the principal researcher and acknowledged by the expert group. The phases took place between September 2020 and December 2021, as specified in Table 1.
Table 1
Research phases and participation.
| PHASE | GROUP | PERIOD |
|---|---|---|
| 1: Policy review | Expert group | September 2020 |
| 2: Identify the available data | Expert group | September 2020 |
| 3: Assessment of the indicators | Expert group | October 2020 |
| 4: Integration in the benchmark models | Expert group | November 2020 |
| 5: Selection of the benchmark model | Two directors groups and general board Aedes | January and April 2021 |
| 6: Data collection | Principal researcher | June–August 2021 |
| 7: Benchmark results | Principal researcher | September–December 2021 |
Phase 1 consists of a policy review, to identify relevant incentives for housing associations. Phase 2 consists of desk research where available benchmark indicators were identified from the available data sources. After this phase, a group session with expert staff members from housing associations was organised, where they discussed the policy review and confirmed the list of available indicators. Phase 3 consists of an assessment of the available indicators using a weighted multicriteria approach with five assessment criteria to narrow down the identified options in phase 2 to a smaller list of viable options. The assessment of the available indicators ended with a group session of the expert group confirming the assessment of the available indicators. Phase 4 consists of a desk assessment and expert group discussion about four integrated models with indicators to measure the sustainable development of Dutch housing associations. The expert group confirmed the four selected models and proposed a final model. Phase 5 consists of a group discussion with two different advisory groups of directors of housing associations. They analysed and judged the four selected models. The general board of Aedes, also comprising directors of housing associations, affirmed the selected final model. Phase 6 consists of the data collection used for benchmarking. Phase 7 describes shortly the results of the benchmark.
The results of this research are described according to the phases in the research method section.
The first phase consisted of a policy review. The aim was to understand the policy context relevant for benchmarking the energy performance of housing associations.
In the summer of 2019, a new Climate Agreement (National Climate Agreement 2019) was agreed between the Dutch government, Dutch companies and organisations to mitigate climate change in agreement with European goals. The Climate Agreement is enforced with legislation and further specified agreements with businesses and sector organisations. For the Dutch non-profit housing sector, several policies apply. A generic goal was formulated to lower CO2 emissions from the built environment. This is framed as the fossil fuel (natural gas) consumption used by the built environment (primarily for space heating and water heating). This is enforced with several policies discussed below. These policies are different from those resulting from the Climate Agreement in 2013 (Sociaal Economische Raad 2013), where it was agreed that the housing stock of housing associations should obtain an average energy label B in 2020, based on the theoretical energy performance of buildings calculation. Newly proposed policies from the Climate Agreement 2019 do not enforce a goal expressed as an energy label. These proposed policies relevant to housing associations are the neighborhood-oriented approach, subsidies to accelerate the rate of the renovation, and the proposed introduction of a theoretical maximum heating demand for dwellings.
The neighborhood-oriented approach is a policy that is driven by municipalities. It aims to enhance a sustainable energy system by coordinating actions of (local) authorities, energy infrastructure companies, local companies, inhabitants and also housing associations. Regional and local development plans are written to combine the availability of heat, the quality of the energy infrastructure, and the energy demand from buildings and companies. With one-third of the Dutch housing stock owned by housing associations, they are an important stakeholder in this neighborhood-oriented approach.
Policy aiming to accelerate the renovation pace of housing associations consists of two main subsidies: one for district heating solutions, and another to increase renovations by bundling demand and coupling it to innovative steady supplies.
The new policy introducing a theoretical maximum heating demand for dwellings in 2050 aims to lower the energy demand of dwellings by improving the quality of the building envelope.
These policies are successors of the old policy where housing associations aimed at improving the energy performance of dwellings to an average energy label B. These new policies form a wider approach with different incentives to improve the energetic quality of the non-profit housing stock. These incentives are mainly aimed at the energy performance of dwellings, with an aim to decrease energy consumption during the operation phase of dwellings.
Following European regulations from the EPBD, an improved calculation method for the theoretical energy performance of buildings has been enforced in the Netherlands from 2021, via the NTA 8800 (NEN 2020). This improved calculation method has some major changes as opposed to its predecessor the NEN 7120 NV, as examined by van der Bent et al. (2021b). Previously, the energy label had a dimensionless value: the theoretical energy index (a calculation of the theoretical energy use divided by a combination of floor area and building envelope area). The new energy label value is still based on a theoretical energy consumption, but divided only by the floor area. Therefore, it can be expressed in the dimension of kWh/m2/yr. Also, the calculation of the theoretical energy consumption is improved and updated regarding the characteristics of building installations and building physics.
In the second phase, the available indicators for benchmarking the sustainable development of Dutch housing associations were identified by the principal researcher in consultation with the expert group. High-quality datasets are recommended as a basis for measuring and benchmarking (Jiang et al. 2014; Duvier et al. 2018; Steadman et al. 2020). Different metrics should be examined (Bordass 2020), where the units kWh/m2/yr and kg CO2/m2/yr are specifically mentioned by Jiang et al. (2014). A combination of desk research, expert knowledge from the principal researcher and expert knowledge from the expert group led to the identification of three main data sources which are available to benchmark the sustainable development of Dutch housing associations: the SHAERE database, the dVi database and data from the Central Bureau of Statistics (CBS). Creating new data sources was not considered feasible due to high overhead costs to collect new data.
From these sources, a list of potential indicators was extracted to measure the development of the energy performance of the housing stock of Dutch housing associations. A proposal was made by the principal researcher to the expert group which discussed and agreed that these 12 indicators are options to consider measuring the development of Dutch housing associations. The potential indicators are listed in Table 2.
Table 2
Possible indicators to benchmark the energy performance of dwellings.
| POSSIBLE INDICATOR | UNIT | SOURCESa | DESCRIPTION |
|---|---|---|---|
| 1. Average theoretical primary fossil energy consumption | kWh/m2/yr | SHAERE | Energy label value after 2020 under the NTA 8800. Based on a theoretical energy consumption divided by floor area |
| 2. Average theoretical energy index | n | dVi | Energy label value up to 2020 |
| 3. Average number of label steps in the energy label | n | SHAERE | Average number of energy label steps after 2020 under the NTA 8800 |
| 4. Average theoretical heating demand | kWh/m2/yr | SHAERE | Theoretical unit of measurement of the heating demand of dwellings |
| 5. Average difference heating demand and maximum heating demand | kWh/m2/yr | SHAERE | Average difference between the heating demand of dwellings (quality building envelope) and the maximum heating demand based on the layout and topology |
| 6. Percentage dwellings complying with the maximum heating demand | % | SHAERE | Percentage of dwellings below the maximum heating demand of the dwelling based on its layout and topology |
| 7. Percentage of dwellings that are gas-free | % | SHAERE | Percentage of dwellings without a gas-fired heating system |
| 8. Percentage of dwellings with photovoltaic panels | % | SHAERE | Percentage of dwellings with photovoltaic (PV) panels, and therewith contributing to the production of clean electricity |
| 9. Indicator mix of building characteristics | Undefined | SHAERE | Undefined combination of building characteristics; however, not yet operationalised |
| 10. Average actual energy consumption (gas + electricity and district heating) | kWh/m2/yr | CBS | Combined average actual energy consumption on a dwelling level per m2 of the three main energy carriers in the Netherlands: gas, electricity and district heating |
| 11. Average actual CO2 emission (gas + electricity and district heating) | kgCO2/m2/yr | CBS | Combined average actual CO2 emissions on a dwelling level per m2 of the three main energy carriers in the Netherlands: gas, electricity and district heating |
| 12. Average actual CO2 emission (gas) | kgCO2/m2/yr | CBS | Average actual CO2 emissions on a dwelling level per m2 of gas consumption. CO2 emission within the building |
Note: Indicators are listed without any order of preference.
In the third phase, the indicators from the list were assessed on suitability by scoring the indicators on five assessment criteria by the principal researcher in consultation with the expert group (see below). Using a weighted method as an analytical approach can contribute to the selection of main indicators (Laaroussi et al. 2020). The aim of this phase was to narrow down the identified options in phase 2 to a smaller list of viable options. The criteria to assess the identified options were determined by the principal researcher and validated by the expert group. The five criteria aim to ensure indicators are effectual (available and comparable), are communicable (recognisable), and relate to policy (both at the national level and as perceived by housing associations). The five criteria are:
Each possible indicator was scored from –2 (totally non-compliant) to +2 (totally compliant). A weighting factor of 1 was given to the criteria availability, comparability and recognisability, and a weighting factor of 2 was given to the criteria the relation to national policy and the relation to housing association policy, because these last two are regarded as more important by the expert group. This led to the scoring table shown in Table 3.
Table 3
Assessment of indicators with a scoring table.
| WEIGHTING FACTOR ASSESSMENT CRITERIA: | 1 | 1 | 1 | 2 | 2 | |||
|---|---|---|---|---|---|---|---|---|
| INDICATOR NO. | INDICATOR DESCRIPTION | AVAILABILITY | COMPARABILITY | RECOGNISABILITY | NATIONAL POLICY | HOUSING ASSOCIATION POLICY | TOTAL | SELECTED FOR THE NEXT PHASE |
| Building indicator | ||||||||
| 1 | Average theoretical primary fossil energy consumption | 1 | 1 | 1 | 2 | 2 | 11 | × |
| 2 | Average theoretical energy index | –2 | –2 | 1 | –1 | 0 | –5 | |
| 3 | Average number of label steps in the energy label | 1 | 1 | 1 | –1 | –1 | –1 | |
| Insulation indicators | ||||||||
| 4 | Average theoretical heating demand | 1 | 0 | 1 | –1 | –1 | –2 | |
| 5 | Average difference between heating demand and maximum heating demand | 1 | 1 | 1 | 2 | 1 | 9 | × |
| 6 | Percentage of dwellings complying with the maximum heating demand | 1 | 1 | 1 | 1 | 1 | 7 | × |
| Installation indicators | ||||||||
| 7 | Percentage of dwellings gas-free | 1 | 2 | 2 | 1 | 1 | 9 | × |
| 8 | Percentage of dwellings with photovoltaic (PV) panels | 1 | 2 | 2 | 1 | 1 | 9 | × |
| 9 | Indicator mix of building characteristics | 1 | 0 | 0 | 0 | 0 | 1 | |
| Effect indicator | ||||||||
| 10 | Average actual energy consumption (gas plus electricity plus district heat) | –1 | 0 | 1 | 0 | 1 | 2 | |
| 11 | Average actual CO2 emissions (gas plus electricity plus district heat) | –1 | 0 | 1 | 1 | 2 | 6 | × |
| 12 | Average actual CO2 emissions (gas) | 0 | 0 | 1 | 2 | 1 | 7 | × |
It was then decided in consultation with the expert group to consider seven indicators scoring > 5 (Table 3). Five indicators scoring < 5 were not considered in the next phase. The most salient trade-offs from the assessment of the indicators are described as follows:
In the fourth phase, the available and ranked indicators were integrated by the principal researcher in consultation with the expert group in different benchmark models consisting of one or several indicators. Examining multiple benchmark models to select a robust model is recommended by Ding & Liu (2020). This was done to be able to assess relations between indicators. The integrated models all have a different bandwidth of benchmarking, ranging from a small model (housing associations are benchmarked on a single indicator) to a wide model (housing associations are benchmarked on six indicators). The different combinations of indicators were determined by the principal researcher and validated by the expert group. The combinations are shown in Table 4. We gave the four benchmark models a name with a description, explaining the nature of the combination of indicators.
Table 4
Schematisation of the four proposed benchmark models.
| INDICATOR NO. | INDICATOR DESCRIPTION | BASIC MODEL | REAL ESTATE MODEL | POLICY PERFORMANCE MODEL | WIDE MODEL |
|---|---|---|---|---|---|
| 1 | Average theoretical primary fossil energy consumption | × | × | × | × |
| 5 | Percentage of dwellings complying with maximum heating demand | × | / | ||
| 6 | Average difference heating demand and maximum heating demand | × | / | ||
| 7 | Percentage of dwellings gas-free | × | × | ||
| 8 | Percentage of dwellings with photovoltaic (PV) panels | × | × | ||
| 11 | Average actual CO2 emissions (gas + electricity + district heat) | × | |||
| 12 | Average actual CO2 emissions (gas) | × | × | ||
Note: The symbol ‘/’ means that one of these two indicators should be chosen.
In the expert group, the advantages and disadvantages of these benchmark models were discussed. These are described as:
The fifth phase consists of a group discussion of two advisory groups of housing association directors. They analysed and judged the four proposed models and agreed with the proposal of the expert group to select the policy performance model as the final model. The combination of three indicators in the policy performance model most accurately reflects the efforts of housing associations to improve the average energy performance of their housings stocks in accordance with the current policies. The general board of Aedes, the umbrella organisation of housing associations, affirmed the selected policy performance model. Therefore, the selected benchmark model consists of three indicators: (1) the average theoretical primary fossil energy consumption; (2) the average difference between the theoretical heating demand (quality building envelope) and the theoretical maximum heating demand; and (3) the average actual CO2 emissions from gas consumption.
Data were collected in June and July 2021, commissioned by Aedes and executed by the principal researcher. Housing associations voluntarily delivered a standardised data export with indicators of the energetic quality of their dwellings, together forming the SHAERE database. A total of 246 housing associations participated with over 2 million dwellings, covering 95% of the Dutch non-profit housing sector. This data source delivered data for indicators 1 and 2. In August 2021, the dwellings were anonymously connected to actual energy consumption data in an analysis environment at the CBS to be able to deliver data for indicator 3. The anonymisation ensured that no ethical issues arose during the collection and handling of data.
Figures 1, 2, 3 show the ranking of the housing associations according to the three indicators of the selected benchmark model. A more extensive description of the benchmark results was published by Aedes as part of a wider benchmark of the Dutch non-profit housing sector (Aedes 2021). No absolute benchmark values are stated for the different indicators. In this wider benchmark, housing associations are scored with an A, B or C, respectively, one-third of the population per indicator, accompanied by a more detailed dataset with secondary indicators to enhance the learning opportunities from the benchmark results.
Housing associations ranked by the average theoretical primary fossil energy consumption.
Housing associations ranked by the average difference between theoretical heating demand and maximum theoretical heating demand.
Housing associations ranked by the average actual CO2 emissions from gas.
This research addressed the question: Which set of indicators can be used to benchmark the energy performance of Dutch housing associations and what can we learn from the process to find these indicators? After an extensive process, three indicators were selected that cover the most important policy aspects for the sustainable change of the housing stock of housing associations. The process was divided into several phases, which helped to arrive at a well-founded benchmark model. During the process several aspects from literature were applied. Different metrics (Bordass 2020) were examined and the metrics kWh/m2/yr and kg CO2/m2/yr were applied in the benchmark, complying with Jiang et al. (2014). High-quality datasets were found and used as a basis for measuring and benchmarking (Jiang et al. 2014; Duvier et al. 2018; Steadman et al. 2020). A weighted method as an analytical approach contributed to the selection of main indicators (Laaroussi et al. 2020), and we found that exploring multiple benchmark models could support decision-making to a final model (Ding & Liu 2020).
Several limitations arose. The analysis is country specific, which means it is only applicable for Dutch housing associations within the current Dutch policy context. However, the process used to create a benchmark model is generic. A policy analysis, selections of indicators, assessment of indicators, integration into models, selection of the final model, data collection and delivering results are applicable to a wide variety of similar questions in other countries that seek energy performance measurement and benchmarking of the built environment.
A second limitation is the Dutch specific policy context where the built environment focus is on the quality of dwellings and the emissions of CO2 related to the energy consumption within the dwelling. For example, no clear goals or targets are formulated regarding the use of materials in retrofitting dwellings (although these exist for new construction). Moreover, no data are available describing the sustainable use of materials for every housing association. This limits the measurement of the sustainable performance to energy consumption. Other countries may have a different policy context with a wider interpretation of sustainable performance.
A third limitation within the Dutch policy context is the presence of the energy performance gap, widely researched for both the Dutch (Majcen et al. 2013; Filippidou et al. 2019; van der Bent et al. 2021a) and European (Laurent et al. 2013; Summerfield et al. 2019) contexts. Theoretical indicators measuring the energy performance of dwellings, derived from the EPBD, all have a performance gap: the theoretical energy consumption deviates strongly from actual energy consumption. In the Dutch benchmark model, this is covered to some extent by having three indicators. Two are theoretical, but the third is based on the actual energy consumption of fossil fuel–natural gas (translated to CO2 emissions). This ensures housing associations also have an incentive to lower actual gas consumption and related CO2 emissions. Other countries measuring and benchmarking sustainable performance should be aware of this performance gap as well.
Finally, Anand & Kodali (2008) state that benchmarks should be understood as a repetitive process. The results of benchmarking should lead to improvement in organisations, so the effectiveness of the benchmark model needs to be reviewed and updated periodically. During our research we were not able to close this loop for time reasons, but a suggestion for improvement would be to include this in future research.
A process was created for formulating an energy performance benchmark for Dutch housing associations. A similar process can be used by other researchers aiming at benchmarking the energy performance between organisations within their policy context. The final policy performance model to measure and benchmark the sustainable performance of Dutch housing associations consists of three indicators closely related to governing policies regarding the sustainable improvement of the Dutch non-profit housing sector: The average theoretical primary fossil energy consumption, the average difference between the theoretical heating demand (quality building envelope) and the maximum theoretical heating demand, and the average actual CO2 emissions from gas consumption. The first indicator is related to the current policy regarding the energy labelling of dwellings, derived from the Energy Performance of Buildings Directive (EPBD), the NTA 8800. The second indicator relates to the policy to decrease the average theoretical heat demand of dwellings. The third indicator is related to the goal for the Dutch built environment to lower actual CO2 emissions. The model was then used to collect data and benchmark the energy performance of dwellings of housing associations. This research contributes to the wider literature by creating a model for benchmarking the energy performance of dwellings within the relevant policy context. This will be increasingly relevant for policymakers and landlords who need to respond to the United Nations Paris Agreement by reducing greenhouse gas emissions.
The creation of the benchmark model was commissioned by Aedes, the Dutch association of housing associations. Aedes is acknowledged for its cooperation in the development and execution of the benchmark among Dutch non-profit housing associations.
HSvdB was responsible for conceptualisation, data collection, methodology, analysis and writing. HJV was responsible for funding, review and editing. AM and NM contributed the review and editing.
The authors have no competing interests to declare.
Data used in the research project are not publicly available due to restrictions of ownership.
Aedes, the Dutch association of housing associations, is a sponsor of the research project. This project was executed with the support of the MMIP 3 & 4 grant from the Netherlands Ministry of Economic Affairs & Climate Policy, as well as the Ministry of the Interior and Kingdom Relations.
Aedes. (2020). Meer tevreden huurders ondanks moeilijke tijden. https://aedes.nl/aedes-benchmark/benchmarkresultaten-en-publicaties
Aedes. (2021). Huurders tevreden, corporaties overbelast. https://aedes.nl/aedes-benchmark/benchmarkresultaten-en-publicaties
Ahern, C., & Norton, B. (2019). Thermal energy refurbishment status of the Irish housing stock. Energy and Buildings, 202, 109348. DOI: https://doi.org/10.1016/j.enbuild.2019.109348
Anand, G., & Kodali, R. (2008). Benchmarking the benchmarking models. Benchmarking: An International Journal, 15(3), 257–291. DOI: https://doi.org/10.1108/14635770810876593
Aranda, J., Zabalza, I., Llera-Sastresa, E., Scarpellini, S., & Alcalde, A. (2018). Building energy assessment and computer simulation applied to social housing in Spain. Buildings, 8(1). DOI: https://doi.org/10.3390/buildings8010011
Autoriteit Woningcorporaties. (2020). Staat van de corporatiesector 2020. https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/rapporten/2021/02/05/staat-van-de-corporatiesector-2020/bijlage-1-staat-van-de-corporatiesector-2020.pdf
Boissevain, J. (1865). Betere woningen voor de arbeidersklasse. In Staatkundig en Staathuishoudkundig Jaarboekje 1865 (pp. 387–411). De Vereeniging voor de statistiek in Nederland.
Bordass, B. (2020). Metrics for energy performance in operation: The fallacy of single indicators. Buildings & Cities, 1(1), 260–276. DOI: https://doi.org/10.5334/bc.35
Cozza, S., Chambers, J., Deb, C., Scartezzini, J.-L., Schlüter, A., & Patel, M. K. (2020). Do energy performance certificates allow reliable predictions of actual energy consumption and savings? Learning from the Swiss national database. Energy and Buildings, 224, 110235. DOI: https://doi.org/10.1016/j.enbuild.2020.110235
Csoknyai, T., Hrabovszky-Horvath, S., Gerogiev, Z., Jovanovic-Popovic, M., Stankovic, B., Villatoro, O., & Szendro, G. (2016). Building stock characteristics and energy performance of residential buildings in Eastern-European countries. Energy and Buildings, 132, 39–52. DOI: https://doi.org/10.1016/j.enbuild.2016.06.062
Ding, Y., & Liu, X. (2020). A comparative analysis of data-driven methods in building energy benchmarking. Energy and Buildings, 209, 109711. DOI: https://doi.org/10.1016/j.enbuild.2019.109711
Duvier, C., Neagu, D., Oltean-Dumbrava, C., & Dickens, D. (2018). Data quality challenges in the UK social housing sector. International Journal of Information Management, 38(1), 196–200. DOI: https://doi.org/10.1016/j.ijinfomgt.2017.09.008
European Commission. (2018). Energy Performance of Buildings Directive. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2018.156.01.0075.01.ENG
Filippidou, F., Nieboer, N., & Visscher, H. (2019). Effectiveness of energy renovations: A reassessment based on actual consumption savings. Energy Efficiency, 12(1), 19–35. DOI: https://doi.org/10.1007/s12053-018-9634-8
Hjortling, C., Björk, F., Berg, M., & Klintberg, T. A. (2017). Energy mapping of existing building stock in Sweden—Analysis of data from Energy Performance Certificates. Energy and Buildings, 153, 341–355. DOI: https://doi.org/10.1016/j.enbuild.2017.06.073
Housing Europe. (2021). The state of housing in Europe 2021. https://www.stateofhousing.eu/#p=1
IEA. (2020). Tracking buildings 2020. International Energy Agency (IEA). https://www.iea.org/reports/tracking-buildings-2020
Jiang, P., Chen, Y., Dong, W., & Huang, B. (2014). Promoting low carbon sustainability through benchmarking the energy performance in public buildings in China. Urban Climate, 10, 92–104. DOI: https://doi.org/10.1016/j.uclim.2014.10.004
Laaroussi, Y., Bahrar, M., Zavrl, E., El Mankibi, M., & Stritih, U. (2020). New qualitative approach based on data analysis of European building stock and retrofit market. Sustainable Cities and Society, 63, 102452. DOI: https://doi.org/10.1016/j.scs.2020.102452
Laurent, M. H., Allibe, B., Galvin, R., Hamilton, I., Oreszczyn, T., & Tigchelaar, C. (2013). Back to reality: How domestic energy efficiency policies in four European countries can be improved by using empirical data instead of normative calculation. ECEEE Summer Study Proceedings 2013, 2057–2070. https://www.eceee.org/library/conference_proceedings/eceee_Summer_Studies/2013/7-monitoring-and-evaluation/back-to-reality-how-domestic-energy-efficiency-policies-in-four-european-countries-can-be-improved-by-using-empirical-data-instead-of-normative-calculation/
Majcen, D., Itard, L., & Visscher, H. (2013). Actual and theoretical gas consumption in Dutch dwellings: What causes the differences? Energy Policy, 61, 460–471. DOI: https://doi.org/10.1016/j.enpol.2013.06.018
National Climate Agreement. (2019). National climate agreement. https://www.klimaatakkoord.nl/binaries/klimaatakkoord/documenten/publicaties/2019/06/28/klimaatakkoord/klimaatakkoord.pdf
NEN. (2014). NEN 7120—Energieprestatie van gebouwen: Bepalingsmethode inclusief Nader Voorschrift (NV). Nederlands Normalisatie-Instituut (NEN). https://www.nen.nl/nen-7120-2012-c5-2014-nl-197669
NEN. (2020). NTA 8800: Energieprestatie van gebouwen: Bepalingsmethode. Nederlands Normalisatie-Instituut (NEN). https://www.nen.nl/nta-8800-2020-a1-2020-nl-278296
Netherlands & Schorer, J. R. (2004). Woningwet: Wet van 29 augusts 1991 …; Wet balansverkorting geldelijke steun volkshuisvesting: Wet van 31 mei 1995. Kluwer.
Overheid.nl. (2018). Woningwet (13 June). Overheid.nl. https://wetten.overheid.nl/BWBR0005181/2018-06-13
Roth, J., Lim, B., Jain, R. K., & Grueneich, D. (2020). Examining the feasibility of using open data to benchmark building energy usage in cities: A data science and policy perspective. Energy Policy, 139, 111327. DOI: https://doi.org/10.1016/j.enpol.2020.111327
Saunders, H. (2015). Recent evidence for large rebound: Elucidating the drivers and their implications for climate change models. Energy Journal, 36, 23–48. DOI: https://doi.org/10.5547/01956574.36.1.2
Sociaal Economische Raad. (2013). Energieakkoord voor duurzame groei. https://www.rijksoverheid.nl/binaries/rijksoverheid/documenten/convenanten/2013/09/06/energieakkoord-voor-duurzame-groei/energieakkoord-voor-duurzame-groei.pdf
Steadman, P., Evans, S., Liddiard, R., Godoy-Shimizu, D., Ruyssevelt, P., & Humphrey, D. (2020). Building stock energy modelling in the UK: The 3DStock method and the London Building Stock Model. Buildings & Cities, 1(1), 100–119. DOI: https://doi.org/10.5334/bc.52
Streicher, K. N., Padey, P., Parra, D., Bürer, M. C., & Patel, M. K. (2018). Assessment of the current thermal performance level of the Swiss residential building stock: Statistical analysis of energy performance certificates. Energy and Buildings, 178, 360–378. DOI: https://doi.org/10.1016/j.enbuild.2018.08.032
Summerfield, A. J., Oreszczyn, T., Palmer, J., Hamilton, I. G., Li, F. G. N., Crawley, J., & Lowe, R. J. (2019). What do empirical findings reveal about modelled energy demand and energy ratings? Comparisons of gas consumption across the English residential sector. Energy Policy, 129, 997–1007. DOI: https://doi.org/10.1016/j.enpol.2019.02.033
Sunikka-Blank, M., & Galvin, R. (2012). Introducing the prebound effect: The gap between performance and actual energy consumption. Building Research & Information, 40(3), 260–273. DOI: https://doi.org/10.1080/09613218.2012.690952
UNFCCC. (2015). The Paris agreement. United Nations Framework Convention on Climate Change (UNFCCC). https://unfccc.int/sites/default/files/english_paris_agreement.pdf
UNFCCC. (2021). The Glasgow climate pact. United Nations Framework Convention on Climate Change (UNFCCC). https://unfccc.int/sites/default/files/resource/cop26_auv_2f_cover_decision.pdf
van der Bent, H. S., van den Brom, P. I., Visscher, H. J., Meijer, A., & Mouter, N. (2021a). The energy performance of dwellings of Dutch non-profit housing associations: Modelling actual energy consumption. Energy and Buildings, 253, 111486. DOI: https://doi.org/10.1016/j.enbuild.2021.111486
van der Bent, H. S., Visscher, H. J., Meijer, A., & Mouter, N. (2021b). Monitoring energy performance improvement: Insights from Dutch housing association dwellings. Buildings & Cities, 2(1), 779–796. DOI: https://doi.org/10.5334/bc.139