Building occupants can provide meaningful insights about the performance of their space. Their direct experiences with the indoor environments they inhabit allow them to determine whether those spaces support or hinder the activities performed. There are several ways to examine this relationship between occupants and their spaces. For example, unobtrusively monitoring occupant behaviors or the environment characteristics of their space provides an objective view of how the environment is being used, but can lack occupant insights that could only be attained by speaking directly to occupants. Alternatively, interviews and focus groups allow for in-depth discussion with occupants around the perception of an environment, but these methods tend to be labor and time intensive to conduct and analyse (Leary 2017). In response to these challenges, surveys have been used to provide a quick and cost-effective way of assessing and analysing occupant experiences with their indoor spaces. Post-occupancy evaluation (POE) surveys are a popular method of collecting these types of building user insights (Leaman & Bordass 2001; Zimmerman & Martin 2001; Zagreus et al. 2004; Candido et al. 2016; Li et al. 2018). Over the years, the content of these tools has shifted (Peretti & Schiavon 2011), and each theoretical approach presents its own unique set of strengths and weaknesses (Vischer 1989).
In 1999, the Center for the Built Environment (CBE) at the University of California, Berkeley created one of the first digital post-occupancy surveys. Known as the CBE Occupant Survey, this web-based tool is designed to measure the satisfaction and self-reported productivity of occupants in a variety of building types (Zagreus et al. 2004; Heinzerling et al. 2013). Originally conceived by a team of engineering, architecture, and psychology experts, the tool has continually evolved in response to industry demands and research agendas, while preserving the core elements of a POE survey.
The CBE Occupant Survey has collected occupant assessments for over two decades and compiled them into an extensive database. One major benefit of this large data set is that it establishes a reference point for the ‘average’ occupant. This reference allows a comparison (or ‘benchmark’) of ratings for a specific building feature (e.g. satisfaction with air quality or acoustics) or to the overall evaluation of a space. Creating this norm is useful for building owners and operators working to make informed decisions about future building adjustments (Vischer 1989). In 2015, the developers of the Building Occupant Survey System Australia (BOSSA) (Candido et al. 2016) presented data on the development of their tool’s measurement and benchmark properties. However, in general, little critical examination has been given to the content and measurement properties of tools such as these. And even more, little discussion has occurred on ways to unify measurement efforts and best practices across POEs widely used in the field.
Over the lifetime of the CBE Occupant Survey, there has been a shift in industry from focusing on delivering satisfactory environments to now making spaces healthy for both occupants and the planet. Building certification programs such as Leadership in Energy and Environmental Design (LEED) in North America, Building Research Establishment Environmental Assessment Method (BREEAM) in the UK, and GreenMark in Singapore all advocate for more sustainable buildings. Yet, in more recent years, there has been a surge in certification systems such as The WELL Building Standard, Fitwel, and the Living Buildings Challenge, which aim to tie occupant health and wellbeing more directly with environmental sustainability, and attention has increased towards crafted spaces where occupants can thrive.
As the focus towards creating spaces that promote occupant wellness rises, it is necessary to examine more critically how the success of a space is assessed. Traditionally, both the CBE Occupant Survey and the analysis of its database have focused on self-reported productivity and occupant satisfaction across a range of indoor environmental quality (IEQ) factors (Zagreus et al. 2004; Altomonte & Schaivon 2013; Frontczak et al. 2012). These data and tool have also been used to compare different building systems (e.g. radiant versus all-air systems or mixed-mode versus air-conditioned) (Karmann et al. 2017; Brager & Baker 2009), or to assess green certification effectiveness (Altomonte et al. 2019). This focus on IEQ is echoed throughout other widely used POE tools (such as BOSSA: Candido et al. 2016; and the BUS Methodology: Leaman & Bordass 2001). These are undoubtedly important factors in understanding the functionality of a space, but as the definition shifts of how a successful building serves its occupants, so must the tools to measure that success. Some existing POE tools have responded accordingly. For instance, the BUS survey (Cohen et al. 2010) uses measures of occupant self-reported productivity and health. Others, such as the Building Assessment Survey and Evaluation Study (USEPA 2003) and the HOPE project (Roulet et al. 2006), incorporate metrics of health, wellbeing, and job characteristics. Though tools such as these incorporate metrics aimed at understanding occupant wellness and the occupant overall in more depth (in addition to their perceptions of IEQ), there is little cohesion within the field in the way POEs measure these variables. This lack of consistency makes it harder to draw broad conclusions about the relationship between a space and these occupant factors.
POEs such as those mentioned above, and their resulting databases, are impacting the ways design decisions are made, building standards and policies are formed, and ultimately the way people live within built spaces. Considering these forces, it is more important than ever to assess the effectiveness of POEs to ensure surveys are achieving the field’s desired aims. A successful survey (Devlin et al. 1993) works to:
Although it can be challenging to know whether a survey has met these criteria, one that has been extensively and repeatedly used can help to shed a light on the efficacy of survey measurement. It seems timely to examine thoroughly the properties of the CBE Occupant Survey tool, and to summarise the resulting data set built over 20 years from hundreds of buildings around the world. Doing so allows a critical examination of where to reduce redundancy and incorporate new questions to measure previously ignored factors in understanding the success of a building. Lessons learned in a critical analysis of this tool can also help illuminate the best practices and future directions for similar surveys and standard POE methods currently in place.
This paper examines the measurement characteristics of the CBE Occupant Survey and the resulting database collected to date. First, the distribution of participant responses to perceived satisfaction and self-reported productivity is examined across several IEQ factors. This helps to determine the representativeness and generalisability of this data set. Next, the structure and measurement properties of the survey are examined to assess critically the psychometrics of this tool that are frequently used to measure space. Further, how the benchmarking metrics align with the tool’s underlying structure is studied to ensure they best meet the needs of practitioners who base design decisions off those scores. Finally, the findings from the analysis are framed through a multidisciplinary lens to suggest future directions and best practices for constructing and performing POEs.
To conduct the analysis, the latest version of the database was retrieved and simple data cleansing was performed to remove unusable records. As of January 2020, the final version of the database (after cleaning) was comprised of survey responses from occupants in 897 buildings from across the US, as well as from Australia, Canada, China, Italy, India, Japan, Mexico, Singapore, United Arab Emirates (UAE), and the UK. Of these buildings, there are 692 offices (77%), 109 K-12 education spaces (12%), 38 laboratories (4%), 30 healthcare workspaces (3%), 18 higher education spaces (2%), and 10 multi-family residential/dormitories (1%). Buildings with incomplete (i.e. ‘Other’) or missing metadata were excluded from the analysis. The clean database contained a total of 93,662 participant responses, with 34,630 self-reporting as male (37%), 48,371 as female (52%), 29 identifying as non-binary (0.0003%), and 10,581 not reporting their gender (11%). Of the survey participants, 14,727 are 30 years of age or younger (16%), 29,278 are between the ages of 31 and 50 years (31%), 15,566 are over the age of 50 years (17%), and 33,858 did not report their age (36%). Table 1 summarises the time occupants spent in their building and the space type that best described their workspace.
|OFFICE||HEALTHCARE||K-12 EDUCATION||HIGHER EDUCATION||LABORATORY||MULTI-UNIT RESIDENTIAL||TOTAL (%)|
|Time in the building||<1 year||14,052||1,302||921||208||213||8||17.9%|
|Time at one’s personal workspace||<3 months||8,553||24||144||43||61||3||9.4%|
|Time in the workspace per week||<10 hours||4,134||41||225||123||76||2||5.0%|
|Type of workspace||Private||17,545||71||586||442||269||10||20.3%|
The CBE Occupant Survey is comprised of 10 sections, each targeting different aspects of an occupant’s space. The first section addresses occupant demographics (e.g. age, gender, type of work) and general experiences with their environment (e.g. how many hours per week they work, how long they have resided in their workspace). Additionally, occupants identify their location in the building, along with the type of workspace (e.g. private office, shared office, low/high cubicle, etc.). Following this is a sequence of sections targeting occupant perceptions of the indoor environment, including satisfaction with office layout, furnishings, thermal comfort, indoor air quality, lighting, acoustics, cleanliness, and maintenance (Figure 1).
When an occupant indicates some degree of dissatisfaction within these questions, they are presented a set of branching questions to gain detailed and specific feedback on the challenges of the space. These branching questions are comprised of checkbox and open-ended response types that aim to understand the reasons of dissatisfaction. Next, occupants see a set of customised, building-specific questions that often relate to aspects of interest (e.g. conference technologies, wayfinding, elevators, etc.). Lastly, occupants are asked to report on their satisfaction with the overall building, satisfaction with their personal workspace, and whether or not they believe the environment overall increases or decreases their productivity. Overall building and workplace satisfaction are measured on a seven-point Likert scale (1 = very satisfied, 7 = very dissatisfied) and productivity is measured on a seven-point interval scale (1 = increased by 20%, 7 = decreased by 20%). The general survey structure is shown in Figure 2.
The structure described is typical of the version of the survey designed for commercial offices. Versions of the survey for other building types (healthcare, higher education, K-12 education, laboratory, multi-unit residential/dormitory) follow the same order, but include several other questions specific to that environment typology. In cases where additional modules (e.g. questions focused on daylighting or wayfinding) are required, those questions are placed within the structure in such a way to reduce the cognitive fatigue of the respondents. For example, the daylighting module would be placed immediately after the lighting section, and the wayfinding module may be placed before or after the layout section.
Percentages of occupant responses were calculated for each item under investigation to better understand occupant perceptions. Specifically, percentages were calculated for each of the 16 satisfaction questions and the nine questions examining occupant self-reported productivity (i.e. whether IEQ aspects enhance or interfere with one’s ability to complete his or her job).
Three different techniques were used to examine the underlying structure of the survey’s measurements beyond the theoretical structure imposed at its conception in 1999. The first was simple correlation analysis based on Pearson’s r. The values suggested by Ferguson (2009) were used to determine and highlight moderate (0.5–0.8) and strong (> 0.8) correlations. The second technique, principal component analysis (PCA), is an unsupervised learning approach to determine a low-dimensional representation of the observations that can explain the largest possible fraction of the variance. The third technique, hierarchical clustering analysis (HCA), looks to find homogeneous subgroups among the observations (Gareth et al. 2013). It does not require the preselection of the number of clusters, and can be visualised in an easy-to-interpret tree diagram known as a dendrogram. An advantage of this technique is that a single dendrogram can be used to represent visually any number of clusters (Gareth et al. 2013). Both PCA and HCA analyses were performed only on items relating to occupant satisfaction (16 questions). An additional question on perceived productivity due to the overall environment was dropped from this analysis because it used a different scale that is conceptually different than the other items. Dissatisfaction (branching) questions—checkboxes used to diagnose problem areas—were dropped due to their structure (different than the Likert scales used in other questions) and because they are only shown to participants with some level of dissatisfaction. Open-ended text questions were also excluded from the analysis. For PCA, varimax rotation was used on normalised data. For HCA, Euclidean distance was used for the measure of dissimilarity. For all analyses, results were considered statistically significant when p < 0.05.
Using the results from the Pearson correlations, PCA, and HCA, benchmark scores of the CBE Occupancy Survey were computed as weighted averages across multiple items, which in most instances are two or three satisfaction questions. For reference, the benchmark scores were compared across the whole database as well as by building type. Because multi-unit residential/dormitory spaces only comprise 1% of the database and are objectively different space typologies that do not focus on occupant work, these buildings were excluded from the analysis.
The following software were used: R (R Core Team 2019; v.3.5.0) and RStudio IDE (RStudio Team 2019, v.1.2.5019), along with the following packages: tidyverse (Wickham et al. 2019), data.table (Dowle & Srinivasan 2019), here (Müller 2017), corrplot (Wei & Simko 2018), and dendextend (Galili et al. 2015). The ‘psych’ package (Revelle 2019) and the ‘dist’ function (base R) were used for the PCA and HCA, respectively.
The breakdown of satisfaction votes for the full database across 16 items is shown in Figure 3. Sound privacy (54% dissatisfied), temperature (39%), and noise level (34%) are the largest sources of dissatisfaction for occupants. Ease of interaction (75% satisfied), amount of light (74%), and general cleanliness (71%) are the three most highly satisfied items within the database.
The CBE Occupant Survey has nine additional questions designed to determine if aspects of the workspace enhance or interfere with occupants’ self-reported productivity. The results in Figure 4 show that acoustical quality has the largest negative impact on self-reported productivity (39% dissatisfied), followed by temperature (35%). Roughly two-fifths of building occupants think acoustical quality and temperature interfere with their ability to get their job done. Almost one-quarter of respondents said that the office layout interfered with their ability to get their job done, which is likely related to perceptions of privacy and amount of space satisfaction. The majority of responses to cleanliness and maintenance, furnishings, and lighting quality items were positive. The neutral votes earned a much larger share for these productivity questions compared with the satisfaction questions, suggesting occupants have less of an opinion about these types of questions or they are harder to answer.
Correlation analysis was first performed on questions measuring occupant satisfaction to better understand the relationship between occupant responses to those items. The results in Figure 5 show the moderate (0.5–0.8) and strong (0.8) correlations between several cognate items. For example, the three questions relating to cleaning and maintenance were all highly correlated. Similarly, satisfaction with the building overall had a strong relationship with satisfaction with occupants’ personal workspace. Other items that reported strong correlations include noise level and sound privacy, furnishing comfort and adjustability, and the amount of light and visual comfort. Furthermore, the results show that enhance/interfere questions all moderately or strongly correlate with their companion satisfaction questions; this indicates homogeneity on what these items are measuring. For this reason, all enhance/interfere questions were dropped from the remaining analyses of the underlying structure and instead the focus was solely on satisfaction votes.
Before performing the PCA and HCA, any incomplete cases were removed from the database. Due to partial responses and slight changes in questions over the 20-year lifespan of the survey, this reduced the data set for analysis from 90,529 to 51,625. Three, seven, eight, and 10 PCA factor solutions were examined for the 16 satisfaction questions, which accounted for 62%, 82%, 85%, and 91% of the total variance, respectively. Multiple criteria were applied to determine the appropriate number of factors to maintain in the PCA, seeking convergence across three methods: scree plot (Cattell 1966), Kaiser rule (eigenvalues of ≥ 1), and the interpretability of the solutions (Zwick & Velicer 1986). After considering all criteria, we proposed a seven-factor solution as best explaining the underlying structure of the CBE Occupant Survey. Table 2 shows the rotated factor loadings for the seven-factor solution; loadings > 0.5 are grouped. The items loading most strongly on the first factor are temperature and air quality. Taken together, these items relate to occupant perceptions of the air, and therefore this factor was named ‘Air’. The factors were all named following this logic, and these thematic mappings are reported as table headers. The PCA results are generally logical and follow the theoretical structure of the survey closely. Although the two global assessments of the environment—building overall and personal workspace—loaded most heavily into the cleanliness and layout themes, they have the lowest loadings of any item across the analysis and tend to load in all the items.
|Amount of space||0.13||0.15||0.85||0.13||0.12||0.20||0.11|
|Ease of interaction||0.10||0.14||0.24||0.90||0.15||0.15||0.16|
|Amount of light||0.12||0.16||0.15||0.10||0.88||0.14||0.12|
|Adjust your furniture||0.14||0.17||0.15||0.08||0.15||0.88||0.13|
While PCA is an exploration of how individual items are interrelated, HCA was used to examine how these items fit together to better understand the overall survey structure. The HCA results using the same data set as the PCA are shown in the dendrogram in Figure 6, where the y-axis displays branches depicting the clustering of items, and the x-axis displays the dissimilarity (or distance) between each item and cluster. The more similar items are to one another, the shorter the distance between the nodes of their branches. Items under nodes that split later in the dendrogram (a lower Euclidian distance) are more similar to each other, and items under nodes that split earlier in the dendrogram (a higher Euclidian distance) are more dissimilar (Gareth et al. 2013).
The results of the PCA were then used to inform the decision on the number of clusters to highlight (seven), and each cluster is color-coded in the dendrogram to aid interpretation. The seven branches (or clusters) in Figure 6 focus on satisfaction with different components of occupants’ workspace (in descending order): air, cleanliness, layout, privacy, lighting, furniture, and acoustics. The themes of the clusters show a logical, strong alignment with the results of the PCA reported in Table 2. The most notable difference between the results of the PCA is that the HCA grouped satisfaction with building and personal workspace together with the ease of interaction and amount of space, while excluding visual privacy.
Using the thematic mapping of the PCA, the clustering reported in Figure 6 suggests that satisfaction with the overall workspace/building is strongly related to the layout of the space. This result was found by Frontczak et al. (2012) in their previous analysis of the CBE Occupant Survey database, but using a different analytical technique. Another way to interpret this finding is that these items are reflective of the broader aesthetic and environment the space affords for daily working and living. Interestingly, visual privacy is distinct from other layout-related questions. This suggests that perhaps a feeling of privacy differs from more activity-related behavior such as interacting with others or having enough space to complete tasks.
Historically, each building in the CBE database is given a set of scores based on responses to some of the satisfaction questions. The average responses for 14 satisfaction items are collapsed into nine categories to benchmark buildings against the entire database (hereafter referred to as ‘pre-2020’ benchmark scores). The analysis of the survey structure suggested that many of these satisfaction items were correlated. Therefore, we recalculated new benchmark scores based on the thematic mapping of the HCA to determine the impact of the underlying structure of the survey on the building benchmarks. We made two important changes to what was suggested by the HCA results: temperature and air quality were kept as different categories, and personal workspace satisfaction was separated from layout. For the remainder of the paper, this new clustering approach is used (referred to as ‘post-2020’ benchmark scores).
Figure 7 shows the benchmark scores for the CBE database using both the pre-2020 (14 items; cleaning service and amount of space were not included) benchmark categories and the post-2020 (15 items; building overall was not included) benchmark categories based on the PCA and HCA analyses. Acoustics, temperature, and privacy had the lowest benchmark scores, with the median scores for all other items above slightly satisfied (5). The most notable difference between the two weighting methods is seen within the theme of layout; the replacement of visual privacy (one of the lowest ranking items) with amount of space led to a higher score for that category. Subsequently, visual privacy was also moved to represent its own category and the ‘building’ category is dropped.
Finally, there is a question of whether or not benchmark scores differ across building types. An exploratory analysis was performed by calculating scores using the clustered weighting by building type. This preliminary analysis suggests that there were differences in the scores between building types (see Appendix A), but these differences may not be large enough to warrant unique POE surveys for each typology. Given the small sample sizes of some building type in this database, these results should be viewed as exploratory.
There have been significant advancements in building science, psychometrics, digital survey methods, and the understanding of occupant psychology and perception of IEQ since the conception of the CBE Occupant Survey. The impact of these advances can be seen in the increased focus on spaces that promote occupant health, wellbeing, and improved productivity. With these changes in mind, the following sections use findings from the analysis of the CBE Occupant Survey to reflect on the wider practice of POE, and to offer recommendations to improve occupant surveys in response to the shifting requirements of contemporary workspaces.
Correlational analysis, PCA, and HCA were employed to assess the underlying structure of the tool and the relationship between items. At a high level, the survey appears to successfully measure occupant satisfaction. This analysis shows that these items examine themes of air, cleanliness, layout, visual privacy, lighting, furniture, and acoustics. It is unsurprising that items fell neatly into their corresponding IEQ factors given the fact that the survey was designed around these themes. There is evidence that the theoretical framework, or even the sequencing of items, affects responses (Lavrakas 2008). That was unable to be tested herein, but it is one possible explanation for the results of the clusters reported in the PCA and HCA. This would apply to all POE surveys, and thought should be given to question order to avoid either thematic linkages or priming effects that could promote bias responses.
These analyses revealed a case for measurement reduction in some items of the CBE Occupant Survey. In particular, high correlation between questions asking whether certain aspects of IEQ enhance or interfere with occupants’ ability to complete their job and their respective satisfaction questions suggest these items measure similar things. The authors believe one item assessing whether the occupant feels the space overall enhances/interferes with their ability to complete their work should replace redundant questions for specific IEQ factors. This could be accompanied by an open-ended question about how occupants feel the space could be changed to improve their ability to accomplish tasks. Doing so would result in fewer questions, but arguably greater insights that better address the shift in industry interest.
It is common practice that scales consist of multiple items to assess the same variable. The idea is that having multiple items to assess a variable (i.e. a multi- compared with a single-item scale) increases a survey tool’s reliability (Wanous et al. 1997). However, it is also thought that in instances where the psychological construct being measured is clear and specific, only a single item may be needed to fully measure a construct (Sackett & Larson 1990). Future work should empirically examine more deeply if single- or multi-item measures for the assessment of environmental quality perceptions are most appropriate. Specifically, as a field, researchers should explore if attitudes (e.g. satisfaction versus perceived impact to work) of environmental factors are equally psychologically complex to discern and require the same scale length. Further, there is a need to conduct measurement research to better explore if all IEQ factors (e.g. thermal comfort versus cleanliness) are perceived with the same level of psychological complexity. The correlation analysis suggests that multiple items around an IEQ theme are not always necessary to address particular constructs when the psychological complexity of the variable is likely low. For example, multiple questions on cleanliness in the CBE Occupant survey could be replaced with one question without losing information.
Many of the questions in the CBE Occupant Survey use a seven-point Likert scale on the assumption it captures all possible levels of variance while maintaining interpretability. However, there may be reason to presume a five-point scale would suffice for some themes (Vischer 1989; Dawes 2008). When considering measures, the most appropriate scale length is often variable specific and requires empirical testing to rule on which scale is most appropriate (Devlin et al. 1993). With this mind, there is a need for research testing whether a five-point scale would produce similar results as those from a seven-point scale when assessing environmental satisfaction.
Benchmarking buildings using survey responses helps to determine the ‘average’ or ‘typical’ space and can be used as a point of reference to better understand the performance of specific buildings (Vischer 1989). These benchmark scores are used to compare buildings with others, track the success of a space over time, and help diagnose strengths and weaknesses in building operation. In the case of the CBE Occupant Survey benchmark, changes to the benchmark calculations herein are proposed based on the underlying structure of the survey that emerged from both the PCA (Table 2) and HCA (Figure 7). Collapsing individual items into the thematic categories proposed in Figure 7 seems appropriate, with the exception of thermal comfort and air quality. Though research shows that these are perceptually connected (Fang et al. 1998, 2004; Melikov & Kaczmarczyk 2012; Schiavon et al. 2017; Zhang et al. 2011), such that occupants conflate air temperature, air speed, and humidity with the freshness and ‘feel’ of the indoor air, these should remain distinct entries in any benchmarking system purely for diagnostic purposes. It is important to note that any changes to benchmarking systems may have flow-on effects for efforts to summarise IEQ measures into a single summative evaluation (Heinzerling et al. 2013) or to determine the weighting factors of individual items on overall satisfaction (Kim & de Dear 2012).
Many assessments of the performance of an indoor space focus on determining whether they meet a minimum standard. For example, ASHRAE Standard 55 (2017) often considers ‘slightly dissatisfied’ responses as falling within the ‘acceptable’ range. While it is important to understand whether or not a space is satisfactory or acceptable, if both the supply and the demand sides are striving for spaces to do more, then it is appropriate to assess if they are meeting these aspirations. Just as when diagnosing issues, lessons can be learned from occupants’ insights around successful elements of a design. Instead of simply asking occupants to focus on factors that do not work within the space, researchers should also be asking them what does work and why.
Spaces matter to the extent to which they support the occupants within them. Questions that address occupant expectations and needs from their environment are essential. For instance, does an occupant expect to socialise or be solitary in a conference room versus their personal workspace? It has been shown that there is a relationship between the environment and the cognitive and emotional state an occupant expects to experience in a specific space to help them complete their desired activities (Graham et al. 2015). The ambiance or environmental features needed to focus are likely different from those needed to evoke a personal connection with others. This is important in the context of office buildings. Asking questions such as ‘How would you like to feel when working in your office?’ or ‘Do you feel creative when you are working in this space?’ helps researchers (and ultimately designers, facility managers, and others) to understand the ways space impacts emotions and supports the ability of occupants to achieve their goals. These questions should be asked at the individual occupant level as well as at an organisational level, when relevant. By combining variables relevant to the individual (e.g. employee status, job commitment, sensitivity, personality) or organisation (e.g. valuing transparency or collaboration) with occupants’ perceptions of space, researchers can more effectively determine the drivers of occupants’ environmental satisfaction.
Using surveys to understand the goals of individuals and organisations can inform design practices that help support the expectations of occupants and the requirements of a space. When developing new questions to address these things, survey designers should incorporate established and validated measures where possible. Both social and health sciences have developed many measures for relevant variables such as stress (e.g. Perceived Stress Scale; Cohen et al. 1983), wellbeing (e.g. Satisfaction With Life Scale; Diener et al. 1985), and personality (e.g. Big Five Inventory; John & Srivistava 1999). Though metrics such as these have rarely been applied to building science, they have been rigorously tested in those fields to maximise measurement integrity. Further cross-pollination between these disciplines would enhance the breadth and depth of POEs in response to recent interest in occupant health and wellbeing.
As indoor spaces become increasing flexible through design, technological advances are making the personalisation or control of aspects of a space more accessible and scalable. Deploying POEs in spaces in which occupants are transient or self-select locations—such as agile workspaces—is much more complex than traditional offices where people are more stationary. Increasing survey frequency may be useful in spaces where occupants move around more freely, but this strategy increases the potential for survey fatigue. Generating shorter, experience-focused surveys known as ecological momentary assessments (EMAs) (Shiffman et al. 2008) through smartphones or wearables could help mitigate this potential issue. These could be paired with unobtrusive measures of occupant behaviors and environmental parameters using strategically placed sensors. While self-reports offer insight into an occupant’s psyche and experiences, unobtrusive measures reduce the risk of personal bias by monitoring how spaces are truly used rather than relying on how occupants recall or believe spaces are used. Even more, the advantage of combining these measures into a mixed-method approach to measurement (e.g. self-reports and unobtrusive measures) is a more holistic view of the occupant–building relationship.
The purpose of a POE is two-fold: to assess a building’s performance and to provide insight into how to improve or maintain the successes of a space. Far too often POEs fall short in accomplishing the latter, and further efforts towards implementing research-led design when possible should be bolstered. Those investing the time and resources into conducting POEs need to work harder to close the gap between assessment and subsequent communication of the findings to building owners, operators, and the design team in an actionable and digestible way. Just as importantly, findings from POEs need to be fed back to the occupants themselves. Educational resources should also be given to occupants to help empower and nudge their behaviors towards using their spaces as effectively as possible. By incorporating investors and occupants in the conversations around design and the change management process following a survey, all parties involved can be more easily set for success.
This critical assessment of the CBE Occupant Survey tool and database provided many new insights. Appendix B outlines some of the changes CBE will be implementing to the tool and the way in which POEs will be supported in the future.
While the costs associated with POEs have reduced over the years, they are typically done in high-end office buildings with available resources to heavily optimise workspaces and pursue certification. The CBE Occupant Survey is no exception, with most surveys performed in Class A office buildings in North America. This bias widens the equity gap in healthy and comfortable buildings for everyone, while limiting our knowledge of how buildings perform. As a research discipline, there should be efforts to measure more diverse populations across a wider range of contexts. One example is to include classifying occupant characteristics (e.g. age, gender, ethnicity, education, experiences relevant to target variables) to better contextualise the survey results and respondents. This will help to identify the gaps in findings and improve the representation and generalisability of data sets and ultimately improve the design requirements for individuals across different populations. A similar push is needed in rating systems (e.g. LEED, WELL, FitWel, Living Building Challenge, GreenMark, BREEAM) to create programs that are accessible to all buildings, not just those who can afford the certification process.
Occupant surveys continue to be a useful method for evaluating the performance of spaces, and much has been learned around how best to conduct these assessments over the last two decades. This paper has provided an updated summary of the size and characteristics of the responses database, and the distributions of occupant satisfaction and self-reported productivity. Roughly two-thirds of respondents are satisfied with their workspace. Occupants are most satisfied with spaces’ ease of interaction, amount of light, and space cleanliness; they are most dissatisfied with sound privacy, temperature, and noise level. Roughly two-fifths of building occupants felt the acoustic quality and thermal comfort interfered with their ability to complete their work, and one-quarter of respondents indicated the office layout was creating interference. Examining the structure of the survey tool using three statistical methods yielded seven clusters or themes of measurement: air, cleanliness, layout, privacy, lighting, furniture, and acoustics. However, new benchmark metrics are proposed based on nine measurement categories (i.e. the seven themes mentioned above with the addition of ‘personal workspace’ and splitting ‘temperature’ and ‘air quality’). Using the findings from this study, it is suggested that single-item measures are used in cases where variables lack complexity (e.g. satisfaction with cleanliness). The use of open-ended questions is encouraged when looking for deeper insights or those more idiosyncratic to a particular space. The inclusion of other questions is suggested that ask occupants about what is positive about a space, in addition to that which is problematic. Further, the inclusion of items should be considered that explore factors beyond satisfaction, such as occupant design preferences, whether or not a space supports the desired activities, occupant emotions, wellbeing, health, and occupant needs. Finally, researchers and the users of research are encouraged to think creatively about how best to measure occupants as space design and technology expands.
The authors express gratitude to (alphabetically): Edward Arens, Fred Bauman, Gail Brager, Toby Cheung, Charlie Huizenga, Michael Kent, David Lehrer, Paul Raftery, and Hui Zhang for their thoughtful feedback and suggestions on this project. Also, they thank the Berkeley Research Impact Initiative (BRII), sponsored by the University of California, Berkeley Library, whose support made publication of this work possible. Finally, the authors offer an immense gratitude to all the Center for the Built Environment’s collaborators, partner members, and survey users for the continued support and use of the CBE Occupant Survey.
All authors contributed greatly in the intellectual development of the concepts throughout this paper. T.P. and L.T.G. performed the data selection, while T.P. led data analysis implementation, with collaboration from L.T.G. and S.S. in interpretation. L.T.G. led the generation of the original draft paper, T.P. led the generation of the visualizations throughout the paper, and all authors collaborated heavily in the final writing and curation of the manuscript.
The Center for the Built Environment (CBE) at the University of California, Berkeley, with which the authors are affiliated, is advised, and funded in part, by many partners that represent a diversity of organizations from the building industry, including manufacturers, building owners, facility managers, contractors, architects, engineers, government agencies, and utilities.
These data is not available for open access in order to comply with the approved IRB protocol approved for this study.
This work was approved by the IRB at the University of California, Berkeley (grant number IRB-2010-05-1550).
This research was supported by the Center for the Built Environment (CBE) at the University of California—Berkeley and the Republic of Singapore’s National Research Foundation Singapore through a grant to the Berkeley Education Alliance for Research in Singapore (BEARS) for the Singapore–Berkeley Building Efficiency and Sustainability in the Tropics (SinBerBEST) Program. BEARS was established by the University of California, Berkeley as a center for intellectual excellence in research and education in Singapore.
Altomonte, S., & Schaivon, S. (2013). Occupant satisfaction in LEED and non-LEED certified buildings. Building and Environment, 68, 66–76. DOI: https://doi.org/10.1016/j.buildenv.2013.06.008
Altomonte, S., Schiavon, S., Kent, M. G., & Brager, G. (2019). Indoor environmental quality and occupant satisfaction in green-certified buildings. Building Research & Information, 47(3), 255–274. DOI: https://doi.org/10.1080/09613218.2018.1383715
Brager, G., & Baker, L. (2009). Occupant satisfaction in mixed-mode buildings. Building Research & Information, 37(4), 369–380. DOI: https://doi.org/10.1080/09613210902899785
Candido, C., Kim, J., de Dear, R., & Thomas, L. (2016). BOSSA: A multidimensional post-occupancy evaluation tool. Building Research & Information, 44(2), 214–228. DOI: https://doi.org/10.1080/09613218.2015.1072298
Cattell, R. B. (1966). The scree test for the number of factors. Sociological Methods and Research, 1, 245–276. DOI: https://doi.org/10.1207/s15327906mbr0102_10
Cohen, R., Standeven, M., Bordass, B., & Leaman, A. (2010). Assessing building performance in use 1: The Probe process. Building Research & Information, 29(2), 85–102. DOI: https://doi.org/10.1080/09613210010008018
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396. DOI: https://doi.org/10.2307/2136404
Dawes, J. (2008). Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50(1), 61–104. DOI: https://doi.org/10.1177/147078530805000106
Devlin, S. J., Dong, H. K., & Brown, M. (1993). Selecting a scale for measuring quality. Marketing Research, 5(3), 12–17. https://web.a.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=f837d25e-4cd6-4fb3-9cc3-d23975619a69%40sessionmgr4007
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with Life Scale. Journal of Personality Assessment, 49, 71–75. DOI: https://doi.org/10.1207/s15327752jpa4901_13
Dowle, M., & Srinivasan, A. (2019). data.table: Extension of ‘data.frame’ (Version 1.12.8). https://CRAN.R-project.org/package=data.table
Duarte, R. C., Schiavon, S., & Parkinson, T. (2020). Targeted occupant surveys: A novel method to effectively relate occupant feedback with environmental conditions. Building and Environment, 184(15), 107–129. DOI: https://doi.org/10.1016/j.buildenv.2020.107129
Fang, L., Clausen, G., & Fanger, P. O. (1998). Impact of temperature and humidity on the perception of indoor air quality. Indoor Air, 8(2), 80–90. DOI: https://doi.org/10.1111/j.1600-0668.1998.t01-2-00003.x
Fang, L., Wyon, D. P., Clausen, G., & Fanger, P. O. (2004). Impact of indoor air temperature and humidity in an office on perceived air quality, SBS symptoms and performance. Indoor Air, 14(s7), 74–81. DOI: https://doi.org/10.1111/j.1600-0668.2004.00276.x
Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532–538. DOI: https://doi.org/10.1037/a0015808
Frontczak, M., Schiavon, S., Goins, J., Arens, E. A., Zhang, H., & Wargocki, P. (2012). Quantitative relationships between occupant satisfaction and satisfaction aspects of indoor environmental quality and building design. Indoor Air, 22(2), 119–131. DOI: https://doi.org/10.1111/j.1600-0668.2011.00745.x
Galili, T. (2015). dendextend: An R package for visualizing, adjusting, and comparing trees of hierarchical clustering. Bioinformatics, 31(22), 3718–20. DOI: https://doi.org/10.1093/bioinformatics/btv428
Graham, L. T., Gosling, S. D., & Travis, C. K. (2015). The psychology of home environments: A call for research on residential space. Perspectives on Psychological Science, 10(8), 346–356. DOI: https://doi.org/10.1177/1745691615576761
Heinzerling, D., Schiavon, S., Webster, T., & Arens, E. (2013). Indoor environmental quality assessment models: A literature review and a proposed weighting and classification scheme. Building and Environment, 70, 210–222. DOI: https://doi.org/10.1016/j.buildenv.2013.08.027
John, O., & Srivastava, S. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and Research, 2, 102–138. Guilford.
Karmann, C., Schiavon, S., Graham, L. T., Raftery, P., & Bauman, F. (2017). Comparing temperature and acoustic satisfaction in 60 radiant and all-air buildings. Building and Environment, 126, 431–441. DOI: https://doi.org/10.1016/j.buildenv.2017.10.024
Kim, J., & de Dear, R. (2012). Nonlinear relationships between individual IEQ factors and overall workspace satisfaction. Building and Environment, 49, 33–40. DOI: https://doi.org/10.1016/j.buildenv.2011.09.022
Lavrakas, P. J. (Ed.). (2008). Encyclopedia of survey research methods. Sage. DOI: https://doi.org/10.4135/9781412963947
Leaman, A., & Bordass, B. (2001). Assessing building performance in use 4: The Probe occupant surveys and their implications. Building Research & Information, 29(2), 129–143. DOI: https://doi.org/10.1080/09613210010008045
Li, P., Froese, T. M., & Brager, G. (2018). Post-occupancy evaluation: State-of-the-art analysis and state-of-the-practice review. Building and Environment, 133, 187–202. DOI: https://doi.org/10.1016/j.buildenv.2018.02.024
Melikov, A. K., & Kaczmarczyk, J. (2012). Air movement and perceived air quality. Building and Environment, 47, 400–409. DOI: https://doi.org/10.1016/j.buildenv.2011.06.017
Müller, K. (2017). here: A simpler way to find your files. R package (Version 0.1). https://CRAN.R-project.org/package=here
Peretti, C., & Schiavon, S. (2011). Indoor environmental quality surveys. A brief literature review. In Proceedings for Indoor Air 2011, Austin, TX, US. https://escholarship.org/uc/item/0wb1v0ss
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
Revelle, W. (2019). psych: Procedures for Personality and Psychological Research (Version 1.9.12). Northwestern University. https://CRAN.R-project.org/package=psych
Roulet, C.-A., Johner, N., Foradini, F., Bluyssen, P., Cox, C., De Oliveira Fernandes, E., Müller, B., & Aizlewood, C. (2006). Perceived health and comfort in relation to energy use and building characteristics. Building Research & Information, 34(5), 467–474. DOI: https://doi.org/10.1080/09613210600822279
RStudio Team. (2019). RStudio: Integrated development for R (Version 1.2.5033). RStudio. http://www.rstudio.com/
Sackett, P. R., & Larson, J. R., Jr. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology, 2nd ed., 1, 419–489. Consulting Psychologists Press.
Schiavon, S., Yang, B., Donner, Y., Chang, V. W.-C., & Nazaroff, W. W. (2017). Thermal comfort, perceived air quality, and cognitive performance when personally controlled air movement is used by tropically acclimatized persons. Indoor Air, 27(3), 690–702. DOI: https://doi.org/10.1111/ina.12352
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4(1), 1–32. DOI: https://doi.org/10.1146/annurev.clinpsy.3.022806.091415
Wanous, J. P., Reicbers, A. E., & Hudy, M. J. (1997). Overall job satisfaction: How good are single-item measures? Journal of Applied Psychology, 82(2), 247–252. DOI: https://doi.org/10.1037/0021-9010.82.2.247
Wei, T., & Simko, V. (2018). R package ‘corrplot’: Visualization of a correlation matrix (Version 0.85). https://github.com/taiyun/corrplot
Wickham, H., et al. (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. DOI: https://doi.org/10.21105/joss.01686
Zagreus, L., Huizenga, C., Arens, E., & Lehrer, D. (2004). Listening to the occupants: A web-based indoor environmental quality survey. Indoor Air, 14(8), 65–74. DOI: https://doi.org/10.1111/j.1600-0668.2004.00301.x
Zhang, H., Arens, E., & Pasut, W. (2011). Air temperature thresholds for indoor comfort and perceived air quality. Building Research & Information, 39(2), 134–144. DOI: https://doi.org/10.1080/09613218.2011.552703
Zimmerman, A., & Martin, M. (2001). Post-occupancy evaluation: Benefits and barriers. Building Research & Information, 29(2), 168–174. DOI: https://doi.org/10.1080/09613210010016857
Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432–442. DOI: https://doi.org/10.1037/0033-2909.99.3.432
Following this analysis, and as a first step towards future measurement practices, a series of changes to the survey tool will be implemented:
In tandem with these changes, the authors are investigating novel approaches to pairing post-occupancy evaluation (POE) surveys with other data types, including unobtrusive measures (such as those collected from sensors) and real-time, in-the-moment occupant feedback (Duarte et al. 2020). The reporting tools have also been improved in order to deliver accessible and meaningful findings to designers, building owners, and occupants. In doing so, it is hoped practitioners will be equipped with the knowledge necessary to craft healthy and supportive spaces and to empower occupants to maximise the ways they use their spaces.