Amplifying weak signals: a method-building approach for inclusive climate resilience strategy making

Socio-ecological inclusion and the impacts of climate change on the built environment are two shared concerns central to the design and planning of the just transition in cities. The just transition leans heavily on inclusive convergence processes that are grounded in knowledge integration and transdisciplinary practice. However, there is a paucity of effective methods for the inclusion of so-called weak signals from actors situated at the periphery of these convergence processes. Building on the concept of structured flexibility, we introduce a building-blocks approach as a modular architecture for constructing methods for distributed engagement and knowledge integration beyond conventional small-group settings. By engaging in research through design, the Amplifying Weak Signals approach was prototyped with students in the context of dealing with heatwaves in The Hague and tested with expert users from the region who facilitate resilience strategy-making processes. Out of 900 possible building block combinations, 18 methods were created during prototyping. The resulting heatwave strategies that were drafted based on the collected peripheral knowledge showed the integration of new socio-ecological issues rather than a drastic departure from the baseline resilience strategy of the city. We discuss the research findings and their use in the production of guidelines for the construction of methods to integrate peripheral knowledge in convergence processes. Ongoing work to develop the guidelines in the form of an open-access, interactive repository of knowledge elicitation methods for urban resilience spatial planning will also be described. Recommendations for scaling the approach are provided.

1 Introduction

Delta cities and regions sit at the frontline of climate change. As temperatures and sea levels rise, increased flooding, heatwaves, critical infrastructure failures and biodiversity loss top the long list of risks threatening these urban spaces. Exposure to climate-related risks is unevenly distributed among city dwellers, both human and non-human. Dealing with the impacts of climate change and doing so in a socio-ecologically inclusive manner have become concerns shared by a broad range of urban actors. But formulating strategies for climate resilience based on such shared concerns is difficult to orchestrate because of the diffuse and often conflicting nature of knowledge. Shared concerns are viewed in participatory design literature as controversial topics of interest that can mobilize citizen participation (Bjögvinsson et al., 2012; Slingerland et al., 2020). Integrating knowledge from citizens and other actors who are peripheral to urban resilience conversations can be particularly challenging.

Knowledge integration entails the development and use of different knowledge types that go hand-in-hand and influence each other directly and interactively (Rotmans, 2006). Various scholars point to the increasing uptake of knowledge integration efforts across scientific fields (Scherhaufer, 2014; Bammer et al., 2020; Daniels et al., 2020). Participation in democratized knowledge integration processes is typically implemented as a narrow, strictly delineated part of the research process (Scherhaufer, 2021). Recently, calls for more coherent, holistic knowledge integration practices have emerged (Daniels et al., 2020; Dannevig et al., 2020; Scherhaufer, 2021). One justification for the broadening of knowledge integration practices is that non-academic stakeholders need actionable research to adapt to and mitigate the effects of climate change. However, creating knowledge that is useful in addressing real-world problems can remain a challenge in academic research (Scherhaufer, 2014). To integrate science with practice, research projects attempt to include non-scientific stakeholders in their knowledge convergence processes (Scherhaufer, 2021).

To guide the convergence process, a variety of frameworks and structures have been proposed. However, many of these structures remain sector-, location-, or context-specific and neglect the involvement of citizens (Daniels et al., 2020; Wamsler et al., 2020). Regardless of the implementation of integration frameworks, the inclusion of non-scientific knowledge into the research process faces another challenge. Stakeholders have a great variety in ways-of-life which affects their worldview, attitude toward learning and extent and degree of participation (Dannevig et al., 2020). Knowledge integration, while urgently needed, risks alienating actors and knowledge that are less conducive to convergence. Moreover, these emerging transdisciplinary practices are difficult to manage, particularly in their early stages. These stages are marked by haphazard actions to identify actors, knowledge needs and appropriate instruments to support knowledge integration around a shared concern. The few examples that exist tend to be oriented toward supporting convergence processes in workshop settings but are difficult to scale.

The research presented in this paper was prompted by the Resilient Delta Initiative's methodology pilot workshops organized in the fall of 2021 with researchers from three Dutch partnering universities. To complement the shared concerns or “strong signals" that emerged from the workshops, we focus on weak signals. Weak signals give early information about future strategic surprises or discontinuities for strategy making (Holopainen and Toivonen, 2012). Investigating weak signals provides a basis for developing alternative pathways that can lead to broadly-informed strategies for dealing with societal challenges. Weak signals are nested in diverse knowledge networks surrounding societal challenges but have not been picked up in the convergence process [see also contingency discussion in Champlin et al. (2018)]. This may be because the knowledge needed to take action on shared concerns is insufficiently concrete, prone to misinterpretation, difficult to integrate with other knowledge or has been overlooked. There is a paucity of effective methods for the inclusion of weak signals from actors situated at the periphery of convergence processes but whose knowledge could have significant bearing on the course of the transition. Engaging the knowledge of peripheral actors who are not typically present during convergence exercises requires a redesign of existing planning tools and processes. In particular, process facilitators require means of gaining insights from these peripheral actors without resorting to one-size-fits-all solutions that lack contextual relevance.

To deal with pressing societal challenges, demand-driven approaches are required for developing information technologies and other planning support instruments (Pelzer, 2015; Geertman, 2017; Champlin et al., 2019). Tailoring the tools for knowledge integration to fit the contextual needs of the spatial planning process and challenge(s) at hand requires greater flexibility. This flexibility is often lacking in research, decision-making and the frameworks used for knowledge integration (Wamsler et al., 2020; Scherhaufer, 2021). Planning support science theory applies a process-oriented focus to determine suitable information, knowledge and instruments that support future-oriented planning (Geertman and Stillwell, 2020). The creation of knowledge integration methods that are responsive to the contextual nuances of these planning processes involves introducing design thinking to convergence. The open-ended, exploratory nature of design methods and techniques makes them suitable means for exploring contexts based on process and user requirements. Design approaches combine analytical and creative processes that allow transdisciplinary researchers to experiment, create, and prototype methods before gathering feedback and redesigning (Razzouk and Shute, 2012).

Infrastructuring is a participatory design (PD) approach that blurs the lines between the design of (technical) artifacts and their use in continuous, on-going situated practices (Karasti and Baker, 2004). This PD process has been picked up in information systems design and urban planning discourses. Infrastructuring entails an ever-evolving process of re-aligning socio-technical system elements, adapting to different contexts and reciprocal shaping happening between infrastructure and human behavior (Dreessen et al., 2017; Pfeffer, 2019; Simonsen et al., 2020). It is an ongoing design activity that anticipates its continuation beyond a project or intervention (Bjögvinsson et al., 2012). Emerging perspectives on inclusive infrastructuring embrace alternative modes of knowing not just to imagine alternative futures but also to prototype and co-create the urban future with different actors in various time-space settings (Pfeffer, 2019).

In a similar vein, we introduce toolkitting as a form of knowledge infrastructuring that supports spatial planning beyond the boundaries of technology based on a broad set of methods and instruments. Toolkitting concerns the continuous adaptation and iterative development of planning support methods and instruments as an integral, open-ended activity that co-evolves with the planning process it supports. It engages three interrelated components in a continuous (re-)design process: the actor, the knowledge type and the means or instrument for supporting a particular planning stage (Figure 1). This redesign process can be made more accessible and engaging through gamification and the division of planning support instruments into subparts, or building blocks. These principles can be applied at different scales from planning street-level interventions to developing city-wide strategies. In geodesign, the latter form of method design has been described as metaplanning, or the design of the planning process and its support techniques (Campagna, 2016). Metaplanning is the scale of interest in this study.

Figure 1

Figure 1. The three interrelated components of toolkitting are continuously rearranged based on strategy-making process requirements.

The notion of toolkitting introduced in this paper is consistent with a process-oriented view of planning support. It extends the notion of planning support usefulness by looking beyond the fit between the planning task, the supporting technology and the target user (Goodhue and Thompson, 1995; Pelzer et al., 2015; Champlin et al., 2019). This conventional focus on technology is reinforced by the neck-breaking speed of technological advancements in the planning field, seen most recently in the rapid development of urban digital twins and emerging AI applications. In what follows, we extend this definition of usefulness to more-than-technology planning support methods and tools derived from various design disciplines to investigate the task-tool-user fit.

The central aim of this study is to open up knowledge integration processes and their settings to actors located at the periphery of these convergence processes. This study explores how facilitators of convergence processes can be supported in collecting and integrating knowledge from peripheral actors into mainstream urban climate resilience conversations through the application of toolkitting. To be responsive to dynamic multi-actor processes notoriously riddled with uncertainty, toolkitting must strike a balance in its provision of both structure and flexibility, or structured flexibility as introduced by te Brömmelstroet (2012). Structured flexibility poses two design criteria for planning support: (1) frameworking capacities for guiding knowledge integration processes based on the knowledge requirements of different planning stages and (2) openness and adaptability to fit the ever-changing process and contextual needs of a project.

The paper continues by introducing the methodology of the study used to design and evaluate the amplifying weak signals (AWeS) approach for peripheral knowledge integration in a case study. The research through design methodology is applied for its capacity to navigate complexity and accelerate integrative learning through flexible, open-ended exploration (Stappers and Giaccardi, 2017). The outcomes of each design stage are subsequently reported describing the emergent features of the method in terms of their structuring and flexibility characteristics. Finally, reflections are provided on the AWeS approach in terms of its scalability and continued development into an interactive, open-access repository of planning support tools for urban climate resilience strategy-making.

2 Research methods

Following the research through design methodology (Stappers and Giaccardi, 2017), this study engages in an exploration of how structured flexibility can be translated from a design principle into an actionable set of guidelines that support facilitators of knowledge integration processes in finding the right tool for the job at hand. The research methodology consisted of a pre-design, generative and evaluative stage as shown in Figure 2. Each stage resulted in a design outcome that was analyzed in terms of its contribution of structured flexibility in building a knowledge integration method before iteration and testing in the subsequent stage. Several curiosities drove the three design stages.

Figure 2

Figure 2. Stages and outcomes of the research through design methodology applied in the study.

First, we wanted to know what the experts were saying about the need for knowledge integration in transdisciplinary processes and how various design instruments contribute to eliciting knowledge from different actors. Therefore, in the pre-design research stage, we conducted a literature review and interviews with experts to explore the challenges of knowledge integration. The literature review was conducted through contributions from participants in the Master's course Research Challenges at Delft University of Technology. In this course, five students conducted literature reviews on the topic of design methods for knowledge integration. As a starting point, students were provided two seminal articles about transdisciplinary knowledge integration for tackling complex problems (Bammer et al., 2020; Daniels et al., 2020). To fill in gaps in the literature review, we consulted four academic experts who are dealing with the shared concerns we were investigating and who apply design methods in developing planning support instruments.

Second, we wanted to experiment with a large group of students in constructing design methods for knowledge collection using a set of building blocks derived from toolkitting principles. And, we wanted to know how an urban resilience strategy built from the knowledge collected from peripheral actors compared to an urban resilience strategy developed by professionals. Therefore, the generative design stage consisted of two method-building sessions and fieldwork. This experimental component of the design process was organized with 90 students participating in the 2022 Summer School “Planning and Design of the Just City" at Delft University of Technology. The students originated from 36 countries and were studying in a range of Master programs dealing with the built environment. The 2-fold aim of the method-building sessions was: (1) to test the use of the toolkitting building blocks to build a customized method for knowledge collection and apply it in a real-world setting and (2) to experiment with the process of integrating the collected peripheral knowledge into a mainstream convergence process.

Third, we wanted to know how professionals facilitating urban resilience strategy-making processes perceived the AWeS approach prototype. During testing with the target end users in the evaluation stage, we explored the usefulness of the prototype by investigating the question: How do end users experience the approach and in what ways can it support their work as facilitators of convergence processes? Two user testing sessions were conducted with researchers and urban designers who are involved in transdisciplinary convergence projects related to resilience of the built environment.

The next three sections report on the pre-design, generative and evaluative stages of the study in terms of (1) the findings of each stage with particular attention given to indications of needs for structured flexibility in the design intervention and (2) the outcome of the design stage that was used in the advancement of the AWeS approach.

3 Pre-design of the building blocks for constructing a method

The literature review was meant to identify studies reporting on design methods applied in the built environment fields of architecture, urban design, civil and ecological engineering and urban planning and to analyze the methods in terms of their (in)effectiveness in integrating knowledge across disciplines. The scope of the review was set to complex challenges related to resilience in urban delta regions. The selected shared concerns were the impacts of climate change and socio-ecological inclusion. The students were informed that their reviews would contribute to designing a better-informed approach for engaging peripheral actors in a climate resilience strategy-making process. The expert interviews aimed to get insights about experiences working with transdisciplinary projects, challenges in knowledge integration, actors whose knowledge is typically overlooked during the process, and methods and tools (both digital and non-digital) the experts use to overcome this exclusion.

From the expert interviews, we learned that in transdisciplinary learning communities, communication and engagement of the actors are common challenges. First of all, communication between different academic experts faces the challenge of different usage of the same language and terminology. A prime example of this is the term urban resilience, which has many definitions that originate from a range of disciplines (Champlin et al., 2023). Means of measuring urban resilience are even more divergent and dependent not only on the discipline but also on the methods applied in studying a particular resilience challenge. Secondly, there is often a challenge to explain scientific information and its relevance to non-scientific actors. This category of actors often includes citizens who are or will be affected by decisions based on scientific information in the future. Moreover, recently, more attention has been paid to ecological actors and the necessity to consider their needs. To overcome these highlighted challenges, the interviewed experts use various digital and non-digital tools. Combining digital and non-digital tools such as workshops and meetings with numerical models and online questionnaires through mixed-method approaches can provide integrated insights and help to address the mismatches between the information needs of citizens and scientists that often occur. It was noted by the experts that in each project, different tools and methods are required. Therefore, it is important to adapt the actor involvement strategy for each case, its characteristics and context.

3.1 Outcome 1. The building blocks

The literature review and expert interviews revealed a range of knowledge types that could be collected from peripheral actors and various instruments that could support knowledge collection. Still, there remains a lack of structured means to guide the arrangement of these components into situated support methods. To explore how these arrangements could take form, we studied a set of toolkitting building blocks consisting of four peripheral actors, five knowledge types, and nine design instruments.

3.1.1 Peripheral actors

Peripheral actors include (1) academics with deep disciplinary knowledge with an inclination toward interdisciplinary collaboration, (2) citizens who are often underrepresented or excluded from expert convergence processes (Fischer, 2000), (3) future actors for whom knowledge cannot be defined nor can individual actors be delineated, and (4) ecosystem agents and in particular sentient non-human animals (Pham and Saner, 2021). Actor types 3 and 4 often require proxies: experts on sustainability, datasets, representatives of civil society, and even scientists who can make informed statements on what may benefit or burden nature and future generations.

3.1.2 Knowledge types

The term “knowledge” is a conundrum, as its types can be categorized in various ways (Pfeffer, 2019). The categorization used in this study was based on the literature review and deemed best suited by the authors for exploring knowledge from the selected peripheral actors. These knowledge types include (5) experiential knowledge, which is shaped by the daily experiences of residents in terms of how their lives are organized, their perceptions of and behaviors in their physical environment (Kahila and Kyttä, 2009), (6) relational knowledge shaped by the actor's interactions with the urban environment and relational values, (7) situated knowledge about specific local contexts (Fischer, 2000), (8) expert knowledge acquired through scientific research or skill development (Knapp et al., 2019), and (9) intergenerational knowledge passed across generations. These knowledge types can manifest in either tacit or explicit forms. Expert knowledge in particular is more conducive to being shared explicitly, that is, in systematic, codified forms. Tacit knowledge may be internally known but has yet to be systematically expressed (King and McGrath, 2004). It is described as the maker's way of doing or knowing demonstrated by hand (Stappers and Giaccardi, 2017).

3.1.3 Design instruments

The range of knowledge types and forms challenges the notion of a one-size-fits-all means of supporting the sharing of knowledge. Therefore, several design instruments that support knowledge and data collection were explored in this study. These design instruments are:

10. Sketch planning is used to indicate spatial relationships in current or future states. Sketch planning provides groups with a visual means of communication when identifying the key features of a spatial system. It is used by planners and designers to describe key points in their understanding and proposals effectively. Sketch planning is often used in the preliminary stages of planning where actors explore the problem and design a solution through shared representations, achieved as a result of mapping and drawing exercises (Harris, 2001; Vonk and Ligtenberg, 2010).

11. Deconstruct-reconstruct is a method to deconstruct the hidden qualities of a place, and later reconstruct its hidden qualities in a co-design setting. As a design method, it provides a set of questions (e.g., “What is it?”, “What does it mean to you/others?”, “What does it exclude?”) used to identify and describe the elements of a given urban space and to reassemble those elements into an improved physical-functional configuration (Forgaci, 2021).

12. Game Co-Design is a group model-building exercise conducted using a tabletop game prototype. The game serves as an engaging, easy-to-understand third-space environment situated between real-world systems and the model of these systems. Involving stakeholders as co-designers of a game allows them to critique how the real-world system is represented in the components and mechanisms of the game. This allows modelers to learn more about each stakeholder's priorities, interests and preferences in a spatial context (Champlin et al., 2022).

13. Open datasets can be publicly accessed and shared. The use of open data allows transparency and there are no limitations to how it is used, modified, combined, and shared. Open datasets can be used to understand social, economic, and environmental phenomena. It allows transparency in governance and is public evidence about how different projects and policies affect real-world conditions. Open data are rather abstract, so interpretation of insights they provide needs to be done with a critical attitude and often needs to be validated with observation on a subset of cases. Approaches in data science can be merged with design practices to capture insights into complex behavior and drivers of behavior (Bourgeois and Kortuem, 2019).

14. Observations allows us to gather data by simply watching events and behavior or noting down details about the physical setting and the human and non-human agents using it. Observation can be covert or overt (Byrne, 2021). Covert observation means people are unaware that they are under watch and tend to act more naturally. In overt observations, everyone is aware of the process, and this is often required for ethical reasons. Observation allows data collection when respondents are unwilling or unable to communicate verbally. Observation notes can be in the form of text, but also drawing notations on a map or plan (e.g., observations of where people gather in a square).

15. Survey-based questionnaires consist of a set of questions design to collect specific information from a target group of respondents. The wording and the order by which they are asked are consistent. It relies on the ability of respondents to remember and convey information accurately, and so it must be relevant to their experiences. Questionnaires are used to gather information from a larger sample of participants than interviews and can be in either paper-based or online format. The instrument presents actors with uniform prompts that can generate comparable responses. Hence, this format requires the wording, and order of questions to remain consistent throughout the study (Martin, 2006).

16. Map-based surveys can be used to capture local knowledge from citizens spatially by combining participatory mapping and survey questions. The inclusion of these elements provides greater structure than sketch planning. Digital versions like public participation GIS can reach a larger pool of participants and enable systematic analysis (Kahila-Tani et al., 2019).

17. 3D land use typology classifications enable the study and communication about urban data that is required to model urban climate. This includes parameters such as land cover, building morphology, building design, building use, socio-economic data, and urban greenery (Masson et al., 2020). These parameters underlie land use typologies ranging from compact high-rise to sparsely built. When combined with socioeconomic data, scientists can use the typologies to evaluate risk in different neighborhoods during a hazardous event. 3D typologies enable scientists to study not only which neighborhoods are at risk but also how different hazards may be experienced and what can be done to mitigate risk by exploring what-if scenarios.

18. Interview-based surveys are used to collect information from individuals in a structured, semi-structured or unstructured format. Semi-structured interviews are most common as they combine a structured set of questions that are asked from all interviewees with a set of open questions that can reveal information (e.g., knowledge, experiences) that is specific to each individual. Interviews are either transcribed or summarized in narrative form by the interviewer before analysis. This method of gathering information requires empathy, a good understanding of the characteristics of participants and the language they use (e.g., expert interviews differ from interviews carried out in a neighborhood community). The latter results in a participatory approach to data collection with an exploration of the roles of different actors in the city (Slingerland et al., 2020).

The building blocks serve as a loosely structured guide for the modular construction of methods for collecting peripheral knowledge. The building blocks afford the user flexibility in determining the sought-after type of knowledge, the target peripheral actor to engage and the appropriate means of acquiring this knowledge through the adoption of one or more design instruments.

3.2 Outcome 2. The playing cards

To encourage exploration and experimentation in building methods, an interactive and playful means of presenting the building blocks was needed. For these purposes, we chose to present the building blocks as playing cards. Illustrations on the front side provide a graphical overview of each building block and a summary of the building block on the back side facilitates the rapid assembly of a design method and initial assessment of its usefulness. Additionally, the playing card format encourages interaction between users to collaboratively construct a data collection method (Figure 3).

Figure 3

Figure 3. Playing cards showing the three building block types and quick-reference information on the backside.

3.3 Outcome 3. The diamond process model as scaffolding

Once peripheral knowledge has been collected using a constructed data collection method, the convergence facilitator has the challenge of introducing these collected insights in a meaningful way into the convergence process. This knowledge integration process is often ad hoc and in need of a structuring mechanism. To structure this knowledge integration, we introduced a planning process model. The diamond process model (Figure 4) is adapted from the strategy-making process framework introduced in Champlin et al. (2019). The framework provides a dynamic view of the strategy-making stages of planning similar to the double diamond design model (Tschimmel, 2012). The model involves iterative cycles of convergent and divergent thinking, where ideas are generated and explored before collectively making choices (Dennis and Wixom, 2002). This process model includes three divergence dynamics—information gathering, scenario designing and evaluating strategies—and three convergence dynamics—objective setting, developing scenarios, and selecting a strategy—involved in strategy making. A shared concern forms the basis for problem formulation and is the result of a prior series of divergence and convergence dynamics. The shared concern is most likely defined based on a shared set of “strong signals". Our approach uses the process model to guide convergence conversations with transdisciplinary researchers and supports them in exploring beyond their disciplinary knowledge to seek out what they do not (yet) know.

Figure 4

Figure 4. Diamond process model, adapted from Franken-Champlin (2019) indicating shared concerns as the starting point for problem formulation.

4 Prototyping the AWeS guidelines

4.1 Prototyping methods

The aim of the prototyping stage was to experiment with the toolkitting principles in constructing design methods that support strategy making. The experiment was organized into two sessions. The first session focused on knowledge divergence, specifically the collection of issues from peripheral actors using a design method constructed by each group. The second session focused on knowledge convergence. Here, the challenge was to integrate the collected issues in a simulated strategy-making workshop.

The case study location and shared concerns introduced to the students was the ability to adapt to intensifying heatwaves in the Scheveningen Harbour district. The case site was introduced to the students by a researcher studying spatial design strategies for dealing with heatwaves in the Harbour area. To frame the strategy-making sessions, the researcher presented a possible heatwave strategy to the students based on literature and current actions of the Muncipality of The Hague. This strategy served as a baseline for the experiment. The students were tasked with creating an alternative heatwave strategy for the development of Scheveningen Harbour based on knowledge they collected from peripheral actors.

4.1.1 Method-building session setup and analysis

The summer school students were randomly divided into 18 five-member groups. The students were given an introduction to the AWeS project, the case study and the workshop setup. The students were challenged to design and test a design method for eliciting knowledge from actors who are typically left out of decision making concerning the impact of the heatwaves in Scheveningen Harbour. Each group was provided the building block playing cards and asked to build a method to elicit knowledge from a peripheral actor whose knowledge is not considered in the mainstream discussion surrounding heatwaves in the case study area. Limits were set on the combination of building blocks to one peripheral actor, one knowledge type targeted from this actor, and one appropriate instrument for collecting the knowledge. Included in the instrument options was a wild card. If students chose the wild card, they could combine a maximum of two instruments or introduce their own instrument. Students were instructed to read the information on the back of the cards to learn about each of the building blocks. We asked the students to fill in a questionnaire describing their chosen method, their motivation for developing it and the expected knowledge collection outcome. Later in the week, students were able to test their method during a field trip to Scheveningen Haven.

We wanted to determine the usefulness of the design methods based on the fit between the method and the strategy-making tasks conducted during the sessions. In our analysis, we aimed to answer the following question: How can facilitators of urban planning projects be supported in collecting knowledge from peripheral actors? We analyzed the questionnaire responses from the students as well as pictures of their method and collected data that the groups posted on a MIRO whiteboard. We conducted a thematic analysis (Clarke and Braun, 2017) of the data using Atlas.ti. From this, we derived inferences about the building blocks and design methods in terms of their potentials, limitations and challenges that may arise when working with them. From these inferences, we developed a set of tables that structure the method-building process to ensure successful data collection from peripheral actors. One of these tables is shown in Section 4.3.

4.1.2 Strategy-making sessions setup and analysis

Following the data collection carried out during fieldwork, summer school participants were invited back to the second part of the workshop series, a simulated strategy-making session. During the session, they integrated the peripheral knowledge they collected into the city's baseline heatwave strategy. The following baseline strategy was introduced:

The City of The Hague plans to launch a long-term, multi-scale strategy to combat Urban Heat Island (UHI) effects. This strategy, comprised of interventions at the micro, meso, and macro levels, will deploy urban heat adaptation strategies ranging from the building scale to the city scale. The primary goal at the micro level will be to minimize solar heat gain in buildings through the use of green roofs, vertical gardens, and heat-reflecting building envelopes. Reducing heat gain may improve thermal comfort for building occupants while also benefiting the surrounding urban environment. The meso-level interventions include designing the microclimate with blue-green infrastructure (BGI). Also replacing hard surfaces with green permeable surfaces are possible. Finally, at the macro level, there is a masterplan for creating urban wind corridors which is a long-term strategy for channelling cool air from the North Sea across the UHI hot spots of the city. In addition, by 2030, a substantial energy-efficient building upgrade will take place throughout the city, ensuring a healthier and more sustainable future for everyone.

To simulate multi-actor strategy making in a workshop setting, the student groups were combined into six meta-groups each consisting of three small groups. To test the potential for peripheral knowledge integration, we organized each meta-group so that three different actor types were represented. To provide structured guidance to the session, the research team served as strategy-making facilitators and provided each meta-group with a template based on the diamond process model, as shown in Figure 4. The meta-groups were provided with an A2 sheet of paper with the three categories of issues, parameters and strategy printed on it. The baseline issues, parameters and strategy previously presented by the urban resilience expert were already filled in. The facilitators asked each meta-group to discuss and select the most relevant issues and parameters (e.g., building heights, tropical day temperature thresholds, etc.) collected from peripheral actors and to place them on the convergence sheet using sticky notes. This peripheral knowledge was then integrated through negotiation and selection to formulate a heatwave resilience strategy. This alternative strategy was compared to the baseline strategy during analysis.

4.2 Prototyping findings

During the method-building session, the 18 student groups generated 17 unique design methods based on a combination of one peripheral actor, one knowledge type and up to three design instruments. These methods represent a fraction of nearly 900 possible methods that could have been created using the building blocks provided. The methods were generated as a result of each group's informed reasoning based on the descriptions of the building blocks on the back of the cards. The documented data we collected described the strengths and weaknesses of each method for collecting issues within the contexts of their application in Scheveningen Harbour. This necessitated us to identify common links and descriptions in these diverse accounts describing the application of the same building block. These descriptions were networked to create the knowledge frameworks for structured flexibility introduced in the approach, see Outcome 5 in Section 4.4.

An exploration of the data showed that nearly one-third of the 18 groups developed methods to collect experiential knowledge from citizens. And, many groups preferred to combine two design instruments to create a customized method for triangulating data collection and ensuring data credibility. Findings were compiled into a set of three tables to relate how specific design instruments could help overcome challenges specific to different actor types. For example, to overcome the challenge of collecting knowledge directly from “ecological agents”, groups chose “observation” as their design instrument. The inability to communicate with ecological agents only leaves the option of using design instruments that do not require direct communication. From the analyzed data, the most notable finding was that most of the issues about ecological concerns (specifically about biodiversity) were found by collecting “relational knowledge” from ecosystem agents through observation in combination with other instruments such as “open datasets” and “Deconstruct-Reconstruct”. Also, the actor group “citizens” was relied on the most to collect issues about the urban form.

Findings from the strategy-making session show knowledge convergence did not result in a radical transformation of the baseline strategy. Instead, more social and ecological concerns were added to the existing strategy centered around green infrastructure. Other studies have similarly found that contributions from non-experts tended to be more additive than integrative (Scherhaufer, 2021). During the analysis, the research team found that the tables used to summarize the findings of the prototyping experiments provided a potentially useful means of structuring learning about and rapid evaluation of the design methods based on the knowledge they collected. But for facilitators to use them in a meaningful way, the tables needed to be incorporated into a structured process that guided users through the construction of a contextualized design method. These findings led to the introduction of a set of frameworks as the next three design outcomes for the AWeS approach.

4.3 Outcome 4. Building block assessment

Analysis of the method-building session findings resulted in a set of tables that provide an overview of the different characteristics of each building block in terms of their strengths, challenges and barriers to using them for knowledge collection. Figure 5 is an example of one of these tables. It shows the different characteristics of each design instrument that was tested by the students. The approach also includes similar characteristics tables for the other building block types, namely peripheral actors and knowledge types. The tables serve in the AWes approach as guides for assessing the building blocks of the method being built. For example, the students claimed that “sketch planning" provides enriched spatial and contextual data, supports inclusive design while increasing empathy and understanding, and can also provide supplemental data.

Figure 5

Figure 5. Overview of the characteristics of each design instrument.

4.4 Outcome 5. Method assessment framework

By analyzing the design methods in relation to the issues that were collected, gathered data was organized into a method assessment table (Figure 6). Similar to the building block assessment tables (Outcome 4), the method assessment table was populated using the findings from the method-building session with students. The table supports future adopters in creating their own design method and judging the feasibility of its execution within the context of their projects. The framework creates a link between the design methods that were constructed and tested to the issues they elicited through a classification system.

Figure 6

Figure 6. Method assessment framework.

This classification is necessary to elaborate the shared concern into a set of core issues and sub-issues. The issue classification system is based on the Guide to the City Resilience Profiling Tool published by UN-Habitat (2018). The issue classification specifically draws from the classification of the elements of the urban system in question which are subjected to various shocks and stresses related to the built environment, mobility, supply chain and logistics, basic infrastructure, social inclusion and protection, economy, ecology and municipal public services. The size of the bubble at each cell in the framework matrix indicates the frequency by which that method was used to collect the corresponding issue type.

A convergence facilitator looking to design a method that is tailored for the collection of specific issues can create preliminary prototypes of their method based on advice provided by this framework. In exchange, the framework provides a uniform system for users (a) to learn from the experiences of past projects and (b) to share their own experiences in method construction and application, thereby further populating the frameworks of the AWeS approach with practice-based insights.

4.5 Outcome 6. Method-building guidelines

The AWeS approach for building a customized convergence support method is intended to be adaptive to user needs at any process stage. To guide users in a structured yet flexible way, the AWeS approach relies on the frameworks mentioned above that provide overviews of (1) the strengths, challenges and barriers of the building blocks and (2) the links between design methods (assemblages of building blocks) and categories of issues that have been collected using similar methods in past projects. However, as the prototyping sessions have shown, the method-building process can be challenging without a structured means of navigating through the frameworks and their advice. Therefore, we introduce the method-building guidelines.

An initial setup of the guidelines for user testing is described as follows: to establish the point of intervention in a planning process, a shared concern is defined. A primary issue of this shared concern and its supporting sub-issues are then selected and classified according to the issue classification system. The method assessment framework helps the user to link these issues to one or more design methods that were constructed to deal with a similar set of issues. Then through an iterative process of trial and error, different combinations of building blocks can be constructed and compared. This step is meant to expand the users' understanding of other issue types that they perhaps have not encountered in their own convergence process so far.

After arriving at an initial method construction, the users have created a visual overview of different combinations of building blocks and are prompted to swap blocks for a better match with the needs of their own convergence process. The users are encouraged to determine a final design method that encompasses the most important features of each of the rapidly constructed methods. In this final step, the assessment frameworks are used to check the compatibility of the selected building blocks and the effectiveness of the method as reported from earlier projects. The end goal is to inspire convergence facilitators to explore new method(s) which could elicit previously overlooked peripheral knowledge. The results of choices at each step are archived on a guidelines game board and the process is repeated multiple times to create different design methods and then select the most useful one(s) (see Figure 7).

Figure 7

Figure 7. Process framework board.

5 Evaluating the AWeS approach with end users

The final design step of this study was to test the AWeS approach with professional end users in two urban resilience cases. Before the sessions, the researchers and designers were asked to describe their respective urban resilience projects that could benefit from bringing peripheral knowledge into the mainstream convergence discussion. They were asked to define the shared concern of their urban resilience challenge using the following formulation: the resilience of what (an urban system or set of systems)—to what (a shock or stress). The shared concern of the researchers was building on un-embanked areas that are vulnerable to sea level rise. The shared concern of the urban designers was resilience of an urban boulevard to drought and flooding.

To gain their perceptions of the usefulness of the approach, the end users were provided with a printed copy of the guidelines, a set of building block cards, and an activity board that was made to document each step of the process. The session was facilitated by the five researchers. The three main facilitating roles were the moderator who managed the flow of the workshop, the facilitator who served as the mediator between the end users and the approach and the integrator who posed critical questions about knowledge integration to the end users. Two additional researchers observed the workshops taking notes of both process and outcome, which together provide comprehensive insights into convergence (Innes and Booher, 1999). At the end of both sessions, a debrief was held to determine the strengths and weaknesses of the approach prototype.

The results of the end user testing indicate that an issue-driven approach to knowledge integration is more desirable by end users than a method- or tool-driven approach. Contrary to our expectations, despite being methods experts themselves, the urban designers preferred to begin with an exploration of planning issues rather than with an exploration of previous design methods. While this request surprised us, it was consistent with the notion of process-driven planning support as introduced in Planning Support Science literature (Franken-Champlin, 2019; Geertman and Stillwell, 2020). Findings from testing with the researchers indicate that the approach can be applied as an accountability tool to assess how inclusive their convergence strategy had been so far. The researchers also shared that the experiences reported by previous contributors in the assessment frameworks could also bring attention to unexpected knowledge integration challenges not yet encountered in their own project.

A major roadblock during the prototype testing sessions was the identification of issues. We use an existing framework [Guide to the City Resilience Profiling Tool published by UN-Habitat (2018)] for the identification of the issues which did not (and might not in future applications of the approach) fully match the specific issues that convergence facilitators are managing. Moreover, testing revealed the need for additional guidelines for selecting a set of issues of a shared concern. Too little attention was given during user testing to the critical planning process stage of problem formulation. While other studies have dealt with the topic of problem formulation (Franken-Champlin, 2019), this was beyond the scope of our research.

The conclusion drawn from the user testing sessions was that in its present state, the AWeS approach does not provide reliable advice for constructing a useful knowledge integration method. This makes sense considering the building block assessment frameworks (Outcome 4) and method assessment framework (Outcome 5) were both populated with data from a 2-day session with students who had limited opportunity to build and test their methods. We expect that as facilitators of convergence processes populate these frameworks with results from their projects, the approach will become more reliable and informative. The urban designers stated that a more mature version of the approach could save them time by accelerating the search for an appropriate instrument. The researchers stated that the approach in its present form including the exercise of following the guidelines can serve as a quality check for their convergence process. Specifically, it could facilitate discussions around shared meanings of important issues and checking that they had the necessary tool-related expertise in house to collect the needed peripheral knowledge.

6 Discussion

Using research through design, we generated the Amplifying Weak Signals (AWeS) approach, a mid-fidelity prototype of a set of guidelines that support convergence facilitators in building their own method for integrating peripheral knowledge into an urban resilience strategy-making process. The approach and its guidelines are based on structured flexibility, a core principle of toolkitting as introduced in this paper. Toolkitting provides a modular and engaging means for developing situated planning support based on knowledge needs from a given (set of) actors at different stages of a planning process.

To address challenges concerning both knowledge collection (divergence) and knowledge integration (convergence), we developed a building blocks-based prototype for eliciting knowledge from a set of peripheral actors. In a two-part method-building exercise with students, we explored how people interact with the building blocks to construct their own customized knowledge integration method. Based on these insights, we developed a prototype set of guidelines for the modular construction of a design method for collecting knowledge from peripheral actors about the impacts of climate change on the built environment and socio-ecological inclusion. We then tested a prototype of the guidelines with expert users during two workshops. Given the infancy of the approach and the limited capacity of this small research project, there are a number of areas for advancement.

First, further testing and iteration of the approach is required. During the divergence stage of the summer school workshop, some participants struggled with understanding the building blocks and had very limited time before the second workshop to use their methods on-site to collect peripheral knowledge. During the data collection process, the issue of data corruption was a concern as the workshop assignment was clubbed with the broader activities of the summer school. Another issue that emerged was the variability in the interpretation of building blocks across the different groups. In particular, the understanding of knowledge types varied, with some groups not fully grasping their use. Further, during data analysis, we became aware of some discrepancies likely due to participants' incorrect understanding of the building blocks while filling out their descriptions. This was in part due to the rather general description of the building blocks themselves. There was little time in the 2-h workshop setting to open a larger discussion about knowledge types or to train the students in the application of the design instruments.

Second, results from the convergence workshop did not show the expected radical departure from the baseline strategy. Instead, the convergence exercise resulted in a more nuanced understanding of issues already present in the baseline strategy and in more emphasis on issues of social inclusion over core baseline that were more related to ecology and the built environment. While this is already a considerable contribution, the AWeS approach can potentially lead to more radical departures from the baseline. Our workshops only demonstrated a very small fraction of the design methods that can be produced with the AWeS approach. And, time constraints prevented participants from a thorough reflection on the baseline strategy—both factors suggest that there is still room for research into a more disruptive form of convergence when peripheral knowledge is included.

Third, findings suggest that different types of approach users lean toward issues as the starting points for constructing a support method. This finding from the empirical research is consistent with planning support research that advocates a process-driven rather than a tool-driven approach. Additionally, the findings highlight the importance of considering the perspectives of different types of users when designing knowledge collection methods. Current momentum toward open-source data and repositories can open avenues for converting the tangible components of the AWeS approach into a digital interface and method repository which could vastly expand the reach of the approach. However, partners from within muncipalities or private firms may have a greater incentive not to share their data and methods due to data privacy issues or in the interest of maintaining their competitive advantage. Such insights can inform the development of future knowledge-collection methodologies that are better suited to the context of the spatial planning challenge and the circumstances of different types of process facilitators.

Next to its capacity to scale the inclusion of peripheral actors and their knowledge in numbers, this study has shown us the potential to scale the approach in a variety of directions. We outline four of these directions:

1. Expanding the building blocks. For this design study, the scope of the AWeS approach was limited to a set of building blocks whose selection was informed by literature and in consultation with experts. This scoping, while necessary for experimentation, should not be interpreted as the fixed boundaries for the approach's future application but as a starting point. For example, the analogue design instruments applied in the study could be expanded to include a digital alternative, as many of the instruments have digital counterparts that can be developed and applied. Map-based surveys, serious games and 3D land-use typologies are prime examples of instruments that translate well to digital environments for distributed participation and knowledge collection. When going digital, it is important to consider the sought-after knowledge type, where digital tools excel and what trade-offs are made in digital vs. analogue tool selection. For example, online serious games enable large-scale engagement for generating big data insights whereas playing tabletop serious games can be highly effective in fostering communication for the sharing of relational knowledge in a small-group setting. Furthermore, the nine instruments studied in this project represent only a handful of design instruments applied in practice. The AWeS approach could serve as the structuring framework to archive and share other on-going initiatives to benchmark tools and methods for planning support. The same holds for the other building blocks. The four studied actors represent only a handful of actors situated at the periphery of multi-actor convergence processes and they each hold a variety of types of knowledge, ways of knowing and values underlying their preferences and behaviors that need to be shared in planning just urban resilience transitions.

2. Supporting other strategy-making stages. This paper is connected to a research project that intervened in an on-going resilience convergence process at the stage of problem formulation. With this as our research challenge, we primarily focused on the task of collecting knowledge about issues related to two shared concerns. However, toolkitting is conceived as a continuous process of redesigning planning support in a longitudinal way at different planning stages and across them, which leads to the next scaling dimension.

3. Integrating knowledge across stages and instruments. While instruments may be designed with a certain task in mind, this alignment is less obvious in practice. Previous work confirms that planning support instruments can serve purposes beyond that of their intended design. One such example is conceptual modeling techniques that can support both issue identification and issue parameterization (Champlin et al., 2018). Such bridging capabilities supported by interoperable technologies are necessary for the seamless flow of knowledge through a planning process.

4. Including more shared concerns. This study dealt with the resilience of the built environment but is not limited to this scope. Results from this study showed that the issue classification system used in the project was too limited. This points to the reality that planning practice does not fit cleanly in the organized taxonomies that science uses to understand phenomena. The real world is far messier and more interconnected. Therefore, the classification of issues as a means of drawing relations to building blocks and comparing different design methods must become more exhaustive. This aim could be achieved by collaborating with a growing community of adopters who can continuously add issues that fall outside the domains presented in the current classification system.

7 Conclusions

The AWeS approach belongs to a growing collection of repositories that support the creation and sharing of both knowledge and knowing (Rasmussen et al., 2019). In the future, we aim for the frameworks in the AWeS approach to be populated through contributions by an ever-growing community of researchers. If this can be achieved, the approach itself can evolve into an online, interactive space for sharing and learning about inclusive planning support tools and methods. Through transparency in the application of methods by this community of researchers, we aim to make the process more scientifically robust. We expect that as the approach matures, positive frictions may arise as a result of how these methods are applied in diverse settings. As a next step, we aim to translate that approach in its current format of downloadable guidelines and process frameworks into an openly accessible, living online framework with interactive features that in part simplify the heavy facilitating role required to use the approach in its current state.

There is growing recognition that complex societal challenges like transitioning to climate-resilient cities require knowledge integration. To achieve this integration, ever-growing knowledge banks of expertise and opportunities to learn from examples of successful convergence initiatives are needed (Bammer et al., 2020). We have attempted to extend the knowledge bank to actors at the periphery of knowledge integration practices to achieve a more just transition to climate-resilient cities. The AWeS approach serves currently as a proof-of-principle that needs to become more robust through adoption and scaling in multiple directions.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

CC: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing—original draft, Writing—review & editing. AE: Formal analysis, Investigation, Validation, Visualization, Writing—original draft. RV: Formal analysis, Investigation, Validation, Writing—original draft. JG: Formal analysis, Investigation, Software, Validation, Writing—original draft. CF: Conceptualization, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing—original draft, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research has been funded by the Resilient Delta Initiative.

Acknowledgments

We would like to extend our gratitude to the Methodology pillar of the Resilient Delta Initiative for early insights into convergence processes in a workshop setting. We thank Arjan van Timmeren for support in developing the research idea and Istiaque Ahmed for serving as a workshop facilitator and for presenting the heatwaves baseline strategy for the City of The Hague. Our appreciation also goes to Roberto Rocco and Caroline Newton for the opportunity to conduct the method-building sessions as a part of the Planning and Design for the Just City 2022 Summer School and to all the scholars who provided their expert knowledge about the challenges they have encountered working in transdisciplinary research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bammer, G., O'Rourke, M., O'Connell, D., Neuhauser, L., Midgley, G., Klein, J. T., et al. (2020). Expertise in research integration and implementation for tackling complex problems: when is it needed, where can it be found and how can it be strengthened? Palgr. Commun. 6, 1–16. doi: 10.1057/s41599-019-0380-0

Crossref Full Text | Google Scholar

Bjögvinsson, E., Ehn, P., and Hillgren, P.-A. (2012). Design things and design thinking: contemporary participatory design challenges. Design Issues 28, 101–116. doi: 10.1162/DESI_a_00165

Crossref Full Text | Google Scholar

Bourgeois, J., and Kortuem, G. (2019). “Towards responsible design with internet of things data,” in Proceedings of the Design Society: International Conference on Engineering Design, Vol. 1 (Cambridge: Cambridge University Press), 3421–3430.

Google Scholar

Byrne, J. A. (2021). “Observation for data collection in urban studies and urban analysis," in Methods in Urban Analysis, ed. S. Baum (Springer), 127–149. Available online at: https://link.springer.com/book/10.1007/978-981-16-1677-8

Google Scholar

Campagna, M. (2016). Metaplanning: about designing the geodesign process. Landsc. Urban Plan. 156, 118–128. doi: 10.1016/j.landurbplan.2015.08.019

Crossref Full Text | Google Scholar

Champlin, C., Hartmann, T., and Dewulf, G. P. (2018). Mapping the use of planning support in a strategy-making session. plaNext Next Gen. Plann. 6, 5–24. doi: 10.24306/plnxt.2018.06.001

Crossref Full Text | Google Scholar

Champlin, C., Sirenko, M., and Comes, T. (2023). Measuring social resilience in cities: an exploratory spatio-temporal analysis of activity routines in urban spaces during covid-19. Cities 135:104220. doi: 10.1016/j.cities.2023.104220

PubMed Abstract | Crossref Full Text | Google Scholar

Champlin, C., te Brömmelstroet, M., and Pelzer, P. (2019). Tables, tablets and flexibility: evaluating planning support system performance under different conditions of use. Appl. Spat. Anal. Policy 12, 467–491. doi: 10.1007/s12061-018-9251-0

Crossref Full Text | Google Scholar

Champlin, C. J., Flacke, J., and Dewulf, G. P. (2022). A game co-design method to elicit knowledge for the contextualization of spatial models. Environ. Plann. B Urban Anal. City Sci. 49, 1074–1090. doi: 10.1177/23998083211041372

Crossref Full Text | Google Scholar

Clarke, V., and Braun, V. (2017). Thematic analysis. J. Posit. Psychol. 12, 297–298. doi: 10.1080/17439760.2016.1262613

Crossref Full Text | Google Scholar

Daniels, E., Bharwani, S., Swartling, Å G., Vulturius, G., and Brandon, K. (2020). Refocusing the climate services lens: introducing a framework for co-designing “transdisciplinary knowledge integration processes” to build climate resilience. Clim. Serv. 19:100181. doi: 10.1016/j.cliser.2020.100181

Crossref Full Text | Google Scholar

Dannevig, H., Hovelsrud, G. K., Hermansen, E. A., and Karlsson, M. (2020). Culturally sensitive boundary work: a framework for linking knowledge to climate action. Environ. Sci. Policy 112, 405–413. doi: 10.1016/j.envsci.2020.07.002

Crossref Full Text | Google Scholar

Dennis, A. R., and Wixom, B. H. (2002). Investigating the moderators of the group support systems use with meta-analysis. J. Manag. Inf. Syst. 18, 235–257. doi: 10.1080/07421222.2002.11045696

Crossref Full Text | Google Scholar

Dreessen, K., Huybrechts, L., Grönvall, E., and Hendriks, N. (2017). “Infrastructuring multicultural healthcare information systems,” in Participatory Design & Health Information Technology, eds. A.M. Kanstrup, et al. (IOS Press).

Google Scholar

Fischer, F. (2000). Citizens, Experts, and the Environment: The Politics of Local Knowledge. Durham, NC: Duke University Press.

Google Scholar

Forgaci, C. (2021). Deconstructing Urban Resilience. Available online at: https://search.edusources.nl/en/materials/483d912c-f454-4df1-b5d2-6d5494f922bb

Google Scholar

Franken-Champlin, C. J. (2019). Contextualizing Planning Support (Systems): Co-designing to Fit the Dynamics of Spatial Strategy Making (dissertation). Available online at: https://research.utwente.nl/en/publications/contextualizing-planning-support-systems-co-designing-to-fit-the-

Google Scholar

Geertman, S. (2017). Pss: Beyond the implementation gap. Transport. Res. Part A Policy Pract. 104, 70–76. doi: 10.1016/j.tra.2016.10.016

Crossref Full Text | Google Scholar

Geertman, S., and Stillwell, J. (2020). Planning support science: developments and challenges. Environ. Plann. B Urban Anal. City Sci. 47, 1326–1342. doi: 10.1177/2399808320936277

Crossref Full Text | Google Scholar

Goodhue, D. L., and Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Q. 19, 213–236. doi: 10.2307/249689

Crossref Full Text | Google Scholar

Harris, B. (2001). “Sketch planning: systematic methods in planning and its support” in Planning Support Systems, Combining GIS, Models and Visualization, eds. R. Brail, R. Klosterman (ESRI Press, Redlands, CA), (2001), pp. 59–80

Google Scholar

Holopainen, M., and Toivonen, M. (2012). Weak signals: Ansoff today. Futures 44, 198–205. doi: 10.1016/j.futures.2011.10.002

Crossref Full Text | Google Scholar

Innes, J. E., and Booher, D. E. (1999). Consensus building and complex adaptive systems: a framework for evaluating collaborative planning. J. Am. Plann. Assoc. 65, 412–423. doi: 10.1080/01944369908976071

Crossref Full Text | Google Scholar

Kahila, M., and Kyttä, M. (2009). “Softgis as a bridge-builder in collaborative urban planning,” in Planning Support Systems Best Practice and New Methods, eds. S. Geertman and J. Stillwell (Springer), 389–411. Available online at: https://link.springer.com/book/10.1007/978-981-16-1677-8

Google Scholar

Kahila-Tani, M., Kytta, M., and Geertman, S. (2019). Does mapping improve public participation? Exploring the pros and cons of using public participation gis in urban planning practices. Landsc. Urban Plann. 186, 45–55. doi: 10.1016/j.landurbplan.2019.02.019

Crossref Full Text | Google Scholar

Karasti, H., and Baker, K. S. (2004). “Infrastructuring for the long-term: ecological information management, in 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the Jan. 5 2004 to Jan. 8 2004 (Big Island, HI: IEEE), 10. doi: 10.1109/HICSS.2004.1265077

Crossref Full Text | Google Scholar

King, K., and McGrath, S. A. (2004). Knowledge for Development?: Comparing British, Japanese, Swedish and World Bank Aid. HSRC Press; Zed Books: Cape Town, London.

Google Scholar

Knapp, C. N., Reid, R. S., Fernández-Giménez, M. E., Klein, J. A., and Galvin, K. A. (2019). Placing transdisciplinarity in context: A review of approaches to connect scholars, society and action. Sustainability 11:4899. doi: 10.3390/su11184899

Crossref Full Text | Google Scholar

Martin, E. (2006). Survey questionnaire construction. Surv. Methodol. 13, 1–13.

Google Scholar

Masson, V., Heldens, W., Bocher, E., Bonhomme, M., Bucher, B., Burmeister, C., et al. (2020). City-descriptive input data for urban climate models: Model requirements, data sources and challenges. Urban Clim. 31:100536. doi: 10.1016/j.uclim.2019.100536

Crossref Full Text | Google Scholar

Pelzer, P. (2015). Usefulness of Planning Support Systems: Conceptual Perspectives and Practitioners' Experiences (PhD thesis), InPlanning.

Google Scholar

Pelzer, P., Arciniegas, G., Geertman, S., and Lenferink, S. (2015). Planning support systems and task-technology fit: a comparative case study. Appl. Spat. Anal. Policy 8, 155–175. doi: 10.1007/s12061-015-9135-5

Crossref Full Text | Google Scholar

Pfeffer, K. (2019). “Techniques of infrastructuring for urban infrastructure planning in the global south, in TRIALOG Conference 2019: Who's Knowledge Counts? The Meaning of Co-productive Processes for Urban Development and Urban Research (Stuttgart).

Google Scholar

Pham, H., and Saner, M. (2021). A systematic literature review of inclusive climate change adaption. Sustainability 13:10617. doi: 10.3390/su131910617

Crossref Full Text | Google Scholar

Rasmussen, S., Fritsch, J., and Hansen, N. B. (2019). “A design archival approach to knowledge production in design research and practice,” in Proceedings of the 31st Australian Conference on Human-Computer-Interaction (Fremantle, WA: ACM), 233–243.

Google Scholar

Razzouk, R., and Shute, V. (2012). What is design thinking and why is it important? Rev. Educ. Res. 82, 330–348. doi: 10.3102/0034654312457429

Crossref Full Text | Google Scholar

Rotmans, J. (2006). Tools for integrated sustainability assessment: a two-track approach. Integr. Assess. J. 6, 35–57.

Google Scholar

Scherhaufer, P. (2014). Bridging the gap between the theory and practices of stakeholder participation in integrated vulnerability assessments of climate change. Syst. Pract. Act. Res. 27, 449–463. doi: 10.1007/s11213-013-9294-8

Crossref Full Text | Google Scholar

Scherhaufer, P. (2021). Better research through more participation? The future of integrated climate change assessments. Futures 125:102661. doi: 10.1016/j.futures.2020.102661

Crossref Full Text | Google Scholar

Simonsen, J., Karasti, H., and Hertzum, M. (2020). Infrastructuring and participatory design: Exploring infrastructural inversion as analytic, empirical and generative. Comp. Support. Cooperat. Work 29, 115–151. doi: 10.1007/s10606-019-09365-w

Crossref Full Text | Google Scholar

Slingerland, G., Lukosch, S., Hengst, M. d, Nevejan, C., and Brazier, F. (2020). Together we can make it work! Toward a design framework for inclusive and participatory city-making of playable cities. Front. Comp. Sci. 2:600654. doi: 10.3389/fcomp.2020.600654

Crossref Full Text | Google Scholar

Stappers, P. J., and Giaccardi, E. (2017). “Research through design,” in The Encyclopedia of Human-Computer Interaction, eds. M. Soegaard, and R. Friis-Dam (The Interaction Design Foundation), 1–94. Available online at: https://research.tudelft.nl/en/publications/research-through-design-2

Google Scholar

te Brömmelstroet, M. (2012). Transparency, flexibility, simplicity: from buzzwords to strategies for real pss improvement. Comput. Environ. Urban Syst. 36, 96–104. doi: 10.1016/j.compenvurbsys.2011.06.002

Crossref Full Text | Google Scholar

Tschimmel, K. (2012). “Design thinking as an effective toolkit for innovation,” in Proceedings of the XXIII ISPIM Conference: Action for Innovation: Innovating from Experience (Barcelona).

Google Scholar

UN-Habitat (2018). City Resilience Profiling Tool (crpt), 31. UN-Habitat. Available online at: https://unhabitat.org/guide-to-the-city-resilience-profiling-tool

Google Scholar

Vonk, G., and Ligtenberg, A. (2010). Socio-technical pss development to improve functionality and usability—sketch planning using a maptable. Landsc. Urban Plan. 94, 166–174. doi: 10.1016/j.landurbplan.2009.10.001

Crossref Full Text | Google Scholar

Wamsler, C., Alkan-Olsson, J., Björn, H., Falck, H., Hanson, H., Oskarsson, T., et al. (2020). Beyond participation: when citizen engagement leads to undesirable outcomes for nature-based solutions and climate change adaptation. Clim. Change 158, 235–254. doi: 10.1007/s10584-019-02557-9

Crossref Full Text | Google Scholar