Stimulating boundary crossing learning in a multi-stakeholder learning environment for sustainable development

Abstract

Purpose

Sustainable development requires multiple stakeholders to work and learn across practices, in other words, it requires boundary crossing competence. To prepare students for their future sustainability professions, higher education should facilitate the development of boundary crossing competence in its curricula. This study aims to confirm whether boundary crossing learning can be stimulated by workshop-based support in multi-stakeholder projects.

Design/methodology/approach

This quasi-experimental intervention study (N = 122) investigates the effect of a series of supporting workshops on students’ boundary crossing learning in multi-stakeholder projects. The workshops allowed students to adopt four learning mechanisms (identification, coordination, reflection and transformation) theorised to stimulate learning across boundaries between practices. Students followed zero, one, or two workshops. By analysing the student learning reports, the study examines the effect of the workshop intervention on students’ self-efficacy for stakeholder collaboration, the number of reported student-stakeholder collaborative activities and the reported boundary crossing learning mechanisms.

Findings

The results show that a series of two workshops increase the number of reported collaborative activities and activates the students’ boundary crossing learning in terms of reflection and transformation.

Research limitations/implications

These findings support the evidence-based design of multi-stakeholder learning environments for sustainable development and contribute to the body of knowledge regarding learning across practices.

Originality/value

Boundary crossing competence receives increasing attention as an asset for sustainable development. The added value of this study lies in its confirmation that the boundary crossing theory can be translated into directed educational support that can stimulate students’ boundary crossing learning.

Citation

Oonk, C., Gulikers, J., den Brok, P. and Mulder, M. (2022), "Stimulating boundary crossing learning in a multi-stakeholder learning environment for sustainable development", International Journal of Sustainability in Higher Education, Vol. 23 No. 8, pp. 21-40. https://doi.org/10.1108/IJSHE-04-2021-0156

1. Introduction

Sustainable development requires collaboration between multiple societal stakeholders representing various practices, disciplines and perspectives (Scholz and Steiner, 2015). Collaboration can lead to tensions when differences between practices are ignored or remain implicit (Akkerman, 2011). These tensions should be recognised, sought, appreciated and used to enhance collaboration and come to new transformative insights (Akkerman and Bakker, 2011). However, this is not an easy task (Akkerman, 2011; Engeström et al., 1995). All stakeholders involved need boundary crossing competence, meaning the ability to operate and communicate across boundaries between different practices (Walker and Nocon, 2007).

To develop boundary crossing competence among future professionals, education should facilitate effective learning environments in which students optimally learn with – and from – multiple stakeholders (Fortuin and Bush, 2010; Webb and Burgin, 2009; Wenger, 2000). This is certainly valid for higher education programmes whose graduates will likely collaborate with multiple stakeholders when working on complex sustainability challenges in their professional lives, such as life sciences programmes.

Various authentic learning environments in higher education (for example, the design studio, service learning and mixed variants of these), potentially foster students’ learning from the collaboration with multiple stakeholders (Jacoby, 2014; Long, 2012; Molderez and Fonseca, 2018). This learning is supposed to occur in these learning environments as students often work in multi-disciplinary groups and/or with various societal stakeholders and are stimulated to reflect on their experiences (Dewey, 1915). Over the years a large body of literature has been published regarding students’ learning experiences in studios and service learning environments (Balassiano, 2011; Hebert and Hauf, 2015; Sletto, 2010). However, knowledge is limited regarding what – and how – students optimally learn from their collaboration with multiple stakeholders in these authentic learning environments (Gerholz et al., 2018; Holmén et al., 2021; Webb and Burgin, 2009). The aim of this study is to close this gap by investigating if and how explicit support of student-stakeholder collaboration by targeting parallel workshops can stimulate students to learn from their stakeholder collaboration in a multi-stakeholder learning environment.

In the Dutch authentic Regional Learning Environment (RLE), students work in collaboration with multiple stakeholders on real-life projects regarding sustainable regional development. As such, the RLE is expected to foster students’ learning from their cross-boundary collaboration with multiple stakeholders (see also Section 2). However, previous RLE studies have shown that students did not automatically – let alone optimally – learn across these boundaries (Oonk et al., 2019). High intense student-stakeholder collaboration within an RLE did not significantly increase students’ learning and perceived readiness for stakeholder collaboration, more than low intense stakeholder collaboration. A plausible explanation for the ambiguous learning effect of high intense stakeholder collaboration was a lack of pedagogical support. Students were simply sent to the stakeholders without being prepared with respect to how to communicate and collaborate with the various stakeholders. Moreover, respective learning objectives were missing and students were not encouraged to use what they learned from working with the stakeholders.

Based on this lack of pedagogical support, we developed supporting workshops that can be offered in parallel to an RLE project. We used the boundary crossing theory (Akkerman and Bakker, 2011) as a framework to design these workshops and analyse their effectiveness. The four learning mechanisms of the boundary crossing theory – identification, coordination, reflection and transformation – offer concrete handles for operationalising student-stakeholder collaborative activities within these workshops and can be used to analyse student behaviour. These workshops were designed and applied as an intervention, with the aim of testing their effect on student-stakeholder collaboration.

The intervention study reported here explores the effect of explicit support of student-stakeholder collaboration in the RLE, by means of workshops, with respect to:

• the students’ self-efficacy for student-stakeholder collaboration;

• the number of student-stakeholder collaborative activities; and

• boundary crossing learning processes.

Results of the study facilitate the design of effective authentic, boundary crossing learning environments for sustainable development and contribute to the body of knowledge regarding learning across practices. The next two sections explain the way in which boundary crossing learning is supposed to occur in the RLE and provide theoretical insights into boundary crossing learning as a steppingstone towards the setup of this intervention study.

2. Practical context: suboptimal boundary crossing learning in the regional learning environment

The RLE has been established in The Netherlands by educational institutes in collaboration with various community partners (Foorthuis et al., 2012). The general aim of the RLE is twofold:

• to support the learning and professional development of students and other parties; and

• to contribute to sustainable regional development.

Many Dutch institutes of higher and vocational education incorporate the RLE in their curricula, including nearly all Dutch schools for higher education urban and landscape planning, in which our study was conducted.

Several characteristics typify the design of the RLE. In the RLE, students work in groups on real world regional (i.e. supra-local) sustainability challenges in a project setting together with various regional stakeholders, such as local and regional authorities, semi-government bodies, entrepreneurs, research institutes, non-governmental organization and citizens (Foorthuis et al., 2012; Meijles and Van Hoven, 2010). All these stakeholders have a vested interest in the challenge at hand. Contributing to overcoming the challenge requires the integration and/or co-creation of new knowledge between students and multiple regional stakeholders. The end result should provide value to the external problem holder and contribute to sustainable regional development, ultimately leading to transformation in the sense of co-creating new hybrid practices. The RLE shows similarities with other authentic learning environments as studios, service learning and mixed variants of these (Oonk et al., 2019).

These design characteristics of the RLE provide students with various opportunities to “cross boundaries” between multiple practices, disciplines and perspectives and learn from that. For example, students discuss their assignment with the commissioner, initiate meetings with various stakeholders, become open to various opinions, integrate knowledge and perspectives from different disciplines and should come up with one co-creative transformative final product that is expected to contribute to actual sustainable development. As such, the RLE is expected to stimulate boundary crossing learning; however, apparently this does not occur automatically.

Previous studies examining whether intense student-stakeholder collaboration in the RLE triggered student learning and outcomes revealed that students did not optimally use the learning potential of the RLE (Oonk et al., 2019). Student self-efficacy for collaborative activities with the stakeholders was low. The students were reluctant to organise collaborative activities with stakeholders. Moreover, the students did neither show more competence development as a result of working in this multi-stakeholder environment nor did they develop more transformative products. These findings seemed to stem from the fact that learning both from and with stakeholders was, in neither of the studied RLEs an explicit learning objective nor was it actively stimulated, coached, or assessed by the teachers. On the other hand, all the teachers expressed that the added value of the authentic RLE regarding student learning lies in its multi-stakeholder nature. This shows the need to provide students with more explicit support that enables them to better work and learn across the boundaries between their own and the stakeholders’ practices.

3. Theoretical framework: boundary crossing learning

Different stakeholders who collaborate in multi-stakeholder processes represent various practices, with each stakeholder having his/her own disciplinary knowledge and perspectives regarding the issue at hand and his/her own stake in the solution (Akkerman and Bakker, 2011). Boundaries between these various practices often lead to “discontinuity in action or interaction” (Akkerman and Bakker, 2011: page 133). This discontinuity hinders learning and knowledge creation in multi-stakeholder processes. However, these boundaries can also be powerful places to learn. At the boundaries between different practices, people are challenged to unravel the mystery of “otherness”, explore the limits of their own competence, revisit their own realities, expand their horizons and co-create new knowledge (Wenger, 2000: page 233); all kinds of capacities that are supposed to support sustainable development (Wiek et al., 2011). To work on sustainable development, people need to develop their boundary crossing competence; in other words, they need the ability to seek, recognise, appreciate and use these boundaries to learn, co-create and innovate together across practices (Akkerman and Bakker, 2011). However, learning at boundaries is complex, does not happen easily and requires explicit support (Akkerman and Bakker, 2011; Fortuin and Bush, 2010; Stentoft, 2017; Wenger, 2000), as also found in our previous RLE studies. Sustainability education should equip students with the ability to learn and work in multi-stakeholder contexts; teach them how to co-create new knowledge for sustainable solutions. We argue that the boundary crossing theory, describing four learning mechanisms that catalyse learning at the boundaries, offers handles for designing supporting interventions in multi-stakeholder – boundary crossing – learning environments such as the RLE.

Akkerman and Bakker (2011) identified four learning mechanisms, including their respective associated sub-processes, that catalyse boundary crossing learning. The first learning mechanism, identification, involves questioning one’s own and others’ core identities, exploring the mutual complementarity of different practices. Identification leads to insights into what the diverse practices concern but does not necessarily lead to actual collaboration. The second learning mechanism, coordination, expresses what people can learn from seeking communicative connections between diverse practices or perspectives, for example, by contacting each other to exchange relevant information or by using languages from different practices. These connections can be established using effective means and procedures, also called “boundary objects” (Akkerman and Bakker, 2011: page 133) that allow different practices to communicate efficiently in distributed work. The third mechanism, reflection, includes making and taking perspectives. People come “to realize and explicate differences between practices and thus to learn something new about their own and others’ practices” (Akkerman and Bakker, 2011: page 144). The fourth learning mechanism, transformation, involves joint work at the boundaries between practices, combining ingredients from different practices into a new (i.e. hybrid) practice. Transformation results into people’s intentions to and visualisations and establishment of new, cross-boundary hybrid practices. Boundary crossing learning is supposed to develop when these four learning mechanisms are adopted while learning across practices.

4. Towards a boundary crossing theory-based intervention study

This study applies boundary crossing theory – and its four learning mechanisms – in two ways. Firstly, we used the theory to develop workshop-based support of higher education students’ learning with and from multiple stakeholders in the RLE. Secondly, the four boundary crossing learning mechanisms were operationalised into a concrete analytical frame that can be used to analyse differences in the students’ boundary crossing learning processes after receiving – or not receiving – explicit pedagogical support. This study is, therefore, a first attempt at empirically testing the usability of the theoretical ideas of boundary crossing and its four learning mechanisms in designing and evaluating effective multi-stakeholder learning environments.

We designed an intervention comprising a series of workshop-based activities. The workshop-activities were designed based on the four boundary crossing learning mechanisms, thus stimulating students’ identification, coordination, reflection and transformation activities within their RLE project. The effect of receiving various amounts of explicit support was examined in this intervention study. The research questions for the study were the following. Does explicit workshop-based support of student-stakeholder collaboration in the RLE result in:

RQ1.

More self-efficacy for student-stakeholder collaborative activities during the ongoing RLE projects?

RQ2.

More student-reported collaborative activities between students and stakeholders?

RQ3.

Differences in number and type of student-reported boundary crossing learning mechanisms?

We hypothesized that engaging in more boundary crossing-based workshop activities leads to an increase in students’ self-efficacy for and the number of, student-stakeholder collaborative activities, as well as to more and different kinds of boundary crossing learning mechanisms reported.

The next section explains the design of the study.

5. Methods

This study used a quasi-experimental intervention design, with the conditions conducted in real-world educational practice by assigning higher education students who participated in eight different RLEs to one of three experimental conditions.

5.1 Intervention

The intervention consisted of the exposure of students to no workshop (condition 0), only a first workshop (condition 1), or a first and second workshop (condition 2) for student-stakeholder collaboration during their RLE project. The two workshops were designed in such a way that they specifically addressed the student-stakeholder collaborative activities needed in various stages of the RLE projects and to stimulate the boundary crossing learning mechanisms at stake in these stages. In addition to boundary crossing theory, proven tools for improving stakeholder collaboration were included in the workshop design (Bryson, 2004; Freeman, 2010; Webb and Burgin, 2009).

Workshop 1 was scheduled at the beginning of the RLE project, shortly after the external client informed the students about their assignment, but before the identification of and actual collaboration with other stakeholders started. Workshop 1 stimulated activities addressing the learning mechanisms identification and coordination as crucial mechanisms in the beginning of the collaboration process. Examples of activities were a stakeholder force field analysis (identification) and a networking role-playing activity (collaboration). Workshop 2 was scheduled halfway through the RLE project after the students had identified and mobilised the relevant stakeholders and performed some of the collaborative activities with the stakeholders. Workshop 2 included activities addressing the learning mechanisms reflection and transformation as important mechanisms coming into play a few weeks after the start of the collaboration process. Examples of activities were a reflective review of stakeholder involvement and collaborative activities to date (reflection), a simulation of an agitated stakeholder meeting (reflection) and a brainstorming session on possible impactful stakeholder collaborative activities intended to trigger transformation in the region (transformation).

A draft design of both workshops was pilot-tested and evaluated in a group of higher education planning students (n = 13) working in an RLE. Student evaluation results were used to modify the workshop design. The modified draft-design was then pilot-tested and evaluated in two other groups of planning students (n = 15; 25) working in RLEs. The evaluations of the modified draft-design were then used to create the final design of the two workshops.

To control for differences in teaching style, all workshops were facilitated by the first author, who served in the role of guest teacher. The design of the workshops was identical between all cases, although the content of the activities was tailored to the specific RLE-projects that the students were working on. To measure the fidelity of the intervention, all workshops were evaluated. Individual evaluation forms asked students to score 27 statements regarding the usefulness and satisfaction of the workshops using a five-point Likert scale (1 = totally disagree and 5 = totally agree). The mean (SD) evaluation scores for workshop 1 (n = 76) were: M_usefulness = 4.03, SD = 0.711; M_satisfaction = 3.96, SD = 0.824. Average evaluation scores for workshop 2 (n = 60) were: M_usefulness = 4.2, SD = 0.84; M_satisfaction = 3.87, SD = 0.791.

5.2 Participants and procedure

A total of 122 third-year undergraduate students at five Dutch higher education spatial planning schools (N = 122), working in eight different RLEs, participated in either no workshop (n = 30; condition 0), workshop 1 only (n = 36; condition 1), or both workshop 1 and workshop 2 (n = 56; condition 2) during their RLE projects (Table 1). In consultation with the RLE teachers, we decided how to integrate the workshops into their RLE projects. As a result of this consultation, students in RLE 3, 4 and 6 were assigned to participate in both workshop 1 and workshop 2; a few of these students could not make it to workshop 2 and therefore participated only in workshop 1. The students in RLE 1, 2 and 5 were assigned to participate in workshop 1 (except for two students who could not make it to workshop 1) and 41%, 72% and 24% of the students in RLE 1, 2 and 5, respectively, also participated in workshop 2 on a voluntary basis. The students in RLE 7 and 8 were in condition 0 and did not participate in any workshop. The mean age of the students was 21.5 years (SD = 2.213) and 75% of the students were male.

A pre-test questionnaire to determine the students’ expectations regarding collaborating with other people during their upcoming projects revealed that the students in the three conditions did not differ with respect to the degree to which they were tuned into working with external stakeholders. During their RLE projects, the students worked in groups of 3, 4 or 5 students on a complex regional sustainability issue that was assigned by an external client. Finding solutions for the issue required the involvement of various stakeholders next to the client.

5.3 Data sources

To answer RQ1, the students completed a post-test questionnaire that included 11 statements addressing their self-efficacy for various student-stakeholder collaborative activities during the projects. Example statements included “During the project I felt able to enthuse stakeholders to contribute to the project” and “During the project I could contribute to the organisation of one or more stakeholder collaborative activities”. The students scored their level of agreement with each of the 11 statements on a five-point Likert scale (1 = totally disagree and 5 = totally agree). The scale was reliable (Cronbach’s α = 0.861).

To answer RQ2 and RQ3, the students’ answers to three open questions in the post-test questionnaire were used to compare the number of reported student-stakeholder collaborative activities between the three conditions (RQ2) and the adopted boundary crossing learning mechanisms per condition (RQ3). The first open question asked students to describe their overall RLE learning experiences. The second open question asked students to provide their learning history regarding the peaks and troughs experienced when working with other people in the project. This second question was expected to highlight critical moments in the students’ collaborative activities and to stimulate the students to report for these moments whether or not they represented boundary crossing learning mechanisms. The third open question asked students to describe how they would approach a new – but similar – project in the future. This question was added based on personal communication with experts in boundary crossing education research (personal communication, January 2015) and based on the following assumption: if students really gained a certain degree of boundary crossing competence, they are likely to use this competence in future situations. Thus, students who adopted boundary crossing learning mechanisms during their project are expected to report on boundary crossing learning mechanisms if they are asked how to approach a new, similar project in the future.

5.4 Analysis

An analysis of variance (ANOVA) compared students’ scores on the 11 statements addressing self-efficacy for student-stakeholder collaborative activities and indicated differences in the levels of self-efficacy between the three conditions (RQ1).

The analysis of the qualitative data, i.e. student reports on the open questions in the post-test, consisted of two parts.

Firstly, to answer RQ2, each separate meaningful expression in a student’s post-test report was coded as “referring to a stakeholder collaborative activity_done”, “referring to a stakeholder collaborative activity_intended in the future”, “referring to another activity”, or “not referring to an activity”. The interrater reliability (к) of the coding between two researchers, both (co)authors of this paper, was 0.87, which represents an almost perfect strength of agreement (Landis and Koch, 1977). After coding was completed, a Pearson chi-square test tested differences between conditions regarding the types of reported activities.

Secondly, to answer RQ3, we used a multi-rater deductive coding process (Gilgun, 2011) to analyse the three types of student post-test reports regarding the four boundary crossing learning mechanisms and their associated sub-processes. We developed a preliminary coding scheme based on a selection of literature regarding boundary crossing learning mechanisms and previous attempts to operationalise the theoretical concepts into coding frameworks (Akkerman and Bakker, 2011; Cremers, 2016). The preliminary coding scheme included the four boundary crossing learning mechanisms and the associated sub-processes. Because none of the existing coding frameworks was used in the context of student-stakeholder collaboration, we expected the need to modify the preliminary framework to optimise it for use in our study. Based on the preliminary coding scheme, 12 raters (ten students in a professional master’s degree programme on “Learning and Innovation” who were following a curricular course on boundary crossing and the two researchers) coded the same random selection of data per condition. Each separate meaningful expression of a student referring to a mental or physical effort (or something learned from that effort) was coded as one of the four learning mechanisms and an associated sub-process, or as “another effort or learning process”. The initial interrater reliability of this coding step was almost perfect (к = 0.85) on the level of the boundary crossing learning mechanisms and substantial (к = 0.66) on the level of associated sub-processes. A discussion between the 12 raters regarding the results of this pre-coding round led to a contextualised adjustment of the preliminary coding scheme to optimise its use in the context of student-stakeholder collaboration. The resulting final coding scheme (Appendix) was used to determine the final inter-rater reliability between the coding of two authors on both the level of boundary crossing learning mechanism (к = 0.98) and the level of associated sub-processes (к = 0.89). Finally, the first author coded all qualitative post-test data using the final coding scheme and selected illustrative examples for each code (Appendix).

After coding was completed, the number of excerpts for each (sub)learning mechanism were summed per condition. A Pearson chi-square test tested differences between the conditions regarding the number of excerpts per learning mechanism. Next, differences between the conditions on both the level of boundary crossing learning mechanisms as well as on the level of associated sub-processes were interpreted.

6. Results

Answering RQ1, the ANOVA showed no significant differences in the self-efficacy for student-stakeholder collaborative activities between the three conditions (M = 3.6, SD = 0.578; M = 3.5, SD = 0.616; M = 3.3, SD = 0.818 for conditions 0, 1 and 2, respectively).

Answering RQ2, the Pearson chi-square test showed that the types of reported activities differed significantly between conditions [χ² (6, N = 1,876) = 49.50, p < 0.001]. Table 2 provides further insights in these results. The percentages of reports on done student-stakeholder collaborative activities were almost similar in condition 0 (15%) and condition 1 (16%) and higher in condition 2 (25%). The percentages of reports on intended future student-stakeholder collaborative activities increased slightly from condition 0 (10%) to condition 1 (12%) and condition 2 (15%). In condition 2, a higher percentage of students referred to a done and/or intended stakeholder collaborative activity (48%) compared to the students in condition 0 (32%) and condition 1 (34%). This means that supporting students in all four boundary crossing learning mechanisms stimulated them to conduct more student-stakeholder collaborative activities.

Answering RQ3, the Pearson chi-square test showed that the reported learning mechanisms differed significantly between conditions [χ² (8, N = 1,876) = 67.86, p < 0.001; see Table 3].

The students’ adoption of the identification and coordination learning mechanisms hardly differed over the three conditions 0, 1 and 2 (12%, 13% and 10%, respectively, for identification and 15%, 15% and 16%, respectively, for coordination). The number of excerpts referring to reflection increased from condition 0 (6%) to condition 1 (11%) and condition 2 (19%). Where transformation was barely addressed in conditions 0 and 1 (both at 1%), this percentage was four times higher – albeit still relatively low – in condition 2 (4%). Finally, the percentage of excerpts referring to other, non-boundary crossing learning processes decreased from condition 0 (66%) to condition 1 (60%) and condition 2 (51%).

With respect to the adopted sub-processes per learning mechanism (Appendix for descriptions), a few results were striking. In all three conditions more than half of the excerpts for identification were sub-coded as exploring mutual expectations. Nearly all these excerpts were related to expectation management in the triangle between students, teachers and external client(s). For coordination, in condition 2 the sub-process translating was coded two and three times more frequently than in condition 0 and 1, respectively. The higher scores on reflection in condition 2 manifested in higher scores on the sub-processes perspective making and facilitating perspective making. For transformation, more emphasis was placed on intending the creation of a new practice in conditions 0 and 1 and more on envisioning new practices in condition 2.

7. Discussion

This section successively discusses the results for the three research questions, as well as implications and limitations.

7.1 Self-efficacy for stakeholder collaboration

The non-significant effect of the workshops on self-efficacy for student-stakeholder collaboration rejects our hypothesis that more support would increase self-efficacy. Moreover, this finding is in contrast with previous studies performed in the context of service learning and other multi-stakeholder learning environments, which suggest positive effects on self-efficacy (Gerholz et al., 2018; Holmén et al., 2021). Our findings are also in contrast with the high average student scores regarding the usefulness of both workshops in preparing the students for student-stakeholder collaborative activities based on the scores in the workshop evaluations. One possible explanation for this discrepancy may be that the items used in the questionnaire referred to the students’ perceived ability to work with the stakeholders during their current project. Because success builds a strong sense of self-efficacy (Van Dinther et al., 2011), the scores may have been different had we asked the students if working in an RLE influenced their self-efficacy for working with stakeholders in a hypothetical new project after they finished their ongoing projects.

7.2 Student-stakeholder collaborative activities

The finding that students in condition 2 reported more done student-stakeholder collaborative activities, as well as the finding that these students reported more stakeholder activities than non-stakeholder activities, supports our hypothesis and suggests that the students who participated in both workshops became more open to stakeholder collaboration compared to the students who participated in no workshop (condition 0) or only one workshop (condition 1). This effect cannot be attributed to differences in the students’ initial level of stakeholder awareness, as we controlled for this. Nor can it be attributed to differences in self-efficacy (Section 7.1). We suggest that the reflection and transformation activities in workshop 2 specifically stimulated students’ awareness of the crucial importance of collaborating with stakeholders, more so than the identification and coordination activities in workshop 1.

7.3 Students’ boundary crossing learning

The increasing number of excerpts referring to boundary crossing learning processes instead of to other learning processes from condition 0 through condition 2 indicates that the workshops increasingly engaged the students into boundary crossing processes. This supports our hypothesis that offering more boundary crossing–based workshops would likely improve boundary crossing learning. Nevertheless, the total amount of excerpts referring to boundary crossing was relatively low compared to the number of excerpts referring to all types of other learning processes. This result can be attributed in part to the fact that we did not explicitly ask the students to refer to boundary crossing in their learning reports; moreover, numerous influential project management–related issues appeared to come to the students’ minds when writing their reports. The relatively low number of references to boundary crossing may also have been caused by the small size of the intervention and the related limited involvement of the RLE teachers in the workshops (Section 7.4).

Little difference was found between conditions with respect to the students’ adoption of identification and coordination learning mechanisms. This lack of difference between conditions 1 and 2 could be attributed to the fact that both groups of students were supported equally in terms of conducting identification and coordination activities (i.e. both groups engaged in workshop 1 activities). However, the lack of difference between condition 0 on the one hand and condition 1 and 2 on the other hand, implicates that this study does not allow to determine any intended effects specific to workshop 1.

In all three conditions, more than half of the excerpts for identification referred to the sub-process exploring mutual expectations. On the one hand, this is a positive finding, as expectation management influences the whole RLE process (Oonk et al., 2019). On the other hand, reviewing the contents of the excerpts for this sub-process reveals the novice level of the students, in that they wanted to clarify everything at the start of the project and were somewhat ignorant of the high levels of uncertainty and constant changes in these types of complex projects in real life (Scholz and Steiner, 2015). This challenges the design of workshop 1. Workshop 1 should support the students to find a balance between discussing mutual expectations with the partners involved and leaving room for necessary adjustments of those expectations during an RLE learning trajectory that is full of unexpected “learning surprises” (Scardamalia et al., 2012).

When referring to coordination, the students in condition 2 reported considerably more efforts of translation, i.e. efforts to understand the (languages used by the) stakeholders. We consider this a side effect of workshop 2, in which the students extensively reflected on their collaboration and – often challenging – communication with the stakeholders and discussed follow-up actions to improve their communication.

Workshop 1 – as well as the series consisting of workshops 1 and 2 – seemed to increasingly trigger reflection, although this learning mechanism was not specifically addressed in workshop 1. This finding implies that workshop support stimulates reflective mechanisms anyway. The relatively high number of excerpts referring to the reflection sub-processes perspective making and facilitating perspective making in condition 2 compared to condition 1 suggests that explicit reflective activities trigger these sub-processes. This is a promising finding with respect to the pre-conditional value of perspective making for transformation suggested by Cremers (2016: page 109).

Although the percentage of excerpts referring to transformation was four times higher in condition 2 than in condition 0 and condition 1, it was still too low to draw any meaningful conclusions regarding the effectiveness of the workshops at facilitating the ultimate transformative aim of the RLE. The low transformative awareness, intentions and activity in the RLE, relate to our observation that RLE projects – although “real-life” in nature – suffer from the “boundaries” that students feel between school and the real world. In most RLEs, the school is the sole – and final – party responsible for the assessment of students’ work and their study progress. Thus, students may not feel equally responsible for both their school projects and the outcome of real-world projects and their follow-ups. This can hinder transformation, for which improvement might be more a matter of institutional redesign of the RLE as of redesigning the workshops.

7.4 Implications for the educational design of multi-stakeholder learning environments

The proven effect of workshop-based support of student-stakeholder collaboration informs the future pedagogical and didactical design of the RLE, as well as similar authentic learning environments used in higher education programmes in which students are expected to actively collaborate with and learn from multiple stakeholders (Gerholz et al., 2018; Jacoby, 2014; Long, 2012). Intervening in the students’ learning processes using these types of boundary crossing workshops appears to be useful in helping make students aware of the mechanisms that underlie working across boundaries, thereby optimising students’ collaborative practices. In addition, explicating boundary crossing and its four learning mechanisms in student-stakeholder collaborative projects appears to enrich essential reflection processes in multi-stakeholder learning environments (Jacoby, 2014; Sletto, 2010).

Our results suggest that a series of at least two workshops and addressing all four boundary crossing learning mechanisms is needed to trigger students’ learning and working with external stakeholders. But the fact that short courses and workshops can have little impact on student practices (Moon, 2001) is reflected in our results. The effects of our workshops were possibly even lower, given that the participating RLE teachers in most cases excused to be present during our guest workshops, while they had agreed in advance to be present. Consequently, the RLE teachers were unable to relate to and build upon the workshops in their coaching during the RLE projects, thus hampering the student-stakeholder processes to be continuously stimulated throughout the projects.

We recommend designing and investigating the effects of more substantial pedagogical support in boundary crossing learning environments. We suppose learning across boundaries will likely be more effective when explicated and supported throughout the entire multi-stakeholder project. This support should include the formulation of boundary crossing learning objectives, offering boundary crossing learning activities performed in parallel with the complete project process and tailored to each specific RLE project, as well as the respective formative and summative assessment activities (Gulikers and Oonk, 2019). Such parallel interventions facilitate scaffolding and ongoing reflection on both the learning and working processes. Scaffolding and reflection were previously reported to be critical elements in this type of experiential learning (Holmén et al., 2021; Webb and Burgin, 2009).

7.5 Boundary crossing and sustainability competence

Boundary crossing competence receives increasing attention as an asset for people’s capacity to work on sustainable development (Gulikers and Oonk, 2019). Boundary crossing caters for collaboration across practices of e.g. disciplines, cultures and university and societal partners. This is exactly what is needed to work on sustainability issues due to the “wicked” and multi-scaled character of these issues (Wals, 2014).

Based on a broad systematic review of sustainability competence frameworks, Wiek et al. (2011) capture which capacities people need to be able to contribute to sustainability issues into five key competencies (i.e. systems thinking, anticipatory, normative, strategic and interpersonal competence). A range of case studies have been reported illustrating examples of learning activities that could possibly stimulate higher education students’ sustainability competencies (Barth, 2014). However, more systematic guidelines on the “how” behind the sustainability competence development are yet scarce. The boundary crossing lens, including its performance focussed learning mechanisms, focusses exactly on this “how” question. Boundary crossing provides an overarching perspective and specific and practical tools on how to appreciate and use diversity in cross-practice collaborative challenges on our way to sustainability innovations. Applying boundary crossing learning mechanisms, at least like we did in our workshop interventions, is supposed to enhance the mastery of all five key sustainability competencies and as such promising on our way to educate sustainability “ambassadors”.

7.6 Implications for theory: exploring and operationalising boundary crossing learning

This study has two important methodological findings with respect to understanding learning and collaboration across boundaries. Firstly, the ability to dive into the actual adaptation of boundary crossing learning mechanisms – albeit self-perceived by the students in this study – enables further studies regarding boundary crossing learning in various contexts, e.g. more effective interventions or individual differences in boundary crossing behaviours.

This study reveals the possibility of collecting rich data regarding the occurrence of boundary crossing learning mechanisms and the related sub-processes in a learning environment that aims to facilitate learning across boundaries. We achieved this by collecting and triangulating three types of learning reports per respondent. The three open questions encouraged students to report on:

1. their actual learning outcomes;

2. the peaks and troughs experienced in their learning processes; and

3. their ability to transfer the learning outcomes to future learning opportunities.

The combined reports indeed appeared to reveal boundary crossing learning processes.

Secondly, the translation of theoretical boundary crossing concepts into a contextualised coding framework (Appendix) appeared to facilitate grasping the studied mechanisms.

Taken together, these two methodological findings respond to the call for more systematic empirical studies designed to uncover and operationalise the theoretical concepts of boundary crossing in working and learning across contexts (Akkerman and Bakker, 2011; Edwards, 2012).

7.7 Limitations of the study

Some limitations of the study may have influenced the results and warrant discussion.

Firstly, the students in RLE 1, 2 and 5 (Table 1) were not assigned randomly to the conditions, as the students voluntarily chose to participate in workshop 2. This may have caused a slight bias in the positive effects of condition 2, as these students may have been more eager to learn.

Secondly, the individual post-test questionnaires were completed in two different circumstances, either before or shortly after the students’ final RLE assessment. This may have influenced the students’ answers. However, to receive the most objective answers possible, we emphasised to the students that our measurements were completely unrelated to their RLE assessment.

Thirdly, the entire study was based on indirect measures of boundary crossing. Although the students could refer to boundary crossing in their reports on the open questions, we did not explicitly prompt them to talk about boundary crossing. Consequently, even if a student did not report boundary crossing activities, (s)he still could have experienced boundary crossing processes. Adding more direct measurements (e.g. observations or questions that explicitly address boundary crossing) may have strengthened the study.

Finally, the study only used data regarding the students’ learning processes, not regarding the learning results of the complete RLE. Large differences in assessment strategies between the RLEs precluded our ability to use the learning results as a comparable measure.

8. Conclusion and future prospects

This study elucidates that pedagogical support can stimulate boundary crossing learning processes among higher education students in a multi-stakeholder sustainability learning environment. Based on our results, we draw two main conclusions. Firstly, a series consisting of at least two boundary crossing workshops regarding student-stakeholder collaboration as part of a multi-stakeholder project stimulates the amount of student reported stakeholder collaborative activities and the adoption of the boundary crossing learning mechanisms reflection and transformation. Secondly, it is possible to operationalise boundary crossing theory into educational activities and an analytical frame that facilitates the exposure of students’ boundary crossing learning in a multi-stakeholder learning environment. These findings can guide the pedagogical design of “boundary crossing” learning environments aimed at sustainable development. More generally, these findings add to the body of knowledge on how to stimulate learning across practices.

The results of this study incite at least two main directions for future research. Firstly, relating learning processes to learning outcomes of “boundary crossing learning environments” would enable researchers to investigate if and how students’ stakeholder collaborative – boundary crossing – activities affect their learning outcomes, for example, the results of the students’ project or their development of boundary crossing competence. Engeström et al. (1995) noted that when the learner is part of different practices simultaneously, the learner combines ingredients from those different practices to achieve hybrid solutions. This study shows that students rarely achieved transformation during their projects, i.e. the intended hybrid result of the RLE that was meant to contribute to sustainable development. Future research should pay attention to what is needed to stimulate transformative results among higher education students both at the personal and project levels, as well as how to measure these results. Our ultimate goal is to support students in their development into “transformation agents” and the students barely showed that they were already acting on that.

A second direction for future research is the professional development of both teachers and stakeholders, who also require support to work effectively in university-community learning environments (Brundiers et al., 2013). A focus group discussion on effective student-stakeholder support, held with 14 RLE teachers after the closure of the eight studied RLEs (February 2016), revealed that the teachers were occupied primarily with the organisational aspects of the school-stakeholder collaboration, with limited urgency and openness regarding further development of actual student-stakeholder support. Moreover, our observations regarding RLE practices reveal stakeholders’ concerns with respect to how they can optimally collaborate with students. All RLE partners need to become more aware of, open to and equipped with the capabilities needed for boundary crossing learning towards transformation. As one student in condition 2 summarised perfectly regarding our final goal in attempting to effectively support boundary crossing working and learning in multi-stakeholder learning environments: “People should definitely cross boundaries to really transform the region. I’m not afraid to cross boundaries and collaborate with the unknowns, since I now feel skilled to do so.”

Table 1.

General characteristics of the studied RLEs

	RLE 1. Planning school A 2014/2015 n = 17	RLE 2. Planning school B 2014/2015 n = 11	RLE 3. Planning school C 2014/2015 n = 12	RLE 4. Planning school D 2014/2015 n = 12	RLE 5. Planning school E 2014/2015 n = 25	RLE 6. Planning school A 2015/2016 n = 18	RLE 7. Planning school E 2015/2016 n = 21	RLE 8. Planning school B 2015/2016 n = 6
General characteristics
Study load	20 weeks fulltime	20 weeks fulltime	8 weeks fulltime	9 weeks fulltime	20 weeks fulltime	20 weeks fulltime	20 weeks fulltime	20 weeks fulltime
# projects within RLE	4	3	1; divided into four sub-tasks	1; divided into three sub-tasks	7	4	6	2
Student group size	4–5	3–4	3–4	4–5	4–5	4–5	4	3
Students’ mean age (M) and (SD)	M = 21.65 SD = 2.94	M = 21.55 SD = 1.44	M = 20.67 SD = 1.07	M = 21.17 SD = 1.75	M = 21.3 SD = 1.94	M = 22.5 SD = 2.71	M = 21.0 SD = 2.18	M = 22.67 SD = 2.58
Students’ gender [# (%)]	♂ 12 (71%) ♀ 5 (29%)	♂ 6 (55%) ♀ 5 (45%)	♂ 10 (67%) ♀ 5 (33%)	♂ 9 (69%) ♀ 4 (31%)	♂ 24 (71%) ♀ 10 (29%)	♂ 16 (89%) ♀ 2 (11%)	♂ 16 (70%) ♀ 7 (30%)	♂ 5 (83%) ♀ 1 (17%)
Conditions
No workshop	n = 1	n = 1	n = 1	–	–	–	n = 21	n = 6
Workshop 1	n = 9	n = 2	n = 1	n = 2	n = 19	n = 3	–	–
Workshops 1 and 2	n = 7	n = 8	n = 10	n = 10	n = 6	n = 15	–	–

Table 2.

Number of reported student-stakeholder collaborative activities as a percentage of all reported excerpts per condition and of all reported activities

	No workshop	Workshop 1	Workshops 1 and 2
Total #excerpts = 1,876	# excerpts = 481	# excerpts = 546	# excerpts = 849
% of total #excerpts referring to:
Stakeholder collaborative Activity_DONE	15	16	25
Stakeholder collaborative Activity_INTENDED in the FUTURE	10	12	15
Other activity	53	55	43
Not referring to an activity	22	17	17
Total	100	100	100
Stakeholder collaborative activity as a percentage of total percentage of activities	32	34	48

Table 3.

Percentage of excerpts referring to boundary crossing learning mechanisms (Akkerman and Bakker, 2011) and to other learning processes (including percentages for distinguished subprocesses per learning mechanism)

	No workshop	Workshop 1	Workshops 1 and 2
Total # excerpts = 1,876	# excerpts = 481	# excerpts = 546	# excerpts = 849
Identification (I)	12.47	12.64	10.13
I_Knowing yourself	0	7.25	0
I_Knowing the stakeholders	13.3	13.04	16.28
I_Knowing other perspectives	23.3	20.29	18.60
I_Clarifying complementarity	10	5.80	1.16
I_Exploring mutual expectations	53.3	53.62	63.95
Coordination (C)	14.55	15.38	15.78
C_Contacting for connection	15.71	23.81	11.94
C_Organising collaboration	24.29	33.33	21.64
C_Using a boundary object	30	29.76	37.31
C_Translating	7.14	5.95	15.67
C_Controlling agreements	22.86	7.14	13.43
Reflection (R)	5.82	10.62	18.50
R_Recognising others	28.57	37.93	15.92
R_Learning from another	46.43	37.93	37.58
R_Perspective making	14.29	10.34	29.30
R_Facilitating perspective making	10.71	12.07	17.20
R_Mutual learning	0	1.72	0
Transformation (T)	1.46	1.47	4.12
T_Intending creation of new practice	14.29	25	5.71
T_Envisioning new practices	0	12.5	22.86
T_Establishing new practices	0	12.5	5.71
T_Enthusing others for a new practice	71.43	25	40
T_Stimulating follow-ups	14.29	25	25.71
Other learning processes (O)	65.70	59.89	51.47
O_Project management	39.56	50.15	53.78
O_Personal development	6.96	13.15	8.24
O_Multidisciplinary group work	0.32	1.83	0
O_Remaining learning processes (referring to e.g. project assignment, methods and supervision)	53.16	34.86	37.98

Table A1.

Boundary crossing coding scheme contextualised in the multi-stakeholder RLE

Learning mechanism and associated characteristic sub-process	Description of the process Illustrative example (Experimental Condition as derived from [0,1 or 2 Workshops])
Identification (I)
I_Knowing Yourself	Knowing and/or explicating the own expertise and its limitations The fact that our group consisted of four agricultural students also contributed to our agricultural project assignment (1)
I_Knowing the Stakeholders	Identifying which stakeholders are relevant in the light of the project assignment Already at the first day of the project, you should identify the stakeholders (0)
I_Knowing Other Perspectives	Identifying stakeholders’ knowledge, interests, perspectives and mutual relations Make sure that you know the interests of each actor and the relationships between the actors (1)
I_Clarifying Complementarity	Clarifying the complementarity of and/or boundaries between the own and others’ possible contributions (i.e. knowledge, skills, etc.) Sometimes you need a lot of parties to realise the intended result (0)
I_Exploring Mutual Expectations	Exploring and tuning mutual expectations at the beginning of a project The client, the teachers and we; we all had different ideas on what and how to do it. We had to align the ideas (1)
Coordination (C)
C_Contacting for Connection	Contacting stakeholders The client, the students and the teacher coach should frequently be in contact (1)
C_Collaboration	Collaborating (including talking) with the stakeholders I learned how to involve citizens in shaping the project (2)
C_Using a Boundary Object	Using and/or explicating the importance and use of a boundary object (including the organisation of activities that facilitate collaboration, e.g. a brainstorm session or a design workshop with village inhabitants) As part of our study, we organised a brainstorm session for the village residents (1)
C_Translating	Understanding the (languages of the) stakeholder The communication was tough; we did not understand the intentions of the client (1)
C_Controlling Agreements	Controlling working agreements on mutual tasks The client and we should have kept each other better informed (2)
Reflection (R)
R_Recognizing Others	Recognizing and making explicit the characteristics of another person or practice There are different levels in the concept of a “hippy”. Some people go as far as they can in ecology and sustainability, others go less far (1)
R_Learning from Another	Explicating something learned from another person or practice Nothing is possible, particularly innovative thinking. They have to defend themselves to the local council, so their word is law (1)
R_Perspective Making	Reconsidering the own perspective as a result of something learned from another person or practice Then we unplugged because the clients were nothing more than negative and did not communicate; they chose for another line of reasoning (2)
R_Facilitating Perspective Making	Facilitating others’ learning and/or perspective making I learned how to open up the clients’ mind for other peoples’ ideas (2)
R_Mutual Learning	Explicating mutual learning The assignment was clear for both sides and so far fine for both parties (1)
Transformation (T)
T_Intending the Creation of a New Practice	Expressing the intention to create a new practice Next time I would try to establish a real connection with the project area (1)
T_Envisioning New Practices	Describing visions on new practices Due to using the local village park as an impetus, it appeared that there were a lot of possibilities to renew the village (2)
T_Establishing New Practices	Integrating interests and perspectives into new realistic practices and establishing these practices Apparently, we opened up a lot of possibilities in the area and brought together various parties (2)
T_Enthusing Others for A New Practice	Enthusing stakeholders for a suggested new practice I learned how to enthuse neighbourhood citizens (0)
T_Stimulating Follow-Ups	Inciting and stimulating follow-ups for the new practice During the project, you already need to make sure that after the project something will really be realised (1)
Other Learning Processes (O)
O_Project Management	Processing and managing the student group process You should make a planning and distribute tasks (2)
O_Personal Development	Learning for personal development I learned to take more initiatives from myself (0)
O_Multidisciplinary Group Work	Explicating something learned from another students’ discipline Because of our different backgrounds, we [students] sometimes clashed (2)
O_Remaining Learning Processes (Referring to e.g. project assignment, methods and supervision)	Remaining other learning processes not to be coded another “Other Learning Process” The teacher changed the assignment and because of that everything became very unclear (0)