This task focused on analysing the project within the framework of Patton’s developmental evaluation (2011).
Developmental evaluation, as outlined by Patton (2011), really resonated with me as an evaluation strategy. We spend so much time in higher education approaching evaluation as a summative activity, to prove worthiness or compliance or outcomes to whatever leadership or funding entity requires it, and focusing on rapid strategic implementation with visible outputs, that we miss opportunities for much deeper more meaningful evaluation to inform practice.
It also strikes me that developmental evaluation is a process that I have already been following, without being explicitly aware of it as such. Throughout this research learning program, the concept of the ‘innovation’, the ‘thing’ that I will actually implement, has shifted significantly over time and still is not fully crystallised. I have continuously been evaluating my initial proposal as I have waded further into literature bodies, expanded my knowledge and capabilities and developed a deeper understanding of the organisational context. It is clear that the intervention I originally proposed – a framework for the intentional design of culture – is not a good fit for UNE’s current context and point of readiness, and it is likely that my intervention design will shift even further in light of the upcoming needs analysis exercise.
This project, which for now I will broadly categorise as ‘an intervention towards organisational learning, culture and change capability’, fits in a number of Patton’s developmental evaluation project categories, depending on the perspective taken.
This category is probably the most natural fit for the project, particularly in light of the action research spiral. Firstly, there is no single model, or “right thing” to implement, and there are likely to be many iterations of an intervention (as there have been throughout this research learning program, where ongoing evaluation in light of my research has taken me away from the original proposal into more murky waters). Secondly, the organisational context I am working in is dynamic, complex and constantly evolving – a rigid, fixed intervention model is likely to be inappropriate in this context.
Developmental evaluation provides a framework for undertaking development and evaluation simultaneously with implementation – effectively, ‘working on the business while you’re in the business’, which is an ongoing wicked problem in higher education.
Adapting effective general principles to a new context
This category speaks most strongly to the transdisciplinary nature of this project. In assembling a bricolage of practical strategies that might inform or support an intervention towards organisational learning, I have taken principles from a diverse range of contexts – organisational learning and culture, pedagogy and individual learning, psychology, design thinking, change management, leadership, systems thinking, complexity theory, coaching, storytelling, music and even the restaurant industry, and worked to create meaningful translations of these into the higher education (and more specifically UNE) context.
This category is also where my explorations of the concept of an organisational learning designer might land, drawing in the principles that inform learning design as a profession and the principles that inform organisational learning as a discipline, and exploring what might emerge between the two.
Developing a rapid response to a crisis
Depending on your perspective, there are two “crises” that one could potentially argue are currently facing UNE that require a rapid response vis a vis organisational learning, culture and change capability:
- A new Vice Chancellor with a dynamic and action-focused approach to strategy
- The ongoing sector narratives regarding the sustainability of universities and calls for significant reform in structure and function Either or both of these could be considered crisis events and throw our organisational learning and change capability into sharp relief. Many challenges in this vein have already emerged much more acutely in recent months in response to proposed changes, and it is clear there is a need for a rapid response to help navigate the coming zeitgeist.
Preformative development of a scalable innovation
This category would apply if I take the approach of using a small prototype zone within the organisation (for instance, LaTT) as a means of keeping the PhDI project manageable, but that could then have a view to later being scaled up to an institution-wide initiative. Patton talks about the notion of ‘preformative’ as developing a model that becomes formal and is subject to formal formative and summative evaluation. There is potential for the end outcome of this project to be a formal intervention model that could then be formally implemented and evaluated, although in my current frame of reference this result does make me feel a little uncomfortable due to the disconnect between a fixed model and a dynamic, complex organisational context.
There are also synergies with Maxwell’s (2003) notion of reconnaissance within the action research frame, using developmental evaluation to develop a model for implementation.
Major systems change and cross-scale developmental evaluation
This category is probably the least applicable to my particular project, although there is certainly room for the argument to be made that, in a context where there has not been a deliberate and intentional approach to developing organisational learning, culture and change capability previously in the organisation, and in a solidly Mode 1/single loop institutional context, addressing organisation learning with a Mode 2/double and triple loop approach is major systems change. Even at a very small level, the idea that we might reconsider, for instance, the language we use to speak to each other, or the mental models baked into the documentation we create, has implications for radical change.
In terms of how developmental evaluation intersects with my other research frames, there is a very natural fit for it within action research, and I’ve already touched on its synergies with transdisciplinary research.
Being acutely oriented towards action, developmental evaluation fits very naturally both within the traditional action research spiral – one could probably argue that the action research spiral is developmental evaluation – and Maxwell’s (2003) modified action research framework, where the reconnaissance phase links with the notion of preformative development.
Aside from the bricolage approach to gathering principles from diverse places outlined above, developmental evaluation also serves the function of doing transdisciplinary research in a Mode 1 institution. The nature of developmental evaluation allows space for the double and triple loop learning around navigating paradigms, mental models and demonstration of worth in a sometimes sceptical (and sometimes outright hostile) Mode 1 context.
Patton, M.Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: The Guilford Press.
Maxwell, T.W. (2003). ‘Action Research for Bhutan?’, Rabsel III, 1-20.