Learning that Lasts? The Alverno Model and Evaluating Graduate Outcomes

How can we be certain that students are integrating their different learning experiences at University and, furthermore, that they are developing the capacity to apply that learning to the wide range of different contexts that they may experience after they graduate?

This is a question that has bearing on what has become a perennial pressure on Higher Education in the UK, most recently restated in the 2016 White Paper, to embed employability and other generic skills development into disciplinary programmes.   While most people working in the HE sector recognise the importance of transferable skills and ensuring students are equipped on graduation to succeed in life, many are also conscious of tensions at work in meeting the expectations of an employability agenda, particularly where there is a perceived risk of skills training usurping disciplinary academic standards, or of economic instrumentalism undermining traditional HE values. It is one that also relates in part to an issue that emerged in the SoTL Network meeting held last March when we looked at student engagement; particularly to the claims made in the research for the need to consider the entire nexus of students’ academic, personal and social experiences. Our conclusion at the time was that many of these areas of experience were outside of our sphere of influence as educators and that we could only realistically concentrate on supporting students’ academic development.

But what if this is not necessarily the case? What if there was already a HE institution running well-established, highly innovative programmes that aim to develop and assess academic knowledge,  generic skills, and even some of the more ineffable learning outcomes implicit in the HE “student experience” –  such as civic virtue and reflection on personal moral values – in a wholly integrated fashion? What if this institution had a substantial bank of longitudinal evidence spanning almost four decades affirming the effective transferability of these learning outcomes to students’ professional and personal development after graduation? Such is claim and reputation, supported by numerous prestigious awards and accolades for innovation in Higher Education, of Alverno College based in Milwaukee, US.

In principle, the US Liberal Arts curriculum that Alverno has adapted is already premised on integrating disciplinary knowledge with a wider range of skills. For example, whether a student is studying Maths, History or Nursing as their major discipline, they would still be required to demonstrate written, verbal, digital and quantitative literacy developed in modules threaded through their major programme. In practice, however, many in the US, and indeed many academics in the UK delivering programmes with a tighter disciplinary focus, question whether students are genuinely integrating the knowledge and skills developed from one module to the next across a whole programme.  Some would even question whether students are truly capable of adapting and applying what they have learned at University to different contexts after graduation. The York pedagogy has evolved as one response to this issue at our institution, by aiming to ensure the design of programmes and student work progressively supports learning across the entire 3-4 years of study, while making it clear to students what their development should look like through the provision of a programme map, and what they should be able to do upon graduation through the clear articulation of Programme Learning Outcomes

For the next session we have chosen two very short papers (4-5 pages each) that establish some context and a broad outline of the Alverno model (Hakel 1997 and Mullin 2001) and one of the College’s own longitudinal studies of graduate outcomes considered in professional, personal and civic contexts (Rogers & Mentkowski, 2004).

We suggest priority be given to reading Hakel (1997) and the Alverno research (2004), although it would also be useful to at least read through the table on p57 in Mullin (2001).

Possible questions for consideration:

  • Hakel identifies typical objections to the implementation he encounters from colleagues when they hear of the Alverno model. What is your take on these, particularly in the light of the Alverno research on graduate outcomes (Rogers and Mentkowski 2014)?Are there components of the Alverno model that could be applied to the York context and/or within your department’s programmes? The framing questions on the penultimate page of Hakel (1997) provide some orientation for reflecting on this question.
  • Rogers and Mentkowski (2004) claim in their conclusion that one of the objectives of Higher Education is to show how a “good society is conceived.” How does this mesh with the overall emphasis in their study on alumna performance outcomes – not just in professional contexts, but also in civic and personal roles? In what ways might this perspective inform UK debates on supporting student employability and the criticism that this objective is driven by instrumental concerns at odds with HE values and traditions?
  • Do you find the evidence in the Alverno research and methodology convincing as a means of capturing learning gain after graduation? What lessons could be learned from this approach to inform debates on evaluating the effectiveness of HE programmes in the UK in developing graduate abilities, knowledge and skills?

What do you think of Mullin’s (2001) claim that the reason educational reform tends not to achieve its objectives is because it is usually implemented incrementally rather than totally?


Meeting Report

Alverno College, in the eyes of enthusiasts such as Milton Hakel (1997)– one of the key readings for this session – is almost idyllic in its fulfilment of a higher education provision.  It is represented as having found a framework for learning (albeit one that is subject to ongoing reflection and development) that engages students from all backgrounds, allowing them to emerge with a set of skills and abilities that not only embody individual maturation and disciplinary expertise, but also provide an invaluable profile for success beyond higher education.  Against such unequivocal praise, the focus of this SoTL Network discussion revolved around whether lessons learned from the specific context of Alverno can be applied to the UK HE system and whether, actually, the evidence posited for the ongoing impact of the Alverno experience in individuals’ personal and professional lives post-graduation stands up to scrutiny.

But it’s different isn’t it …?

Alverno’s approach was immediately recognised as being ‘different’ – something that does not exist within a UK context.  It was equally swiftly noted that the context within which Alverno operates is fundamentally different as well, perhaps in ways that are too intractable to allow their model to be applied in the UK.  In particular, in the US there is an existing  tradition of central departments providing support for the achievement of co-curricular outcomes – knowledge-based but also intra- and inter-personal based; Alverno is a liberal arts college and its liberal arts curriculum – which perhaps lends itself better to a focus on non-subject specific learning outcomes – is not a feature of contemporary UK higher education; Alverno is a Catholic college and the explicit values which underpin the approach must be understood in that context, which may be very different from the explicit values expressed in non-Catholic institutions; Alverno is small – only slightly bigger, really, than some big departments in UK universities  – and scaling up the approach would be unfeasible.  A critique of these responses might return to Hakel’s article, and in particular the reactions he observed on reporting about Alverno to colleagues back in his home institution.  In other words, do such responses represent an almost reflexive reaction, perhaps one grounded in the almost seismic rethinking that would be required to actually apply the Alverno model in any meaningful way to our own practice?  Perhaps, but such a reaction is not necessarily unwarranted, particularly if, as one of the other articles for the session implies, it is precisely because change and development in Higher Education reform tends to be incremental, as opposed to fundamental and substantial, that beneficial impact is rarely seen (Mullin 2001).

Don’t we already do it?

In response to the discussion around the applicability of the Alverno model, it was suggested that, actually, many UK institutions arguably take similar approaches, or at least share this way of thinking, via the identification of graduate attributes.  It is perhaps fair to say that the surge of attention on graduate attributes over the last 15 years or so is reflective of a shift in the way that higher education has been conceived of in the UK and this is  itself no doubt a result of the bow-wave left in the wake of the Dearing Report (1997).  This shift in thinking, tied to massification, is that universities do not simply have a purpose by virtue of their existence as educational establishments; they are not wholly about learning for learning’s sake. In fact a key purpose of higher education is to prepare young people to be positive members of society, not least in the realm or work and economic productivity.  The problem, as was pointed out in the discussion, is that the reach of the graduate attributes agenda has often failed to extend beyond high-level institutional strap-line documents – perhaps because the shift in thinking towards employment is, not insignificantly, the result of top-down Government policy.  There is certainly little evidence to suggest that any institution in the UK shares the fundamental embedding of lifelong learning skills and attributes in the curricula as manifested in the Alverno model.  When employability and graduate attributes are detectable within subject curricula, they tend to relate to more vocational disciplines. Indeed, it was a point of discussion as to whether Alverno’s approach, if it does have traction in the UK, is perhaps more suited to vocational and professional subjects and, even more specifically, vocational and professional subjects delivered at Masters level.  This does raise the question, perhaps, of whether all learning that occurs at a higher education level needs to be formally identified within mainstream curricula.  It certainly does not necessarily have to be, as there can be many wider learning benefits to attending university: academic, personal, and professional. However, the essential problem is then whether we wish to measure such learning and, if so, how this can be possible in any systematic way outside of optional, non-credit programmes. The York Award was one notable example of an effective extra-curricular scheme, which prompted a discussion of whether it would be feasible  –  or even desirable –  to interweave this kind of scheme into disciplinary curricula, perhaps coordinated by colleges. It was noted, for example, that when programmes aspire to doing so, they often risk overloading the curriculum  – with all the problems, in terms both of delivery resource and academic coherence, that can ensue.

King Discipline

Drawing back to the liberal arts curriculum, an important point raised during the group’s discussion was that the Alverno approach makes the discipline subservient; the discipline is simply the vehicle through which students achieve the abilities that form the backbone of the Alverno curriculum and these abilities are non-discipline specific.  They are not even general abilities couched in disciplinary terms.  In many ways, this is perhaps one of the most alien aspects of the Alverno model vis-à-vis UK higher education where the discipline is central. It is, in fact, difficult to conceive of a situation where disciplinary outcomes are, in effect, done away with.  In this context, the York Pedagogy is perhaps interesting in that it hones down on a tight set of specific outcomes characterised by ‘doing’ rather than ‘knowing’ – very much aligned to the Alverno philosophy – but allows these to be couched in disciplinary terms. In some ways, it therefore appears to provide an adapted vision of some of Alverno’s thinking. (Robinson 2015) Certainly, there was some sympathy in the group for the argument that knowledge, per se, is often transient in terms of currency.  Of course, the test of the new approach will obviously  be the extent to which the thinking is transposed into tangible practice and to what extent. Questions were  raised, for example, about whether students will be entirely comfortable with a concrete move away from ‘knowing’ to ‘doing,’ and whether it is more problematic to be told that you can’t do something than it is to be told you don’t know something.

The Art of Assessment

An interesting characteristic of the Alverno model concerns assessment.  UK higher education is perhaps familiar with a shift from assessment of learning to assessment for learning, but as one member of the group highlighted, Alverno emphasises assessment as learning and perhaps this represents an additional shift which is worthy of consideration.  The focus on assessment chimes with earlier SoTLN discussions looking at Carol Dweck’s work and focusing on failure as a productive exercise in learning and whether the UK Higher Education system, with a focus on modularised assessment, really allows and enables this.  The issue around failure and how failure is perceived is captured by Hakel:

One political reality of our time is that failure is intolerable, that those who fail should be investigated, humiliated in the media, and punished.  Yet the reality of all times is that eventual failure is inevitable … we need to learn how to fail intelligently and that doing so quickly gives us the chance to improve what we are doing and to adapt to changing circumstances.  Assessment is part of the answer – we cannot learn how to improve without it … (Hakel 1997)

Assessment as learning in the Alverno model means making students’ experience of undertaking assessment, receiving feedback, reflecting on assessment and feedback, and adapting (learning) in response to feedback so routine, so part of everyday learning and teaching, that the term ‘assessment’ loses its ‘fear factor.’  It turns assessment from an experience of measurement, judgement, failure, and inter-student competitive behaviour, to one that equates to learning.  Assessment, rather than being associated with fixed points of stressful time, is normalised as an ongoing, productive, and everyday process.  There was some scepticism within the group as to whether inter-student competition was genuinely removed, and also what the impact of non-grade orientated summative assessment has for a world characterised by league tables and institutional competition; however, assessment as learning is an interesting lens through which to view our practice.  Of course, a recurrent issue when considering how to make undergraduate education more effective is the extent to which pre-university education conditioning can be broken down to enable more productive learning (in the event that change appears to jar against the expectations that students will have from the secondary experience).  Many might argue that changes in deep conceptual approaches require stimulus further down the curriculum and cannot simply be introduced at tertiary level.  Then again, and presumably, this challenge has been no less encountered, and potentially overcome, at Alverno.

The susceptibility of ‘evidence’

An important strength underpinning the proposed merits of the Alverno model, and highlighted by the Rogers and Mentkowski paper (2004), is that its impact has been studied through longitudinal research. This research purports to show that the Alverno curriculum has had a demonstrable impact upon the actions and attitudes of graduates in terms of their engagement with employment and in terms of their personal and domestic lives. The group spent some time critiquing this research. In particular, there was a discussion around whether a causal link between educational outcomes and employment outcomes is actually identifiable. The studies involved coding workplace behaviour using pre-existing frameworks and asking academic staff to rate this behaviour according to the outcomes expected at Alverno. Because the research wholly focuses upon Alverno graduates, however, it is difficult to determine whether the actions and attitudes of such individuals in the workplace are a symptom of the Alverno experience per se or whether (or to what extent) it reflects other influences. The problem here is that shifting the context from Alverno may need the development of an alternative codebook to reflect the values of other higher education providers, which in turn makes comparative analysis suspect.  What can be said, perhaps, is that Alverno graduates do communicate in the language of the Alverno curriculum when reflecting upon their work and their engagement with society.  This could be identified as evidence for some form of impact, but it is difficult to quantify or qualify.

Conclusion

Overall, the Alverno model presents a thought-provoking, and potentially useful, case study for thinking differently about curriculum design. The overall sense from the group was that it is unlikely to be a model that can be transported wholesale into the UK system but, perhaps (and bearing in mind the arguments of Mullin), it is possible that some valuable insight for development can be gained. It is also the case that the York Pedagogy chimes with many of the concepts underpinning the Alverno model and that there may therefore be potential for evolving our framework through considering the experience of, and taking from the lessons learnt by, Alverno College.


Works Cited

Dearing, R. (1997) Higher Education in the Learning Society.  National Committee of Inquiry in Higher Education Report 1.  London: HMSO.

Hakel, M. (1997) What we must learn from Alverno, About Campus July/August.

Mullin, R. (2001) The undergraduate revolution: change in the system or give incrementalism another 30 years?  Change 33: 5, 54-58.

Robinson, J (2015) The York Pedagogy: What and why, and how and why. Forum. 39. Autumn.

Rogers, G. and Mentkowski, M. (2004) Abilities that distinguish the effectiveness of five-year alumna performances across work, family and civic roles: a higher education validation, Higher Education Research and Development 23:3, 347-374.

Copies of the texts are available in the SoTLN Paperpile folder for Alverno.

Advertisements

One thought on “Learning that Lasts? The Alverno Model and Evaluating Graduate Outcomes

  1. Pingback: Making Life Difficult: Testing to Improve Learning | Scholarship of Teaching and Learning (SoTL) Network

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s