Remembering My Hat

22nd June 2016

Learning Design categories – a list of ideas

This is one of my posts that will probably make no sense at all to people beyond the OU, so apologies if that is you. But for those who are at the OU, and especially those who are academics involved in the production of our materials…

7473098586_de920c600a_z

(cc) Rain Rabbit

 

I run a group within the HSC department on production for academics new(ish) to the OU. We were talking recently about working with the Learning Design categories, and decided it would be useful to try to generate a list between us of different ways in which you could design activities of each type. This isn’t exhaustive, nor definitive, and we make no promises that these are always good suggestions – some of them would have to be done very carefully to to get over the bar of ‘but why on earth would students actually bother to do this?’. But we hope it’s useful to other people scratching their heads to think of non-assimilative activities (although we did include those too).

Assimilative

  • Readings – academic and more everyday kinds of texts
  • Audio
  • Video
  • Poetry
  • Maps and infographics based on maps
  • Images and artwork
  • Newspaper headlines
  • Personal stories
  • Case studies
  • Diagrams, inforgrahpics and graphs

Productive

  • Filling in a grid (gives more structure than free text ‘take notes’)
  • Numerical calculations
  • Make a powerpoint or other presentation
  • Do an elevator pitch
  • Draw a spider diagram or concept map
  • Write a briefing for a named audience
  • Write a tweet or headline
  • Write a blog entry
  • List of key points
  • Use the existing sticky notes tool on the VLE
  • Diagram which you can write on or manipulate or put sticky notes on
  • Make some notes (boring!)
  • Precis activities (e.g. rewrite in your own words, not more than 200 words)
  • Take a photo
  • Caption competition or cartoon bubble filling
  • Curating a collection of images or something else
  • Highlighting parts of text (highlighter tool in Word or offline versions)

Finding and Handling information

  • USE THE LIBRARY’S EXISTING TUTORIALS ON Digital Information Literacy
  • Access databases and other data sources and then extract some information
  • Finding a journal article or book from a catalogue
  • Doing a citation search
  • Following up a reference of your choice from a set reading
  • Generate your own data (avoid anything that’s close to interviewing people because of research ethics!)
  • Finding and evaluating infographics
  • Working with graphs and other pictorial data

 Communicative

Experiential

  • THIS ONE IS HARD TO DO and we were least happy about the definition of this one
  • Reflective activities
  • Trying out a productive output on someone you know and getting feedback on it.
  • Trying an activity on yourself e.g. relaxation techniques, you could even include a pre and post test.

Interactive/adaptive

  • Drag and drop where it bounces back if incorrect
  • Quizzes with feedback on incorrect answers
  • Choose between two positions on a complex (often ethical) issue, feedback says ‘that’s valid, but have you also thought about …’ and then summarises the arguments for the opposite position.
  • Games and simulations (very time consuming to develop though)

 

What have we missed? Please do suggest more. And of course let us know if you think we’ve got anything completely wrong.

Advertisements

18th June 2015

Module Workload Workshop

Encouraged by the positive responses on Twitter to my liveblogging of the Queer Kinship conference, here are some notes from a workshop I attended this morning at the OU about managing student workload. Written in slightly fuller sentences, but still a quick and partial account based on what interested me and what was said in my group, rather than aiming to be representational of the whole workshop, still less the OU position. And most likely to be of interest to other educators, especially OU-types.

The workshop was aiming to address these questions:

  1. What factors impact on perceived student workload and how can we manage these to effectively support our students in keeping up with their workload?
  2. How do we effectively manage student expectations around workload?
  3. What elements should be included in IET guidance to module and production teams around managing student workload?
  4. How do we effectively categorise study texts in terms of ‘easy’, ‘medium’ or ‘difficult’ and how do we ensure module teams adopt consistent norms in relation to these?
  5. What should student-directed learning look like and how do we support students in effectively engaging with it

It’s clear that lower workload increases retention (I think the statistic cited was that you get about 4% drop-out per extra hour of content per week) but how low can you go and still deliver graduate-ness by the end? We at the OU still work with the QAA norm of 600 hours per 60 points of credit, but some other HE institutions are moving towards 480 hours. But it’s not always clear how that relates to student-directed study v. module-directed activity.

It is also clear that consistency of workload helps retention. [I’d come across this statistic before and we’d wondered whether that meant you shouldn’t have break weeks, but the people I spoke to today said that this wasn’t the case. Phew.]

What factors impact on perceived student workload and how can we manage these to effectively support our students in keeping up with their workload?

Perceived v. objective is an important distinction, but perceived is very important.

Materials designed to be supportive but non-core (such as guides to referencing, assessment guidance) can add to perception of workload. The trick is to be clear about what it core and what is ancillary or just for support if needed.

Anxiety, stress and feeling of lack-of-control all contribute to perceived workload. Collaborative working can be especially problematic in this respect. So it’s really important to make clear to students the benefits they gain from collaborative working.

Things the module team thinks are easy can be surprisingly hard for students and add to workload unintentionally (e.g. getting to grips with eTMA submission for new students, taking a screenshot).

Referencing module materials! Especially when different modules have slightly different conventions due to having different types of materials.

2460302676_7f4e73aebb_z

This picture, from Normanack thanks, is actually relevant to this post, because a librarian in my group said that he had been asked by a student how to reference a seed packet. An excellent example of unplanned workload for students.

How do we effectively manage student expectations around workload?

At different points – pre-registration, before module start, early weeks, TMA preparation, EMA/exam preparation, for the whole qualification. Knowing what students’ expectations are is itself challenging. Helping them to be realistic without making them think they can’t succeed.

ALs have a crucial role to play here. Module teams can include activities encouraging students to plan out their workload. Peer accounts can be useful (like on the Moon MOOC, apparently, which has a real student popping up throughout). Being explicit about workload.

What elements should be included in IET guidance to module and production teams around managing student workload?

[nothing noted here]

How do we effectively categorise study texts in terms of ‘easy’, ‘medium’ or ‘difficult’ and how do we ensure module teams adopt consistent norms in relation to these?

[I was a bit sceptical of the project of this question. I do think the distinction is really important and one that authors have to constantly think about. Some texts are clearly easier (e.g. newspaper articles) and other harder (journal articles) and this then affects student workload. But it’s very contingent on the student, the module, the level, the qualification context, what the activity is that surrounds the text and what the student has to do with the text. I’m not persuaded this is the kind of thing that can be categorised a priori or assessed without seeing the context]

What should student-directed learning look like and how do we support students in effectively engaging with it?

What do we mean by student-directed anyway? How is it different from ‘studentship’? University norms are 80% module-directed and 20% student-directed at level 1, 70:30 at level 2 and 60: 40 at level 3. We need to tie student-directed study into assessment, otherwise why should they bother? We also need to be clear to ALs that they should give credit for materials which the student has found (especially at higher levels).

24th October 2013

Designing collaborative learning activities better

Notes from a workshop I attended today, organised by the METIS project. Appropriately enough, we were working collaboratively in groups to design a collaborative learning activity. Because this is The Open University, we were designing for an online distance-learning context, where the issues are quite different from face-to-face settings.  But I suspect some of the principles could also be applied to face-to-face teaching.

The first activity we were asked to undertake involved thinking about how to *ruin* a collaborative learning activity. I really liked this form of activity – it made you act as a kind of devil’s advocate to your own favourite ideas. I might reuse this form at the start of my next workshop with the K118 team, but in relation to a different topic (probably assessment).

How to ruin a collaborative learning activity :

  • Unclear about what students need to do so they spend all their time as a group debating this
  • Award credit to whole group regardless of individual input
  • no clear link to learning outcomes
  • make it too big or too small

Barriers to collaborative learning for students:

  • time – students can’t see benefit and see collaborative activity as optional extra so don’t do it.
  • technological problems
  • confidence/anxiety
  • why should they bother? Is more complex than working on your own for many people – needs to have a bigger pay-off
  • group dynamics – some people being unsupportive to others, some people not pulling their weight.
  • students’ different timetables (and possibly time zones)

Possible solutions:

  • Make benefits very clear e.g. in Module Guide, in teaching materials whenever collab activity is introduced, assess it (although this creates its own problems), tie-in to learning outcomes and assessment (even if actual activity is not assessed), workplace benefits.
  • Build confidence through starting with an easier collaborative activity (but make sure it isn’t facile – maybe something a bit reflective (but not too personal) would work well at this stage).
  • Have some flexibility in what technology they can use – email as a back-up?
  • Build up to it carefully
  • Include an example of what the end product should look like, so they know what they are aiming for.
  • Some discussion of group processes and roles.
  • Allow group to form a little before they start work (or include introductory activity as part of task)
  • Careful grouping – people in interest groups? Nation/region/area groups? [Anything more fine-grained creates significant work for ALs or central academics]
  • Clear guidance on what they need to do and when.
  • Possibility of individuals swapping groups?
  • Signpost significant collab activity v clearly in module description.

(cc) Brenderous

Verbs that might be particularly relevant to Learning Outcomes for collaborative learning activity:

  • build on
  • co-create
  • contribute
  • debate with
  • discusss with
  • engage with
  • enhance
  • improve on
  • motivate
  • perform
  • share
  • challenge

Then we did an activity designing a collaborative learning activity. Luckily for me, my group was asked to focus on my module, K118 (Perspectives on Health and Social Care), so I chose something form the Ageing and Later Life block that I thought it would particularly benefit students to do collaboratively.

Learning outcome – share different experiences  to increase awareness of the diversity of older people and challenge preconceptions.

Learning output – create shared wiki (resources and experiences of using wikis at Level one in Arts A150) [My hunch is that this is too technically complex to actually use on K118 but I will investigate it] Simpler output would just be discussion within the tutor group forum. Further discussion of this later in the day.

Presentation from Mary Thorpe, IET, The OU

Some forum-based examples of good-practice.

Simulation / role play

Students have been learning about 3 main theories about X in first 12 weeks of module.

  1. Tutor allocated one of three ‘hats’ to each student in their tutor group.
  2. Students watch a DVD clip and have to answer a question, wearing their particular hat. Given structured questions to help them do that.
  3. Post answers to forum and discuss, use clip timings to be clear. [Nice way of getting them to think about different theoretical perspectives, and possibly making them engage with ones they are not naturally attracted to].
  4. Then tutor group forum discussion.
  5. Then watch another clip, wearing the same hat.
  6. Post answer to forum in same way but also adding reference to other module materials to support arguments.
  7. Take off hats. Watch clips again, think about your own practice. Which theory best fits your practice?

Another example:

Pyramid or snowball – individual study of problem, then compare discuss, propose shared solution, all agree shared solution.

Do individual report on topic of choice (or allocated by tutor) and upload to tutor group forum. Then role play some kind of meeting where have to argue the case for their topic/group. Get marks for participating and reflecting on process, as well as for outputs.

[Stuff I’m already very alert to about importance of making it possible for students to pass assignment even if don’t take part in collaborative activity. And benefits of marks for ‘reflecting on the process’ as well as the outputs]

In example discussed, students hadn’t met before worked together – day school was culmination of activity, not beginning. Did some evaulation – students were fine about working with people they didn’t know because it was so clearly structured. Didn’t need much input from ALs because was so clearly structured. Students felt able to disagree (which is often a worry about collaborative work, that it ends up being too bland and consensual), because of ‘hats’/simulation kind of nature  – it wasn’t personal. They supported their arguments well with evidence [key skill! Great if you can enable that through this kind of thing].

Notes of caution from MT: be very clear about why collaboration is beneficial. Have to have a very good reason to cause students to loose the benefits of individual and flexible study to their own timescales. A tall order at level 1. Optional collaboration can be better (but then of course you’ll get much lower participation rates).

[My thoughts about how we could apply this to K118 – could maybe do ‘hats on’ kind of activity in Mental Health Block where they are looking at competing explanations for mental distress (and hence conflicting prescriptions for help and support). Or in Ageing activity idea, could be asking students to take on persona of particular older person from a set of case studies / characters they have already met (Molly and Monty?) and arguing for what is most important for them for good quality of life in later life (or similar). This could work well in the final week of the block, when they will have already met all the characters.]

Then we had a very brief lunch break (this was a hard and long day’s work, hence the rather long post)

(cc) Terence S. Jones

Back to the design of the possible activity for K118:

We were given a large sheet of flip-chart paper and some special Learning Design post-it notes which are coded as ‘resource’ ‘activity/task’ ‘learning outcome’ and ‘tool’. We then had to categorise all the parts of the activity appropriately on the post-it notes and storyboard them.

  1. Start with explanation of benefits of this activity: revision, employability skills of team-working, what else?
  2. Module team identifies 4 or 5 diverse older people students have already met during Block (Molly, Monty, others)
  3. AL allocates students to characters (or students choose for themselves? Might not get a good range then).
  4. Students review material on characters and fill in a set of questions to help them think about that person and their experiences.
  5. On forum (organised by threads) students discuss salient features of this person.
  6. Then they are posed a question that gets at diversity (something like ‘what is the most important thing in ensuring their quality of life?’) and have to argue that from the perspective of their character.

After we’d done this, I got a bit worried that this was too character-driven, given that we are already worried that students remember the characters, not the theories they are supposed to make intelligible. Also that it might be too easy. We then talked about how you could make it more challenging by getting students to come up with new case studies. Perhaps something like ‘write a short description of an older person who is completely different from [some stereotyped media portrayal of an older person]’. Then, once they had posted their case study, they would have to answer the same question about ‘what’s most important’ or whatever. Or, even better, could get them to find a real-life example from newspapers, publications from voluntary organisations or people they know. Write a summary, read other students’ summaries (this gets them some immediate benefit from . collaborative activity – they get to read more case studies of diverse older people than they could research themselves). Then debate the question (about quality of life or whatever) in the persona of their case study, in groups of 4 -6 within their forum.

So when we came to the next bit of the workshop, which was translating the storyboard into a prototype online tool called ‘WebCollage’, we further refined it to this last idea. This is all captured on the tool, but I don’ t think that’s publicly visible. I can see it here

After we’d done this (in our collaboratively-learning groups, naturally), we had some whole group discussion of our experiences. There was a theme of people finding the tool quite difficult to get to grips with. Several people said that the tool helped them design better activities by forcing them to be systematic and sequential [I don’t think that is a huge benefit for me because I’ve always designed activities in a fairly structured and systematic way, without using such a tool. I worry there might be a bit of a Hawthorne Effect going on here]. The tool did force me to think about what the tutor would be doing to support this activity, because that was one of the fields you had to complete, which was a useful prompt. In my group we thought that there might need to be some separate guidance for tutors on how to run activities like this [investigate whether we are allowed to do this nowadays].

Then we did a ‘heuristic evaluation’ which was defined as team of experts assessing by using a set of heuristics (or ‘rules of thumb’). A low-fidelity rapid evaluation to pick up design flaws at an early stage. Experts ‘walk through’ activity as if students. The rules of thumb were the ‘ways to ruin a collaborative activity’ and ‘barriers’ we identified at the beginning of the workshop. The heuristics we picked out to be evaluated against were:

  1. Clear benefit to students
  2. Build skills gradually in small steps, to prepare for collaborative learning activity
  3. Learning Outcomes need to be clear so students can see what the benefits are (?link to assessment)
  4. Be clear about time needed and ensure this is explicitly built into week’s workload

The rest of my team evaluated another group’s activity but I stayed with the potential K118 activity, to explain it to my evaluators. We used a basic grid: where’s the problem, what’s the problem, what heuristic does it violate, how severe is the violation, recommended action. The potential K118 activity came out pretty well, except what they thought was a minor issue about needing to be a bit more specific about the time needed by students and how many weeks this activity would be spread across. I thought that was fair enough. This would have been a more valuable activity to do once the author thought the activity was fully-designed – it was too easy for me to say ‘we were going to cover that, we just hadn’t had time to get on to that bit yet’!

At the end of the day we had to fill in a lengthy evaulation questionnaire (of course). I managed to press ‘reset’ instead of ‘submit’ at the bottom. That was a very bad end to what was otherwise a good and useful day.

28th March 2013

More principles of assessment

I’m thinking a lot about assessment at the moment, especially in relation to K118 (Title TBC) the new module I’m chairing. So this seminar last week was extremely timely and useful. The notes that follow are, as ever, not full sentences and not a full representation of what was said, just the things I was particularly interested in.

(cc) rileyroxx

Releasing creativity in assessment – New models of assessment and tuition project

First of all we were asked to write down two challenges in designing assessment that were particularly occupying our minds at the moment. Mine were:

  • The desire to be creative v. the danger of doing new stuff which is more likely to fail catastrophically and is generally much harder for staff (especically at the OU where everything is done at scale)
  • Getting the minutiae of weightings, substitutions, thresholds, formative/summative, exam v. EMA etc. right, and being aware of the knock-on unintended consequences on everything else of tweaking one variable.

How transparent is your guidance? All visible to students or separate tutor guidance notes to make marking easier for ALs (I’ve worked on modules with both. I’m not sure which is better).

Common problems:

  • Hard for students to see how the assessment links to the Learning Outcomes
  • Narrow range of assessment tasks and repetition across qualifications – students feel they are being asked to do the same thing again and again.
  • Gap between the students and the Module Team / AL understanding of task
  • Not much use of formative assessment
  • Students don’t pay attention to the feedback – they only care about the grade.
  • Limited use of student self-assessment
  • Delays in feedback mean they sometimes get it too late to be useful for the next TMA
  • No feedback on EMA sometimes
  • Difficult to find critical friend reviewers of assessment and difficult to find good practice examples
  • Tensions between the costs and rewards of innovation
  • Progression in assessment within a qualification

Need to reward the student for taking notice of feedback on previous TMA [could do this either with reflective Part B like K101, or with formative then summative on same essay, and probably other ways too]

Assessment should be [fairly uncontentious, hard to achieve]

  • Central to teaching
  • Part of an ongoing conversation between student and AL
  • Support student’s progress
  • Authentic, engaging, exciting
  • Be linked to specific learning outcomes and other outcome criteria

[Interesting argument between participants in the seminar about whether it is all about testing the learning outcomes, or whether it’s about supporting student learning, and whether those are in fact the same thing]

Life crises argument against formative assessment [if you have too few summative assessments, students drop too many marks if they miss a TMA due to life-crisis kind of factors (bereavement, illnesses, divorce etc.)]

Could maybe do optionality of TMAs? Set 6 but they can chose 4? But would have to think AL workload implications though very carefully [Although is that any different from TMAs that give students two options for questions, in terms of AL impact? Possibly not]

Make all of OCAS formative? Then an EMA that tests the whole module. (Science has one S141)

When assessment is content driven and Blocks are quite different, students can find it hard to build on the feedback they are given on TMAs, especially if it’s an end of Block assessment. [This is a danger for K118 but I think it can be avoided with careful planning]

Lot of people keen on peer-assessment and self-evaluation. Start at level 1 but gently – contribute to a wiki for example. [But all the practical advice I get is against]

Create a free website or blog at WordPress.com.