Clawing my way up through the trough of disillusionment with learning analytics

512px-Gartner_Hype_Cycle.svg

(image: Jeremykemp at English Wikipedia [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)%5D, via Wikimedia Commons)

Warning -this is a bit of a moan post.

Last week I attended the Jisc Learning Analytics Network meeting. It was a really good day, lots of people there, lots of good sharing, moaning, asking where next-ing.  One of the reasons I find these events useful is that they help focus my mind and give me a sense of relief that some of the challenges that I face are similar, if not exactly the same, as many others in the sector.

In terms of learning analytics, my experiences to date have been metaphor-tastic: (ever decreasing) circles, slopes, dead ends, stop-starts . . . I feel that it’s appropriate reflect on my journey via the well trodden Gartner hype cycle.

I’m the first to admit I enjoyed being swept up to the peak of inflated expectations. Exploring the potential of data and learning analytics was probably the last piece of innovation work I was involved in when I work with Cetis. I really enjoyed trying to figure out the practical applications and meanings for mainstream learning and teaching of the swirly twirly graphs at early LAK conferences. It was great to support the emerging UK community via early SoLAR meeting.  I learnt a huge amount being involved in the Cetis Analytics Series.  I always think I brought a  healthy degree of scepticism to some of the hype of learning analytics, but I could  (and still can) see the benefits of extracting, exploring and understanding data around learning and teaching.

From the giddy heights of the peak of inflated expectation, I knew when I moved to a “proper job” within a university I would have a bit of a slide down the slope to the trough of disillusionment. It’s getting out of the trough that I’m finding real difficulty with. Changes in senior management, have meant going through a bit of a treadmill in terms of gaining institutional support and understanding. That’s before even accessing any data.

The Jisc Effective Analytics Programme has been a bit of ray of light and hope for me. Towards the end of last year we took part in the Discovery phase of the programme. This involved a consultancy exercise, onsite for 3 days with a cross section of institutional stakeholders to assess our “readiness” for analytics. At the end of the exercise we got a report with our readiness matrix and some recommendations.  You can view our report here.

At the meeting last week a number of institutions who have gone through the Discovery phase took part in a panel discussion about the experience.  One common thread was the reassurance that the exercise gave to everyone in terms of being “on the right track” with things.  I was pleasantly surprised that we got such good score in terms of our cultural readiness. The validation of having an external report from a nationally recognised agency such as Jisc is also incredibly useful for those of us on the ground to remind/cajole (hit people of the head – oh wait that’s only in my dreams) with in terms of what we should be doing next.

I think one of the main problems with analytics is finding a starting point. Going through the Discovery phase does give a number of starting points. My frustration just now is that my institution is now going through a major rethink of our overall data architecture. So on the one hand I think “hurrah” because that does need to be done. On the other I feel that I am almost back to square one as terms of “business needs” anything to do with learning and teaching seems to fall off the list of things that need to be done pretty quickly.  It’s difficult to juggle priorities, what is more important, getting our admissions process working more efficiently or developing ways to understand what happens when students are engaging (or not) with modules and the rest of the “stuff” that happens at University? Or updating our student record system, or updating our finance systems?

Amidst all this it was good to get a day out to find out what others are up to in the sector. Thanks Jisc for providing these networking events. They really are so useful for the sector and long may they continue. UEL who hosted the event have been doing some great work over the past four years around learning analytics which has emerged from their original BI work with Jisc. The work they have been doing around module attendance (via their swipe card system and VLE data) and performance is something I hope we can do here at GCU sometime soon.

In the morning we got updates from 3 mini projects just have funded starting with the University of Greenwich and their investigations into module survey results and learning outcomes. The team explain more in this blog post. I was also very interested in the Student workload model mini project being developed at the OU.  You can read more about it here.

The other mini project from the University of Edinburgh, was interesting too, but in a different way. It is more what I would term, a pure LA research project with lots of text data mining, regression modelling of (MOOC) discussion forums. Part of me is fascinated by all of this “clever stuff”, but equally part of me just thinks that I will never be able to use any of that in my day job.  We don’t have huge discussion forums, in fact we are seeing (and in many ways encouraging) less use of them (even with our limited data views I know that) and more use of wikis and blogs for reflection and discussion. Maybe these techniques will work on these areas too, I hope so but sometimes thinking about that really does make my head hurt.

I hope that we can start moving on our pilot work around learning analytics soon. ’Til then, I will hang on in there and continue my slow climb up the slope, and maby one day arrive at the plateau.

Some reflections on “mapping the learning and teaching”

I’m big fan of the Visitors and Residents (V&R) mapping methodology. As well as being part of the HEA Challenges of Web Residency programme, I’ve been using it with staff in various ways from an ice breaker to a teaching activity.  It’s a really great way to get discussions started around how/where/when we use technology in our professional and personal lives. It allows us to explore and begin to articulate our comfort and not so comfortable technology zones.

I’ve always thought of it as a personal reflective tool which can then lead to discussions about how one might think about using technology for learning and teaching.  James Clay has written a great blog post on his recent experience of a mapping workshop and how he has started to think about developing he process to map learning, teaching and explore learning/curriculum design.

 . . . I then started to think about how we could use a similar concept to map teaching practice and curriculum design. This lead onto thinking about mapping the “learning” of our learners. Where are they learning, is that learning scheduled and formalised? Is that learning ad-hoc? Is it individual, group, collaborative? So the next stage was to map this in a similar manner to the Visitor and Residents, but what axes could we use when mapping learning?

On the horizontal axis we have a spectrum from broadcast to engagement. Broadcast could be considered one way, and could be one to one, or one to many. So a formal lecture would be considered broadcast, one way to many students. If lectures have opportunities for discussion and questions, then you can see how that would move down the continuum into engagement.  . . . The mapping provides an insight into how the curriculum is designed and how learners interact and engage with the different spaces, tools and delivery mechanisms.

screen shot of Vand R map

What needs to happen to inflate and expand the VLE on the map? How do you push (or expand) the use of the VLE into the engagement side of the continuum? 

As I was reading the post this immediately got me thinking about situations where you don’t use a VLE.  The recent #BYOD4L is a great example of this open, non VLE delivered learning experience.  Using James’ template, I did a quick mapping of it.

You can see that twitter moves into the centre of the formal, engagement quadrant. In terms of broadcast, instead of lectures it’s the main BYOD4L blog (wordpress site) which provides the static, broadcast content/structure/methodology/resources.  The daily update blog posts are shared via twitter and in the Google+ community.  I’ve put the interweb at the centre as it is the heart of delivery/interaction/sharing – in fact the 5C’s (connecting, communication, collaborating, curating and creating) that underpin BYOD4L.

Engagement is driven through Twitter, with the nightly tweet chats comprising the bulk of communication between participants.  The tweet chats are unfailing a really lively, highly engaged activity which is curated then shared (broadcast?) via Storify.   Personal reflection is encouraged (though not mandatory) and takes place in a number of spaces (again mostly “informal”web based such as blogs),  and generally shared via Twitter and Google +.

The Google + community acts as a hybrid discussion/reflection space. It’s a place for more considered commentary out with the very fast paces tweet chats. This year a number of people, myself included experimented with Periscope to broadcast ourselves over the week. I’ve written before about the potential and challenges of this personal broadcasting. In the context of this mapping exercise it is another hybrid space where you can broadcast and interact/engage simultaneously.

BYOD4L is an informal learning event, so there is no formal requirement for engagement. However, even a cursory analysis of twitter shows that there are a number of very active contributors. With appropriate evidence participation in the event can be acknowledge through a series of digital badges.  These can be link to more formal CPD such as PGCerts. etc.

I know from my own experience that outside the informal, web space(s) #BYOD4l exists in there are lots of other types of “real world” engagement activities. For example at my own institution we run a number of f2f sessions over the week where we engage with colleagues in a variety of ways.  The engagement in personal and professional learning and teaching practice as a result of these is hard to measure. Anecdotally I know that a number of colleagues have experimented with using twitter more formally and informally in their modules as a direct result of their experience of engagement with #BYOD4L.

The map James created has made me think about what happens here in terms of more “traditional’ learning and teaching. As we move more to fully online delivery I think that this type of mapping exercise would be really valuable in terms of getting staff (and in my context as I work in staff development they are my first port of call) to think about their online presence in a different way. It might even get them to see the potential of integrating 3rd party services such as twitter, google+, Facebook, Periscope, BoB into their overall learning designs as well as integrating more of the institutionally provided services such library services too.

Going back to James’s questions  “What needs to happen to inflate and expand the VLE on the map?”  I would like to see the VLE more at the centre, it’s the container where we can bring things  (people and services) together. It’s also the place where we manage the boring but unfortunately essential administration and quality assurance part of formal learning.  So maybe I would further adapt James’ map and change the Informal/formal axis labels to personal/institutional like below.  NB this is a quick sketch and misses out loads of things!

IMG_0012

In terms of fully online delivery I think the broadcast/engagement axis could be really useful to help work through some issues around online tutor presence. To get staff to questions when,why and how they would use a particular tool/service. For example,  some announcements are fine to be broadcast mode, but at times you might want a bit more interaction/engagement/feedback.

Similarly when thinking about lectures, just using the terms broadcast and engagement could be a really easy way to get staff to reflect on their current practice. To think through how a broadcast resource could become the basis for more engaging activity and vice versa.  How could the outcomes/knowledge of any activity be curated, shared and re-used effectively with future cohorts. How do different delivery modes (f2f, blended, fully online) impact on the balance/structure of broadcast and more engaged activities?

Not rocket science but finding ways to facilitate and enable these types of conversations is really important.  I’m now thinking sliding scales and and fridge poetry magnet sized cards for people to play with.

I need to think this through a bit more, and try it out with some colleagues, but I can certainly see the potential of this approach to extending the use and relevance of the original V&R mapping.

Exploring #BYOD4L and #BYOD4Lchat

Last week’s #BYOD4L, and in particular, the nightly #byod4lchat tweet chats were fun, at times frenetic and fascinating insight into how our community is connecting, communicating collaborating, curating and creating using a range of mobile apps and devices.

Thanks to Martin Hawksey’s fantastic, open TAGSexplorer tool,  we can delve into the twitter stream a bit more.

In the main #BYOD4L there were 1028 unique tweets sent over the week, and 813 links which is quite a bit of sharing. Unsurprisingly, the #BYOD4lchat was a bit more active with 2467 unique tweets sent over the week. The screenshot below shows the list of top tweeters.

Screen Shot 2016-01-19 at 11.24.32

When we go into the TAGSExplorer interactive view we can get a far richer picture of the the community and its connections.  Screen Shot 2016-01-19 at 11.26.31.png

As well as the “swirly twirly” diagram,  we can explore not only the top tweeters, but also the top conversationalists (there are some users in both but differences too).

screen shot of top #byod4l conversationlists

We can also explore and search a time line archive of the tweets which is fascinating and gives a really nice visual overview of the periods of twitter activity each night last week.

screen shot of archive timeline view

We really did hit twitter every night last week between 8 and 9 pm.  We may or may not have been responsible for twitter going down on Friday night. . . .

I know I enjoyed all the tweet chats I managed to be part of. I found out about lots of new (to me) things and practice, made some new connections as well as catching up with old friends. That’s really the point of events like #BYOD4l, we all learn best by sharing with each other. I hope that the new connections made over the course of last week grow from strength to strength and I look forward to the next round of #BYOD4l.

 

 

#BYOD4L and Blending Learning

Last week was a bit of a whirlwind of activity for me. As well as being one of the organisers for the latest iteration of #BYOD4L, and running a number of drop in sessions throughout the week here at GCU, I was also in Sligo on Thursday and Friday giving a keynote at Sligo Institute of Technology’s Learning and Teaching Symposium.

I was very impressed to hear about the way fully online delivery has been developing at the  Institute of Technology Sligo. They now have 55 fully online courses, nearly half their staff teaching on fully online courses, and  are one of Ireland’s largest providers of online learning. This has all been done from a very small beginnings with a very bottom up, pragmatic approach. Now their online courses are beginning to generate some considerable income, they are investing back into staff development and infrastructure which is always good to hear.  I think we could all learn a lot from their approach, and I’ll certainly be keeping in touch.

I shared some of the approaches we take to blended learning here at GCU which seemed to go down well. My slides are available below.

 

 

#BYOD4L 2016

Once again #BYOD4L is back to brighten up the dark, wet January days and nights with 5 days of 5C’s getting people connecting, communicating, curating, collaborating and creating.

I’m delighted to be part of the organisation team this year along with Neil Withnell, Alex Spiers (with some help from Chrissi). We have a great team of community mentors too, so I’m sure the week is going to be another whirlwind of activity.  The BYOD4L open model really is a great way to get people to share their practice, experiment with new things and get swept up in the twitter chats each evening. There is of course the opportunity to earn a digital badge or two too.

So even if you only have 5 minutes over the next five days, I’d urge you to have a look at the #byod4l twitter stream, check out the website, and google+ community. I think you’ll find it worth it.

I’ve had a really worth while CPD experience from being a participant, facilitator and now organiser of the event. You can read more about my, and others. experience of open facilitation, in this special edition of Learning and Teaching in Action.  As Alex says in his video introduction to the event it really does bring a little ray of sunshine to January.

Screen Shot 2016-01-10 at 13.57.09

 

 

 

Digital presence, persistence, Periscope and mince pies

I don’t know about you, but one of the things I enjoy about holidays is a bit of a digital switch off.  Much as twitter is my “go to” network, it has evolved into a primarily work related space for me. I try not to “be” on twitter out of office hours/weekends and holidays. It’s too much like being at work.  My automatic daily update ensures I still have one tweet a day, and I do share the odd Instagram picture.  There are of course exceptions such as the Eurovision Song Contest which is just so much better with a twitter feed. Watching Question Time live with twitter on the other hand is not such a good plan, it is just a spiral of rage and despair imho.

Like many others my digital presence is constantly evolving.   A couple of years ago, when Twitter was new and exciting, I would check my stream and tweet along with everyone else. Now I tend to use Facebook more for connections with friends and family “out of office hours”.  Over the past few years I’m actually sharing photos more via instagram for quick sharing to FB and sometimes Twitter. Over the past three years I’ve been using the Blipfoto site to share a photo a day. This is a totally different network for me and one I really value. I also get my best stats there. I wish as many people read this blog as looked at my pictures! Like others have pledged to pay to keep the service alive.

Screen Shot 2016-01-05 at 12.21.32

Over the holidays I did decided to experiment  with Periscope the broadcast service from Twitter. When I say experiment, I mean it was more a spur of the moment thing.  I have used Periscope once before at the last Open Data Glasgow meet-up as an emergency streaming solution.

A few days before Christmas, my Mum who was staying with me decided to make mince pies one evening. Instead of trying to explain Periscope to her I decided just to show her, and so our first (and I have to say funniest) broadcast began.

Much to my surprise we got 34 viewers – I think I actually knew 2 of them. Fuelled by our success (and perhaps a glass of wine) the next day we decided to do the same as she made some meringues.  We got about 40 viewers this time including Lawrie Phipps with whom I had the brief twitter chatter below (click the image to see the storify version) and prompted this post.

Screen Shot 2016-01-05 at 10.47.01

I’m still trying to figure out what Periscope means to my digital presence but I do think it has huge potential for education. Like anything without any control over who views, or interacts with you, it will take a bit of time, and initially some pretty digitally confident people to experiment with it.  I don’t know if I am digitally confident or digital careless – probably a bit of both, so later in the holidays I did another broadcast with my 8 year old niece as she and my Mum made pancakes.

Screen Shot 2016-01-05 at 12.11.20

This time we hit the giddy heights of 130 viewers, and got the first taste of the down side of open broadcasting in this way. Some idiot and that’s the only polite word I can think of to use, started randomly typing “sex” then “porno”. In my best school teacher voice I told them, politely but firmly to go away and get their kicks somewhere else.  They did.

On reflection I can see how that kind of interaction would have freaked a lot of people out, and would have turned them off using periscope or other services all together. It exemplifies in a very “lite” way the scary/nasty side of making yourself openly available online. It just made me sad that people feel the need to behave in that way, and have that kind of negative, pointless digital presence.

In terms of education I think we need to be taking control of these kind of spaces, not running away from them or trying to lock them down all together. Of course, I wouldn’t advocate every child to start broadcasting themselves all the time at school and or home. However we could use this type of instant broadcasts in lots of creative, positive ways in learning and teaching; particularly around practical demonstrations, critiques/review etc as well as formal/informal group working). I think HE should be leading the way in this as we all increase our fully online delivery.

In terms of recruitment, there’s huge potential for interviews (formal and informal), virtual open days etc. At GCU we actually did Periscope bits of our last open day.

It is going to take time to figure it all out, again open practices, research and auto-ethnographic approaches are going to become increasingly important. It’s only by sharing and developing our narratives of these types of experiences that we will be able to make any kind of sense of the educational potential of services such as Periscope and how they fit into our evolving notions and experiences of our own and our students digital presence(s).

My holiday experiments have definitely encouraged me think more seriously about how to use Periscope in my context and with my colleagues. My Mum and I may still try to get a Mary Berry type deal somewhere and give up all this education stuff . . .

Third time lucky – 365 blips on the wall 

For the past three years I’ve been trying to share  a photo a day on the blipfoto site. The first year I tired I missed a few days, and last year just one. Thisyear I have actually managed to post a picture everyday.  

In the grand scale of things this isn’t a particularly amazing, life changing feat, but for me it has become a really enjoyable habit. My photos (or blips) are more personal reminders of each day. Some are mundane, some include shoes, some are actually quite nice photographs. Together they are a visual record of the past few years for me, my visual diary  if you like. They are an increasingly important part of my digital presence/ identity. I’ll be writing more about that next week when I have my work head back on and have had more time to think about my holiday periscope adventures. 

I am going to continue to try and share a picture a day, but in the meantime I’m raising a glass to my achievement this year and wish everyone a happy and peaceful New Year.

 blipfoto image