Video - Authentic Assessment of Critical Skills and Career Readiness
In this conversation style interview, Margo Griffith and Doris Zahner, PhD from the Council for Aid...
In this conversation style interview, Margo Griffith and Jeffrey Lehrer, National RTO Compliance Manager from Scouts Australia, discuss the challenges and opportunities around aligning non-formal learning to formal learning pathways for both their youth members and adult volunteers. They also discuss what youth organisations globally are doing to help participants gain recognition of transferable skills learnt through their programs, to be used as credits towards getting into the course they want or when looking for work.
Watch on Channel Edalex (YouTube)
0:00 - Introduction
0:28 - Informal, Non-formal and Formal Learning Within Scouts Australia
6:32 - Evidence of Learning and the Complexity of Mapping and Alignment
9:42 - Building Tools for the Learner to Articulate Their Skills and Competencies
11:52 - The Rio Declaration on Non-Formal Learning From Global Youth Organisations
15:31 - Recognition of Learning Involves a Translation Piece - the Scouts Australia Example
20:19 - Key Solutions for Recognition of Non-formal to Formal Learning Pathways
25:01 - The Role of Policymakers in Recognising Non-formal Education Across All Sectors
34:20 - Next Steps in Creating Non-formal to Formal Pathways
Click on the videos below to view or watch on our Channel Edalex YouTube channel - Subscribe to receive updates on new videos:
(This transcript has been lightly edited for readability)
Margo Griffith (MG) - I am really happy to have Jeffrey Lehrer with me today. Jeffrey is the National RTO Compliance Manager for Scouts Australia. Jeffrey - thanks so much for taking your time to chat with me today.
Jeffrey Lehrer (JL) - No - Thank you for the opportunity.
MG - So we have, and could pick from a multitude of different topics today. And we had some really great discussions in the past but I actually know there were a few close to your heart … so I’m sure that we’ll be touching on those today. Before we dig in to anything though, I am wondering if you might be able to just give us an idea of your role at Scouts Australia, and also what learning looks like within Scouts Australia as well.
JL - So I guess the starting point there - being my role in Scouts Australia - my paid role is as the National Compliance Manager for the Registered Training Organisation. I am also a volunteer but that aside, that paid role means that I manage the training organisation. There are about 60 volunteers, who also contribute to that, but I am the one paid employee. So … that definitely puts me across what’s happening within the qualifications and that formal education side but you actually asked about what the training is like.
Well, the training, I would say, is very different. So, that’s where we come into a very big difference; because scouting worldwide is an educational movement, whereas educational movement is about developing youth to contribute to society. To do that, we need lots of adults to volunteer to develop those youth, so there is a lot of training of those adults. So scouting in itself is an educational movement, often not seen that way but it is an educational movement worldwide.
I think the big difference that needs to be considered is, we do a lot of non-formal education in scouting. Now I look at non-formal education, so what is that? Unfortunately I can’t describe non-formal education without defining formal or informal education. That is the key aspect to this. So, the formal education, let’s think, national qualifications frameworks, schooling, universities, formally recognised qualifications. Informal – ‘I just happened to learn it along the way, I don’t really know where I picked that up. I just … I learnt that'. The non-formal is sort of somewhere between the two of those. It is a defined, clear curriculum that outlines what will be achieved in a fairly flexible manner, but not recognised like those schools, universities or vocational education qualifications are. So it sits somewhere between.
MG - Fascinating, yeah – fascinating stuff, isn’t it? So there are so many complexities within Scouts then, where you have this non-formal, deliberate but non-formal learning; the formal learning which takes place, I guess, through the RTO component, and then a whole bunch of informal learning that’s just happening as you’re going along. Hmm wow – how do you recognise all of that, like, how do you manage all of that?
JL - That probably … that’s the tricky bit. So … we look it in two ways: we’ve got the youth program, so how do we develop our youth - which is very flexible, very open ended, and it is … while there’s defined criteria define things can be done along the way, it is very open ended, very flexible and goes from 5-year-olds to 26-year-olds so you really are looking at a broad range of experience and knowledge in age groups.
And then we got the adults, anything from 18 to 98 that are helping develop those youth. So the difference in our adult training is highly defined, very clearly outlined, a very clear definition as to what needs to be achieved by when, to be registered as a leader in particular sections and to be able to do the job or volunteering, even. So … the great example of that - and you asked about how the formal work - well, first the training organisation recognises it by its formal qualifications, but it does it entirely out of recognition of prior learning.
Now a lot of that we know, what that learning has been, so I referred to the adult training a moment ago as being very clearly defined; we’ve got a very clear online learning, very clear face-to-face workshops, what the curriculum is, what’s in it. And beyond that some very clear guidelines about what must be achieved as practical experience, before they can be signed off as a leader to take responsibility for those young people and help them in their journey. So having that really clearly defined training that is towards a scouting outcome, the scouting outcome being “you’ll be a leader”. But that very clearly defined outcome gives us very clear information to say: “Oh! Units of competency in these qualifications have very clearly defined outcomes: clearly defined outcome A from scouting; clearly defined outcome B from a unit of competency - where do they line up? Where’s the gaps? Is there any difference? How can we make it work?” And therefore we then use that scouting outcome as evidence towards qualifications.
MG - Fascinating stuff, really. So unpacking that a little bit - in capturing all of that and recognising it and having that overtly aligned and displayed, is it an understatement to say that’s a bit of a challenge?
JL - It’s a lot of the challenge. So, there’s a few challenges in there. In saying that, what I’ve described is the adult training, great - clearly defined. And yet even there there’s a big challenge. So one of the challenges is, changing qualifications. Qualifications change, criteria in qualifications change, criteria in units of competency change – wait a second, does that still align to what we had, against the previous one? But recently, we’ve also changed our adult training curriculum. So, we’ve changed our internal curriculum, and we also have some recent changes against some of the qualifications we have on scope that used to be aligned against our old training. Now we have new qualifications, new training - as you can imagine that means a whole lot of new mapping and working out where the cross-overs are, where they’re the same, where they’re different, and are there gaps and what we need to do differently to fill the gaps.
MG - So what I am hearing Jeffrey is that you don’t have enough to do that perhaps...
JL - That’s it. Yes – and any who wish to volunteer and help out, yes please do.
MG - Nice plug, good work.
JL - But the other side of that is as I said the youth program is even more open ended. So because it’s more open ended, because it’s got lots of flexibility, it means that what they’re coming forward with as evidence could vary vastly from individual to individual. Adults – we know the learning pathway they’ve come through – great! So we know roughly where they’re aligned and what their evidence is because we’ve set the curriculum. Youth program has got a lot of consistency within it but that is very open ended. And being so open ended, yes there’s a level of leadership they’ve had to have done, yes there’s a level of interacting with others and communication and dealing with work health and safety, and sustainability and a whole range of things that are useful information. But how they do it, in their local group; how each individual chooses to do it within their local group varies immensely. So the assessors need to do a lot more to get that evidence out of the individuals to then recognise it with a formal qualification.
MG - Do you think the youth coming through those programs actually fully understand or can fully articulate all the richness of the skills and competencies that they’re actually acquiring? And I’m saying that from the context that - how does that set them up to, you know, progress, I guess, a lifelong learning journey whether that’s a formal learning pathway or whether that’s, you know, an industry pathway, whichever way they choose to go. Do you think that they can do it… do they recognise the skills and competencies and can they talk to it?
JL - No, is the simple answer. And that is because it’s a complex thing to understand. And many of them on reflection at a later point in life can easily articulate that. But at the point in time, and I’m thinking 18-year-olds here, then often the answer’s: “no, they can’t articulate that”. Many of them after... and I had a great conversation with a girl who’s about 20, 21 just the other day, and I said: “Look, you’ve done this, you’ve done that - here’s how we can look at it, here’s how we can align it, here’s how it works.” And at the end of that conversation, she went “Oh my goodness, I get it now” – but it was that point of conversation of … well, you’ve got all of this evidence – it shows you do that, it shows you know that. It’s clear. Now, here’s how we’re lining it up to the qualification. And it was only at the end of the conversation that she then understood that articulation and was able to articulate it herself. But it required, in that case it was myself, but plenty of other times it’s other assessors and other volunteers who are having these conversations to get them to that point of understanding. So, doing a non-formal education – do they get it? Are they understanding? No. Can they? Yes.
MG - So if we talk about, I guess that recognition of non-formal learning, I was reading on one of the links that you gave to me actually that the OECD have said that “recognition of non-formal and informal learning is an important means for making lifelong learning a reality” for, you know, the global population. Can you talk to, I guess, what the wider Scouts movement, but other youth organisation movements are doing globally?
JL - Yes – so I guess the very end of 2019 – so December 2019, there was a meeting of what has been described as the ‘Big 6’ – now I think they described themselves that way but in reality it is the 6 largest youth organisations globally. Scouts, Girl Guides or Girl Scouts, depending on which country you’re in, the YMCA, the YWCA, the Red Cross or Red Crescent, again depending on your country and the Duke of Edinburgh’s International Award are the 6. They came together with a couple of other smaller organisations and they’ve gone: “we agree, we need to get better recognition for all of this because the non-formal education, in particular, we know we’ve got some good programs, we know we offer a lot of good learning and training to our members” - but how does that get recognised? How can they then use that? How do they understand that recognition, and be able to articulate that as we just said a moment ago, is not as strong as it could be.
And therefore as organisations, unfortunately, as I said that was right at the end of December 2019 and only a few short months later, well, the world went into chaos. So there’s been little come of that as a result, but that doesn’t mean that there aren’t people working towards - at their national and international level - to get a better recognition of how non-formal ties into formal. Now I agree with the OECD that it also needs to be informal. The difficulty there is: informal is anything and everything, and there’s not necessarily any structure to it in any way, shape or form.
Recognition of what you’ve learnt is still important. Non-formal has a structure - whether it’s Duke of Edinburgh’s Award, whether it’s Scouts, whether it’s another thing, there’s a clear structure around what’s in that program. There’s a clear structure about what the outcomes are that we’re aiming for. So having that clear structure for something that isn’t within the formal education system does make it easier to recognise it against formal education systems.
MG - Yes, got you – makes sense.
JL - Yep. So, non-formal and informal are just as important as OECD have indicated, it’s just from a perspective of being able to assess that, it’s much easier to assess the non-formal because you’ve got that clear structure already in place.
MG - Got you.
JL - So, how does that play out? Well, using the Scouts as an example again - and our internal RTO and our non-formal education - I’m gonna use the Certificate III in Business as an example. I said earlier that we’ve got very structured adult training. It can be a shorter 6 months that someone goes through that adult training and has completed all the requirements and has all the evidence to complete that adult training and use that as primary evidence against a Certificate III in Business. They might have to add a couple of other things but they’re most of the way there. A young person, I’m thinking, the Venturer Scout age group 14-18 years old, would typically take 3 and half to 4 years, and it’s only about 15 to 18 percent of them that get to this Certificate III level, many are at Certificate II but about 15-18% get to that Certificate III level, where they achieve their Queen’s Scout Award, have got about 98% of their evidence towards a Certificate III in Business.
It’s taken them much longer, we’re talking 3 and a half, 4 years versus what could be a shorter 6 to 12 months, very structured but non-formal, very unstructured but non-formal. Both are clear programs, both are clear outlines that we can refer to. But what that means, from my perspective and I’ll throw this out in relation to a lot of information coming out of the European Youth Forum, is actually about: “we want recognition for the non-formal education we’ve done.” Now in the context I’ve just referred to, that might be young person who’s achieved a Queen’s Scout Award, “right - I just want recognition for that; whether it’s a university application or going into a VET qualification, then it’s about can you recognise my Queen’s Scout Award because there was a lot of things I did there that’s relevant and I’ve got that so why do I need to do anything more?”
And I get that, and it makes sense. And particularly as a young person who’s put many years into those outcomes, can they just recognise it? Within Australia and a lot of other countries, most of the European and Asian countries, we have clear national qualifications frameworks. Other education institutions that are within that formal education setting, understand the formal outcomes. And so, when we recognise, as the Scout’s RTO, when we recognise that non-formal award with a formal qualification, that translates very easily for the universities. The other registered training organisations – they look at that certificate, great – I understand it. They look at the Queen’s Scout Award – “what did you do, to get that? I guess that there was probably a lot of work in it, but what does it really mean to me?” So the RTO is our translation piece.
MG - Got you. And that alignment of outcomes or competencies is crucial, isn’t it?
JL - Very crucial – and it’s crucial for a number of reasons. So, I’ve got an example of a Venturer Scout who did her Queen’s Scout Award, she then went into ... she left school... still a member of the movement but left school and went into a forestry course. Her two qualifications she got through Scouts, she was able to provide them to the registered training organisation and get credit for some of the units that she’d already completed. So straight away there’s less time in the new qualification she’s doing and in many cases, less expense. So, there are others who’ve, because they got that formal qualification, have had that leg up to get into university. So they might not achieve the Australian tertiary entrance rank that they needed to get the qualification they wanted, but by having they’re additional Certificate III’s or II’s they were able to go: “Right, here’s something else that I’ve done” and have been able to get the course of their choice.
MG - If you had your ideal world, your wish list – what would, in Australia and perhaps globally, what would those non-formal to formal pathways look like? How would you articulate what that would look like from your perspective?
JL - The… it differs. It does differ, because one of the areas that we utilise a lot within the adults that get formal outcomes, comes down to organisational risk management. So, as an organisation that’s no secret, Scouts does a lot of outdoorsy stuff - we go bushwalking, we go abseiling and canoeing and, some of them go whitewater kayaking and multi-pitch climbing, and some ... some risky activities. Now that doesn’t mean there’s risk every time they go out, some are very low level risk, others are higher risks - but to ensure that we manage that risk, make sure that the adults who are responsible for that have qualifications that are relevant and accepted in the community.
Well, they don’t always need the full qualification, they might just need a component of it. So it might be: “I’m running abseiling, I need things that are relevant to running abseiling.” Great. So, we utilise their non-formal learning, we give them recognition of prior learning, after they’ve reached a certain level, they’ve got enough evidence together, give them that recognition of prior learning, they’re then able to run those activities. So what does non-formal to formal look like? It varies, it varies a lot.
So, as I said earlier, we do have a strong distinction between the youth and the adults, because for the same outcome it might take 4 to 5 times as long for a youth member to achieve the same things as an adult’s achieved – just the nature of how it’s done and also that life experience they bring with it.
One of the ideas we are looking at at the moment is to do with digital badging. Now that is only for the adults but that’s also about our risk management – you’ve done certain components and therefore you’re OK to run that activity. And how do we easily show that to everyone. Well, digital badge is one of the answers there, so that works.
With youth, they tend to complete a full qualification rather than a partial qualification. That’s because they’re looking at it for a different reason. They’re not just looking at it for risk management within an organisation. They’re looking at: “I can get one or two qualifications here, that’s going to set me up to make it easier to enter into my career or the course of my choice.” So, they’re looking at it from a very different perspective.
So how do we make it easier? We’ve got - as I said - we’ve got alignments that we’ve mapped. There are some other organisations we’ve worked with, where, we’ve aligned their non-formal education and shown how it aligns to the formal outcomes and we’re able to give those formal outcomes. So from my perspective, it really comes down to that work of mapping that non-formal program to the formal. It’s a significant piece of work but once it’s done, as long as: A. The qualification and B. The program don't change - then it stays consistent for everyone moving through that program until either of those occur. So, it is an easy way to do it – to get non-formal recognised in formal. So that is one option and one way that we use quite well. How else would you do it? That’s where I pointed out earlier that the elsewhere such as European Youth Forum, just want that straight-up recognition for achievement and that should be recognised. Well, I think that’s problematic because there’s not that translation piece for the other formal providers.
MG - The Big 6 Declaration and joint position piece that you’ve indicated did get derailed a little bit around implementation because of COVID, understandably, is there anything from a policy or a government perspective or a funding perspective that would make or help make that a reality in Australia? I guess what I’m saying is, what is needed in Australia to help make that a reality?
JL - So I see a number of pieces there. The OPEC Summit that was going to occur last year that didn’t occur for obvious reasons around COVID, actually had non-formal education on the cards and therefore, obviously our representatives of Government around the world within that were going to be talking about it. We provided some advice to that, but obviously that didn’t go ahead, for good reason. So hopefully that stays on the agenda, and that’s a key piece because that becomes an international piece of conversation.
As you mention, Big 6 have this international agreement, in many ways we could then localise that here in Australia. Now we are – there’s a level of working together around that, but as we said, COVID has disrupted that slightly so we do need to come back and put some more effort into that at this point. A lot of that is around policy, a lot of that policy piece at the moment is: The Australia Qualifications Framework is under review, the ‘how qualifications are developed in the vocational education and training sector’ is under review, standards within vocational education and training and in higher education - that is the university sector - are under review. All of that’s a good thing, but they’re all under review.
The trend, however, in Australia has been that we’ve got reports going back over the last 10 years and longer, particularly, even within the last 3 to 5 years there’s been a lot of reports that have said that industry wants more recognition processes. “We don’t want to go off and do a course and that’s how people get to their formal outcomes - there’s skilled people out there, that we need to recognise them better.” Totally agree.
The funding mechanisms that we have in Australia - there’s a lot of training that relies on government funding in one way, shape or another. Recognition of prior learning is a bit more complex than a training ... so the assessment of recognition of prior learning is a little bit more complex than a training and assessment pathway. If we have: provided the training, been assessing them along the way - formative and summative assessment along the way - and we’ve concluded the course, here’s your qualification - That assessment in many ways is very predictable, very easy because we have clear curriculum, clear course work we’re working with. The recognition of prior learning is a bit more complex because people can present with anything.
However, the funding mechanisms have said, if you’re only assessing, then we won’t pay you as much because you’re only assessing, you’re not training and assessing. So as a result, that in itself as a funding mechanism has encouraged many training providers in the formal sector to encourage their students away from a recognition only pathway, into a training and assessment pathway. So, in Australia we’ve got the National Centre for Vocational Education Research, who put out reports every year and see our data is fantastic and it’s been showing a great decline consistently with the number of recognition of prior learning outcomes. Meanwhile, the number of reports saying that we need more recognition has been growing. So the funding mechanism doesn’t help that.
I said earlier that informal, which is where I sort of bring my learning from everywhere, regardless of what it is, can be complex and has lots of information, and how do I represent that and how do I show that - that’s difficult for the assessors, it’s difficult for students too. Because I’m trying to pull bits and pieces together from everywhere. When we utilise non-formal, then we’re able to go: “Right, well we know there’s a clear pathway, a clear program, we know what it aligns to - even my example earlier of the Scouts Youth Program which is more open-ended, we still have some clear guidelines around it.
So therefore recognition of prior learning is easier for both the participant and the assessor. I’m not saying it’s a walk in the park, but it is certainly easier than taking at more even informal learning and trying to pull all that together and turn that into a formal outcome.
MG - Yes, I can see that.
JL - So that’s how non-formal can actually fit in nicely. Talking industry now, so, forget community organisations and youth programs - talking industry - and I’ll use the US as an example because they really don’t have a national qualifications framework and they have states doing their own thing, and in some cases counties doing their own thing, and they have industry bodies doing their own thing - and the amount of credentials in the marketplace in the USA is enormous and the last count I heard I cannot remember the numbers, but it was scary.
MG - Yes - according to Credential Engine, in 2020 I think it was something like a million credentials were out there.
JL - Yeah, so, how do you interpret all of that? What does this credential say, that I can manage a team mean, versus that credential that says I can manage a team? It gets really tricky. Now, all right, we’re lucky – here, and in many countries where we have a national qualifications framework, we’ve even got, particularly within the VET sector, we’ve got clearly defined unit of competency requirements, publicly available. So you can compare that non-formal workshop you did as part of your work, or that internal training, that with clearly defined outcomes, that it was part of your workplace training, you can easily compare that against the requirements of the formal outcome.
Now, you might do three pieces of workplace training, clearly defined outcomes, clear information “you have to do certain things to meet the requirements of that internal training,” you might have to do three different things to meet your requirements of a unit of competency, which is only one component of a qualification. But, you’ve got the ability to look at that non-formal education and line it up against formal, and therefore have something that stacks towards a formal outcome. That’s a significant part of how we can use the non-formal to get to formal. Those national qualifications framework are a key part of that, so without that, then what are we actually stacking against? And it might be internal outcomes we’re stacking against, but you know, to make transferability so I can leave one company and go to another company, a national system that I’m stacking against makes sense.
MG - Or at least some framework alignment no matter what that is – yes, I agree 100%. Thanks for breaking that down for us, that was really clear and also you know, an important distinction and an important walk-through so thanks Jeffrey, I appreciate that.
Jeffrey you mentioned before around all of the different policies that are in flux, which, you know, is from a RTO perspective and from, staying across that, and I guess from an accreditation auditing perspective - chaos, right? And a nightmare. What do you think needs to happen or what do you see as the next step from when all of these different policies shake out, what happens then?
JL - I guess my concern is not waiting till they shakeout and making sure we get some of the information in there upfront, because the question you asked earlier about connecting the non-formal to the formal, how do we make it happen? At the moment, 97% of registered training organisations, about - I think it’s 97 point something - do training and assessment. So there’s not a lot of recognition of prior learning, there’s not a lot of assessment only pathways.
What that means is a lot of our policies are written around ‘train’ and ‘assess’. They might allow for assessment only or recognition of prior learning; a lot of our current policies have at least standards for registered training organisations that have the fact that non-formal and informal learning could contribute to a formal outcome. That’s embedded in the forward of our current standards. It’s embedded in the Australian Qualification Framework. We don’t necessarily use that, but it’s there in our documentation. So, making sure that what the new policies, the standards are, still keeps that embedded but in a manner that is useable for both the vocation and higher ed training providers, because when we talk about training and assessment pathways and we don’t consider assessment only pathways, then we’re not considering people could be bringing their non-formal education into the system.
When every report and survey that’s commenting directly to the training industry, is neglecting to talk about assessment only pathways, that goes against what’s actually in our standards and our policies - national standards and policies - because they allow for it. But we’re not implementing them well because we’re not being asked to. And where we’re being asked to, we can get around it by pushing students to do a train and assess pathway instead of an assess only pathway, and on a different perspective where we are funding it, we’re pushing a different perspective, so, we need ensure that it stays in our standards and our policies, but we need to ensure it stays in in a way that it is actionable from the regulators, from the training providers, from the funding bodies that everyone actually understands the context, and what comes with converting non-formal to formal. Regardless of whether that is ... I’ve done an Udemy course, or I’ve been a Scouts member, how does that non-formal education then potentially tie into a formal outcome, if the way we implement all our policies and standards is based on you’ve done the training and assessment as one thing.
MG - Right, Jeffrey – thank you so, so much for stepping me through it but I know there’ll be a ton of people out there that appreciate that kind of clarity as well. It’s been fascinating talking to you. I know non-formal education and the various, informal and formal continuum is really passionate to your heart, and I just want to thank you for your time today, and I look forward to our many other discussions we probably have into the future. Thanks Jeffrey.
JL - Thank you.
The world's first Credential Evidence Platform
Launched In 2019, Credentialate is the world's first Credential Evidence Platform that helps discover and share evidence of workplace skills and analyse competency achievement across an institution like never before. It automatically extracts data from your LMS and issues a digital badge with an embedded personalised evidence page. This unique page details learner achievements using qualitative and quantitative data and can be issued together or separate to the digital badge. Available for the first time globally through Credentialate, is a suite of Skills-First Evidence Alignment tools - such as a Skills and Competency Library, Evidence Matrix builder and Framework Alignment - which recognise and support the burgeoning skills economy, and enable educators to apply backwards-design to micro-credential development. Credentialate was developed by Edalex - an edtech company on a mission to surface learning outcomes, digital assets and the power of individual achievement. Founded in 2016, Edalex brings together the team behind the CODiE award-winning openEQUELLA open source platform that centrally houses teaching and learning, research, media and library content.
Find out more at: edalex.com/credentialate
In this conversation style interview, Margo Griffith and Doris Zahner, PhD from the Council for Aid...