Home » Competency Based Learning

Category Archives: Competency Based Learning

Personalized Learning Plans

In June of 2013, the Governor of Vermont signed the Flexible Pathways Initiative calling for the creation of personalized learning plans.  When I was student at Empire State College in the 1990s, that’s exactly what I had to create for myself, and it is still something that the College promotes. What personalized learning plans do, first and foremost, is put an individual’s education directly into their own hands. What greater method can there be for the buy-in and ownership of an individual’s learning, than their own learning plan. It changes the way students perceive their education, especially in the K-12 world where education is not an option and attending school is mandatory. This changes education from a top down model, to a collaborative model: a collaboration between the student and the school. But, can it succeed, and how does a school district deliver instruction to students with individualized learning plans? This is the challenge of the future, and the only solution to moving education in the direction it needs to go. As technology advances personalize learning will become less and less difficult, more and more what is expected.

This week, the Wall Street Journal published an article entitled Shaking up the Classroom which discussed to move, by some schools, to a competency based model and The Journal, posted an article on evidence that competency based models are improving learning: http://thejournal.com/articles/2014/03/13/states-show-improvement-on-digital-learning-report-card.aspx

The move to personalized learning environments is gaining ground, and it will be interesting to see how schools implement these.

PLA, Test-Outs, and CBE Pretests

Standard VIII, in CAEL’s Ten Standards for for Quality Assurancestates “Fees charged for assessment should be based on the services performed in the process and not determined by the amount of credit awarded.” Anyone who has gone through the portfolio process of earning credit through PLA appreciates the work involved in articulating the learning and being assessed on it. Some PLA programs include an option that allows students to test out of a course. Usually this involves the construction of a well thought out assessment, built by the faculty and assessed under very secure circumstances. The student generally does not pay the tuition for the course, but a set fee for the exam. There are standardized exams for this process (CLEP, ACE, etc), but colleges often construct their own as well. How, then, does this compare with pretests in competency based learning?

A lot of thought needs to be put into the pretest–what is it purpose? Is the student paying for credit hours, or is there a set fee regardless? Many colleges and universities, offering CBE, have gone with alternative tuition fees. Considering Standard VIII above, this makes sense. But what if the institution hasn’t changed the way they charge tuition? If the pretest occurs before the student enrolls in the course, and provides the student with the option of testing out of the course (is given credit for the course ), then there is no conflict, especially if the student is allowed to complete any final artifact of competency that enrolled students will complete. However, if the pretest occurs within a CBE course, and allows a student to  “test out”, then there is a conflict, because the student is paying by the credit hour. In this situation, my own recommendation is that students who can prove they’ve achieved the required level of competency in any given course, should be allowed to test out. Schools developing a CBE program, should consider having this option available for all students.

Pretests that occur within the context of a CBE course, should have the primary purpose of being a diagnostic tool. They can provide students (and instructors) with important information, especially because by its very nature CBL is self-paced. A well constructed pretest can and should have questions that are matched to the competencies in the given course. Additionally, the competencies themselves should have an allotment of time associated with them–the time it would take the average student to cover the content required to master the competency (what we once thought of in terms of carnegie units). When the student completes the pretest, he/she should receive a report that indicates their current level of competency in each of the related areas (so they know how much time and energy they may need to spend in a given competency), and the actual calculation of time they would need to devote to the tasks. This will allow the student, who may need more than a semester to complete the course, the opportunity to request more than a semester and/or know how many courses they can reasonably complete in a given amount of time.

In summary, test-outs and PLA should be used to give credit up front for a particular course/competency, while pretests should be considered diagnostic and not used for awarding credit. All CBE programs should offer some sort of PLA credit. This does not mean that a student, who has not tested out, can’t move through the course rapidly; they can still skip practices and content (if they choose) yet will need to complete the graded assessments associated with all the given competencies within the course.

I’ve attached a suggested template for planning your pretest, which you can download: Pretests_Maps

And here is a  small sample of the kinds of data that properly constructed pretests can provide to students: table showing question #, competency, difficulty, score, time on question

data showing how student scored against competencies and approximate time they will need to spend in order to master the competency

Let’s really talk about assessment

picture of highway ramp with white line on the left and yellow line on the rightThis is the picture of an off-ramp taken from the rear window, and cropped out of the picture is a car that has been totaled.  A car in which a group of teens were killed when they got on the highway going the wrong way. If you were to get on an offramp, going the wrong way, how would you know? The clue is the yellow line on the right side and white line on the left. The white line should always be on the right-hand side, and the yellow on the left. I’ve found that most people don’t know this, and yet that piece of knowledge can be a matter of life and death.

If this piece of information is critical, then shouldn’t it be taught in driver education? And, if it is taught in driver education should we make sure it is assessed? A common method of assessment is the use of quizzes constructed of random questions selected from a larger test bank. This type of assessment does not guarantee that the student will be assessed on critical information. If the drivers test consisted of a subset of random questions selected from a test bank, students may or may not get a question on white lines. Should they? And, if so, should it matter if they get it correct?

These are the kinds of things we should consider when constructing assessment. What is it that students need to have mastered in a course in order to pass? Do we directly assess it, and do we insure students are not passed along until they actually master it? If a course does not have specific competencies that a student needs to master, as some individuals will claim about their course, then why are we asking students to take it? There must be some skill or knowledge, at some level of mastery (even if it is on the beginner level) that we want students to get from that course.

Today, I came across a wonderful document on CBE posted on the Merrimack High School web site:

http://mvhs.mvsd.k12.nh.us/MathPresentation.pdf.

Why use competencies: 90%takeoff, 86%turbulence, 83% panel knowledge, 50% landing. Still passing, but would you want to get on the plane?

I took a screenshot of one of the slides, because I think it illustrates an important point about the weights we assign to various

assessments. I’ve seen a good many rubrics, and far too many of them don’t weigh the most important items properly. I’ve seen a good many category weights that allow students to completely fail the final exam and still pass the course–sometimes with a grade as high as an B.

A good part of the problem education now faces is due to the way we assess education, the way we credential it as well, and maybe even the way we think about it.  Much  more thought needs to go into what we assess, how we assess it, and how much weight we put on those assessments.

Cohorts in self-paced environments: do MOOCs make it easier

On Friday, I had a conversation with a colleague who asked how you create cohorts in a self-paced course/program. After all, in most courses you have a group of students theoretically all at the same place in the content, who should be able to carry on discussions and debates. Such a setup allows for the creation of a community of learners within the course structure–one of the  circles in the Community of Inquiry Model.  But, the traditional cohort (or should we say: default cohort model) is not easy, sometimes not even possible, to achieve in a competency-based, self-paced or personalized learning environment.

The easiest method is to have a large enough base of students that you can group students according to where they are in the content. It would function like the gaming sites that match you with awaiting opponents. The group can be as little as two students, who work together throughout the course or, if the course is not linear, two students who happen to enter the “game room” (or discussion forum) at approximately the same time. The LMS could be set up to notify students when someone has entered and posted a comment, for example…most do this now, so the student expectations of how this will work would be the only thing that would need some tweaking.

But, students can also create their own cohorts. The meetups that have developed around some of the MOOCs are one example of this. Cohorts are not the only way of creating a base of individuals that the student can interact with, and gain insight from. In reality, social networks can replace the cohort model. With a bit of research and/or guidance, there are many places in the authentic world where students could get feedback on ideas, thoughts, assignments, etc. and interact with individuals around a given topic. The benefit is that it can increase their professional and personal network.

There are many places where students can engage in a debate on a topic, post a comment, or submit their research for scrutiny. One site in particular comes to mind: PLOS ONE: http://www.plosone.org/ and all of the sister sites. Today, I found an article on PLOS Biology entitled “Right Brain Left Brain: Fact and Fantasies“. It was published on January 21st (less than a month ago). Below are the statistics for this article. Note that there is an area for comments. And because it is published using an Open Access license, others can build and contribute to this research easily.

PLOSBiology data on article: 7801 views, 917 PDF downloads

Manufactured cohorts are another option. These take creativity and time on the part of the ID and/or SME to create. They will not work in a project-based environment, but are possible in discussion forums. Students can make posts and respond to posts.

A bit of creative thinking can create a highly effective and engaging course without the use of a traditional cohort. In many cases the traditional cohort model was not that effective anyway. Students were only theoretically at the same point in the course, and points are often given for less than optimal participation.  The construction is really meant to serve schools, as it allows for easier grading (easier than assessing social networking assignments), easier construction of courses, and quicker ways of assessing the effectiveness of the given content as everything is normed. What all of this is, however, is not authentic, not experiential, and though often designed to use constructivist methodologies, its effect is often less that ideal–time is a factor that significantly impacts the ability of students to construct knowledge.

 

Types of Assessments–to get you thinking

Generally speaking there are two types of assessments in learning: formative and summative. Formative assessments provide useful feedback to the student and are used for the purposes of increasing understanding. Summative assessments measure achievement. In traditional courses, students are given points for formative assessments and summative assessments. However, in the case of true outcomes based education, especially  CBL,  formative assessments should not receive points as they not measure mastery, but help build mastery. Formative assessments involve self-assessment and/or practice of competencies.

Let’s look at specific activities and assessments to see how this works in true outcomes based learning (competency based).

Tests:
Q: Can a test be used to measure a competency?
A: Yes, with caveats. It cannot be an open book test, must have a time limit. Students should not be able to change answers, or be allowed multiple attempts if the test is measuring a competency. It is best to have each question on it’s own page and not allow backtracking. They should be built using ell-constructed questions to measure competency.

Q: What kind of competencies can a test measure?
A: Tests are particularly good for measuring knowledge, or being able to define terms. For example if a competency states: Student will be able to identify the parts of a cell and their function.

Q: When do points apply to tests?
A: Test and quizzes can be wonderful tools for self-evaluations, practice, and even for teaching content especially if they include comprehensive feedback, and (in the case of practice and teaching) allow for multiple attempts. However, these formative assessments should not have points associated with them. When tests are used to measure a competency, they should have points associated with them. The points should be an indicator of the weight and level of the competency it is measuring. Generally, there should be an assignment somewhere in the course that measures the application of the knowledge the test is measuring. Since that assessment will measure both the knowledge and the ability to apply that knowledge, it should have a greater weight.

Discussion Forums:

Q: Should discussion forums be eliminated from a CBL course
A: That depends. The problem with the construction of most discussion forums is that they require at least a small cohort of students who can take part in the discussion. Depending on the construction of a forum, it can be used to measure a competency (summative) or to develop deeper understanding (learning activity or formative assessment).

Q: How can a forum be used to measure competency?
A: Forums (or any social networking activity) requires a great deal of thought in design, and generally a great deal of time and effort on the part of faculty in monitoring the discussion. Here is an example of a competency that might be measured using a forum: “Student demonstrates the ability to debate and argue a case…” The difficulty comes when the courses are designed for self-paced learning. How does one debate if there is no one to debate with? We tackle that in the next question

Q: Can a forum be used in a self-paced course?
A: Yes, with a great deal of planning and assistance from technology. For example, if the technology can work like gaming rooms, a student can enter the forum and “wait” for another student (or group of students) to “enter” the room. This would require a course that is not completely linear, so students can go back to that module when there are enough students in the forum to make it work efficiently.

Research Papers:
Q: Can a research paper be used to demonstrate competency?
A: Yes, but again that depends on how the assignment is constructed and the competency it is measuring. In general they are a learning activity–the student dives deeply into a particular subject and learns about it. But if there is a competency for demonstrating the ability to find and cite sources then a research paper then the that assignment would be good method for assessing the competency. However, generally speaking, the assignment should also include a presentation of the research with an extensive Q & A session (defend their conclusions)—to insure the student has mastered the material, and not merely regurgitated material.

Homework:
Q: Should we assign points for homework?

A: Homework should never be used to measure an outcome/competency. Homework does not measure mastery. It is practice and therefore should not have points associated with it. Homework should, however, receive feedback or be followed by a practice quiz that provides feedback. (test for understanding)

Projects, Scenarios, Simulations:
Q: We use these for learning activities, but can they also be used for measuring mastery?
A: Yes, projects, scenarios, and simulations can provide for authentic (or close to authentic) summative assessments. With scenarios and simulations, there should be a small margin or error for demonstrating competency (or mastery). A scenario or simulation that was used for training purposes, should not be reused, as is, for assessment of mastery. They should be changed somewhat. Also, while there might’ve been hints or other feedback provided during the training, hints should not be allowed in the summative assessment. Projects should be constructed in such a way as to mimic one that a student would need to complete in the “real” world, with the same or similar expectations.

Interviews & Observations: live, video and audio:
Q: What other methods can be used for assessing competency?
A: Interviews are excellent methods for assessing mastery level learning, especially if the student is not given the questions beforehand and is not allowed to use notes. Observing the student, particularly in clinical/classroom/workplace environments are also excellent ways of measuring competency.

Q: Can observations also be a useful tool for helping students gain competency?
A: Yes, observations can be an excellent tool for learning. For example, when an athlete or performer watches a video of themselves, they can observe where they need improvement and/or what is working well for them. It is an excellent tool for self-assessment. For example, having students record themselves giving a speech, allows them to hear how many “ums” they say, as well as their cadence. When others use an observation to provide constructive feedback, it can also be an excellent learning tool.

Blogs and other forms of Journaling:

Q: What about blogs, reflections and personal journals–those are mostly for processing and learning, right?
A: Journals and blogs can be excellent tools for measuring certain types of competencies, especially when they are accompanied by an artifact of learning. For example, if a student in a language acquisition program creates an audio recording of themselves speaking in the language, and accompanies that with a reflection that includes: what they are saying, why they chose that, how it is expressed culturally, etc, you get a much better understanding of the student’s grasp of the language. Reflections and blog posts can also be powerful tools for assessing mastery gained in internships. For example, in a blog post the student can discuss their experiences and what they are learning. Tests can no be designed to measure somethings that only blog posts and personal reflections can reveal.

I hope you found this helpful. Feel free to post your suggestions in the comment section.