Category Archives: Uncategorized

EFL Gamification 2: Triggers, Ability, and Motivation.

Posted on by
Reply

Gamification is a buzzword.  Gamification is being widely–and often mistakenly–deployed in business situations recently. But because of the haphazard way it is being deployed and the mixed results it seems to be achieving, it is often viewed suspiciously by many in the world of business, and most in the world of education, where the very mention of games seems to suggest an offensive lack of seriousness. There are good and bad reasons to be suspicious of gamification, and it is not surprising that many game designers have hesitations about it.

The main problem, as I see it, is that superficial features of gamification (especially points, badges, and leaderboards) are being applied without enough thought being given to the underlying cognitive and emotional constructs people –customers, employees, learners–bring to any situation. For education, gamification, game design, and user experience (UX) design present an opportunity to re-examine the mechanics and dynamics of motivation and behavior change. Yes, they apply to teaching, including language teaching.

The first post in this series looked at intrinsic and extrinsic motivation and rewards.  My main point was that depending on the type of task or behavior, the use of extrinsic rewards–like many of the the tools of gamification–can help or hinder the development of intrinsic motivation. This is a critical point, but motivation is not the only factor in behavior change. So this post will focus on behavior change and the other factors involved: triggers, ability, and context. Yes, we are still in theoretical territory here, I’m afraid. But I promise to get more practical in future posts.

B.J. Fogg, a professor at Stanford U., is someone you might not have ever heard of if you are an EFL/ESL educator. He does not do research on motivation in education. Instead, his research is in how technology changes human behavior. And his main audience is business people involved in Internet-related start-ups, trying to get people to give their ideas or services a try. EFL teachers are often faced with a challenge that is really not so different from these young entrepreneurs: how do we get learners to engage in specific behaviors, in our case, ones  that we know will help them improve in proficiency? High school students in Japan are limited in the number of classroom hours of English. They are limited by a lack of technology infrastructure in schools. They are limited by the priorities of schools that want more kids to pass certain university entrance exams. That means that a lot of lesson time is spent on teacher explanations of language that is too difficult for many learners in the room, and not enough level-appropriate input is given, and not enough meaning-focused output activities are attempted. One of the possible answers to this problem is in everyone’s hands–mobile devices. But learners have no idea of how to make use of them and teachers are really really reluctant to even try to get learners to do so, fearing accusations of unfairness, steep digital learning literacy curves, and chasms of technology coordination and control issues. And the whole undertaking would require a massive shift in educational culture to begin with. But, ah, if only it were possible…Teachers could flip lessons, focus more on engaging learners in tasks requiring language production, and concentrate much much more on giving good formative feedback on comprehension or skill development in class before  the final summative tests. What I would like to suggest is that by making use of better design–and that almost certainly includes some game design techniques, but will also likely include user user experience (UX) design knowledge–we can increase engagement, push learners toward being more active participants in language learning, create a better learning experience, and hopefully get increased time on task and increased effort, and (eventually) increased target language encounters outside of class. Seriously, who wouldn’t want their classes to be more fun and more effective at the same time?

But before we get into the specifics of how gamification might be able to help with behavior change, we need to look at what Mr. Fogg has to say about changing behavior. There are, of course, other researchers working on habit and behavior change. But I think Mr. Fogg has the easiest to understand and most usable of ideas. As an introduction, let’s listen to the man himself summarizing his work: a short video is available on this page (sorry, the video is not embeddable into this blog).

Mr. Fogg sees behavior change as habit formation. His lab has produced a wonderful  chart that  lists the different types of change by whether it involves starting a new behavior, stopping a current behavior, or increasing/reducing a current behavior. He also distinguishes the duration of the behavior change, whether it is to be temporary (dot), for a fixed period (span),  or lasting (path). For language teachers in Japan dealing with low-proficiency learners (in general, students at ‘lower-level’ schools tend to have poor study skills in addition to poor language proficiency), green span or green path behaviors are what we should be aiming at. Duh, you might say at this point. But wait, because the simplistic beauty of Mr. Fogg’s model starts now. For a behavior to change, 3 things have to be present: a trigger, the ability to do the behavior, and motivation. And the last two, motivation and ability, are trade-offs. That means if you have low amounts of ability, you need to have more motivation. If you have low amounts of motivation (which is usually the case for the learners in our target group), you need to make the behavior steps really small. According to Mr. Fogg, behaviors are always the result of sequences. But you need to think and plan them carefully to be sure they meet certain conditions. That is,  you need to have an appropriate  trigger while you target a doable behavior with sufficient motivation available. Here is another graph that visually represents this.

First, think carefully about the target behavior. Is it simple/easy enough? Do you have a trigger? Because you need one. In the classroom, triggers can be certain events. Set fixed activities in your routines that will act as triggers. One teacher I know has his students get out their dictionaries at the beginning of class for an activity that requires them. Like clockwork, the class starts and the students get out their dictionaries as the teacher writes the day’s three vocabulary items on the board. Target behavior: use dictionaries. Trigger: vocab activity at beginning of every class. Ability: getting out the dictionaries and looking up only three words is doable.  Motivation: the students want to improve at English and the teacher has convinced them that using dictionaries is important.

Think about the current motivation your learners have. Is the behavior in sync with the learners’ goals? If not, you’ll need smaller steps, like in the dictionary example above. Target behaviors in small steps (Mr. Fogg calls them tiny habits). You really can’t go wrong making your steps really really small. Once one is established, you can target a subsequent behavior. An established behavior can  be used as a trigger for another behavior. You can also go the other way and work on the motivation. Explain to learners why a behavior is important. Make the activity more desirable. Or make the behavior more attractive (fun, social, meaningful, etc.). But Mr. Fogg suggests focusing on ability and triggers. Keep in mind that although people will rarely do things they don’t like, it is sometimes the case that people come to like what they do, rather than do only the things they like.

Or leverage a context change. Changes of context are times when humans are more willing and able to lose existing habits or form new ones. So, plan your big changes from the beginning of the school year. Or if you are looking to establish some new behavior, such as pair or group work, reconfigure the classroom seating and move the desks.

Here is Mr. Fogg’s list of patterns for success. These points are well worth keeping in mind as you try to get learners to change behaviors.

“Help people do what they already want to do.”
“Put hot triggers in the path of motivated people.”
“Trigger the right sequence of baby steps.”
“Simple. Social. Fun.” (You must have at least two of these.)
“Harness the motivation wave to make future behavior easy.
“Simplicity matters more than motivation.”

Another presentation of his looks at common pitfalls of behavior change. I’ll embed it below for easier access.

 

Now that we have covered the theoretical ground, it’s  time to look at the actual application of gamification mechanics. That will be the topic of the next post.

 

Also in this EFL gamification series:

Part 1: Intrinsic and Extrinsic Rewards

Part 3: Mechanics

Part 4: The Downside and How to Avoid It

Part 5: The Whole Hog

Part 6: ARGs

 

 

 

EFL Gamification 1: Intrinsic and Extrinsic Rewards

Posted on by
Reply

 

I am interested in gamification. That is not to say I buy into everything that gets labelled as such. Gamification is a concept being applied scattershot  recently in both marketing and education  in the hopes of making something more marketable, attractive, palatable. But gamification is NOT a set of techniques that can be used in any situation to add a little coercion or motivate customers or learners. In a recent Mind/Shift blog post on using games for education, gamification gets this rather abrupt dismissal:

Gamification is the use of game-based elements or game mechanics to drive user engagement and actions in non-game contexts. In gamification, the game mechanics are divorced from the content being taught and are instead added in the form of some sort of reward element after completion of an activity. For example, a short-form math game that involves answering math questions where correct answers are followed by a badge or the reward of playing a “dunk the clown” game would be called gamification. David Dockterman, Ed.D., Chief Architect, Learning Sciences with Tom Snyder Productions/Scholastic is concerned about this use of game mechanics, stating “Gamification can begin to undermine a kid’s desire to learn” (CS4Ed interview, March, 2012).

Read that part in red again. I’m afraid it’s true sometimes.  But not always. The effectiveness or lack thereof (or even detrimental impact) of gamification lies in the approach that it becomes part of when deployed. The key is educational design, and crucial to that is feedback, but I am getting ahead of myself. As  a recent participant in Coursera.org’s Gamification course who has to give a few presentations on gamification in EFL in Japan this year, I have an obvious interest in finding out just what it can do for education. So for the next few posts, I plan to look into different aspects of gamification and language education. First up is motivation. Because that is really the reason gamification exists at all.

 

Motivation is a topic I have written on many times–here in describing the general problem of motivation by English language learners in Japan, and here describing what most teachers mean when they talk about motivation. The literature on motivation in ELT is not always that helpful because it focuses on issues of identity and tends to ignore the realities of the classroom and the role that engineered instructional environments and learning situations can have on forming motivation. I don’t want to reduce the importance of Mr. Dornyei’s work in any way, but with the teachers I work with, when they talk about motivation, they are talking about motivation problems that require behavior change. For that reason, I have found Self-Determination Theory (SDT) to be more functionally applicable. Essentially, it posits that humans are motivated by needs to be competent at things, have autonomy, and be part of a group/society/meaningful unit.

Recently, on Julie Dirksen’s blog by way of Amy Jo Kim (instructional designer and game designer respectively) I found out about Chris Hecker’s 2010 presentation entitled Achievements Considered Harmful?. He is, by the way, another game designer. You can watch it here, (or just skim the page for the most relevant bits). If you have an interest in motivation, it is well worth your time because he grapples with the problem of intrinsic vs. extrinsic rewards. You see, activities are intrinsically motivating if they help you fulfill your inherent desire for personal growth by achieving some kind of competence (“I am good, getting better, mastering this”); if they help learners feel they are working towards their own set of goals with some amount of autonomy (“I am in control and doing things that match my values”); and if they contribute to the sense of relatedness that learners feel by being part of a group, or some kind of purposeful movement  larger than themselves (“I am a part of something here that I think is kind of cool or important”). Some creative and engaging activities just do this naturally. Dan Pink, in his famous TED talk promoting his book Drive, both explains this nicely and makes a pretty strong case for it. Intrinsic motivation springs from within when people are engaged in work/study/activities that align with their needs for competence, autonomy, and relatedness. It is  powerful,  wonderful, and  fragile, and you really really want it to grow in your learners. Though you have some ways to cultivate it, intrinsic motivation does not come about as a result of tool kit rewards that you can just pull out. It emerges as a result of the learner’s experience–from the teacher’s success in designing interactions and engineering instructional environments. Extrinsic motivation, on the other hand is all the trifling enticements and punishments that are used to make subjects do what they are told to do: salaries, grades, threats of prison time, as well as points, badges, leaderboards, and other tools of gamification. Amy Jo Kim has a nice slide from  great presentation she made in 2011, illustrating the two groups in the world of games (and gamification).

Everyone agrees that improving intrinsic motivation is the name of the game in education. Indeed, overt use of extrinsic rewards can actually damage intrinsic motivation when the tasks are interesting or require creativity!  That is Mr. Dockterman’s point above. But it is not true that the presence of extrinsic rewards necessarily kills intrinsic motivation. You can still enjoy your job even though you are being paid, and you can still get seriously interested in an assignment even though it will eventually result in a grade.  Here is what Mr. Hecker says the research definitely says (and he has obviously waded through a lot of it) about  rewards when people are engaged in activities that are interesting:

  1. Tangible, expected, contingent rewards reduce free-choice intrinsic motivation, and
  2. Verbal, unexpected, informational feedback, increases free-choice and self-reported intrinsic motivation.

So cheap rewards such as points or prizes will not work only on their own when the tasks should be interesting and engaging as they are. And you’d be dumb to try. But real gamification lies not in the scattershot application of points (or badges, or whatever) but in the design of a learning experience that engages (and delights!) learners and helps them to see where they are going and how they are doing at any one time (feedback). So a better way to define gamification–effective gamification–is that it is the use of game mechanics and game elements to drive engagement and provide meaningful feedback for learners when it is appropriate to do so.

The above paragraph and most of the literature on this subject make a very important distinction regarding the nature of the task learners are engaged in: is it interesting (requires creativity) or is it boring and repetitive? For the former, you can do real damage by breaking out your extrinsic rewards. For the latter, well, maybe a game of dunk the clown could come in handy. Extrinsic rewards “…can encourage positive behavior and outcomes when one is dealing with dull, repetitive, and/or tedious activities”  says Mr. Werback in For the Win (pg. 62). And yes, extrinsic rewards can actually help to nudge people toward more intrinsic motivation. They can help a learner to feel progress toward competency, for example. Or they can make a playful social environment that learners can feel part of. The crucial point for learners like EFL students is to tie gamification to feedback. That is, make learning clear (clear assessment of where the learner is and needs to go; and clear, effective advice on how to get there). Clear formative feedback is essential, and if it can be done in a way that is fun (and leads to improvements in competency, autonomy, and relatedness), then we are well on our way to having a humming learning system in place. As an instructor, it is best to think in terms of feedback loops for our target behaviors (skills, use of strategies, etc.). Good loops make progress clear and they do in a way that is delightful. So we might even better describe gamification as the delightification of feedback. Of course, if your lessons are a series of tedious slogs, then the deployment of extrinsic rewards will eventually flounder and fail. But to get over the tedious bits, gamification can help. Mr. Werbach has three important lessons or guidelines for use of extrinsic rewards:

  1. Unexpected, informational feedback increases autonomy and self-reported intrinsic motivation
  2. Users like to get reinforcement about how they are doing
  3. Users will regulate their own behavior based on which metrics are provided to them

If you have points or badges or leaderboards, you have to have them for something. It is the choice of those categories and the setting of manageable (attainable yet meaningful) steps that are perhaps the biggest part of the teacher’s job with gamification in education. Lee Sheldon, in The Multiplayer Classroom states that “game design, at it’s heart, is deciding what the player can do.” That means both what and how much. And that is true for formative feedback as well. Notice also how both Mr. Werback and Mr Hecker mention verbal, unexpected feedback. This gives us a good idea of what we should be doing with our learners while they are learning. It is not enough to throw gamification tricks or treats at them. Engagement in their learning by the instructor is essential. Focus on feedback. Focus on learning. And try to have some fun!

In this EFL gamification series:

Part 2: Triggers, Ability, and Motivation

Part 3: Mechanics

Part 4: The Downside and How to Avoid It

Part 5: The Whole Hog

Part 6: ARGs

 

 

 

Formative Assessment Pt. 4: Getting Other Learners Involved

Posted on by
Reply

This is the fourth post considering the implementation of Dylan Wiliam’s ideas on formative assessment in EFL classrooms in Japan. The ideas come mostly from his wonderful 2011 book titled Embedded Formative Assessment. You can learn more about Mr. Wiliam from his website or from a BBC documentary titled The Classroom Experiment (available on YouTube: Part 1 and Part 2). The first posting in this series looked at learning intentions. The second looked at eliciting evidence of achievement. The third looked at how and when teachers can best provide feedback to learners. This post will look at cooperative learning and peer involvement in learning. Mr. Wiliam’s point is that when learners are working together and helping each other, they are naturally giving and getting formative feedback.

Real cooperative learning is a little like real communism. It’s a nice idea but in actual practice, too many people just game the system for their own benefit to get maximum reward for minimum effort. Teachers have serious–and well-founded–concerns about the amount and quality of participation that is brought to the group table by all members. Mr. Wiliam’s comparatively short  chapter on activating students as instructional resources for one another approaches the topic with a tone that makes you think he shares at least some of that sense of trepidation. The research is clearly positive, and Mr. Wiliam presents the profound effects that have been found for cooperative learning, if it is done right (which it usually isn’t). Mr. Wiliam explains how it works (motivation, social cohesion, personalization, and cognitive elaboration) and what two elements are crucial (group goals and individual accountability) before ending the the first part of the chapter with a discussion on how many teachers have a problem with pure, uncut cooperative learning (holding everyone accountable by giving everyone in a group the same score as the lowest-scoring member) and then citing stats that show how few teachers are actually making use of real cooperative learning in their classrooms (very, very few). And on that mixed note of confidence, he begins listing his techniques. I’ll get to the techniques I think might work in Japanese high school EFL classes in a moment, but first an educational culture point needs to be addressed.

There seems to be a strong sense that Japanese classrooms are naturally more cooperative because, well, Japanese group culture makes it easier. Mr. Wiliam states the same thing in his book, listing as “proof” the contrasting proverbs of the squeaky wheel gets the grease (US) and the nail that sticks out will get hammered down (Japan). In addition to the book containing  a mistake with the Japanese version of the proverb, I think this generalization is more than a little stereotypical. Anyone who has seen Japanese students “unmotivated” in regular classes come together in a club activity or festival project knows that  group power and individual accountability are impressive but cannot be taken for granted; and anyone who has seen PTA mothers–dedicated, concerned parents all–trying to avoid being elected for positions that require work knows that Japanese, like anyone else I imagine, can go to pretty great lengths to remain uninvolved, despite being a members of a nation known for being responsible and group-oriented. But I don’t want to get on Mr. Wiliam’s back because his main point is sound: we want to get everyone more involved with helping each other because there are great benefits when that happens; and it really matters how you do it.

One idea that any school can use is the “Secret Student.” You can see it in practice in the BBC video. It is a devious bit of peer pressure judo teachers can use to promote better behavior in the classroom and I think it would work brilliantly in Japan. Each day one student is chosen at random as the secret student and his/her behavior is monitored by the teacher(s). If the student’s behavior/participation is good, his/her identity is revealed to the class at the end of the lesson or day. And the whole class gets a point that goes toward some reward (a trip to an amusement park in the video!). If the behavior/participation of the student is unsatisfactory, the identity is not revealed and the class is informed that they did not get a point for that day. This would almost certainly help to improve participation and reduce disruptive behavior (two really big problems in most high school English classrooms). The only problem is what reward can be offered. It would have to be something possible yet motivating.

One technique to get started with cooperative learning is “Two Stars and a Wish.” Students give feedback on other students’ work  by stating two things they like and one thing that they think could be improved. Mr. William suggests using sticky notes for this feedback. He also suggests picking up some of the feedback comments from time to time to teach students how to give better feedback. This last point is important because it is precisely the generally poor quality of student or peer feedback that makes many teachers to unenthusiastic about peer feedback. There are many times in a language course when students are just out and out unable to provide good feedback. But learning how to give feedback well when it is possible to do so is a real learning opportunity that can benefit the giver and (eventually) the receiver. This technique could be used well for anything students write, translate, present, or any time students produce anything in the target language.

One activity that he suggests, “Error Classification,” probably wouldn’t work in a language classroom as he suggests. This activity requires learners to pour over writing examples to classify the errors made. It sounds nice, but it is unlikely the learners would be able to do this at all but the most proficient of classes. And even if they could, spending so much time on superficial mechanical errors  may not be a good idea. Another activity, “Preflight Checklist,” might be much better for student writing assignments. For this activity, students are given a list of requirements for the writing assignment (things like proper format, clear topic sentence, logical organization, subject-verb agreement, or whatever the teacher is focusing on at the time). Another student is responsible for checking the writing and signing off, meaning certifying that the first student’s writing meets all the requirements.

And a final activity that I think would work well in EFL classes is providing a little time at the end of a lesson or section for pairs or groups to discuss and report on what they have learned. This can be a nice student-led review, and a chance for teachers to see what has and has not been grasped well.

To really get the benefit of cooperative learning, teachers need get learners to have group goals and accept the idea of shared responsibility and accountability. This may be problematic in many situations for many reasons, depending on the year, the course, and the proficiency and motivation differences of learners. I have recently observed a class where the teacher was making extensive use of group cooperative learning. Out of six groups, it was working for three but not really working for the other three. For it to work, it seems that some training, some acceptance of the approach, some accountability, and a fair bit of time are all necessary. When it comes to cooperative learning in Japan, perhaps introducing more chances for learners to see, formatively assess, and then communicate that assessment might be the best way to start. Real cooperative learning is hard, takes a serious commitment, and can all be for naught if not done (and embraced) well.

Next: Part 5, Encouraging greater autonomy and ownership of learning.

 

Formative Assessment Pt. 3: Moving Right Along

Posted on by
Reply

This is the third post considering the implementation of Dylan Wiliam’s ideas on formative assessment in EFL classrooms in Japan. The ideas were gleaned from his wonderful 2011 book titled Embedded Formative Assessment. The first post in this series looked at learning intentions. The second looked at eliciting evidence of achievement. This post will consider how and when teachers can best provide feedback to learners. This part of the book takes up the theoretical rationale for giving feedback.

Let’s start with a question: is a grade feedback? That is, is it information–meaningful, understandable, actionable information–that contributes to the learning process? Mr. Wiliam says usually it is not. In the language of assessment, there is summative assessment and formative assessment. And grades are not formative assessment. And in Mr. Wiliam’s view, formative assessment is really all that matters.

If we think carefully about it, and Mr. Wiliam has, we can see that there are four possible responses to feedback: the learner can change his behavior (make more or less effort); the learner can change his goal (either increase or reduce his aspiration); the learner can abandon his goal altogether (decide that it is too hard or too easy); or the learner can just reject or ignore the feedback. As teachers, we know which of these actions we want learners to take, but what the learner actually does depends on how he sees the goal, the feedback, the feedback giver, and a host of other factors. Feedback seems straightforward in the teaching/learning culture we grew up with. But it is not. In fact, getting it right is really hard. But before we even try to get it right, a more fundamental mindset change is necessary. We have to understand that much of the “feedback-giving” we have traditionally done as teachers has been a waste of time–our time mostly–and has not contributed to learning. Much of or the “feedback-giving” we thought was so important, turns out to either have negligible effect or even negative effect. Yup, negative.

Feedback needs to “cause a cognitive rather than emotional reaction in learners”. It must “make learners think”, and it is only effective “if the information fed back to the learner is used by the learner in improving performance.” And this is why just giving grades is problematic. Students first look at their grades, then they look at the grades of other students, and they generally don’t even read those elaborate comments you spent all that time writing. Providing good feedback is difficult. It requires breaking down each learning intention into micro-skills, or micro-concepts, or significant units, and then being able to identify exactly what the learner is not doing right and how he can improve. The timing is also important. Performance must be fresh in a learner’s mind and there must be time to make use of that feedback on subsequent performance. The amount is important. It must be focused enough to be understandable and actionable. And learners need to believe they have the power to make changes that lead to improvement. They have to trust the teacher and believe in themselves. These are not givens. Teacher praise of effort (see Carol Dweck, who Mr. Wiliam cites often in this chapter) affects this, but so do task perception and the social atmosphere of the classroom.

For language classes with their combination of knowledge learning and skill building, this is a challenge that will require at least two distinct approaches. For skill building, the teacher must act more like a coach. Speaking, writing, listening, and reading must be broken down into micro-skills and learners need to be given feedback on each one so that they and the teacher get a picture of how they are doing and what they need to improve. Let’s take listening as an example. Mr. Wiliam suggests a chart of micro-skills based on the rubric of learning intentions for the course and a score of 0, 1, or 2 for each. 0 means no evidence of mastery; 1 means some evidence of mastery; and 2 means strong evidence of mastery. Both the learner and the teacher get a good idea of what is being done well and not so well, and the rubric (provided earlier) clearly states the conditions of mastery performance. The teacher can then concentrate on giving advice for improving performance. Let’s say the micro-skills include  genre identification, understanding reduced speech, identifying transition signals, or keeping up with native speed levels. The teacher has ways of checking all of these and knows ways of improving each of them.

For productive skills like conversation skills or presentation skills, the same can be done. In addition, video can be used to give feedback and provide a marker against which future performance can be judged (though Mr. Wiliam doesn’t specifically suggest this in the book). I tried this back in the day of VHS analog video and it worked really well, though it was very difficult to get learners to watch critically and reflect on their performance and think about how to improve it. The original idea came out of work done at Nanzan University in the 1990s by Tim Murphey, Linda Woo, and Tom Kenny (here is a later article explaining how it is done). Recently, with digital video and with every student sporting a smartphone or a tablet, this can be done much more easily. Techsmith has a brilliant app available for exactly this purpose, called Coach’s Eye. It allows you shoot and annotate a video and then share it.

For any kind of written work (translations, example sentences, paragraphs, essays, culture notes, etc., something Mr. Wiliam suggests is providing feedback without the grade. This can be done individually or in groups. For groups of four, for example, essay comments can be handed out separately on four sheets of paper. The four corresponding essays are also handed out and the learners in the group must work to match the comments to the paper. This forces them to consider the comments and it gives them a way to compare their performance on specific criteria against that of others. After that–and this is a critical step–the learners are given a chance to make adjustments to their papers and resubmit them for actual grading.

Mr. Wiliam quotes Alfie Kohn in the chapter: “never grade a student while they are still learning.” This is good advice. It can help a teacher to get into the best mindset to move learning along. Mr. Wiliam provides a strong case for doing this. The differences in learning outcomes between classes that employ formative assessment and those that do not are stunning. Teachers should be coaches, encouraging, developing, and training essential skills for performance. Formative assessment is the key, I believe, to getting teachers to assume a more effective role in the classroom and to building a community of learning. More on that last point when we look at what Mr. William has to say about leveraging peer feedback in the next post.

Next: Part 4, Getting other learners involved.

Formative Assessment Pt. 2: Eliciting Evidence of Achievement

Posted on by
Reply

This the second post of a series  considering  Dylan Wiliam’s ideas on embedded formative assessment in EFL  classes at high schools in Japan. Mr. Wiliam is a proponent of assessment for learning, a system where teachers work closely with learners to guide them to better learning. In the previous post, I looked at learning intentions, picking up some of his recommendations and describing how they might fit in EFL classes.  To learn more about Dylan Wiliam, you can visit his website, or read this article from The Guardian, or read his latest book about why and how to make greater use of formative assessment, Embedded Formative Assessment. A BBC documentary of his initiatives called The Classroom Experiment is also available on YouTube (Part 1 and Part 2). Some of the ideas in this post are observable in the TV program and I encourage you to watch it. The book is much richer than the program and I encourage you to get and read it.

In Chapter 4 of his book, Dylan Wiliam illustrates a problem with a nice story. Ask small children what causes wind and they might say it’s trees. They are not being stupid, they are using their observations and creatively making sense of what they see. But it’s a classic confusion of correlation and causation. Your own students would never do such a thing, would they? Oh, yes, they would. They misunderstand, misinterpret, overgeneralize, oversimplify, etc. etc.  probably more often than you think; and it’s your job as a teacher to catch them when they do. By eliciting evidence of learning (or lack of learning), we make it easier for learners to stay on the path of learning. It is important–crucial–that learners and teachers know if learners are on that path, or are veering into the trees (as it were). Very often students manage to achieve the correct answer without really understanding why. But by cleverly using questions and other techniques and attentively listening to learners, we (and they) can get a better idea of how they are progressing. Teachers, aware of some of the common problems learners regularly experience, should give learners the opportunity to make those common errors. This garden path/trap technique is unfair on a test, but is a useful tool in assessment where the goal is to make error salient to  the learner, his peers, and the teacher.

At present, too few of the questions teachers ask in class help to do this. In research cited by Mr. Wiliam for an elementary school class, 57% of teacher questions were related to classroom management, 35% was used for rehearsing things students already know, and only 8% required students to analyze, make inferences, or to generalize–in a word, to really think. For Mr. Wiliam, this represents a good place to make some changes and he suggests things that we do two things at the same time: promote thinking (to see if it’s happening), and increase engagement in that thinking by a larger percentage of the class. These things, unsurprisingly, have “a significant impact on student achievement.”

So how can things be done in language classrooms? Looking at the chapter, it seems that most of this applies to science, history, and math courses, the ponder and wonder courses. Language courses, especially if we have a strong skill focus element as I pushed for last post, seem to require a different kind of learning. But all disciplines are a combination of skills and knowledge. And the techniques Mr. Wiliam describes can be adjusted for different parts of different courses. I will use the terms question and answer when illustrating the techniques, but they could be used for knowledge questions of usage or application of strategies or skill demonstrations of listening or reading comprehension or pronunciation, etc. So let’s get to them. Once again, I am selecting the techniques I think best match high school EFL courses. This is not a comprehensive list and the examples are mostly illustrated by how I imagine they could be used here in Japan. Here we go.

1) The No Questions By Role Rule (my variation of the No Hands Up Rule). Fortunately in Japan we are not oppressed by that small clique of students in every class who seem to raise their hands to venture answers or provide extra comments for almost every question and statement that comes out of a teacher’s mouth. I’ve probably had fewer hands up to answer questions in my 25 years of teaching here than the average North American teacher experiences on a Monday morning. In order to prevent that small clique from monopolizing class (and learning) time, Mr. Wiliam came up with his No Hands Up rule. In Japan, teachers typically go down the role list or go up and down rows picking the students to answer questions. It’s more democratic, yes, but there are similar problems. I’ve regularly observed students counting the questions and students so they can focus on getting their answer to their question right, completely disregarding every other question. What we need is for all students to be engaged in answering all questions. So instead of the roles or the hands, Mr. Wiliam recommends a pot of popsicle sticks, each with the name of one of the students written on it. The teacher asks a question and then pulls a stick from the pot and asks that student to answer it. It forces all students to pay attention and try to come up with an answer since they don’t know when their name will be called.  Of course, variations on this can make it better for your class (see the video for some of these). Adding more than one stick for some special  students is one way, and putting a student in charge of pulling names is another. Another technique is the Pose-Pause-Pounce-Bounce, which can be used along with the sticks. Ask the question, and give everyone a bit of time to think; then choose a student to answer; then ask another student to evaluate the first student’s answer.

Some of you are already shaking your heads. Too many students will answer with “I don’t know.” It’ll be stick after stick after stick of “I don’t know.” What’ll you do then, huh? Well, Mr. Wiliam offers a few suggestions for that, too, because it is essential that all students be brought into the ring of engagement. Get more students to answer and then ask the I-don’t-knower to choose the best answer. Or gamify things a little. Use game techniques like “Phone a Friend“, or give the student two choices and let him or her gamble on the answer. The key point is: keep them engaged and thinking, no matter what it takes, for as long as it takes. Don’t let them slip into drowsy disengagement in the warm sunlight in the back corner of the class. Sleeping students are a real problem in Japan. Sleeping should not be allowed. A policy of zero tolerance for disengagement should be embraced. It’s not easy and it might negatively impact the brighter, more motivated learners for a while, but in the end it is a better approach, Mr. Wiliam argues. Watch the video of  The Classroom Experiment to see some blowback, though.

2) Hot Seat and Waiting Time. In the Hot Seat technique, one student is chosen to answer several teacher questions. Another student is then chosen to summarize or report on what the first student answered. The teacher then gives his or her evaluation. The reason for this is to give learners enough waiting time to process and evaluate in their own heads the answers of their peers before the teacher provides the “correct” answer. Without that waiting time, learners just listen and wait for the right answer from the teacher rather than develop the habit of evaluating ideas themselves. This, of course, can be done with any questions. Be sure to allow enough time for everyone to hear, process, and assess an answer before you,as the teacher, pronounce judgement on it.

3) Multiple Choice Questions For Thinking. Give the learners a set of three, four, or five sentences and ask them to answer a question about them. Which are grammatically correct? Which are academic and which are more casual? Which grammar rules are true? How are the items related? Which one doesn’t belong in the set? Etc. These are all questions that can stimulate pair, group, or whole-class discussions.

4) Variations for No. 3. There are many ways of making use of questions or multiple choice questions or statements for evaluation mentioned in the book. Giving learners cards (A,B,C,D, for example) that everyone can hold up to display their choices can be a nice way of getting the whole class involved in answering. Exit passes are another variation. Each student must write and submit an answer or an opinion on a paper before leaving the class. This forces all students to participate and gives the teacher something concrete in his/her hands.

Many of these techniques are second nature to many teachers, but it is amazing how many do not ever dip their toes in the waters of achievement checking until they are slashing stokes on the the final tests. Making thinking visible has been a buzzy concept for the last few years. One book promotes many ideas for doing so, one of which fits right in with what Mr. Wiliam is suggesting. When eliciting student responses, ask a follow-up question: What makes you say that? This is something I’ve tried successfully in my own classes. It makes students think more deeply and justify their ideas more. Ideas like this are not only effective in L1 content courses. Ideas and approaches like Mr. Wiliam’s  could work very nicely in EFL classrooms in Japan. At present there is a strong tendency for the teacher to just teach, imparting (or so he/she believes) knowledge to learners. Learners are asked to “study.” But aside from memorizing words, or memorizing the text, they usually don’t know what to do. Next year, the Ministry of Education is pushing for all teachers to teach English in English. Mr. Wiliam’s techniques can fit really nicely with that. The techniques listed above could allow for more meaningful use of English in the classroom, more engagement by all learners, and very possibly more learning.

Next: Part 3, Moving right along.

What Works

Posted on by
Reply

The holy grail of English language teaching in Japan, the thing every educator wants to know, is what is the most effective and most efficient way to successfully teach English. It’s an impossible question, however, since there are many learners at different levels studying for different purposes. Success is rarely defined clearly: it might mean getting through a textbook, or reaching March without losing too much sleep or hair. For higher level schools, it usually means being able to obtain ‘certification’ by passing one of the many proficiency tests offered, or for seniors, doing well on the Center Exam or other university entrance exams.

But tests are only one measure of success, and not always the best one. Japanese students are pretty good at taking tests. As I’ve said before, “you learn what you learn in the way that you learn it.” And if you spend a lot of time focusing on tests, well, then you become familiar with test formats and develop strategies for attacking questions. But you don’t get good at English–that is, practical English for use between humans like the kind used in international  business,  academics, or government–without learning to use it and then actually using it. Companies are frustrated by the large numbers of people with certification but no practical skills. This has led a number of companies, especially ones hoping to expand beyond Japan, to enact pretty strong policies.

Often, the biggest lost opportunities involve the more capable students. It is nothing short of a national tragedy that the most capable learners often spend their high school English hours listening to droning explanations of grammar and vocabulary mechanics, line after line, paragraph after paragraph, all in the name of entrance exam preparation. And when they finish at school, they head off to cram school for more of the same. This system may help kids get into university, but it does them no other favors. It is not good for the learners, it is not good for the teachers, and it is not good for businesses that need a workforce with better English (and other) language skills.

But there is hope for these higher level schools. On the weekend I observed the early rounds of the All Japan High School English Debate Tournament held this year in Chiba. Visit the website for more info, but you can’t really get a sense of the amazing things these kids can do without seeing them debate. I mean, wow! Granted, these are kids from the best schools in the country, but still, it was very impressive to see them perform. I was expecting the prepared speeches that opened the debates to be very good and then everything else to be a series of pauses, mumbles, and jumbled utterances. But I was so completely and pleasantly surprised. They presented arguments, they listened, they analyzed, they asked questions, dissected arguments, and summed up. It was all in English, sometimes choppy, sometimes with phonological problems, but always understandable and often stunningly good. What I saw was the best of the best, but I became convinced that debate is a valuable activity for students at higher level high schools. I think it is a good alternative  to the English for Exam Purposes that passes as the be-all and end-all at most higher level schools now. It allows the students to practice all fours skills, develop critical thinking and higher level thinking skills, and develop research and presentation skills. In many ways, I think that debate may be just what shingakkou need.

 

Thinking or Not: Daniel Kahneman’s Thinking, Fast and Slow

Posted on by
1

Thinking, Fast and Slow book cover

What you see is all there is, WYSIATI, is one of the problems you have. It’s one of the problems we all have. It has to do with the  way the brain shows too strong a preference for information available in the immediate environment over information not in the immediate environment. Ever been stuck in a meeting where you are trying to to generate ideas and the same things keep popping up? That’s just our brains spinning their wheels in the immediate environment. It’s one of the heuristic techniques our brains are prone to rely on. It’s one–one of many–of our foibles as a species. And the fact that we keep running meetings in the same way suggests that we have a desperate need to become more aware of it.

But we probably won’t.

Daniel Kahneman has been researching the way way people think and make decisions for years. As a young officer in the Israeli Defense Forces armed with a fresh B.Sc. in psychology, he screened recruits for officer training using the leaderless group challenge. As they struggled under the hot sun to complete some sort of task involving getting over a barrier, he watched them and took notes on the performance of the individuals. He checked boxes and gave high ratings to kids who took charge and organized their fellow soldiers. But the artificial nature of the test, and the fact that all ratings were based on one observation made him suspicious of the limitations–or more correctly, the tendencies, the bad habits–of our minds. As a rater, he began to suspect how powerful WYSIATI is and how it muscles aside any doubts of how things could be with the same recruits on a different day. For a while he was very happy and very confident with his ratings. But that confidence itself made him suspicious. He began to see it an an illusion, an illusion of validity the mind presents us  with. It is seductive because hey, we’re busy and we’ve got an important job to do. We let our brains go with their little shortcuts. We avoid the hard work, the hard thinking. This was the first of many illusions he uncovered or encountered in his career and he goes through his experiences with each one in delightful and insightful detail.

It has been a long and illustrious career  (including a Nobel prize), and the amount of research and discovery here is impressive. This book  is more than just a pop psychology best seller. There really is a lifetime of wisdom here and you would be well to read it. A lot of the content has been covered elsewhere, that’s true (see my reviews of The Invisible Gorilla and Why Everyone Else is a Hypocrite, for example, or take a look at  You are Not So Smart by David McRaney). But I don’t know of any book that is as comprehensive as this one in explaining our limitations. In a series of five units he covers a lot of ground and a lot of years of research. He introduces us to the two thinking systems of the brain, the fast, automatic System 1 and the slow, careful, and reluctant (lazy) System 2. In the second part, he talks about the ease with which some thinking occurs (metaphorical thinking, associative thinking, and causality) and why it is so difficult for us to think in different ways (statistically, for example). In the third part, he really dresses humanity down. Our bad habits,  ignorance, and unwarranted overconfidence get addressed nicely and this section is great fun to read. Later sections take the book in a different direction–economics. I found them to be  less engaging than the earlier parts after a while, with the exception of the part about the experiencing self and the remembering self near the end, which is also the topic of a TED talk he made in 2010.

As a teacher and as a human I found a lot to think about here. Awareness of the tendencies we have is really our only weapon against the habits of our thinking processes. Each chapter ends with little lines of dialog, little bits of wisdom or little rules for being diligent. Print them out and pin them all over your cubicle or kitchen. It might be ultimately a little hopeless, but there are much much worse ways to make use of paper. This book will convince you of that.

A Month and a Half of MOOCs

Posted on by
Reply

I enrolled myself in two MOOCs through Coursera.org., one on gamification and one on statistics. I wanted to experience a  MOOC. I wanted to see how it compared to the self-directed exploration and research I do regularly; specifically, I wanted to see if following a structured approach with 70,000 other people could be beneficial.

These two courses ran at roughly the same time, with Gamification starting a week earlier. Gamification demanded about three hours of time each week, more later when some of the assignments got longer. Statistics demanded more than double (triple?)  that from the first week. The lectures were longer, the concepts more difficult, and the assignments required working with some rather non-intuitive open-source software called R. I found it impossible to continue both courses at the same time. I stopped trying to stay current with the syllabus and just downloaded the Statistics videos and assignments, planning to return to them when Gamification is finished. And that’s what I’m doing. It feels strange, like starting a marathon a few hours late, but it still works.

Which brings me to my first point: MOOCs are as course-like as you make them. If you want the experience of an online course, there are boards for discussion, meet-up groups, wikis, and Facebook groups you can get really active in. You can also just follow along with the videos and the assignments at your own pace. There is actually a lot of flexibility. Professor Werbach who taught the Gamification course  mentioned that only about 12% of the enrolled students were actually handing in the assignments. There are a lot of lurkers. There are a lot of participation options. And that’s not a bad thing, I think.

Next, I really didn’t have high expectations for the courses as e-learning experiences, as courses. But the ones I took were much more interactive and responsive to students than I thought they would be. I thought they would be more canned. I thought they would be finished products produced months before the actual delivery. But instead I realized that they we being constructed–and adjusted–as the course went on. There were very real changes made in response to technical problems (system outages, for example) and content challenges (extra instructional support added on the fly when learners found some things especially difficult). There were e-mail updates and previews, messages of encouragement, bits of behind-the-scenes information, and  Facebook and Twitter participation by instructors.

But were they worthwhile? I have to say yes. They were more challenging than I expected. The lectures were video-based and re-created the feel of a professor and a powerpoint (some room for improvement there, I think, but not a problem), and the assignments were well designed in that they really required application of course content. Gamification relied on quizzes and multiple peer assessments of assignments–a logistically ambitious and mostly successful idea. Statistics relied on quizzes, and assignments that generated a lot of engagement and interaction as learners scrambled to complete them and help each other. But there were things that I knew and couldn’t skip, and things that I wanted to look at more in depth but had to move on from. Course forums and wikis helped a little with this, but an extended reading/resource list would have been helpful. Gamification provided one but it was at little shallow. the biggest “problem” was perhaps the focus of the course. There was a small difference in focus between the courses and my own interests. I wasn’t in Gamification to learn about business, for example, and I would have preferred a more educational focus. But that’s a minor point.

Compared with online courses I’ve taken before, I felt more like I was going through it alone because there weren’t as many regular discussions/assignments with interaction among a group of people you get to know virtually. That was obviously missing. But as I said earlier, there were forums and Facebook, etc. interactions that I chose to not get active in. The MOOCs were more limited in the range of interaction, but they did not compare that unfavorably.

Bottom line: these MOOCs were a great resource and I plan to take more. I’ve been recommending them to everyone. I think they represent something potentially very exciting. A recent presentation by John Daniels (via Tony Bates) lists up some of the potential effects MOOCs could have. I recommend the report and trying a MOOC yourself.

The New Normal

Posted on by
Reply

 

I’ve been thinking a lot recently. And that has been changing me. It’s the same for everyone–an amazing ongoing transformation, an alteration of thought patterns and perceptions as we go about our daily lives and our brains try in their own quirky ways to make sense of things.

I’ve been reading Daniel Kahneman’s Thinking, Fast and Slow. It’s a wonderful  book that reminds me a little of In Search of Memory by Eric Kandel in that it is written by a giant nearing the end of his career. There is wisdom and balance and a lifetime of knowledge in it. I previously posted on a TED talk he gave about a year before Thinking, Fast and Slow was first published on the experiencing self and the remembering self. But something jumped out at me from the book the other day–the idea of how the brain makes things seem normal, even things that are very unusual. In the book he tells a story that reminded me of one of my own experiences. I was travelling around Europe, a 20-year-old kid on break from uni, with a Let’s Go guide. I moved along a fairly worn path at one point: Venice-Florence-Rome-Naples-Pompei-Corfu-Athens-Istanbul. I met a guy–I don’t remember his name so I’ll call him Guy X–in Rome, at a youth hostel we were staying at. He told me how he kept on meeting the same people as he traveled through Europe. His story sort of freaked me out. It was full of weird coincidences with different people in different parts of his trip. I didn’t know it at the time, but I was to become another paragraph in a story I’m pretty sure he’s been telling ever since. You see,  I later met him again at the youth hostel in Naples. Nothing strange there. The Let’s Go guide only listed 5 or 6 accommodation options for a city. But a few weeks later, I was sitting in a cafe in Athens, and there he was at the next table. And a week or so later, I took a ferry up the Bosphorus toward the Black Sea. And there he was, sitting in the seat beside me as we chugged past the Rumelihisari Castle. Mr. Kahneman, in describing a somewhat similar experience, notes that although it was unusual to meet Guy X a second (then a third, then a fourth time), it is even weirder to think about how normal it became to do so. When I met him on the ferry in Turkey, I was not in the least bit surprised. Guy X had become The Guy I Meet Unexpectedly. Indeed had I met someone else on that ferry that I had previously met once in my travels, it would have felt infinitely more surprising. Because that’s what brains do–they adjust to new events and make them seem normal.

And this got me thinking about culture again. Not culture in some quasi-national sense. And not culture in some don’t-stick-your-chopsticks-in-your-rice kind of nuggets of behavior. But culture in the sense of what we find acceptable; what we choose to include and what we choose to embrace, and what we decide we don’t need. That is to say, what we think is normal at any given time. Culture is the water we fish swim in. We notice changes, pause for a moment to let it register, and then swim on. The changes just become part of the environment if they are not life-threatening or similarly significant. But to go back and see what was normal before can be very shocking. A  kind of a Holy-cow!-Were-we-really-like-that? feeling arises because the new normal has become, well, so very normal. Watching the wonderful  drama series Mad Men makes this clear. In the early episodes, it is shocking to find people smoking in office meetings, or in offices at all for that matter. Men smoke, women smoke (even a pregnant woman!), and men drink in the office (“Should we drink before or after the meeting–or both?” one character asks). It was shocking for me to see this, and yet I when I thought about it, I do remember my father smoking in the car with the four of us kids in the back, unbuckled and inhaling almost as much as my dad. I remember my parents putting out cigarettes at parties the way my wife and I put out plates of olives. I remember smoke rising here and there at cinemas, the light of the projector catching it, or people smoking on airplanes. And I remember people doing almost everything with a cigarette in their hand or lips. You just don’t see that anymore. And we’ve all gotten very used to not seeing it anymore.

At my job, I often think of teaching as a collection of cultural activities. What is normal in the world of teaching is highly relative, but there are general trends and practices that are part of the ebb and flow of English teaching culture. And Japan has a lot of peculiarities when it comes to teaching culture. The widespread acceptance of of communicative language teaching, coupled with an almost equally widespread lack of implementation of its methodology, is one of the most glaringly peculiar, for example. I am always suspicious of teachers who make the claim that what they do in class is the “way Japanese teachers teach.” That’s because I have seen that change, and I know it will change again in the future, even if I am not happy about the speed (or tack) of that change. I once watched a teacher teach two back-to-back English lessons, one an English I lesson and the other an Oral English lesson. For the first, the teacher spoke almost completely in Japanese, and focused on explaining the words and grammar of the text. In the second, the same teacher spoke almost completely in English and led the students through a mostly communicative lesson. This seemed perfectly normal to the teacher in charge; yet it seemed very strange to me. At some point, this teacher will likely gravitate to one type of teaching–a type which is both effective and matches the needs of her learners and her own personality and style–and forget that she ever thought it was necessary to use two completely different approaches in two different classes. It’ll probably be proceeded by a change in the curriculum that acknowledged that a four-skills course like English I and a listening and speaking course like Oral Communication should be such separate critters. And then it will seem as natural and normal as anything ever was.

That’s how the new normal will feel, I’m sure. But getting there is still going to take some time.

 

MOOCs

Posted on by
1

I would very much like to do my doctorate. I have several ideas and have spent  time scrounging around on the web for some place where my budget of time and cash and my schedule can all be accommodated in a blissful combination, and the fetters of my everyday responsibilities will fall away and I will be able to concentrate on learning and writing with a chorus of heavenly voices as BGM. Yup, it’s a fantasy. I know it. It just ain’t gonna happen. I’m too busy and too poor and too invested in my family to commit myself to an Ed.D. program at this time. So I just study and research by myself. I’m on my own, with my books and my copied articles, and my PDFs on my ipad, and my hop-scotching from topic to topic as I follow my inclinations without discussions or deadlines.

But now there is a new game in town. MOOCs, or Massive Open Online Courses, are crawling out of the academic woodwork and getting all sorts of attention. Game-changers, they’re being called by some enthusiastic folk (especially the people involved in the three most prominent at the moment: Coursera, Udacity, and Edx). Of course, there is dissent as well, this being the internet and all (this article sums up the pros and cons pretty well). They don’t really level the playing field and allow wider access–real access–to higher education. Well, they kind of do, but point taken. They are not real courses with interaction with classmates and the professor! Yup, with thousands of people enrolled,you can’t expect to actually get to talk to anyone. They are just advertising for big name unis! Probably true, at least partially. I don’t care that much. The fact is that they are a niche right now for people like me who want to learn, who are learning on their own anyway, to add a little structure and direction to their studies. And that’s a good thing. The world is like that anyway now. You can learn almost anything you want on your own; but you only get a degree if you pay and work at it long and hard enough. It is the way it is.

So, I’ve enrolled myself into two courses at Coursera. I want to give it a spin. I’m hoping that I learn something, something that I may have overlooked if if I tried to learn the subject on my own. Anything good that emerges from the experience will be welcome, and unexpected, and precious. I know, like everyone knows, that MOOCs are not the answer to the problems of higher education. But they are a nice gesture, from a lot of people who don’t really need to make such gestures. For the time being, I’m still thinking of it as a slightly less lonely version of learning alone.