Formative Assessment Pt. 3: Moving Right Along

Posted on January 23, 2013 by Marcel

This is the third post considering the implementation of Dylan Wiliam’s ideas on formative assessment in EFL classrooms in Japan. The ideas were gleaned from his wonderful 2011 book titled Embedded Formative Assessment. The first post in this series looked at learning intentions. The second looked at eliciting evidence of achievement. This post will consider how and when teachers can best provide feedback to learners. This part of the book takes up the theoretical rationale for giving feedback.

Let’s start with a question: is a grade feedback? That is, is it information–meaningful, understandable, actionable information–that contributes to the learning process? Mr. Wiliam says usually it is not. In the language of assessment, there is summative assessment and formative assessment. And grades are not formative assessment. And in Mr. Wiliam’s view, formative assessment is really all that matters.

If we think carefully about it, and Mr. Wiliam has, we can see that there are four possible responses to feedback: the learner can change his behavior (make more or less effort); the learner can change his goal (either increase or reduce his aspiration); the learner can abandon his goal altogether (decide that it is too hard or too easy); or the learner can just reject or ignore the feedback. As teachers, we know which of these actions we want learners to take, but what the learner actually does depends on how he sees the goal, the feedback, the feedback giver, and a host of other factors. Feedback seems straightforward in the teaching/learning culture we grew up with. But it is not. In fact, getting it right is really hard. But before we even try to get it right, a more fundamental mindset change is necessary. We have to understand that much of the “feedback-giving” we have traditionally done as teachers has been a waste of time–our time mostly–and has not contributed to learning. Much of or the “feedback-giving” we thought was so important, turns out to either have negligible effect or even negative effect. Yup, negative.

Feedback needs to “cause a cognitive rather than emotional reaction in learners”. It must “make learners think”, and it is only effective “if the information fed back to the learner is used by the learner in improving performance.” And this is why just giving grades is problematic. Students first look at their grades, then they look at the grades of other students, and they generally don’t even read those elaborate comments you spent all that time writing. Providing good feedback is difficult. It requires breaking down each learning intention into micro-skills, or micro-concepts, or significant units, and then being able to identify exactly what the learner is not doing right and how he can improve. The timing is also important. Performance must be fresh in a learner’s mind and there must be time to make use of that feedback on subsequent performance. The amount is important. It must be focused enough to be understandable and actionable. And learners need to believe they have the power to make changes that lead to improvement. They have to trust the teacher and believe in themselves. These are not givens. Teacher praise of effort (see Carol Dweck, who Mr. Wiliam cites often in this chapter) affects this, but so do task perception and the social atmosphere of the classroom.

For language classes with their combination of knowledge learning and skill building, this is a challenge that will require at least two distinct approaches. For skill building, the teacher must act more like a coach. Speaking, writing, listening, and reading must be broken down into micro-skills and learners need to be given feedback on each one so that they and the teacher get a picture of how they are doing and what they need to improve. Let’s take listening as an example. Mr. Wiliam suggests a chart of micro-skills based on the rubric of learning intentions for the course and a score of 0, 1, or 2 for each. 0 means no evidence of mastery; 1 means some evidence of mastery; and 2 means strong evidence of mastery. Both the learner and the teacher get a good idea of what is being done well and not so well, and the rubric (provided earlier) clearly states the conditions of mastery performance. The teacher can then concentrate on giving advice for improving performance. Let’s say the micro-skills include genre identification, understanding reduced speech, identifying transition signals, or keeping up with native speed levels. The teacher has ways of checking all of these and knows ways of improving each of them.

For productive skills like conversation skills or presentation skills, the same can be done. In addition, video can be used to give feedback and provide a marker against which future performance can be judged (though Mr. Wiliam doesn’t specifically suggest this in the book). I tried this back in the day of VHS analog video and it worked really well, though it was very difficult to get learners to watch critically and reflect on their performance and think about how to improve it. The original idea came out of work done at Nanzan University in the 1990s by Tim Murphey, Linda Woo, and Tom Kenny (here is a later article explaining how it is done). Recently, with digital video and with every student sporting a smartphone or a tablet, this can be done much more easily. Techsmith has a brilliant app available for exactly this purpose, called Coach’s Eye. It allows you shoot and annotate a video and then share it.

For any kind of written work (translations, example sentences, paragraphs, essays, culture notes, etc., something Mr. Wiliam suggests is providing feedback without the grade. This can be done individually or in groups. For groups of four, for example, essay comments can be handed out separately on four sheets of paper. The four corresponding essays are also handed out and the learners in the group must work to match the comments to the paper. This forces them to consider the comments and it gives them a way to compare their performance on specific criteria against that of others. After that–and this is a critical step–the learners are given a chance to make adjustments to their papers and resubmit them for actual grading.

Mr. Wiliam quotes Alfie Kohn in the chapter: “never grade a student while they are still learning.” This is good advice. It can help a teacher to get into the best mindset to move learning along. Mr. Wiliam provides a strong case for doing this. The differences in learning outcomes between classes that employ formative assessment and those that do not are stunning. Teachers should be coaches, encouraging, developing, and training essential skills for performance. Formative assessment is the key, I believe, to getting teachers to assume a more effective role in the classroom and to building a community of learning. More on that last point when we look at what Mr. William has to say about leveraging peer feedback in the next post.

Next: Part 4, Getting other learners involved.

Formative Assessment Pt. 2: Eliciting Evidence of Achievement

Posted on January 18, 2013 by Marcel

This the second post of a series considering Dylan Wiliam’s ideas on embedded formative assessment in EFL classes at high schools in Japan. Mr. Wiliam is a proponent of assessment for learning, a system where teachers work closely with learners to guide them to better learning. In the previous post, I looked at learning intentions, picking up some of his recommendations and describing how they might fit in EFL classes. To learn more about Dylan Wiliam, you can visit his website, or read this article from The Guardian, or read his latest book about why and how to make greater use of formative assessment, Embedded Formative Assessment. A BBC documentary of his initiatives called The Classroom Experiment is also available on YouTube (Part 1 and Part 2). Some of the ideas in this post are observable in the TV program and I encourage you to watch it. The book is much richer than the program and I encourage you to get and read it.

In Chapter 4 of his book, Dylan Wiliam illustrates a problem with a nice story. Ask small children what causes wind and they might say it’s trees. They are not being stupid, they are using their observations and creatively making sense of what they see. But it’s a classic confusion of correlation and causation. Your own students would never do such a thing, would they? Oh, yes, they would. They misunderstand, misinterpret, overgeneralize, oversimplify, etc. etc. probably more often than you think; and it’s your job as a teacher to catch them when they do. By eliciting evidence of learning (or lack of learning), we make it easier for learners to stay on the path of learning. It is important–crucial–that learners and teachers know if learners are on that path, or are veering into the trees (as it were). Very often students manage to achieve the correct answer without really understanding why. But by cleverly using questions and other techniques and attentively listening to learners, we (and they) can get a better idea of how they are progressing. Teachers, aware of some of the common problems learners regularly experience, should give learners the opportunity to make those common errors. This garden path/trap technique is unfair on a test, but is a useful tool in assessment where the goal is to make error salient to the learner, his peers, and the teacher.

At present, too few of the questions teachers ask in class help to do this. In research cited by Mr. Wiliam for an elementary school class, 57% of teacher questions were related to classroom management, 35% was used for rehearsing things students already know, and only 8% required students to analyze, make inferences, or to generalize–in a word, to really think. For Mr. Wiliam, this represents a good place to make some changes and he suggests things that we do two things at the same time: promote thinking (to see if it’s happening), and increase engagement in that thinking by a larger percentage of the class. These things, unsurprisingly, have “a significant impact on student achievement.”

So how can things be done in language classrooms? Looking at the chapter, it seems that most of this applies to science, history, and math courses, the ponder and wonder courses. Language courses, especially if we have a strong skill focus element as I pushed for last post, seem to require a different kind of learning. But all disciplines are a combination of skills and knowledge. And the techniques Mr. Wiliam describes can be adjusted for different parts of different courses. I will use the terms question and answer when illustrating the techniques, but they could be used for knowledge questions of usage or application of strategies or skill demonstrations of listening or reading comprehension or pronunciation, etc. So let’s get to them. Once again, I am selecting the techniques I think best match high school EFL courses. This is not a comprehensive list and the examples are mostly illustrated by how I imagine they could be used here in Japan. Here we go.

1) The No Questions By Role Rule (my variation of the No Hands Up Rule). Fortunately in Japan we are not oppressed by that small clique of students in every class who seem to raise their hands to venture answers or provide extra comments for almost every question and statement that comes out of a teacher’s mouth. I’ve probably had fewer hands up to answer questions in my 25 years of teaching here than the average North American teacher experiences on a Monday morning. In order to prevent that small clique from monopolizing class (and learning) time, Mr. Wiliam came up with his No Hands Up rule. In Japan, teachers typically go down the role list or go up and down rows picking the students to answer questions. It’s more democratic, yes, but there are similar problems. I’ve regularly observed students counting the questions and students so they can focus on getting their answer to their question right, completely disregarding every other question. What we need is for all students to be engaged in answering all questions. So instead of the roles or the hands, Mr. Wiliam recommends a pot of popsicle sticks, each with the name of one of the students written on it. The teacher asks a question and then pulls a stick from the pot and asks that student to answer it. It forces all students to pay attention and try to come up with an answer since they don’t know when their name will be called. Of course, variations on this can make it better for your class (see the video for some of these). Adding more than one stick for some special students is one way, and putting a student in charge of pulling names is another. Another technique is the Pose-Pause-Pounce-Bounce, which can be used along with the sticks. Ask the question, and give everyone a bit of time to think; then choose a student to answer; then ask another student to evaluate the first student’s answer.

Some of you are already shaking your heads. Too many students will answer with “I don’t know.” It’ll be stick after stick after stick of “I don’t know.” What’ll you do then, huh? Well, Mr. Wiliam offers a few suggestions for that, too, because it is essential that all students be brought into the ring of engagement. Get more students to answer and then ask the I-don’t-knower to choose the best answer. Or gamify things a little. Use game techniques like “Phone a Friend“, or give the student two choices and let him or her gamble on the answer. The key point is: keep them engaged and thinking, no matter what it takes, for as long as it takes. Don’t let them slip into drowsy disengagement in the warm sunlight in the back corner of the class. Sleeping students are a real problem in Japan. Sleeping should not be allowed. A policy of zero tolerance for disengagement should be embraced. It’s not easy and it might negatively impact the brighter, more motivated learners for a while, but in the end it is a better approach, Mr. Wiliam argues. Watch the video of The Classroom Experiment to see some blowback, though.

2) Hot Seat and Waiting Time. In the Hot Seat technique, one student is chosen to answer several teacher questions. Another student is then chosen to summarize or report on what the first student answered. The teacher then gives his or her evaluation. The reason for this is to give learners enough waiting time to process and evaluate in their own heads the answers of their peers before the teacher provides the “correct” answer. Without that waiting time, learners just listen and wait for the right answer from the teacher rather than develop the habit of evaluating ideas themselves. This, of course, can be done with any questions. Be sure to allow enough time for everyone to hear, process, and assess an answer before you,as the teacher, pronounce judgement on it.

3) Multiple Choice Questions For Thinking. Give the learners a set of three, four, or five sentences and ask them to answer a question about them. Which are grammatically correct? Which are academic and which are more casual? Which grammar rules are true? How are the items related? Which one doesn’t belong in the set? Etc. These are all questions that can stimulate pair, group, or whole-class discussions.

4) Variations for No. 3. There are many ways of making use of questions or multiple choice questions or statements for evaluation mentioned in the book. Giving learners cards (A,B,C,D, for example) that everyone can hold up to display their choices can be a nice way of getting the whole class involved in answering. Exit passes are another variation. Each student must write and submit an answer or an opinion on a paper before leaving the class. This forces all students to participate and gives the teacher something concrete in his/her hands.

Many of these techniques are second nature to many teachers, but it is amazing how many do not ever dip their toes in the waters of achievement checking until they are slashing stokes on the the final tests. Making thinking visible has been a buzzy concept for the last few years. One book promotes many ideas for doing so, one of which fits right in with what Mr. Wiliam is suggesting. When eliciting student responses, ask a follow-up question: What makes you say that? This is something I’ve tried successfully in my own classes. It makes students think more deeply and justify their ideas more. Ideas like this are not only effective in L1 content courses. Ideas and approaches like Mr. Wiliam’s could work very nicely in EFL classrooms in Japan. At present there is a strong tendency for the teacher to just teach, imparting (or so he/she believes) knowledge to learners. Learners are asked to “study.” But aside from memorizing words, or memorizing the text, they usually don’t know what to do. Next year, the Ministry of Education is pushing for all teachers to teach English in English. Mr. Wiliam’s techniques can fit really nicely with that. The techniques listed above could allow for more meaningful use of English in the classroom, more engagement by all learners, and very possibly more learning.

Next: Part 3, Moving right along.

Formative Assessment Pt. 1: Learning Intentions

Posted on January 16, 2013 by Marcel

This the first post of a series on considering embedded formative assessment in EFL classes at high schools in Japan. In previous posts (here and here), I mentioned some of the potentially powerful reasons for making use of this type of formative assessment. Dylan Wiliam, a teacher/administrator/researcher/teacher training from the UK believes that the single most effective (and cost-effective!) way of improving learning is for teachers (and learners) to provide assessment for learning, not assessment of learning. This requires a rethinking of the purposes, timing, and techniques of assessment. In Japanese EFL classes, it will likely involve more than this…In this series, I will look at the possible application of Dylan Wiliam’s stages of formative assessment here in Japan. To learn more about Dylan Wiliam, you can visit his website, or read this article from The Guardian, or read his latest book about why and how to make greater use of formative assessment, Embedded Formative Assessment. A BBC documentary of his initiatives called The Classroom Experiment is also available on YouTube (Part 1 and Part 2). But before we go on, it is important to clear one thing up: assessment for learning is perhaps not the assessment you are thinking of if you are thinking about grading. It has very little to do with grading and everything to do with informing the teacher and the students (and possibly others, including peers and parents) about how to learn. So the topic of testing for grading will not be addressed here.

Where are we going? Or more precisely, where am I going? This is the question that should be on the minds of all learners as they select a course or arrive for the first lesson. It is a question that needs to be kept in mind as learners proceed through courses as active monitors and agents in their own learning. But often in institutionalized settings, it is not. Instead, the learners do not voice any expectations they may have and just flip through the textbook for a hint of the things they will learn. It’s frustrating for some, but years of similar starts to courses have made it unquestionably normal.

Too often in high schools in Japan, the teachers actually have a fairly similar experience. They flip through the textbook to see what it is they are going to teach in the upcoming year. That is, many schools fail to create a curriculum with specific skill targets for each year and instead they let the textbooks (OK, Ministry-approved so they must be appropriate, no?) decide what they are going to teach. It is the content of the textbook that becomes the de facto syllabus for the course. Having students learn–usually meaning “memorize”–the content of the textbook becomes everyone’s purpose. And it is at the point of this decision to not make a syllabus with specific skill targets and instead just teach the textbook from start to however far we get, that the first obstacle to deploying embedded formative assessment emerges. For once the textbook becomes the object of learning, it changes the course content into a body of knowledge or information. It shifts the goals of the course from the learner’s skills and ability to something outside the learner. The starting and ending point of learning is no longer the learner, but the percentage of the textbook that the learner can “master.”

That is not to say that the textbook content cannot be a good part of a syllabus for a course. Used flexibly, by a dedicated teacher, a good textbook contains enough interesting activities and content that it can provide structure for a course and facilitate learning. But there’s an expression in English that we need to keep in mind: when the only tool you have is a hammer, every problem begins to look like a nail. For HS teachers in Japan, the textbook becomes the hammer with which they address the needs of every unique learner in the class. It is not the most effective way to teach and it doesn’t have to be this way. With clear skill targets, the teacher and the learners get a way of talking about learning. The teacher gets something she can show, demonstrate, and measure the progress of. The learners get a model and a yardstick. Of course all language courses feature a combination of knowledge content, skill content. But a greater emphasis on skills by everyone in the classroom is necessary to prevent the course from focusing completely on knowledge and understanding, things that will not actually matter that much when learners try to make use of the target language in the real world.

“It is important that students know where they are going in their learning and what counts as quality work, but there cannot be a simple formula for doing this,” says Mr. Wiliam. Look at that first part again: “know where they are going in their learning and what counts as quality work.” The learners need to have a better idea of what they can do now and and what they will be expected to be able to do and know by the end. They need to see it. They need to see themselves, the target, and the gap. This is, at present, not a common way that schools, English departments, or individual teachers approach the kids who come to them to learn. The focus of Mr. Wiliam’s book and assessment for learning (AFL) is entirely the classroom and the learners in it. He does not spend any time discussing placement tests or proficiency tests. Instead, the learners are asked to consider learning intentions for every unit, topic, or module the class will encounter. And he provides several concrete suggestions as examples for how this can be done. Many of them are collaborative in nature. I went through them and pulled out the ones that I thought could be adapted for use in English language classes in Japan. In most cases, the actual example is described as how I would imagine using the technique in HS English classes. If you want the complete list of original examples, you’ll just have to get the book, something I recommend anyway.

First up is passing out 4-5 examples of student work from the previous year. In the book it is done with lab reports, but it could be done with any kind of student writing (or if you have recorded examples of presentations or student speaking, that would work, too). In groups, the learners rank the works and report on how they assessed them. This lets the criteria for better performance become salient through comparison and discussion. Teachers may want to provide some topics or questions to guide the learners’ attention to specific aspects.

A variation of the above involves the work of the present class. After the writing assignment is completed, the teacher collects them all and reads them, selecting what he thinks are the three best examples of student writing. No other feedback is on the paper at this point–no grade and no comments. The teacher hands out copies of the best student writing. The learners are asked to read them for homework and then discuss why the teacher thought these were the best. Then–and here’s the important step–all the students (including the authors of the best papers) get their papers back and are given time to redraft their writing. They then, finally, submit them for a grade.

In “Choose-Swap-Choose” learners choose a good example of their own work from several they have made (a short recorded speech, for example). They then submit these to a partner who then chooses the one he/she thinks is best. The two students discuss their choices if there is disagreement.

One good idea of reading aloud or pronunciation classes has learners in groups practicing the recitation of a short passage in the target language. Each group then chooses the learner who they think has the best accent and the whole class listens in turn to the representatives of each group. The teacher comments on the strengths and weaknesses of each one.

And finally, have the learners try to design their own test or test items for mid-term or end-of-term tests. Of course, this should be done while there is still time to make use of the feedback that emerges from this activity. But in making test items, students clearly show what they think they have learned and what they think is important.

The main thing to point out from all of the above techniques is that they provide feedback to both the learners and the teacher. The learners can use that information to make adjustments to their learning. And the teacher can use it to see what has been learned and how well in order to make adjustments to teaching. All of these techniques promote meta-cognitive skills. They also contribute to the creation of a community of learners. According to Mr. Wiliam, they also definitely lead to better learning. But would this approach work at high schools in Japan? The answer is a great big “it depends.” It depends on the levels of motivation and trust in the classroom. It depends on whether the teacher can afford the time it takes to allow learners to examine and discuss the work of others. And it depends on the mindset of the teachers. They need to be willing to try out a more learner-centered approach to teaching and learning, one with a greater emphasis on skills. Many–too many–teachers prefer to teach content at the students and leave the learning up to them. Too many have their syllabus strapped to the ankles of the syllabuses of the other teachers teaching the course in a given year. There is nothing to do but move along in lockstep. But I think that some of these ideas could be put into practice in almost any school in the prefecture where I work.

In the next post, we’ll look at what Dylan Wiliam says about how to elicit evidence of learning. Part 2: Eliciting evidence.

MOOCs

Posted on August 25, 2012 by Marcel

I would very much like to do my doctorate. I have several ideas and have spent time scrounging around on the web for some place where my budget of time and cash and my schedule can all be accommodated in a blissful combination, and the fetters of my everyday responsibilities will fall away and I will be able to concentrate on learning and writing with a chorus of heavenly voices as BGM. Yup, it’s a fantasy. I know it. It just ain’t gonna happen. I’m too busy and too poor and too invested in my family to commit myself to an Ed.D. program at this time. So I just study and research by myself. I’m on my own, with my books and my copied articles, and my PDFs on my ipad, and my hop-scotching from topic to topic as I follow my inclinations without discussions or deadlines.

But now there is a new game in town. MOOCs, or Massive Open Online Courses, are crawling out of the academic woodwork and getting all sorts of attention. Game-changers, they’re being called by some enthusiastic folk (especially the people involved in the three most prominent at the moment: Coursera, Udacity, and Edx). Of course, there is dissent as well, this being the internet and all (this article sums up the pros and cons pretty well). They don’t really level the playing field and allow wider access–real access–to higher education. Well, they kind of do, but point taken. They are not real courses with interaction with classmates and the professor! Yup, with thousands of people enrolled,you can’t expect to actually get to talk to anyone. They are just advertising for big name unis! Probably true, at least partially. I don’t care that much. The fact is that they are a niche right now for people like me who want to learn, who are learning on their own anyway, to add a little structure and direction to their studies. And that’s a good thing. The world is like that anyway now. You can learn almost anything you want on your own; but you only get a degree if you pay and work at it long and hard enough. It is the way it is.

So, I’ve enrolled myself into two courses at Coursera. I want to give it a spin. I’m hoping that I learn something, something that I may have overlooked if if I tried to learn the subject on my own. Anything good that emerges from the experience will be welcome, and unexpected, and precious. I know, like everyone knows, that MOOCs are not the answer to the problems of higher education. But they are a nice gesture, from a lot of people who don’t really need to make such gestures. For the time being, I’m still thinking of it as a slightly less lonely version of learning alone.

The Social State of the Union: David Brooks Recounts the Life Story of The Social Animal

Posted on August 14, 2012 by Marcel

Like New York Times columnist David Brooks, I have read a lot of books and articles on evolutionary psychology, social psychology, behavioral economics, and neuroscience in the last couple of years. Their influence seems to be everywhere, even if not recognized as such. There is a sense of a paradigm shift, particularly in education, but in other areas as well. People are talking now about learning as a social process influenced by–if not driven by–emotion, for example. So I’ve been listening to podcasts (The Brain Science Podcast, Radiolab, All in the Mind, etc.), following websites (Mind/Shift), following certain people on Twitter (BJ Fogg, for example), and reading books (Brain Rules, The Invisible Gorilla, Incognito, Why Everyone Else is a Hypocrite, etc.).

But David Brooks has read more.

And in The Social Animal, he attempts to summarize all of what he has read in a narrative following two people of different backgrounds and temperaments–from before birth right up until death. Along the way, he weaves in research findings from quite a dizzying amount of disciplines. The life stages and experiences of the two characters allow him the opportunity to expound the new insights researchers and philosophers (let’s be generous here; though we could call the authors of some of the books he cites armchair social and political critics). He talks about physical attraction, attachment, learning, self-control, culture, intelligence, choice, morality, and more. While he is an excellent writer and insightful critic, some sections are better than others. His passion and expertise just come through better. But along the way, we encounter little nuggets of research wisdom, clever descriptions of characters and character types, and a forceful argument that often challenges conventional wisdom. If you’ve read a lot in the field, you’ll see a lot that is familiar. But Mr Brooks’s historical background will help you situate these concepts in the landscape of ideas. That alone makes the book worth reading.

In the end, the book was for me the equivalent of a State of the Union address. It told me where we are, how it is different from what was thought before, and–in a way–how widely these ideas are being accepted. I say “in a way” because the book’s very existence is a kind of testament to the last question. The book is a welcome overview at a time when a lot of new ideas and new perspectives are coming onto peoples’ radar screens. Its acceptance (bestseller) and the status of its author (TED talk here, for example), make this book a high-profile publication. That in itself says a lot.

I feel like a should warn non-Americans who are considering this book to be aware that this book is firmly rooted in American culture. Immigrants, the American school system, and business, and American politics are all covered in significant detail. People looking for something universal or global, may find that that they get a lot of American culture along with human universals.

I bought this book almost a year ago. I let it sit on my shelf too long before I finally got around to reading it. But once I started I found it pleasant enough to get caught up in. The reviews on Amazon are not all together positive, but I thought this book was well worth reading. It is a comprehensive overview demonstrating the extent to which neuroscience and related disciplines have permeated thought in the last few years. The story structure works, and Mr Brooks is a good enough writer with enough to say as he bounces from topic to topic. Oh, and he also manages to mention the chick sexers again!

Where is the Joy?

Posted on April 20, 2012 by Marcel

We have been trying to come up with a checklist we can use in teacher observations ( for HS teachers of English in Japan). It’s been harder than I imagined. We want to help teachers and their students. But any list of criteria always reflects the preferences of its creators. It reflects an assessment. It reflects an agenda.

After observing many classes last year, I came away with a few depressing impressions, one of which–sorry–was that I am so glad I am not a student in any of those classes. OK, not all, but many…um, no…most. The classes that were pleasant experiences generally achieved this through the power of the performance personality of the teacher. It was the teacher’s jokes, movement in the classroom, and energy that drove the lessons. The same syllabus taught by another teacher would probably have resulted in another snoozer. I know it is difficult to separate teachers from their lesson plans, but if we do that and stand back, some rather distinct features emerge.

Teachers don’t have enough confidence in their own English; and/or have great difficulty effectively using English in the classroom even when their English skills are extremely good (more on this in a future post…)
There is a clear preference for efficiency at the expense of process (that is, activities are tweaked to make them faster, smoother, and more efficient to set up and conduct in the classroom, to get all learners to move quickly at the same pace)
Control is always with the teacher (it is full-frontal presentation and explanation-based instruction)
Learners are not given opportunities to experiment with the language (meaning-focused output-based activities are extremely rare; there are always clear correct answers in activities)
And there is very little joy in the classes (with many sleeping or distracted students and very low levels of student energy or participation); emotions are rare ( rarely are textbook stories milked for their emotional resonance); and only briefly did I get any sense of the shared journey of learning the class was making

In a single sentence, there is a preference for teacher-centered cognitively efficient classes. So when we tried to make our observation checklist (dripping with our own preferences and agenda, and functionally short), these are the categories we went with:

Viscerally Engaging?
Cognitively Engaging?
Communicative?
Pedagogically Sound?
Creative?
Secure?

You might disagree with some of the items or the weighting of some concepts, but this list represents a view that classes should be more interesting and fun and communicative. Some of the sections are very general, to be sure. Each one would really take a book or a course or two to understand. And feedback is not addressed. So, yes, it is still a work in progress.

There is more and more recognition of the role of affect in education. There is also greater recognition that learning should be a joyful experience. A recent article in the Mind/Shift blog looked at some recent research from Finland. The authors of a study described in the article (Rantala and Määttä) tried to identify what produces joy in the classroom:

No doubt many pupils would agree with this example of their findings: “The joy of learning does not include listening to prolonged speeches.” Such teacher-centric lessons are much less likely to generate joy than are lessons focused on the student, the authors report. The latter kind of learning involves active, engaged effort on the part of the child; joy arrives when the child surmounts a series of difficulties to achieve a goal.

They also mention there is greater joy when learners are allowed to work at their own level and when they are allowed to play more (the study looked at elementary school students).

This is hard to do. It is hard to set up an activity that is a doable challenge for learners and then let them experience achieved success. It is especially hard when probably no one in the room–teacher or student–has ever experienced this kind of lesson before. It also hard to let learners work at their own pace when exam periods are fixed. And making things playful or game-like is also more difficult than just assigning points and keeping score (as Stephen Anderson makes clear in this presentation or his book, Seductive Interaction Design).

For a more scholarly approach to the topic of emotions, google Antonio Damasio, Helen Immordino-Yang, or Kurt Fischer to get a wide variety of articles, or look at any of the neuroscience and learning initiatives that have come online recently). There is also this article on Cognition Affect and Learning by Barry Kort. It looks at stories and emotions among other things. And here is a link from the Eide Neurolearning blog that gives a nice summary of humor and affect in learning and links to several other resources.

Changing from an approach that focuses on cognitive efficiency to one that focuses on greater learner control of the process of learning will not be easy. But the increasing number of studies coming out that highlight the need for greater consideration of emotions and the social nature of learning are pointing to a shift in pedagogy. Adding humor to lessons is much easier to do right away. Taking inspiration from Apple’s slogan: Think quirky, think playful.

Two Articles on Innovation: The Blue School and Sony

Posted on April 16, 2012 by Marcel

Innovation is the creation of better or more effective products, processes, services, technologies, or ideas that are accepted by markets, governments, and society.

The above quote comes from Wikipedia. It came up as I checked the spelling of “innovation”. What sent me looking for the spelling of innovation was an encounter with two articles from the New York Times, one from April 13 called At the Blue School (from which I also borrowed the image above), and the other from a day later called How the Tech Parade Passed Sony By. Both of these articles are very interesting and worth a few minutes of your time if you are interested in education and Japan. And both are focused on topics getting a lot of coverage recently.

Articles on neuroscience appear almost daily in the news, and several groups/sites/schools/programs have come into existence in the last few years. There’s the Neuro Education Initiative at John Hopkins University (mentioned in the article), The NeuroLeadership Institute (associated with author and Blue School board member David Rock, and also mentioned in the article), USC’s Brain and Creativity Institute, Harvard’s Mind / Brain / Behavior initiative, and an Annenberg Learner Resource called Neuroscience and the Classroom, to name some that I’ve come across in the last half year or so.

Sony has also been in the news, partly for the transfer of power to a new CEO (from Howard Stringer to Kazuo Hirai), and partly for making a record loss ($6.4 holy smoke billion!). The Times article focuses on Sony’s problems, the subject of so many books (here and here in Japanese), TV spots, and articles that there are probably families who discuss it at the dinner table regularly. Well, maybe not dinner table, but certainly it is present at the heart of the debate about what is “wrong” with Japan economically and what can/should be done about Japan, Inc., the economic model many people grew up with.

Seeing these two articles on the same day got me thinking, wondering if there is any connection we can make between a school that tries a new initiative, garnering both academic praise (my neuroscience Twitter gallery went quadruple post on that link) and serious parental acceptance (it costs more than $30,000 a year to send your young elementary-age child to the Blue School), and a company that seems to have forgotten how to innovate. It is tempting to make the leap that schools in Japan, like their business compatriots, are resistant to change and are struggling to find their way in the face of a changing global environment (blah they can’t lose money but they can certainly waste it on ineffective English language lessons blah blah). It is tempting, but probably a gross (as in icky) generalization, and mostly incorrect. It is tempting because in my job as a teacher trainer for a local government in Japan, lack of innovation is something I see quite a lot of. It is tempting because the Sony article makes the following claim that seems to hook the two articles together:

Sony’s woes mirror a wider decline in Japanese electronics. Though executives here are quick to blame a strong yen, which hurts exports, a deeper issue is that once-innovative companies seem to have run out of ideas. And when a nation can no longer compete on abundant labor or cheap capital, ideas and innovation are paramount.

It is probably incorrect to make these connections too quickly because a single boutique school does not represent a nation, and while applying neuroscience findings to classroom settings is something I am obsessing about myself recently, I’m not sure a) people have completely figured out how to do so effectively yet, and b) good teachers probably do a lot of what neurofanatics say teachers should be doing anyway. Compare the Blue School classroom depicted in the article with this elementary school classroom in Kanazawa, for example. It may very well be true that the focus on creativity and the process of learning practiced at the Blue School may be exactly what more Japanese need educationally to get out of the past and into the global future. Certainly I would like to see more of that, more application of skills and less rote learning, in language classes in Japan. And I think we can say that adding more fun, personalization, emphasis on affect, and the collaborative, social side of learning, would make lessons more bearable for a lot of learners. But how well these things can be instituted, and how effective they can be when they are instituted, is still not certain.

When I read the definition from Wikipedia, that last part really stood out: “…that are accepted by markets, governments, and society.” I copied it and pasted it here because it raised a few questions, both inward and outward. Like most people (I think anyway…) I had always sort of assumed that innovation was all about creativity and newness. But acceptance is a crucial part of innovation, not necessarily at first, but at some point, or else it is not innovation. Innovation is the process of social acceptance of creative initiatives. That makes me feel better about my job (where I do face rejections of my initiatives). Change is a process, not an event (a quote I found attributed to Barbara Johnson, but repeated often). Yup. What the Blue School and Sony have in common is they have to go through the same process. Size, culture, structure, personalities, and the power of the idea behind the initiative all impact on this process.

Steven Thorne at IATEFL 2012: New Media and Language Learning

Posted on March 23, 2012 by Marcel

Steven Thorne’s March 22 plenary is available online (thanks to the British Council).

Here’s the description:

Awareness, appropriacy, and living language use

There has been a great deal of research and pedagogical experimentation relating to technology use within second and foreign language (L2) education. This presentation broadens the scope of inquiry to examine entirely out-of-school L2 digital engagement in environments such as social media, fan fiction communities, and online gaming. The presentation argues first for the efficacy of a usage-based model of second language development and the benefits of explicitly addressing genre awareness and pragmatic appropriateness as core assets in the language learning process. I then present a pedagogical framework designed to increase the relevance of instructed L2 education through the structured juxtaposition of digital vernaculars with more formal ‘classroom’ genres of language use, an approach I and colleagues are calling bridging activities (e.g., Thorne & Reinhardt, 2008). In conclusion, an argument is made for the continued exploration of new media genres of language use and their selective inclusion into instructed L2 pedagogy, processes, and curricula.

Her e’s the link.

For me, there’s a lot to like in this presentation. The world of the web provides great (language) learning opportunities for both social and linguistic reasons. Everything is in place: technology, access, digital skills. Bridging learners from the classroom into the real world should be part of any language program, anywhere. For that to happen, real questions about the “content” of courses needs to happen, and teachers themselves must break out of a very established culture of what language teaching and learning is and what it is for.

A Trolley Named Desire

Posted on September 14, 2011 by Marcel

OK, everybody knows the little trolley thought experiment, right? It even has a Wikipedia page, which I shall now quote liberally from:

A trolley is running out of control down a track. In its path are five people who have been tied to the track by a mad philosopher [or workers working on a track in the less dramatic version I first heard]. Fortunately, you could flip a switch, which will lead the trolley down a different track to safety. Unfortunately, there is a single person tied to that track. Should you flip the switch or do nothing?

And so most people say they would indeed flip the switch and sacrifice one for the safety of the five. But then comes the second version (the description of which at Wikipedia is under the rather blunt heading “The fat man”:

As before, a trolley is hurtling down a track towards five people. You are on a bridge under which it will pass, and you can stop it by dropping a heavy weight in front of it. As it happens, there is a very fat man next to you – your only way to stop the trolley is to push him over the bridge and onto the track, killing him to save five. Should you proceed?

And most people recoil at the thought. Hmm, smile the ethical philosophers, you do realize that the two scenarios are identical, do you not? Killing one saves the five. But there is a difference and an important one that keeps bringing people back to this problem. Michael Sandel uses it to start his wildly-popular course on Justice (available online here, BTW). The difference between the problems makes you think. I am not alone in the reaction I had, judging from the opinions of Sandel’s students and the tabulated results of people answering the questions. I originally felt that the difference between just flipping a switch and actually running up to and grabbing, hoisting, and possibly forcing a bewildered obese person over the bridge railing was enough difference to make me take a less active role in the second scenario.

But this morning, while reading David Eagleman’s engaging Incognito on the train to work, I read about the neuroscience take on the problem (also listed on the Wikipedia page as I discovered later). Eagleman introduces Joshua Greene and Jonathon Cohen, who have done a lot of research on neuroethics, much of it in Greene’s Moral Cognition Lab at Harvard. According to them and the neuroscience perspective, the difference is that interacting with the fat man up close activates the emotional networks of the brain . The brain, Eagleman explains, is a “team of rivals” and two large groupings of rivals are the emotional and rational systems. In fMRI screenings of people considering the two trolley problems, the first scenario causes areas of the brain connected to rational thinking to be more active and the second scenario activates areas involved with motor planning and emotion.

Cognitive or emotional? Advertisers know the difference between these two systems well. And so do politicians. But in education, the cognitive is still everything. The phrase that suddenly grew larger for me on the page this morning, the one that makes all the difference, is “personal interaction.” In re-thinking the classrooms of language lessons in Japan, I wonder if this is not the key issue. I think creating a cacophony of social/personal interaction to activate the emotional parts of the brain might be a goal for these classes, though rational minds might recoil at such an approach. It seems fair to say that affect is woefully under-addressed in courses, coursebooks, and pedagogical literature (the latter probably because of the difficulty of doing research). There is this book, now 13 years old and counting, but much of the literature seems concerned primarily with reducing learner anxiety in the classroom. But in terms of neuroscience, 13 years is a lifetime ago…Jane Arnold, the editor of Affect in Language Learning, says on her website that” she is convinced that when teaching is affective it becomes more effective” but there seems to be little in the way of literature that really demonstrates that. I guess I’ll just need to keep looking.

Sept. 24, 2012 Update: This article from NPR reports on how the visual/emotional aspect of the second part of the trolley thought experiment makes it very different.

“Some dilemmas produce vivid images in our heads. And we’re wired to respond emotionally to pictures. Take away the pictures — the brain goes into rational, calculation mode.”

Understanding the Modular You: Robert Kurzban’s Why Everyone Else Is a Hypocrite

Posted on April 4, 2011 by Marcel

This reatively short and very entertaining book packs a message that can change the way we look at people and the minds that make us who we are. Robert Kurzban is an evolutionary psychologist. And in one long breath, here is his message: the mind is modular and it makes no sense to talk about “you” or “me” because there is no conscious, single, controlling “you” or “me” running the show inside our heads, and when”you” and “me” talk, it is basically my “press secretary” talking to your “press secretary,” one of many modules of our mind, modules that act with their own rhyme and reason and don’t necessarily talk to each other and aren’t necessarily aware of each other, but do act in a way that is in accordance to the goals for which the module was genetically selected for. Got that? Basically, it means that there are lots of parts of “you” and they do what they were designed (selected) to do, usually without regard for other parts of “you.” Behavioral inconsistencies can be explained through understanding this modular specialization structure of the brain. Deep down, I think we already know this. This is why people on diets lock their fridge doors at night, why George Reckers and Elliot Spitzer walk(ed) a different walk from their public talk, etc. But Kurzban’s theory–and he repeatedly states that it is an exploratory theory– is a way of conceptualizing how the mind can manage to be so blatantly contradictory, and as such it has great explaining power. People can hold very strong beliefs, impossibly contradictory beliefs, often for no reason. People can say one thing and do another, they can waffle between patience and impulsivity, have overinflated and unrealistic views of themselves (almost everyone, for example, believes themselves to be an above-average driver), and hold others to high moral principles that they prefer not to apply to themselves. Seen through Kurzban’s evolutionary psychology spectacles, the selfish little modules trying to gain advantage any way they can form a logical–if depressingly dispicable–portait of the organisms we are.

Aside from being an interesting and entertaining read, this hand-grenade of a book may eventually shake a lot of long-held belief fruit out of the trees that social psychologists and behavioral economists have been feeding from for years. For example, the notion that motivation and preferences are fairly constant (two that have long bothered me personally), are debunked pretty thoroughly here. Quoting LIchtenstein and Slovic (2006b): preferences “…are labile, inconsistent, subject to factors we are unaware of, and not always in our own best interests. Indeed, so pervasive is that lability that the very notion of a ‘true’ preference must, in many situations, be rejected.” What this means, is that things like motivation and preferences are more complex than we think they are now; different modules of the brain work differently “depending on context, state, and history,” and they–“we”– aren’t aware of it. So instead of constant motivations or preferences held by an individual, we should probably think of people as collections of independently-acting modules, each module “…designed to bring about certain states or affairs.” Motivation is a design to bring about a goal, but it is better conceptualized as a bunch of design goals located in a bunch of modules. The L2 Self may be not much more than a handful of sand. Actually, I don’t think the implications of Kurzban’s theory go that far, but at the least, we have to acknowledge that we are only looking at a small part of an individual whenever we look at him, no matter how closely.