What can Data do for EFL?

image of glasses

In the US, something very interesting is happening as an indirect result of standards-based testing and the establishment of charter schools: a certain interesting set of research conditions has emerged. Charter schools are themselves often experiments, established in part to be “…R&D engines for traditional public schools” (Dobbie & Fryer, 2012). As such, they often eschew many elements of traditional education practices, and instead attempt to institute the results of research into educational effectiveness from the last several decades. For many charter schools, this research-informed pedagogy assumes a unifying or rallying role, and the ideas are woven into their mission statements and school cultures, as they try to take underprivileged kids and put them on the road to college, through grit training, artistic expression, or higher expectations. Even from a brief visit to the websites of charter systems such as Uncommon Schools, MATCH, or KIPP, you can see what they are intending to do and why. And some of these charter schools have become extremely successful—in terms of academic achievement by students, and in terms of popularity. And both of these have led to new research opportunities. You see, the best charter schools now have to resort to lotteries to choose students, lotteries that create nice groups of randomly-sampled individuals: those who got in and received an experimental treatment in education, and those who didn’t and ended up going to more traditional schools. And that provides researchers with a way to compare programs by looking at what happens to the students in these groups. And some of the results may surprise you.

Let’s play one of our favorite games again: Guess the Effect Size!! It’s simple. Just look at the list of interventions below and decide whether each intervention has a large (significant, important, you-should-be-doing-this) impact, or a small (minimal, puny, low-priority) impact. Ready? Let’s go!

  1. Make small classes
  2. Spend more money per student
  3. Make sure all teachers are certified
  4. Deploy as many teachers with advanced degrees as possible
  5. Have teachers give frequent feedback
  6. Make use of data to guide instruction
  7. Create a system of high-dosage tutoring
  8. Increase the amount of time for instruction
  9. Have high expectations for students

Ready to hear the answers? Well, according to Dobbie & Fryer (2012), the first four on the list are not correlated with school effectiveness, while the next five account for a whopping 45% of the reasons schools are effective. Looking at the list, this is not surprising, especially if you are aware  of the power of formative feedback.

Some people might be a little skeptical still. Fine. Go and look for studies that prove Dobbie and Fryer wrong. You might find some. Then look at where the research was done. Is the setting like yours? Just going through this process means we are putting data to work. And that is much, much better than just going with our own instincts, which are of course based on our own experiences. I work teaching English in Japan, and I know that is a far cry from the hard knocks neighborhoods where Dobbie and Fryer looked into the effects of interventions in Match schools. But I think there are enough similarities to warrant giving credence to these results and even giving them a try at schools Tokyo. I have several reasons. First, extensive research on formative assessment, high expectations, classroom time, and pinpointed direct instruction is very robust. Nothing in their list is surprising. Second, in Japan, English is often as foreign from the daily lives of most students as physics or math are from the lives of many American teens. The motivation for learning it is likewise unlikely to be very strong at the beginning. Many of the students in the Match system are less than confident with their ability with many subjects, and are less than confident with aiming at college, a world that is often quite foreign to their lives. Many English learners in Japan similarly see English as foreign and unrelated to their lives, and the notion that they can become proficient at it and make it a part of their future social and/or professional lives, requires a great leap in faith.

But through the Match program, students do gain in confidence, and they do gain in ability, and they do get prepared for college. Given the demographic, the success of Match and the other “No Excuses” systems mentioned above is stunning. It also seems to be long lasting. Davis & Heller (2015) found that students who attended “No Excuses” schools were 10.0 percentage points more likely to attend college and 9.5 percentage points more likely to enroll for at least four semesters. Clearly the kids are getting more than fleeting bumps in scores on tests. And clearly the approach of these schools—in putting to work proven interventions—is having a positive effect, although not everyone seems to be happy.

And it’s not just that they are making use of research results. These schools are putting data to use in a variety of ways. Paul Bambrick-Sotoyo of Uncommon Schools has published a book that outlines their approach very nicely. In it we can find this:

Data-driven instruction is the philosophy that schools should constantly focus on one simple question: are our students learning? Using data-based methods, these schools break from the traditional emphasis on what teachers ostensibly taught in favor of a clear-eyed, fact-based focus on what students actually learned (pg. xxv).

Driven By Data book cover

They do this by adhering to four basic principles. Schools must create serious interim assessments that provide meaningful data. This data then must be carefully analyzed so the data produces actionable finding. And these findings must be tied to classroom practices that build on strengths and eliminate shortcomings. And finally, all of this must occur in an environment where the culture of data-driven instruction is valued and practiced and can thrive. Mr. Bambrick-Sotoyo goes through a list of mistakes that most schools make, challenges that are important to meet if data is to be used to “…make student learning the ultimate test of teaching.” The list feels more like a checklist of standard operating procedures at almost every program I have ever worked in in EFL. Inferior, infrequent or secretive assessments? Check, check, check. Curriculum-assessment disconnect? Almost always. Separation of teaching and analysis? Usually, no analysis whatsoever. Ineffective follow-up? Har har har. I don’t believe I have ever experienced or even heard of any kind of follow-up at the program level. Well, you get the point. What is happening in EFL programs in Japan now is very far removed from a system where data is put to work to maximize learning.

But let’s not stop at the program level. Doug Lemov has been building up a fine collection of techniques that teachers can use to improve learning outcomes. He is now up to 62 techniques that “put students on the path to college,” after starting with 49 in the earlier edition. And how does he decide on these techniques? Through a combination of videoing teachers and tracking the performance of their classes. Simple, yet revolutionary. The group I belonged to until this past April was trying to do something similar with EFL at public high schools in Japan, but the lack of standardized test taking makes it difficult to compare outcomes. But there is no doubt in my mind that this is exactly the right direction in which we should be going. Find out what works, tease out exactly what it is that is leading to improvement (identify the micro-skills), and then train people through micro-teaching to do these things and do them well and do them better still. Teaching is art, Mr. Lemov says in the introduction, but “…great art relies on the mastery and application of foundational skills” (pg.1). Mr. Lemov has done a great service to us by videoing and analyzing thousands of hours of classes, and then triangulating that with test results. And then further trialing and tweaking those results. If you don’t have copy of the book, I encourage you to do so. It’s just a shame that it isn’t all about language teaching and learning.

Interest in using data in EFL/ESL is also growing. A recent issue of Language Testing focused on diagnostic assessment. This is something that has grown out of the same standards-based testing that allowed the charter schools in the US to thrive. You can download a complimentary article (“Diagnostic assessment of reading and listening in a second or foreign language: Elaborating on diagnostic principles”, by Luke Harding, J. Charles Alderson, and Tineke Brunfaut). You can also listen to a podcast interview with Glen Fulcher and Eunice Eunhee Jang, one of the contributors to the special issue. It seems likely that this is an area of EFL that will continue to grow in the future.

Lessons from Training

Image

I came across another interesting article on the BBC website today. It was temptingly titled Can you win at anything if you practise hard enough? It told the story of a young table-tennis coach from the UK, named Ben Larcombe, who attempted to take his lumpy and “unsporty” friend and turn him, over the course of a year, into a top competitive player. As part of the process of writing a book on the topic of training and improvement, the pair documented Sam Priestley’s transformation from a sort-of player to an impressively good one, at least to my eyes. You can watch the whole thing unfold before your eyes in this video. The article includes, however,  a rather bubble-bursting comment from an English tennis coach and expert named Rory Scott:

“He is nowhere near the standard of the top under-11 player in the UK.”

So the BBC writer goes on to ask this question: “Why did the project fail?” What? Just because Sam didn’t meet his goal of getting into the top 250 table tennis players in the UK in one year of practicing every day, doesn’t mean it was a failure at all. It shows the potential for people to learn when they are persistent and work hard regularly with good strategies and feedback. The rest of the article goes on to explore exactly that potential, largely from a cultural viewpoint of different attitudes to natural ability and the need to persist versus instantaneous gratification.

I’ve been seeing similar sorts of studies a lot lately. The last few years have seen a plethora of books and talks on similar topics: how far does practice take you? You can read about Josh Kaufman’s attempt to learn something in 20 hours, or watch him tell about it at TED. Or you can read about Joshua Foer attempting to get better at memorizing things in his book, Moonwalking With Einstein. Or if you are really serious about the source of greatness, whether is comes from genes or training, try The Sports Gene by David Epstein. And don’t forget Doug Lemov’s Practice Perfect, a book which has a focus on learning teaching.

Practice, I’m convinced, is important. But so are attitudes to practice, and so is the kind of practice you do and the the kind of feedback you get. If we can get these right, our learners will learn better and faster, which will lead to other benefits. Practice is a touchy issue in language teaching, a field still trying to come to terms with the “drill and kill” of the audio-lingual approach. But intense, focused practice with constructive feedback and repeated opportunities to incorporate that feedback and improve is something very important to the learning process. It takes a lot of time, to be sure, maybe even 10,000 hours (though see Mr. Epstein’s book for a good discussion on amounts of time), but impressive results are possible. That is something I want my learners to understand and buy into.

Mr. Larcombe has a website with more detailed info about the process of teaching table tennis. He is also currently preparing a book.

Making EFL Matter Pt. 3: The Challenges and Benefits of Discussion

image of balls in a tray

 

Well, what do you think? This question and answer form a basic opinion exchange that is sometimes called a discussion. And it is, sort of. But just as a single decontextualized sentence is of limited use in understanding grammar, so too a brief opinion exchange does not have enough context–with all its intentions, personalities, and sociolinguistic depth–to really be called a discussion. A discussion is more complex, and ultimately more powerful, because it has a goal and requires input and interaction from multiple members, which should allow them to collectively generate better ideas (solutions, plans, etc.) than any one of the participants could have done alone.

This sounds good in theory, but it is difficult to achieve in classrooms–especially EFL classrooms where learners have a layer of linguistic difficulty on top of the conceptual and procedural challenges inherent in establishing a system of rich academic discussions. The first thing we must acknowledge is that academic discussion skills, like Rome, are not built in a day. As I mentioned earlier, they need to be incrementally developed, starting with basic conversation and interaction skills.  Without basic conversation skills, discussion is not attainable. Students who are used to pairwork and are able to use the basic greetings, openings, and closings of common conversation “scripts” (Hi. How’re you doing? So, what did you do on the weekend? Well, nice talking to you!), and can react to each other’s utterances (Uh-huh, Really?! Oh, I love that!, Really? How was it? etc.) will find discussions accessible. Absolute speaking beginners will struggle and likely fail. Speaking must be taught, skills must be developed, and regular opportunities for fluency development given, or else activities like academic discussions, and the opportunities to flex critical thinking muscles that go with them, won’t be achieved. A little bit of speaking tagged on to the end of a lesson won’t get you there (as programs in high schools in Japan are slowly waking up to).

So now we know it’s difficult and requires a program of incremental skill development starting with a foundation in basic interactive conversation skills. One question we might ask is: is it worth the trouble. Given a limited amount of time, why should so much be devoted to conversation and discussion skills development? Well, the answer comes from sociocultural learning theory. As Daniel Siegel puts it in his forward to the wonderful Social Neuroscience of Education: “We evolved in tribes, we grow in families, and we learn in groups.” Walqui and van Lier (2010), in listing up the tenets of sociocultural learning theory for their QTEL approach, focus on some of the key points: “Participation in activity is central to the development of knowledge; participation in activity progresses from apprenticeship to appropriation, or from the social to the individual plane; and learning can be observed as changes in participation over time” (pg. 6). That is to say, we learn through active participation (engagement and collaboration) with others. “Language is primarily social”…and “…learning…is essentially social in nature” (pg. 4-5). This learning does not happen by chance, however. The really really hard thing to do is to get students into that sweet spot where they are developmentally ready and linguistically scaffolded  up to the point where they can function and learn. Development becomes possible when “…teachers plan lessons beyond the students’ ability to carry them out independently” (pg. 7), but create the proper community and provide the proper scaffolding to allow for success with such lessons. To answer the question that started this paragraph, the potential benefits of learning in groups are great enough to warrant using this approach. Students can learn content and language, and collaboration skills, essential skills for the 21st century according to Laura Greenstein (who also helpfully provides a rubric and suggestions for assessing collaboration, as well as other skills).

One more potential objection to focusing on academic discussion comes from Doug Lemov. Actually, it’s not so much of an objection as request to rethink and balance your choices. In Teach Like a Champion 2.0, he suggests that both writing and discussion can be strong tools for “causing all students to do lots of the most rigorous work,..but if I had to choose just one, which admittedly I do not, I would choose writing. Hammering an ideas into precise words and syntax and then linking it to evidence and situating it within a broader argument are, for me the most rigorous work in schooling” (pg. 314).  Writing is great, and cognitively more “precise” perhaps, and definitely needs to be part of the syllabus. Discussion is by nature more social. You can do both and you should do both; finding the time to do so is challenging, however.

So what do we teach? And what are discussion skills that students need to learn and develop? The best list can be found in Academic Conversations by Jeff Zwiers and Marie Crawford. It needs to be said, however, that this book is made for L1 learners and you will need to adapt as you adopt. But the basic list and framework make it easy and intuitive to do so. Here is the list of skills:

  1. Elaborate and Clarify: Make your thinking as detailed and clear as you can, carefully explaining the rationale behind your thinking.
  2. Support Ideas with Examples: Use examples to illustrate thinking. It is a particular and powerful way of elaborating and clarifying ideas.
  3. Build On or Challenge Partners’ Ideas: Actively respond to and develop the ideas that arise, either by expanding on them or tweaking them, or pruning out the bad ones through well-considered disagreement.
  4. Paraphrase Ideas: As ideas arise, paraphrase them to both show your understanding and create a springboard for idea development and improvement.
  5. Synthesize the Discussion Points: Bring all the ideas you’ve been discussing to a conclusion. Produce a group decision or plan.

These skills need to be introduced incrementally. Some of them are more difficult to teach and practice than others, particularly with students who lack proficiency or fluency, but also for cultural reasons. Some students in Japan find it challenging to disagree, and may have trouble ranking or pruning some ideas from their synthesis. These can be trained and taught. In my experience, students are not used to making their thinking so explicit and considering  ideas so carefully. Once they get the hang of it, however, they clearly become better listeners, better collaborators, and better thinkers. And after gaining some fluency with the formulaic expressions required to do academic discussions, they sound considerably more proficient.

Assessment is not as difficult as you may imagine. Ms. Greenstein’s rubric provides a nice overview of expected performance and can be used together with teacher observations, and peer or individual reflection and feedback. Performance tests are easy to organize because we can check several students at once in their group. Focusing on the success of the group in terms of process and outcome is actually not that hard to measure.

This post is part of a series considering ways to add more focus and learning to EFL classrooms by drawing on ideas and best practices from L1 classrooms.

Part 1 looked at the importance of goals.

Part 2 looked at using data and feedback.

 

 

Fine-tuning Your Classroom Skills: Doug Lemov Catalogs How to Teach Like a Champion

Teach Like a Champion Book Cover

Hey! EFL teachers in Japan! No disrespect, but you probably need this book.  If you  are a native speaker EFL teacher in a country where there is more interest  than accomplishment in learning English, you probably need this book. You might be one of the few educated, trained, and disciplined among us, but chances are, you are not. More likely, you belong to the majority– by and large and by most accounts, a ragtag bunch of misfitsMost of us fell into this profession and were not trained as teachers exactly; I mean, not in the sense of teachers who went to teachers college, studied theory and techniques, did practicums, and got jobs in school systems run by school boards, with standards, supervisors, paperwork, and training. Many of us were waiters, travelers, English majors, or other such mostly unemployables who began teaching EFL and then learned how to do it in a combination of certificate courses, graduate courses, conferences, and keeping our eyes open on the job. Or you may be a Japanese teacher of English as a foreign language, teaching in senior or junior high school after gaining a license that required only a few extra undergraduate courses, completion of a two or three-week placement at your old school, and passing the tests for the prefecture you are now employed by. You know your basic grammar, have a fairly good vocabulary, and are pretty confident that with a little prep time can explain anything between the covers of a textbook. But you have gaps in your knowledge of English usage and classroom management almost as impressive as the unwarranted overconfidence your native English-speaking colleagues bring with them into your school, so full or self-righteousness and irresponsibility. Both you groups of teachers would benefit from this book because not only were you never trained specifically for classroom and learning management, a lot has been learned about teaching and managing classes since you were at school. There is now a good mass of knowledge and techniques that I bet you never took a course in and most likely are unaware of. 

This book is not aimed at EFL teachers, not even a little. But it will tell you, and show you, how to do things with your students that will get them to pay more attention, learn more deeply, and make the most of the time you are now probably doing a good job of frittering largely away. This book by itself will not solve your problems, and you’ll likely find it pushes more regimentation and rigor than either  you or your students need, but man ‘o man there is a lot to be learned here. 49 techniques, roughly half of which you can adapt very nicely for an EFL classroom, are presented by Mr. Lemov. He gleaned them from observing teachers who were getting better-than-expected results in schools whose demographics made those results look almost miraculous. These are techniques that took chunks of a doomed demographic and put them on the Path to College. Some of them are techniques you might indeed be using. But I have managed enough native-English speaking teachers and observed enough Japanese teachers of English in high schools that I can say with quite a bit of certainty, you (and your students!) will benefit from this book. Mr. Lemov provides us with clear explanations, warnings against pitfalls, and short illustrative video clips to make the points easy to understand. Putting them to use will be more difficult, but not impossible, as most of the techniques require only slight adjustments in timing or language. But it is those small changes that can make a huge difference. Techniques such as Cold Calling (picking the student after you have asked the question), Pepper (asking a rapid sequence of questions as review), and concepts such as Ratio (getting students to do more of the thinking) and At Bats (maximizing practice opportunities) can transform the one-way slog of some English classes or the whimsical frivolity of others into real learning environments.

The book has a website. Go there first and look around.  http://teachlikeachampion.com/ Learn a little about Uncommon Schools or Mr. Lemov himself and what he does. Watch some of the videos and see the control and confidence and learning on display. I am quite certain you will begin to see where you may be falling short now. These are schools and teachers who refused to let demographics get in the way of success, who challenge their learners to perform at a higher level, and demand their attention and effort. If you don’t know how to get to that point with your learners, then you probably need this book. It is not a panacea, however, and you will still need to adapt the ideas to fit your subject and classes. In general, the techniques are very cognitive and very behaviorist and so Japanese teachers of English will likely find them more immediately appealing as they will mesh more nicely with current practices. Teachers would do well to not forget the affective and emotional when planning lessons, something Mr. Lemov spends too little time on (from a language teacher’s perspective). But this book can give you a lot of help learning and improving techniques you probably were never exposed to in your education and your own training. It can help you make whatever you are doing more efficient and more effective.