Successful Spoken English – interview with authors

The following is an email interview with the authors, Christian Jones, Shelley Byrne, Nicola Halenko, of the recent Routledge publication Successful Spoken English: Findings from Learner Corpora. Note that I have not yet read this (waiting for a review copy!).

Successful Spoken English

1. Can you explain the origins of the book?

We wanted to explore what successful learners do when they speak and in particular learners from B1-C1 levels, which are, we feel, the most common and important levels. The CEFR gives “can do” statements at each level but these are often quite vague and thus open to interpretation. We wanted to discover what successful learners do in terms of their linguistic, strategic, discourse and pragmatic competence and how this differs from level to level.  

We realised it would be impossible to use data from all the interactions a successful speaker might have so we used interactive speaking tests at each level. We wanted to encourage learners and teachers to look at what successful speakers do and use that, at least in part, as a model to aim for as in many cases the native speaker model is an unrealistic target.

2. What corpora were used?

The main corpus we used was the UCLan Speaking Test Corpus (USTC). This contained data from only students  from a range of nationalities who had been successful (based on holistic test scoring) at each level, B1-C1. As points of comparison, we also recorded native speakers undertaking each test. We also made some comparisons to the LINDSEI (Louvain International Database of Spoken English Interlanguage) corpus and, to a lesser extent, the spoken section of the BYU-BNC corpus.

Test data does not really provide much evidence of pragmatic competence so we constructed a Speech Act Corpus of English (SPACE) using recordings of computer-animated production tasks by B2 level learners  for requests and apologies in a variety of contexts. These were also rated holistically and we used only those which were rated as appropriate or very appropriate in each scenario. Native speakers also recorded responses and these were used as a point of comparison. 

3. What were the most surprising findings?

In terms of the language learners used, it was a little surprising that as levels increased, learners did not always display a greater range of vocabulary. In fact, at all levels (and in the native speaker data) there was a heavy reliance on the top two thousand words. Instead, it is the flexibility with which learners can use these words which changes as the levels increase so they begin to use them in more collocations and chunks and with different functions. There was also a tendency across levels to favour use of chunks which can be used for a variety of functions. For example, although we can presume that learners may have been taught phrase such as ‘in my opinion’ this was infrequent and instead they favoured ‘I think’ which can be used to give opinons, to hedge, to buy time etc .

In terms of discourse, the data showed that we really need to pay attention to what McCarthy has called ‘turn grammar’. A big difference as the levels increased was the increasing ability of learners to co-construct  conversations, developing ideas from and contributing to the turns of others. At B1 level, understandably, the focus was much more on the development of their own turns.

4. What findings would be most useful to language teachers?

Hopefully, in the lists of frequent words, keywords and chunks they have something which can inform their teaching at each of these levels. It would seem to be reasonable to use, as an example, the language of successful B2 level speakers to inform what we teach to B1 level speakers. Also, though tutors may present a variety of less frequent or ‘more difficult’ words and chunks to learners, successful speakers will ultimately employ lexis which is more common and more natural sounding in their speech, just as the native speakers in our data also did.

We hope the book will also give clearer guidance as to what the CEFR levels mean in terms of communicative competence and what learners can actually do at different levels. Finally, and related to the last  point, we hope that teachers will see how successful speakers need to develop all aspects of communicative competence (linguistic, strategic, discourse and pragmatic competence) and that teaching should focus on each area rather than only one of two of these areas.

There has been some criticism, notably by Stefan Th. Gries and collaborators that much learner corpus research is restricting itself factorwise when explaining a linguistic phenomenon. Gries calls for a multi-factor approach whose power can be seen in a study conducted with Sandra C. Deshors, 2014, on the uses of may, can and pouvoir with native English users and French learners of English. Using nearly 4000 examples from 3 corpora, annotated with over 20 morphosyntactic and semantic features, they found for example that French learners of English see pouvoir as closer to can than may.

The analysis for Successful Spoken English was described as follows:

“We examined the data with a mixture of quantitative and qualitative data analysis, using measures such as log-likelihood to check significance of frequency counts but then manual examination of concordance line to analyse the function of language.”

Hopefully with the increasing use of multi-factor methods learner corpus analysis can yield even more interesting and useful results than current approaches allow.

Chris and his colleagues kindly answered some follow-up questions:

5. How did you measure/assign CEFR level for students?  

Students were often already in classes where they had been given a proficiency test and placed in a level . We then gave them our speaking  test and only took data from students who had been given a global pass score of 3.5 or 4 (on a scale of 0-5). The borderline pass mark was 2.5 so we only chose students who had clearly passed but were not at the very top of the level and obviously then only those who gave us permissions to do so. The speaking tests we used were based on Canale’s (1984) oral proficiency interview design and consisted of a warm up phase, a paired interactive discussion task and a topic specific conversation based on the discussion task. Each lasted between 10-15 minutes.

6. So most of the analysis was in relation to successful students who were measured holistically?  


7. And could you explain what holistically means here?

Yes, we looked at successful learners at each CEFR level, according to the test marking criteria. They were graded for grammar, vocabulary, pronunciation, discourse management and interactive ability based on criteria such as  the following (grade 3-3.5) for discourse management ‘Contributions are normally relevant, coherent and of an appropriate length’. These scores were then amalgamated into a global score. These scales are holistic in that they try to assess what learners can do in terms of these competences to gain an overall picture of their spoken English rather than ticking off a list of items they can or cannot use. 

8. Do I understand correctly that comparisons with native speaker corpora were not as much used as with successful vs unsuccessful students? 

No, we did not look at unsuccessful students at all. We were trying to compare successful students at B1-C1 levels and to draw some comparison to native speakers. We also compared our data to the LINDSEI spoken learner corpus to check the use of key words.

9. For the native speaker comparisons what kind of things were compared?

We compared each aspect of communicative competence – linguistic, strategic, discourse and pragmatic competences to some degree. The native speakers took exactly the same tests so we compared (as one example), the most frequent words they used.


Thanks for reading.



Deshors, S. C., & Gries, S. T. (2014). A case for the multifactorial assessment of learner language. Human Cognitive Processing (HCP), 179. Retrieved from



IATEFL 2016 – corpus related mini-interviews

Following on from what could be described as a corpus carnival this year, some of those presenters kindly answered 5 questions. Oh and if any other corpus related presenters want to be added let me know. I list the mini-interviews in approximately chronological order:

Teaching the pragmatics of spoken requests in EAP
Christian Jones (University of Liverpool, UK),

Answering language questions from corpora
James Thomas (Masaryk University), @versatilepub

Using English Grammar Profile to improve curriculum design
Geraldine Mark (Gloucestershire College/Cambridge University Press) & Anne O’Keeffe (Mary Immaculate College, Limerick/Cambridge University Press), @TEFLclass

Electronic theses online – developing domain-specific corpora from open access
Alannah Fitzgerald (Concordia University) & Chris Mansfield (Queen Mary University of London),

Guiding EAP learners to autonomously use online corpora: lessons learned
Daniel Ruelle (RMIT University Vietnam), @danrmitvn

Teacher-driven corpus development: the online restaurant review
Chad Langford & Joshua Albair (University Lille 3, France)

Christian Jones
1. Who are you?
I am a Senior Lecturer in Applied Linguistics and TESOL at the University of Liverpool.
2. Who should come to your talk?
EAP or EFL teachers interested in research into spoken language (in this case the speech act of requesting) and the implications for teaching.
3. Why should they come?
Well, hope it will be interesting (!) and will make people think about their own teaching in regard to spoken language. In EAP in particular, a lot of attention us given to writing and reading and while this is understandable, I also think that the way learners interact when they speak in academic settings is important. I’m not  giving a workshop but I hope I will apply theory to practice in a useful way.
4. Which talks are you looking forward to?
I can only attend on the day I am speaking but two things I would like to see are Mike McCarthy talking about spoken language in EAP, my ex-colleague Tania Horak talking about lexical profiling in tests. I will catch up with things I miss online.
5. Top tip
I don’t go to that many conferences so can only give fairly obvious advice: 1) Don’t try to see everything – pick 3 or 4 key talks a day and go from there 2 )The conversations you have in the breaks and the people you meet are a key part of the experience 3) Caffeine is vital! [back]

James Thomas
1. Who are you?
I’m a university teacher trainer, who doesn’t only talk about aspects of teacher development, but in our department, we actually do it: our trainees working with real live students for a whole semester. Internal Practice Teaching’s a buzz for everyone concerned. I’m also the author of a book that does something no other does. This should be my big expose at IATEFL.
2. Who should come to your talk?
Teachers who are interested in the L in ELT and TEFL and TESOL, etc. Everyone knows that dictionaries, grammars and intuition are not enough to answer every language question. By searching for answers in corpus data, we are in effect, asking thousands of native speakers at the same time.
3. Why should they come?
– The audience will observe language activities that involve learning about language, as well as learning language.
– They will see guided discovery activities in action.
– They will see  another avenue for using internet tools in the classroom.
– We develop strategies for dealing with students saying “but I’ve seen it somewhere”
4. Which talk(s) are you looking forward to?
Well, the last time I heard Mr Crystal speak, he impressed a lot of people, so I’m looking forward to being reimpressed. The work of Diane Larsen-Freemen has been quite pivotal in our field, so hearing it straight from the horse’s mouth … And my colleague, Nikki Fortova, has a poster about our Internal Practice Teaching, which uses a bit of augmented reality, so I’m keen to see how people react to the medium as well as the message!
5. Top conference going tip?
I saw Jan Blake perform about 10 years ago at a NILE event – outstanding. [back]

Anne O’Keeffe
1. Who are you?
I am an academic who has an EFL background. As an academic, I work in the area of Corpus Linguistics, at Mary Immaculate College, University of Limerick, Ireland. I am particularly interested in the applications of Corpus Linguistics to language learning.

I regularly give talks about the application of Corpus Linguistics to language teaching as I think that it important to spread the word to those who do the real work of teaching languages. If research is to have any impact, then we need to think about what our findings mean for the classroom. My most recent work has been with Cambridge University Press. This has involved working with Geraldine Mark on a four-year research project which entailed looking in great detail at learner grammar, across the CEFR, using the 55 million word Cambridge Learning Corpus. This has led us to create the open resource English Grammar Profile – Please check it out and let us know what you think!
2. Who should come to your talk?
My talk, which is co-presented with Geraldine Mark, is about the English Grammar Profile resource. We will talk about how its findings can help inform syllabus design. Essentially, we looked at the Cambridge Learner Corpus and identified over 1,200 different grammar competencies in the learners’ writing across the six levels of the CEFR. The database has some surprises about what learners know and when they know it. It also sheds light on what advanced (C level) students can do with grammar and pragmatics.

This talk will be of interest to 1)  anyone who is interested in researching learner grammar competency; 2) anyone who is interested in findings about what grammar learners know at different levels; 3) language teachers who want to hear about this new resource which might be of use to them in their syllabus design.
3. Why should they come?
If you are interested in knowing more about the English Grammar Profile and how it can help you think more strategically about what grammar you teach, this talk is for you. If you are doing MA or PhD research into learner grammar or learner corpora, this talk might give you some ideas and if you are interested in getting a different perspective of grammar syllabi, there is something in this talk for you too.
4. Which talk(s) are you looking forward to?
The plenaries include some really big names! They are definitely not to be missed. The programme looks so interesting. There are so many talks and so little time!
5. Top conference going tip?
Don’t try to overdo it by attending a session at every slot. Allow time to just mingle around the exhibition area and meet people. IATEFL is a very friendly conference and you can make some new friends from around the world. [back]

Alannah Fitzgerald
1. Who are you?
I am an open education practitioner and researcher working in the area of technology-enhanced English language education. Being somewhat nomadic, I have gained experience and understanding from learning, teaching and researching across different educational contexts, including Higher Education institutions in the United Kingdom, Canada, Korea, and New Zealand (my country of origin). Increasingly, I have been drawn to devising and delivering online language learning interventions that can be scaled and assessed across both formal and informal education.
2. Who should come to your talk?
Language teachers, language learners, subject specialists, instructional design and e-learning support teams who want to build their own language collections.
3. Why should they come?
See what you can do with open content for building dynamic online English language collections for any target learner group. Our latest open collection in collaboration with the British Library is made up of 50,000 PhD abstracts for learning English for Specific Academic Purposes.
4. Which talk(s) are you looking forward to?
Unfortunately, I’m only going to be there for the interactive fair as I have two open education conferences on either side of that day. If I were going to be there for the whole gig I’d most likely want to attend a variety of sessions to get a sense of what the wider ELT community is currently concerned with.
5. Top conference going tip?
My greatest experience and tip for conference going is to find people you can work with on projects who are at different schools or institutions. This will help you to get a wider sense of your field or how different fields can intersect in interesting ways, for example, FLAX brings computer science and language education together. [back]

Daniel Ruelle
1. Who are you?
I’m a EAP / IELTS preparation teacher and program coordinator at the Vietnam campus of an Australian university called RMIT – Royal Melbourne Institute of Technology.  It’s actually one of the – if not the – largest offshore universities in the world with two campuses in Vietnam and over 5,000 students.  During my time here I have become quite interested in vocabulary acquisition, especially using corpora in the classroom to encourage learners to autonomously use English more naturally and focus on collocations.
2. Who should come to your talk?
Those who are interested in hearing about my experience training learners to autonomously use free online corpora.  Actually I am one of three presenters in a forum on corpora on Friday, April 15th at 14:10 – 15:15 in Hall 11a, and my co-presenters will be presenting about two complementary themes: “Learning academic vocabulary through a discovery-based approach” and “Exploring EAP teachers’ familiarity and experiences of corpora.
3. Why should they come?
There has been a lot of research on corpora and they are frequently used by researchers, but to date there has been quite a disconnect between research and practice.  These tools have come a long way and are much more user-friendly and intuitive then what many teachers may have experienced in the past.  This forum on corpora will hopefully demystify these incredibly useful tools for both teachers and learners, and give teachers some ideas on how they can use them with their learners.
4. Which talk(s) are you looking forward to?
The keynote speakers this year all look incredible and of them, I’m most excited to see Dr. David Crystal and Scott Thornbury – both legends in the field!  I’m still perusing the myriad parallel sessions and it’s proving very difficult to choose just one in each slot.  I plan to finalise my choices on the long flight to Birmingham.
5. Top conference going tip?
As a presenter – prepare handouts (paper or digital – I often share a link to a Google Drive folder with materials that I can update after the conference) and try to stick around after your presenter for questions.
As an attendee – don’t feel the need to attend every single session.  It can be quite fatiguing rushing from one room to another and sometimes 45 minutes of idle time to check out poster sessions or publishers’ offerings, or just grab a coffee, can really rejuvenate you. [back]

Chad Langford & Joshua Albair
1. Who are you?
We are both linguists working in France; we work primarily as EFL teachers in a university setting with different kinds of learners of all levels. We’ve recently become more interested in developing in-house materials and in moving towards our learners’ needs and away from external syllabi and course books. We find corpora fascinating and really useful.

We teach a variety of courses, but it is our experience in teaching General English to adult learners in the Adult Education department of our university that has provided us the greatest stimulation for this project. Our experience as linguists means that we’re interested in research, and we both have had experience working with large corpora. This is the first time we’ve embarked on creating a corpus of our own, and we’re really excited about it and what it can bring to the profession.
2. Who should come to your talk?
Anyone who is interested in how the manipulation of data can give teachers a better idea of how the language they teach works will find our talk interesting, as will anyone interested in genre studies and in exposing learners to different genres as readers and writers.

The online restaurant review is an interesting genre to look at: on the one hand, some of our more basic intuitions about it are borne out; on the other hand, an actual searchable corpus reveals important characteristics of the genre that might otherwise not be so obvious. Finally, anyone interested in corpora and in the possibility of creating their own corpus should come – we would definitely like to encourage others to pursue same sort of project.
3. Why should they come?
The best reason to come would be for us to meet each other. We would love to meet, and then stay in contact, with other people who are interested in what we’re interested in and in developing a network where we can help each other and share experiences and materials.
4. Which talk(s) are you looking forward to?
We are looking forward to the plenaries, of course. Others corpus-related talks are at the top of our list, too. Grammar and discourse are two other areas we’re interested in. It’s going to be hard to choose, and I think we’ll be pretty busy listening to other people for a lot of the time we’re there.
5. Top conference going tip?
You can’t do everything. Take the time to meet people, and exchange contact information with them. And do it when you meet them – you’re never sure to run into them later, and there will be huge number of delegates this year. [back]

A huge thanks to all the presenters, break a leg folks : )