Assessment of ESOL Learners

Mini-Grant Report

Nancy Fritz, an ESOL teacher at the Genesis Center in Providence, completed this minigrant project in the fall of 1998. Since then, she has shared her findings with others during ESOL sharing sessions, and has participated in other discussions around assessment, standards and accountability. Nancy's report provides useful background information for our consideration as the state enters into a process of (re)considering its adult education standards as part of the changes inherent in the Workforce Investment Act.

[September 20/1999]


Assessment of ESOL learners' progress is problematic for many reasons. Commercially available standardized tests often fail to adequately asses learner strengths and weaknesses particularly at the lower ends of literacy skills. Such tests also do not necessarily measure what students have learned in class or address learner goals (Lytle & Wolfe, Adult Literacy Education: Program Evaluation and Learner Assessment, 1989). In addition, some standardized tests focus solely on reading/writing skills and do not address listening/speaking progress. Some alternative evaluation instruments (such as portfolio assessment) do a good job of demonstrating students' growth, but are difficult to implement effectively with large classes because they are time-consuming and require a significant amount of staff training. Furthermore, many alternative assessment tools do not quantify outcomes, which may be a desirable feature for funding sources, learners and teachers, and program administrators.

For my mini-grant project, I researched the literature to try to find assessment tools that could be used to measure ESOL learner progress specifically in the areas of listening/speaking and writing. I was interested in finding instruments that would measure these skills in a way that is fair to students, understandable and usable for teachers and administrators, and quantifiable for everyone who is interested. In this report, I will share what I found.

My search began with a review of traditional assessment measures, standardized tests that are widely used in ESOL programs such as the BEST test and the CASAS. The attractiveness of these tests comes from the fact that students can be compared to other students in the same or other programs with a standardized score. The tests are generally quite cost effective. They can be administered to large numbers of students at the same time. In addition, some tests such as the BEST test require very little training for teachers to administer.

There are, however, many serious objections to using standardized tests as a measure of students' knowledge of or progress in English.

 

 

 

 

 

 

 

 

 

For people who are interested in reading more about the objections to standardized tests, I recommend Elsa Auerbach's Making Meaning, Making Change for an excellent discussion.

In spite of the fact that there are so many objections to standardized tests, their ease of administration and the ability to compare students' progress over time with a numerical score are very attractive features, especially to teachers who do not have a lot of time. They are also usually necessary to report data on students' progress to funders. For more meaningful measures of progress, many teachers and students use various alternative measures such as student-teacher interviews and conferences, checklists and questionnaires and portfolio assessment. A portfolio is a purposeful collection of student work that exhibits the student's efforts, progress or achievements in given areas. The strengths of portfolio assessment are that it can:

1) capture a wide array of complex student outcomes

2) display the processes which the student uses to produce work

3) enable the student to monitor his/her own progress and become part of the assessment process

4) provide ongoing and continuous information of student progress and

5) provide feedback to the teacher for instructional planning and improvement.

 

Some potential limitations or problems with portfolio assessment are:

1) Staff training is required to successfully implement the idea. Portfolio assessment is a new concept to many teachers who were trained to use more traditional methods

2) A portfolio assessment needs to be carefully developed so that it does not become a mere collection of student work samples

3) Instructors need to be sure that what is included in portfolios portrays a representative picture of student performance.

 

I think that in addition to samples of students' work, a portfolio could show the progress a student has made as measured on different checklists or indexes. Looking at samples of work or listening to tapes made of a student's conversation may or may not clearly show progress to an interested person. Furthermore, a lack of time often precludes a detailed examination of a student's portfolio. The ability to see progress in terms of changes in a numerical score is therefore sometimes desirable. The following tools seem to me to be useful for the purpose:


Aural/Oral:

 

A. For measuring speech intelligibility and communicability, the index presented by Joan Morley in Pronunciation Pedagogy and Theory, New Views, New Directions (1994, Alexandria, VA: TESOL) seems to me to be a simple and usable scale. Teachers could tape record short conversations with students and rate their speech intelligibility at various intervals during the course.

B. The Literacy Volunteers of America ESLOA test is brief and easy to administer. It measures aural comprehension and oral production from beginner through intermediate levels. My objection to the test is that some of the pictures are difficult to understand, but as a rough measure of students' oral/aural abilities, I think it is pretty good.

C. The Aural-Oral Language Observation Matrix (See Figure A) is a more elaborate tool that could be used to show students' progress. The interviewer should be consistent over time in order to make this matrix reliable, but I think this scale captures the important features of aural-oral language.

Figure A - Aural-Oral Language Observation Matrix (based on Student Oral LAnguage Observation matrix, CA Department of Education, Office of Bilingual Bicultural Education).

Language Area

1

2

3

4

5

observations

A. Comprehension

does not understand even simple conversation

has great difficulty understand-

ing what is said and only with many prompts, repetitions

understands most of conversa-

tion with prompts - repetition at slower than normal speed

understands most conversations at normal speed; trouble with some diioms and vernacular forms

understands everyday conversations and beginning to function in most non-context embeded social discourse

B. Speech Production

1. speech flow

speech is limited to few halting words of tries to mimic

hesitant, awkward; silent or very limited at times

fairly constant flow but disrupted or halted by apparent limitations

generally fluent in conversation with periodic hesitations and awkwardness

fluency and ease in conversation and discussion, approxmiates native speaker

2. vocabulary

little or not basic, functional vocabulary

very basic functional control and very liited self-expression

ifrequent misues of words; awkward self-expression

for his/her age, only periodic semnatic misuse or inadequacy

for his/her age, near-native use and or words and idioms, with an occasional inadequacy

3. word-order and usage

little or no ability to string words together

restricted to chunks, phrases; much rephrasing

numerous syntax, usage + grammar errors

periodic misuse of syntax, usage + grammar without loss of meaning

occasional lapse or inadequacy but near native approximation

4. pronunciation

barely or not compre-

hensible

very difficult to understand; needs to repeat often

difficult to compre-

hend; many misunder-

standings

intelligible with periodic intonation and stress errors but understand-

able

intelligible with occasional intonation and stress errors

C. Communication Management

1. discourse management

does not express self in English

instrumental expression with little control of conversation management

functional expression with some conversa--

tion management control

adequate and clear expression with periodic inadequacy in discourse management

clear expression with near-native use of discourse management

2. pragmatics and paralinguistic forms

does not recognize or use

recognizes but minimal use

comprehends and uses periodically

comprehends and uses often

applies with near-native approximation


Writing:

Assessment in Adult ESL is a tricky issue and I appreciate having had the time and support to investigate it a little further. I think that the tools I found while doing this project will be useful to me in the future and I hope that other teachers may find them useful as well.


Figure B- Writing Mechanics Checklist

Date:

Begins sentence with a capital letter

Uses capital for I and names

Uses punctuation at end of sentence

Writes complete sentences

Uses correct verb tense

Subject and verb agree

Correct use of prepositions

Correct use of articles

Sentences are understandable

Handwriting is legible

Spaces between words

Spelling

 


other assessment resources:

Adeventures in Assessment - Adventures in Assessment provides a forum for adult literacy practitioners to critically reflect upon a range of issues and experiences pertaining to alternative assessment, published by SABES. Subscription information and selected articles appear on line.

Assessment and evaluation digests from the National Clearinghouse for ESL Literacy Education (NCLE).

Assessment of English Language learners in adult education programs (November 1995) - Compiled by Kristin Carl, Maggie Rosen, Christina Cavaluzzi Evaluation Assistance Center East, George Washington University

Assessing Success in Family Literacy Projects: Alternative Approaches to Assessment and Evaluation Daniel D. Holt, Editor


back to ESOL

lr/ri home