An experience, some possibilities
The COVID-19 pandemic required most educational institutions in Colombia to implement emergency remote education as a mechanism to temporarily provide students with access to instruction in times of crisis . This alternative allowed classes to continue from home and kept the entire educational community safe in accordance with the 2009 Ministerial Directive of the Ministry of National Education
Even with the urgent need to migrate face-to-face to virtual English classes through emergency remote teaching, teachers understood that online learning, beyond these circumstances, required rigorous design and implementation research Means et al. (2014) identify nine dimensions that make this understanding even more complex. These dimensions are modality, rhythm and sequence, student-teacher interaction, pedagogy(s), teacher´s role, student´s role, online communication, monitoring and learning assessment and feedback. So, where to start?
In the following lines, I describe some visible steps of the process we followed with my team of English teachers to first discuss what we should prioritize in terms of monitoring and learning assessment and, secondly, incorporate appropriate principles of the use of technology to carry out relevant, flexible and equitable assessment. My intention, on the one hand, is to share this experience that has happened in an overwhelming way and on the other, to extend the discussion about what this process of analysis and adaptation has meant. Accordingly, this reflection is limited to offering a space of self-knowledge from which English teachers in our diverse and multiple contexts, can continue to explore and feed.
Stage one: the meaning of learning assessment and monitoring
Assessment is at the heart of the learning continuum. This involves being able to observe, value, feed and measure the actions, skills and attitudes of students in the light of the pedagogy(s) proposed in our course. This follow-up is present in all moments of learning: in the principles, in the objectives, in the sequence and gradation of the contents, and in the activities and learning materials. For this reason, evaluation does not mean exclusively that it is dependent on a grade or product. The nature of this definition would not change in virtual environments, but if we were to use technology, we would need to know how it would help to enrich this process in an articulated way and thus enhance meaningful learning experiences.
This is why in the current context we had to embrace the principles of flexibility and equity to know what types of monitoring and evaluation activities we could consider as alternatives other than numerical note reviews. Workshops, oral and written interventions, open-ended comprehension exercises, practice and consolidation of linguistic and cultural skills, projects, among others, include these options.
- Flexibility: This principle of learning assessment encompasses both summative evaluation (quantitatively measurable or as a quantifiable product —by means of a numerical scale) and formative (measurable qualitatively or as a process— by means of indicators, descriptors or performance criteria). Flexibility is advocated by the incorporation of different forms of evaluation envisaged in the pedagogical exercise, by opportunities of various moments of evaluation, by the forms of monitoring and feedback that promote the improvement of performance, and by the ways of teaching and learning in a foreign language.
- Equity: This other principle guarantees the minimum conditions for all students to account for their learning progress without being harmed by factors that affect their performance potential. (Montoya, 2020). This principle involves re-analyzing the context.
García Santa-Cecilia (2008) defines the analysis of the context as the inquiry of information necessary to logically plan the foreign language syllabus, the community to which it is directed, the formative scope, the resources available, among others. For this first moment of the analysis in our English course, students and teachers answered a survey that identified emerging needs due to the crisis, such as the linguistic, social and emotional needs of students, as well as the technological resources available. All this allowed us to understand the most effective way to review and/or adapt learning objectives, content, methodology, materials, activities, and evaluation.
Types of assessment
Nation (2010) identifies 6 types of assessments that can provide teachers and students with information about students’ knowledge and progress as well as information about their involvement. These types are placement, diagnosis, proficiency, learning observation, short-term achievement, and achievement at the end of the course. In this context, we use the principles of the last three of the list.
Learning observation: Throughout the course, the activities students develop are monitored to see if they are likely to meet the learning goal. Observation can be carried out using analysis techniques for task design, metacognition, or classroom observation.
Short-term achievement assessment: At regular intervals during the course, students’ progress can be monitored to check course learnings. They observe the products of the activities or a small group of activities. Examples include weekly tests, rubrics, and oral or written production checklists for teacher evaluation, or self-assessment and co-evaluation by the student and his or her peers.
Achievement assessment at the end of the course: This assessment is of a criterion nature, which means that they are evaluated from a previously established construct and not in relation to other students. For this reason, the main goal of this type of assessment is to value students based on what they have learned throughout the course. This happens at the end of the course and at one or two more points throughout the course. This evaluation is intended to examine the effectiveness of course content as well as student learning. This type of assessment can assess linguistic and sociocultural skills either separately or integrated.
Separate skills assessment or integrated skills assessment
General ly, when English courses develop reading, writing, listening comprehension, oral and lexical-grammar production skills in an integrated way, students’ progress verification processes take on a supportive character for feedback if they are done separately and/or discreetly. Based on this, what we want is for students to be aware of their strengths and weaknesses without the weaker skills affecting each other at the time of evaluation. For example, writing an email to a teacher requesting information about the task for next class indicates evaluating a single skill type: written production.
However, if the decision is to evaluate skills in an integrated way, it is necessary to analyze what is expected of the evaluation task by constructing specific criteria for the skills to be evaluated. This type of evaluation reflects more closely on how we use the language in real life, but it is more complex to track it. For example, reading a short article on ways to recycle usable items and then discussing orally the messages left by the article, indicates the evaluation of two or even more skills such as reading, oral production and vocabulary. Defining the construct, the task and its format and the evaluation mode is the activity that we were responsible for doing next.
Stage two: The Assessment Design
In any follow-up or evaluation process, it is essential to determine the expected minimum of the activity to promote learning.
- Defining the construct: According to the guidelines of the Common European Framework of Reference (MCRE), the construct is a set of concepts and skills that are deliberately and consciously decided for students to interact linguistically in an appropriate and creative way in all possible social and cultural contexts. Construct helps us identify a behavior change that can be achieved at the end of a task or a set of tasks. The construct is based on learning objectives.The goals set allow us to determine what we expect students to understand and do at the end of the course. These objectives are usually expressed in the form of actions that indicate a linguistic ability, competence, or attitude. In our case, we had to consider the suggested objectives of the CEFR already set out in the course program for the purpose of adjusting which of them were essential and which were expendable. The students’ exit profile indicated more clearly how to proceed. For example, at the end of the course, the student should demonstrate their overall written production competency at the A2 level.
- Define the follow-up or evaluation task: The task is defined as the set of actions and activities that allow us to produce or reproduce the use of the language in real-life situations. For example, an oral production activity as describing a place using an image, gives students the initial input to talk in a natural way without having to resort to reading. This type of activities connects students to the task from the first moment and motivates them to use the language authentically.Similarly, the materials used for the task must be appropriate for their level. If teachers have a textbook in their curriculum, they can use this resource, but when choosing materials either analog or digital from different sources, they should consider whether they are appropriate for their particular context. This and other aspects will be summarized in Table 1.
- Define the evaluation format: This corresponds to how the task is to be presented and the type of interaction that can be used. For example, for a listening comprehension activity, students should listen to an audio in which two people talk about the weather in their city. Then, individually, they will complete a true-false short quiz. At the end, they can independently verify answers automatically, using a key of answers that the teacher has previously designed.
- Define the mode: This aspect involves the use of technological resources that have positive effects on the student’s learning, that increases their commitment, keep their attention going and supports the pedagogies associated with the evaluation task. Adapting online activities should be a key skill in tackling these new ways of moving in space and time. We acknowledged that the assessment moments change in this mode and that not everything would be assessed in the same way as it was done in the face-to-face class. Synchronous and asynchronous modes were used to corroborate these learnings.
Stage three: using technological resources
Having done this analysis in this context, we thought it was a good decision to take a practical approach to virtuality using the instructional strategy model underpinned by Fiock (2020). In this model, three types of presences are fundamental: cognitive presence, social presence, and the teaching presence.
- Cognitive Presence: It is the ability to build and confirm meaning through reflection. This is characterized by four phases: (1) identification (in which the student examines the situation), (2) exploration (in which the student explores the components of that situation), (3) integration (in which the student integrates emerging meanings of the exploration), and (4) resolution (in which the student can apply new skills, knowledge and/or attitudes learned from the previous phases to situations in the real world). (Anderson, Rourke, Garrison, & Archer, 2001).
- Social Presence: It is the theory that explains the ability of human beings to present themselves as “real people” through means of communication. Garrison et al. (2000) argue that three categories of indicators of social presence can be identified: emotional, open communication and group cohesion. The first indicates how students express their identity and values; the second points to the way students develop aspects of recognition of themselves and others; and the third recognizes that students along with their teachers build a learning community.
- Teaching presence: It is the model composed of the design, guidance, and direction of cognitive and social processes to allow learning. According to Anderson et al., (2001), this presence has three components: (1) instructional design (curriculum, methodology, activities, etc.); (2) appropriate use of speech (establishing a social climate in the student group, recognizing and reinforcing student contributions, etc.); and (3) direct teaching (summarizing the discussion, introducing the content, providing instructions, giving timely feedback, etc.)
Table 1 illustrates the previous steps with some specific examples of activities and the technological resources that support them. This outline proposal could help other contexts understand the preliminary decisions made in relation to evaluation as the focus of all other aspects of learning.
| Language Skill | Learning Objectives (Can do statements) MCER) | Task and Materials | Format |
|---|---|---|---|
| Reading | Can generally understand straightforward factual texts on familiar topics. (B1) | Leer el texto de 5 párrafos “El Agua” y determinar información clave que indique la idea general del texto. Materiales: lectura, cuestionario, clave de respuestas (pueden estar en el libro de texto, libro del profersor o en un recurso en línea) | Leen el texto de forma individual y responden un cuestionario corto de comprensión de lectura. Consolidan de forma grupal para verificar las respuestas correctas e incorrectas. |
| Speaking | Can describe people’s everyday lives using a short series of simple phrases and sentences. (A2) | En una intervención de un minuto, describa el trabajo de un miembro de su familia. Use el presente simple y el vocabulario visto en clase. Materiales: Dispositivo de video (teléfono celular o computador), material de clase (libro de texto, guía con criterios, rúbrica y lista de verificación), recursos en línea. | Preparan la intervención de forma individual y graban un video de ellos mismos con su telefono celular o computador. Envían el video al profesor o lo suben a una carpeta en la plataforma. Los compañeros todos o algunos de los videos y preparan dos preguntas para tres diferentes compañeros. En la clase grupal, harán comentarios de aspectos positivos y aspectos a mejorar y se harán las preguntas. |
| Language Skill | Type of assessment | Modality |
|---|---|---|
| Reading | Observación del Aprendizaje: los estudiantes recibirán retroalimentación automática con una clave de respuestas. En una sesión en vivo, el grupo discutirá las respuestas y resolverán posibles dudas con sus compañeros. | Asincrónico: Actividades individuales de lectura; Sincrónico: Consolidación grupal. |
| Speaking | Evaluación de Logro a Corto Plazo: Esta actividad puede monitorear aprendizajes adquiridos después de la introducción, práctica y aplicación de contenido linguisticos como el uso del presente simple para describir y vocabulario acerca de actividades cotidianas en el entorno del trabajo. La evidencia de aprendizaje se refleja en el video grabado por los estudiantes, la interacción de pequeños grupos en la clase en vivo y la rúbrica analítica con los criterios de calificación de la actividad. Esta incluye aspectos de uso de la lengua como gramática y vocabulario, cumplimiento de la tarea o de la actividad comunicativa y pronunciación de las palabras clave aprendidas en el período a evaluar. | Asincrónico: Actividades individuales de producción oral (presentación en video); Sincrónico: Trabajo en grupos y preguntas. |
| Presence | |
|---|---|
| Cognitive | Aprendizaje y uso de estrategias de lectura como skimming, scanning y vocabulary in context para la comprensión general del texto. |
| Teaching | Trabajo individual de reconocimiento de su aprendizaje y construcción de conocimiento con sus pares. Participación en clase significativa y discusión de los contenidos y su desarrollo. |
| Cognitive | La lectura calibrada en nivel B1 asi como las preguntas del cuestionario. las instrucciones sobre el procedimiento, el tiempo de la tarea, entre otros, están consignadas por diferentes vías: email, por medio de un recurso LMS o explicadas oralmente por el profesor. El libro de texto trae una lectura ejemplo con las características del nivel. Sin embargo, si la lectura es adaptada de un recurso en línea, se deberá analizar el nivel de lengua, la construcción y la complejidad de las preguntas para cumplir el objetivo de aprendizaje. El profesor prepara las respuestas a las preguntas con anterioridad y las deja disponibles después de cumplirse el tiempo de entrega de la tarea. |
| Social | Trabajo individual de reconocimiento de su aprendizaje y construcción de conocimiento con sus pares. Los estudiantes pueden tomarel rol de expertos por turnos para ofrecer retroalimentación efectiva a sus compañeros. La participación audiovisual permite la comunicación interpersonal entre profesor-estudiante, estudiante-estudiante y profesor-estudiantes |
| Teaching | Las instrucciones sobre el procedimiento, el tiempo de la tarea, entre otros, están consignadas por diferentes vías: email, por medio de un recurso LMS o explicadas oralmente por el profesor. Los estudiantes con notificados son suficiente tiempo para la ejecución de la actividad individual y de la actividad grupal. La rúbrica es compartida con anticipación para que los estudiantes tengan claridad sobre los criterios y mínimos esperados. El profesor provee retroalimentación sobre la actividad y da comentarios adicionales sobre los puntos positivos y los aspectos a mejorar. la co-evaluación es guida por el profesor por medio de una lista de verificación que el estudiante deberá llenar. El profesor aquí ofrede diferentes oportunidades de interacciones públicas y privadas. hay comentarios explícitos generales sobre los resultados de la actividad y del objetivo de aprendizaje. |
| Technological Resources | |
|---|---|
| Reading | Opción 1: Libro de texto: recurso de presentación interactiva del texto para proyectar las respuestas. Opción 2: Lectura adaptada: Un recurso como ActivelyLearn.com o como ReadWorks.com ofrece lecturas de acuerdo al tema, el nivel de lengua se identifica ayuda de otras herramientas (Lexile.com y Textinspector.com) el tipo de texto, el número de palabras y la audiencia (primaria, secundaria o universitaria). Igualmente, estas herramientas ofrecen la oportunidad de interactuar con los estudiantes en vivo, dar retroalimentación automática, dar comentarios a las respuestas simultáneamente y ofrecer la opción de trabajo en colaboración y visualización grupal. |
| Speaking | Video: Teléfono celular o computador, carpeta gestionada por la plataforma LMS o Google Classroom Rúbrica: Plantilla de Excel en línea con cálculo automático de los criterios (uso de fórmula) Co-evaluación: (Además del recurso de la clase en vivo) Usar una lista de chequeo en forma colaborativa donde todos pueden editar y comentar en simultánea como jamboardgoogle.com , Padlet.com o Nearpod.com los estudiantes pueden ver sus comentarios en línea al igual que los de sus compañeros. |
A few recommendations
These latter reflections depict the enormous challenge of thinking about the evaluation in the English class, “without a classroom”, during this pandemic time.
- Think about the context: knowing who our students are is a key aspect. Identifying your skills, attitudes, and resources can make a difference. We should forget about the myth that students are “digital natives” and recognize that they may be uncomfortable with some technological resources and some situations in remote classes.
- Be flexible: It is necessary to be aware of the processes of learning a language, giving consistent times and tasks to these principles so that students can identify, understand, practice and apply linguistic and sociocultural skills in spaces and times other than the face-to-face that are reevaluated in virtual scenarios.
- Provide clear and timely communication and feedback. Give explicit instructions on learning goals and expectations. Notice these small transformations in students’ learning through a systematic review. Choosing activities that indicate these transformations and delegating feedback to themselves are tasks that promote their autonomy and alleviate the work of teachers. Teachers should in turn strip themselves of this characteristic of omnipresence, because, although students are demanding more than ever the teachers´ accompaniment, this support must manifest itself at decisive moments in students’ learning.
- Be realistic: Evaluation should be reconfigured to a more positive and less punitive framework Formative evaluation must be present at many more times of learning and summative evaluation should be representative and consistent with our pedagogy. Building trust among all and looking for new ways to evaluate are ways to reduce the anxiety that this concept produces. Evaluating the “open book” style should be a valid condition for students to demonstrate the achievement of the goals at this time.
- Make things simple: This pandemic emergency made us move to use various technological resources in unexpected ways, and although we have learned much about it, we cannot take technology lightly. Technology enhances our pedagogies, but does not replace them. This, is our starting point. RM
Think about the context: knowing who our students are is a key aspect. Identifying their skills, attitudes, and resources can make a difference.
