THE EFFECT OF THE USE OF VIDEO TEXTS ON ESL LISTENING TEST-TAKER PERFORMANCE” BY ELVIS WAGNER (2010) CRITICAL REVIEW

 

Part one

Problem statement and research questions

The research article “The effect of the use of video texts on ESL listening test-taker performance” by Elvis Wagner (2010) premise is employment of the video texts to enhance on second language listening ability.  The research question is necessitated by the fact that for the longest time visual components have been perceived vital for the listener to understand the aural information while video texts were rarely utilized to perform the same duty.  While some of the scholars have been arguing that video text has the potential to increase the performance of L2 listening test, some of the scholars have been against the concept terming it and distraction that affects negatively the L2 listening test. Therefore, this report uses the quasi-experimental design method to conduct a research that will validate the research question.

 Research method, participants and data collection

The article employed Quasi experimental deign method non-randomized both in pretest and post-test.

Data collection was divided into two categories the pre-test and post-test

All the statistical data were analyzed using SPSS tools.

The sample population 103 test-takers (experimental group) and 99 test-takers (control group) were from the Community English program (CEP) and the adult non-credit language program in a University based in America.  Notably, the classes in CEP are divided into six different languages ability levels and the occupants are in the criteria of the outcome of the placement tests. The student’s age ranges from 18 to over 60 years and with different nationalities and cultural backgrounds. Over 20 natives’ languages are presented in these classes.

 Findings and discussions

The summary of the results is as follows: the mean for the audio only group was 7.27 out of 12 (60.6%) and the standard deviation 2.42(notably, n=99). For the video group, pre-test mean was 7.34 (61.2%) and standard deviation 2.4. Also reliability was α=0.77.

 On the side, for the post-test, the results were as follows: audio only mean 20.02 (50%) , SD 7.30 and internal consistency reliability  α=0.88. The results of the video group were as follows: mean 22.76 (56.7%), SD 7.86 and ICR α=0.90

 The results indicate the following findings: the video group post-test scores were higher by at least 6% compare to the audio-only group score’s. The scores supported the notion that the visual aspect of the video text contributes to increment of the scores of the listening test. Therefore, the test takers who received the input of both the visual and the aural channels had higher scores compared to their counterparts.

 

Conclusion

 The use of video texts in ESL listening tests can have a positive effect on test-taker performance. Video texts can provide a more engaging and realistic context for students to practice their listening skills, as they can be more easily related to real-life situations. Additionally, video texts can provide visual cues that can help students better understand the content of the test, as well as provide a more natural setting for the test-taker to practice their listening skills. Furthermore, video texts can help to reduce test anxiety, as students may feel more comfortable in a more natural setting. Finally, video texts can help to reduce the amount of time needed to complete the test, as students can more quickly process the information and answer the questions.

 

 Future studies and limitations

 The report suggests that the future studies should focus on the utilization of the video texts on a myriad of genres of texts, precisely one on one interactive conversation.

However, the research had limitations that include but not limited to the study only focusing o the group performance at the expense of individual test-taker performance and issues with generalizability of the results. The generalizability issue was due to the fact that though differences in scores between the experimental and control groups were statistically significant, the impact sizes were relatively smaller and there was need for more studies to investigate if the video texts effects on the tests in the wide range of other contexts.

 Part two

 Research question and literature review

The research article “The effect of the use of video texts on ESL listening test-taker performance” by Wagner (2010) premise is employment of the video texts to help improve the performance of L2 listening test of test-takers.

To further develop on the research question Wegner (2010)  divided the main research question in the following questions:  Question one is casted on asking how much utilization of video texts on L2 listening test affect  the test performance of the text taker and what is their performance compared to the test-taker in the audio-only situation?

  The second question to structure to have insights on how higher or lower is the scores of test-taker in the video condition compared to those in the audio-only condition. The question tries to figure out the manner in which ‘non-verbal’ information provided by the video channel impact on the performance of the test-taker and precisely, to explore the components of the non-verbal information that has potentialities to affect the performance of those test items individually.

Notably, the research question for every study should be concise and clear.  It has been achieved by the author dividing the main question into smaller manageable questions. The questions mentioned above are trying to create a relationship between video texts and higher performance in L2 listening tests.  The questions are clear and precise and also concise since they are addressing not only higher score but the range of score.

 Literature review

The literature review begins by theoretical rationale the use of video and non-verbal components in listening the second language in a classroom and its ability to increase performance L2 listeners’ comprehension.  These justification leads to development of the critical question of whether video texts on listening test many have an influence in the listening performance. From the literature review they are many past studies with conflicting results.   However, Wegner (2010) literature evaluates both positive and negative influence on the L2 listening performance. This approach is significant by linking the foundation of the research subject to the wide interest and attention. Furthermore, looking at the positive and the negative influence the research tries to indicate though the video texts have its shortcomings, on the other side, the positive outdo the negatives.

 To begin with Wegner argues that video texts have the ability to allow listener to watch the kinetic behavior of speakers.  To support the argument the report included at least three citations hence producing a strong argument. However, there was only general evaluation of the kinetic behavior of speakers without engaging more arguments to expound on the concept.

 The second argument on the salient of video text is that the component allows the integration of still pictures in listening texts.

 However, the assumption that video text improves the performance of the L2 listening test has received criticism. Some of the scholars have argued that test-takers tendency to be glued on the question paper inhibits them from watching the video text monitor. To support the argument, the criticism cited that through evidently video enhance the face validity and authenticity; they have potential to cause distraction. The distraction can results to lowering comprehension since it distract the attention of the test-takers from the aural input.

The article was written in 2010. Looking at the references most of them were dated in 1990s and early 2000 which provide an impression   that the article may be written during that time. It is good research practice to include references which are recent and within a particular range of years for instance no late than five years from the current date. Therefore, the article could be more credible if the sources could be up-to-date and more recently.

Research design

To investigate on impact of video texts on the performance of second language test-taker, the article employed quasi experimental design method.  This method design resembles an experimental design and even though an independent variable is manipulated, participants are non-randomly assigned to conditions with the aim of making the resulting group to be dissimilar in some ways (Rodgers et al., 2019). Furthermore, by the article employing the method design it enables usage of pre-test,-post-test design that enabled the dependent variable to be measured once before the implementation of the treatment and once after the implementation (Miller et al., 2019).  The video conditions and the text only conditions groups were given pretest and post-test and scores of both groups were compared.

 The Pre-test and post-test is utilized due the need of the experimental manipulation. In the spectrum the same participants are measured on the variables of interest at different points of time.

 Data collection procedures

 Data collection was divided into two categories the “pre-test and post-test”. On one side the, pre-test were administered at the beginning of the four new CEP semesters while on the other hand, post-test was across four CEP semesters. For the test-takers they were given booklets and the video text was played.

The procedural collection of the data makes it easy for the research to have ample time to collect data also; the research can organize data in a structural and orderly manner.

Questionnaires (research instrument)

The research instrument used in the report is the questionnaires.  The administered questionnaires are used purposely for collection of data both at pre-test and post-test. There were 18 multiple choice questions right/wrong that was administered. The marking scheme awards appoint to right score and zero point to a wrong score. Also, they were 22 short answer items that was based on the wrong/right rubric. The scores ranges from 0, 0.5 and 1 developed scoring key were followed to give guidance to provide the list of acceptable responses and how much credit they carried. Furthermore, the tests were scored second time to enhance element of accuracy and there were correlation of the two set of answers items by computing the percentage agreement of the scores. The data was inserted into a computer for intensive analysis.  Technically, the utilization of the wrong/right criteria to award points to correct and incorrect answers simplify the entire process of finding the scores. The methodology has no complexity and only requires the input of scores according to the assigned marks (Omolade et al., 2022).  The highest marks means high performance and lowest marks means low performance. Therefore it becomes easy to compare the results of test-testers o video-text condition and audio-only condition and conclude which has the higher performance.

Therefore, to assess whether the questionnaires measured what they were supposed to measure we use validity and reliability. These aspects are prerequisite for research integrity and quality of the research instrument. On the research question whether video –text group test takers had a higher performance than the audio-only test taker group, the report uses multivariate tests to justify the assertion.  The results of the data indicates the F-value =5.15 and p< 0.01 meaning that the video-text group significantly scored higher scores in comparison with the text only group. Therefore, it purports the research instrument was valid and reliable.

 Method used to analyzing data

All the statistical analysis was carried out using the SPSS version 12.0. The tool enabled the study to carry out descriptive analysis for both pre-test and post-test. There was also computation of the internal consistency aiming to figure out on the variance of the test and eventually the measurement error. Also, since the research groups might not be equal to the second language learning ability, the study used ANCOVA and MANCOVA to analyze the data.

 The utilization of the SPSS was ideal for the study since it enables the utilization of the diverse of data sets and also conducts the different analysis such as the T-test, ANCOVA and MANCOVA (Suresh, 2015). Therefore, the SPSS data analysis has the following advantage to analysis of the report findings.

It is easy to use the software. In other words, there is much needed effort for the researchers to use the software. Also, the duration needed to analyze the data with the help of SPSS is relatively less compared to the other statistical tools that are prerequisite to conduct an in-depth critical analysis by inclusion of all data and information correctly. Notably, the basic application of the software is to analyze the scientific data is imperative to achieve the objective of the article.  Therefore, based on the report, the SPSS enabled the in-depth statistical analysis an evaluation. The use of the mean and  SD summarizes the scientific research. Through the use of mean it was clearly that the video group post-test scores were higher by at least 6% compare to the audio-only group scores. Therefore

 Secondly, SPSS tool is central in data transformation.  The author collects raw data and needed to transform to meaningful information. The tool has the ability to handle large volume of data and information with the purpose of progressing in the research and draw the research findings and conclusion critically (Lee, 2005). Therefore, the software is useful in the study for the data analyze as it aids to analyze, transform and produce the results data set of the data collected between different data variables. Furthermore, as indicated in the report, the output can be through graphical representation to enable readers to understand and easily interpreted the results.  This is evident by the report providing various tables to indicate various statistical data and the results of the statistical analysis using the tools

 Finally, it is impactful in the regression analysis.  Through the ANCOVA it is possible to compare the groups and MANCOVA where there is comparison of the data with random variables in which the values are unknown. The regression using the SPSS helps the report to understand the relationship between the dependent and interdependent variables for a data stored in a synchronized manner. The tools have been vital in the article to produce some of the statistical analysis report such as F-value and the p-value

Conclusion

The article is a well-written paper focusing on the salient issue in L2 learning.  The article is well-structured from the abstract to the conclusion and details were well written and referenced. There is prevalent of the use of high-frequency words such reliability experimental and statistical analysis. Furthermore, the paper has to some considerable extent provided detailed discussion and descriptions of the major parts of the article (“introduction, literature review, data collection, method, findings, discussions and conclusion”).

 Generally, the article is compelling, interesting and informative. Notably, the author to support the research question provided every detail of the findings in the clear and precise manner using simple words and terminologies that is easily understandable by the readers. However, the major shortcoming of the article is citing old references.

 References

 

Lee, D.E., 2005. Probability of project completion using stochastic project scheduling simulation. Journal of construction engineering and management, 131(3), pp.310-318.

Miller, C.J., Smith, S.N. and Pugatch, M., 2020. Experimental and quasi-experimental designs in implementation research. Psychiatry Research, 283, p.112452.

 

Omolade, O.K., Stephenson, J., Simkhada, P. and Keely, A., 2022. Is this a good questionnaire? Dimensionality and category functioning of questionnaires used in nursing research. Nurse researcher, 30(3)

Rogers, J. and Revesz, A., 2019. Experimental and quasi-experimental designs. In The Routledge handbook of research methods in applied linguistics (pp. 133-143). Routledge.

 

Suresh, D., 2015. Important of SPSS for social sciences research. Available at SSRN 2663283.