초급 수준 대학생의 영어 능력 평가를 위한 챗봇 기반 말하기 평가 실행 (Implementing a Chatbot-Based Speaking Assessment of Beginner College Students’ English Language Proficiency)

서지정보

· 발행기관 : 한국멀티미디어언어교육학회
· 수록지 정보 : 멀티미디어 언어교육 / 27권 / 4호 / 19 ~ 44페이지
· 저자명 : 김정태

초록

There is a lack of research on the use of the AI voice chatbot in speaking tests. This study compared ESWT (English Speaking & Writing Test) and CEST (Chatbot-based English Speaking Test) scores, analyzed the impact of interactional competence on the wholistic scores of CEST, and examined test-takers’ perceptions of item difficulties. Twenty-seven Korean university students, primarily at novice English language proficiency levels, participated in the study. They took both ESWT and CEST followed by surveys asking their self-perceived item difficulty for each test. The results of paired ttest showed that ESWT yielded higher mean scores across all analytical constructs and holistic scores with significant differences in pronunciation and coherence. A Ridge regression revealed that the interactivity and fluency constructs of CEST strongly influenced its wholistic score, highlighting the potential of AI chatbots to enhance EFL learners' speaking abilities. Participants, however, perceived CEST as more challenging than ESWT, pointing out the need for more preparation and response time due to their low proficiency levels. Regarding item difficulty, read-aloud and selfintroduction tasks were considered the easiest, while opinion-giving and long consecutive questions were the most challenging in ESWT. Participants also found all items challenging overall, reflecting their limited language competence and underscoring the necessity of targeted speaking skill development. Based on these findings, the study presents discussions and practical implications for integrating AI chatbots into speaking assessment.

영어초록

There is a lack of research on the use of the AI voice chatbot in speaking tests. This study compared ESWT (English Speaking & Writing Test) and CEST (Chatbot-based English Speaking Test) scores, analyzed the impact of interactional competence on the wholistic scores of CEST, and examined test-takers’ perceptions of item difficulties. Twenty-seven Korean university students, primarily at novice English language proficiency levels, participated in the study. They took both ESWT and CEST followed by surveys asking their self-perceived item difficulty for each test. The results of paired ttest showed that ESWT yielded higher mean scores across all analytical constructs and holistic scores with significant differences in pronunciation and coherence. A Ridge regression revealed that the interactivity and fluency constructs of CEST strongly influenced its wholistic score, highlighting the potential of AI chatbots to enhance EFL learners' speaking abilities. Participants, however, perceived CEST as more challenging than ESWT, pointing out the need for more preparation and response time due to their low proficiency levels. Regarding item difficulty, read-aloud and selfintroduction tasks were considered the easiest, while opinion-giving and long consecutive questions were the most challenging in ESWT. Participants also found all items challenging overall, reflecting their limited language competence and underscoring the necessity of targeted speaking skill development. Based on these findings, the study presents discussions and practical implications for integrating AI chatbots into speaking assessment.

참고자료

· 없음