日本テスト学会誌 Vol.17 No.1 Abstract
トップ>学会誌>既刊号一覧>既刊号（17-1）>Abstract
JART Vol.17 No.1
| ▶ General research | |
|---|---|
| Evaluation of score adjustment using the adjusted median method | |
| Kenichi Kikuchi, Naoko Nakaune | |
| Toho University | |
| In recent years, many universities have adopted the adjusted median method as a way to adjust scores between the optional subjects on university entrance examinations. However, there has been little discussion of its nature from a test theory perspective. Therefore, we examined the kinds of adjustment results that are produced by using the adjusted median method to adjust scores according to the difference in mean and standard deviation between the optional subjects. Results indicated that when test scores are distributed in normal distributions, the adjusted median method generally adjusts the score so that subjects which originally have a high mean distribution with a large standard deviation are more advantageous to the top performers. In principle, score adjustment should be carried out to eliminate such inequity. Meanwhile, the adjusted median method is advantageous in that even after adjusting the score, the zero and perfect scores remain unchanged, and the adjusted score can be easily calculated by using the median. The advantages and disadvantages of this method must thus be considered when being implemented. | |
| Keywords: adjusted median method, university entrance examination, optional subject, score adjustment, test score equating | |
| ▶ General research | |
| The applicability of the CEFR (Common European Framework of Reference for Languages) to Japanese Language Education | |
| Hiroyuki Noguchi1, Atsuko Osumi2, Ryuichi Kumagai3, and Megumi Shimada4 | |
| 1Nagoya University, 2Japan Foundation, 3Tohoku University, 4Nihon University | |
| In this study, we examined the possibility of applying the CEFR to Japanese language education from the perspective of the scale construction theory. Specifically, our study was carried out as follows: ⅰ) Based on the illustrative descriptors specified by the CEFR for each communicative activity ('listening', 'reading', 'speaking', 'writing', and 'interaction'), we developed a questionnaire in which learners self-assessed the degree to which they 'can do' specific communicative activities in Japanese and in which they provided answers according to a 4-point rating scale. ⅱ) A survey of Japanese learners was conducted in Japan and overseas countries. ⅲ) Based on the data of this survey, the descriptors were put on the IRT scale by GRM (Graded Response Model), and the order of difficulty of the descriptors for each communicative activity was compared with that of the original CEFR. As a result, it became clear that in order to utilize the CEFR in Japanese language education, modifications and supplements are necessary, especially in the area of 'interaction', so as to reflect the differences of levels, the need for inclusion of unique elements of the Japanese language, and the sociocultural differences between Europe and Japan. | |
| Keywords: CEFR, Descriptors, Can-do Statements, Uniqueness of the Japanese Language, IRT Scaling, Graded Response Model (GRM) | |
| ▶ Case study | |
| Development of CBT Reading Items to Elicit “Aggressive Reading” ―Using Language Test of the University of Tokyo Entrance Examination― | |
| Hiroyuki Masukawa1, Hajime Shirouzu2, Moegi Saito3, Shinya Iikubo3, and Takuya Amano4 | |
| 1University of the Sacred Heart, Tokyo, 2National Institute for Educational Policy Research, 3The University of Tokyo, 4Saitama Prefectural Hanyu Daiichi High School | |
| Aggressive reading is a prerequisite cognitive activity in college life, in which readers deconstruct a text into pieces of information and restructure them into a coherent understanding. University entrance examinees, however, often rely on shallow processing including test wiseness. We therefore developed CBT reading items to elicit aggressive reading for previous language tests of the University of Tokyo entrance examination. We compared this revised version with the original version transplanted from the examination to study the effects on items requiring knowledge integration using two groups of students, totaling 79, who differed in their experiences of aggressive reading. A two-way ANOVA revealed significant interactions between the versions and the groups. CBT also enabled process analyses that revealed a hidden difference in the solution processes of the two sub-groups of students who demonstrated equivalent good results. The results indicate the potential of CBT in assisting and assessing the reading processes. | |
| Keywords: Computer-Based Testing (CBT), aggressive reading, deep processing, control and assessment of solving process, entrance examination items | |
| ▶ Case study | |
| Development of a multiple-choice fill-in-the-blanks assessment of English with a Q-matrix for cognitive diagnostic models | |
| Kentaro Fukushima, Nao Uchida, Kensuke Okada | |
| Graduate School of Education, the University of Tokyo | |
| Cognitive diagnostic models (CDMs) have been attracting attention as a method for diagnosing the learning subgoals mastered by examinees. In particular, CDMs for multiple-choice item assessments have been developed to elicit more information on examinees when they choose an incorrect option. However, the CDMs for multiple-choice items require test developers to specify a Q-matrix for each option of the items. Owing to the difficulty of developing such multiple-choice tests as well as the unavailability of a public dataset, existing studies have often conducted only simulation studies. Nevertheless, they have not examined their application to real data. Hence, knowledge regarding the empirical performance of CDMs for multiple-choice items is limited. To address this problem, this study composed a multiple-choice assessment of English with a Q-matrix and evaluated the performance of a CDM for multiple choices. The results imply that, although the model passed the predictive check, the examinees’ answering process to multiple-choice items could not be fully captured by the model, indicating the need for further model development. | |
| Keywords: test development, cognitive diagnostic models, multiple-choice, Bayesian inference | |
| ▶ Case study | |
| COVID-19 Prevention Measures in University Entrance Examinations -A Comparative Case Study of Standardized Tests in Japan, China, and South Korea - | |
| Hongyu Nan | |
| Tohoku University | |
| In 2020, the university entrance examinations of Japan, China and South Korea are entering a revolutionary transforming period. The new university entrance examinations reformation saw various prevention measures against COVID-19 being implemented. In this study, COVID-19 prevention measures for university entrance examinations taken by Japan, China and South Korea were studied and compared. Through comparison, it was found that Japan, China, and South Korea were either conducting or planning to conduct their national unified entrance examinations in spite of the corona virus pandemic. In China, examinations were postponed until July, one month behind usual, whereas for South Korea, examinations were postponed until December, behind by two weeks. Japan’s examinations were scheduled to be held in January, but the examination period has been extended. In this paper, the similarities and differences of measures taken by each country will be discussed under the perspectives of "ensuring fairness" and "determining relevant abilities". | |
| Keywords: University entrance examination, COVID-19, international comparison, Japan , China, South Korea | |
| ▶ Review | |
| Defining Past Exam Papers and Their Relationship to Fairness and Validity: An Analysis of English Test Preparation Materials for University Entrance Examinations in Japan | |
| Shotaro Baba1, 2 | |
| 1Tokyo University of Foreign Studies, 2Japan Society for the Promotion of Science | |
| In the context of high-stakes testing, test takers often use a test preparation material called a “past exam paper.” Although this term is used to refer to a series of questions distributed on the past examination day, no study has given its operational definition. The present study examined the contents of English test preparation materials for university entrance examinations in Japan, and proposed four definitions: (a) an original entrance examination paper which has been distributed on the examination day; (b) a test preparation material which is composed of original entrance examination questions, model answers, and trend analysis of questions, all of which are published per university per year; (c) per university per subject; and (d) per subject. With these definitions, this study explored the reason why previous studies could not reach consensus on the effectiveness of a past exam paper in terms of fairness of test preparation and validity of test score interpretation. | |
| Keywords: past exam paper, test preparation, university entrance examination, fairness, validity | |
| ▶ Report (Special feature) | |
| An attempt to apply simultaneous multi-population analysis to examine the factor structure of the same examinee group data | |
| Yuta Akita | |
| Graduate School of Education, Tohoku University | |
| ▶ Report (Special feature) | |
| Review of Japan-China Comparative Study on Parent-Child Relationships during College Selection | |
| Yihan Guo | |
| Tohoku University | |
| ▶ Report (Special feature) | |
| Comparison of the application process for the university admission between China and Japan：Focusing on self-scoring system | |
| Ruixian Zhou | |
| Tohoku University | |
| ▶ Report (Special feature) | |
| A comparative study on the selection of Advisers about University Admissions between Japanese and Chinese High School Students | |
| Ruyu Lin, Naoki T.Kuramoto | |
| Tohoku University | |
| ▶ Report (Special feature) | |
| Relationship between National University’s Individual Academic Achievement Examinations and Course of Study: Changes in the Question Items of Geography, History and Citizens | |
| Atsuyuki Takagi | |
| Tohoku University | |