日本テスト学会誌 Vol.13 No.1 Abstract
トップ>学会誌>既刊号一覧>既刊号（13-1）>Abstract
JART Vol.13 No.1
| ▶ General research | |
|---|---|
| Comparison among cognitive diagnostic models and item response models in a Japanese sample of TIMSS 2007 mathematics data. | |
| Kazuhiro Yamaguchi1, Kensuke Okada2 | |
| 1Graduate School of Education, the University of Tokyo, 2Department of Psychology, Senshu University | |
| Although Japan has been participating in the Trends in International Mathematics and Science Study (TIMSS) since its inception, it is still not known what kind of cognitive diagnostic models best describes the Japanese sample. Therefore, in this study, we compare the relative fit of a variety of cognitive diagnostic models in the Japanese sample of the TIMSS assessment. We also investigate the possible reasons for the differences in model fit. The results of the comparison and investigation are as follows. The reduced reparameterized unified model (R-RUM) was found to be the best in terms of both AIC and BIC. The good fit of the R-RUM can be attributed to the many cognitive elements required to solve the TIMSS items, as well as their interactions with each other. In addition, the diagnostic models fitted better to the data than the item response theory models. | |
| Keywords: cognitive diagnostic models, IRT model, model comparison, TIMSS 2007 | |
| ▶ General research | |
| The effects of Q-matrix misspecification in a cognitive diagnostic model on the diagnostic accuracy: Examination by considering cognitive hierarchy structures | |
| Kazuhiro Yamaguchi | |
| Graduate School of Education, the University of Tokyo | |
| Cognitive diagnostic models(CDMs) are useful to assess student’s attribute mastery patterns but misspecification of a Q-matrix could lead to inaccurate measurement. The purpose of this study was to assess how misspecifications of a Q-matrix with cognitive structures called attribute hierarchy structures (AHSes) affects student’s attribute mastery patterns estimates and item parameter estimates by simulation experiment. We got four main results: 1. misspecification declined students’ mastery pattern accuracy estimate for all AHS, especially divergent structure, 2. overestimate condition had more effect than underestimate condition, 3. we could relax the effect if Q matrix had fewer misspecification, 3. sample size did not have effects on misspecification’s results, 4. misspecification would affect right setting item parameters in some AHS. | |
| Keywords: Cognitive Diagnostic Model, Attribute hierarchy structures, Q-matrix, misspecification | |
| ▶ Case study | |
| Detection of Differential Item Functioning Using Item Response Theory in Aptitude Test Data Implemented Multiple Countries | |
| Yutaro Sakamoto1,2, Shino Sako1, and Shiho Imashiro1 | |
| 1Recruit Management Solutions Co., Ltd., 2Graduate School of Education, Tohoku University | |
| The methodological and applied studies of differential item functioning (DIF) have been accumulated, but the study focused on three or more groups and polytomous item data is not enough. The purpose of the present study was to detect DIF using IRT in polytomous item data with four ordered categories of the four scales measuring extroversion, leadership, autonomy and adjustment. As a result, the authors detected DIF in extroversion and leadership scale. In addition, the authors compared standard error before and after item purification and confirmed that they did not significantly differ. The need for further study of the effect of multidimensionality on DIF was discussed at the end. | |
| Keywords: differential item functioning (DIF), item response theory (IRT), standard error | |
| ▶ Case study | |
| Developing Lectures about Testing in a General Education Course Measuring the Effectiveness of “the Pedagogy of Testing” | |
| Takuya Kimura1, Dai Nishigori2 | |
| 1Kyushu University, 2Saga University | |
| In this paper, we discuss the possibility of an “Pedagogy of Testing”. First of all, we developed lectures on the topic of testing in a general education course. And then, we measured the effectiveness of this “Pedagogy of Testing”. Our purpose in these “Pedagogy of Testing” lectures is to encourage a change in consciousness from a passive attitude toward testing to an active (scientific and analytic) attitude. In these lectures, we teach the points of view of the academic disciplines, such as philosophy, history, law, sociology, psychology, mathematics, and statistics. After that, we teach test evaluation technologies and the knowledge of test theory. Knowing the elements of test theory such as reliability and validity and the points of view of a variety of test analyses and designs, freshmen or sophomore students can experience of test analysis and design through a report, and experience the dilemmas and the limitations of testing. When one recognizes that “testing” as an academic or research subject, the impression of “testing” improves well. We were thus able to understand the structure of the recognition of the “Pedagogy of Testing”. | |
| Keywords: Pedagogy of Testing, General Education, Class Development, Measurement of the Effectiveness | |
| ▶ Case study | |
| Classification of Item Types in the Individual Scholastic Achievement Tests of National Universities | |
| Tomohiro Miyamoto, Naoki Kuramoto | |
| Tohoku University | |
| As part of efforts aimed at examining the ability of Japan’s national universities to provide achievement tests used for admissions, the present study classified items executed in the individual scholastic achievement tests of every national university ( except graduate university ) during the general admission process of the 2015 academic year. Specifically, we collected approximately 24,000 test items that covered practically all of the items provided by each of the 82 national universities. The result showed that all but one national university executed writing-based test items and that half of the universities provided more than 150 writing-based test items excluding simple items such as fill-in-the-blank or short-answer type items. Writing-based test items dominated the objective items in all subjects, thereby accounting for approximately 80% of all subjects except for “English language” studies. On the other hand, we also found some objective items in the “Japanese language”, “Essay”, and “Compound” subjects, which indicates it would be misleading to discuss this issue based on the coverage rate of those three subjects alone. The results obtained thus far have again reminded us of the continuing importance of evidenced-based discussion in educational issues. | |
| Keywords: national university, item construction, writing, individual scholastic achievement test, item type | |