日本テスト学会誌 Vol.16 No.1 Abstract
トップ>学会誌>既刊号一覧>既刊号（16-1）>Abstract
JART Vol.16 No.1
| ▶ General research | |
|---|---|
| Examination of Item writing guidelines for multiple choice questions | |
| Ayano Tsubota, Hidetoki Ishii | |
| Nagoya University | |
| Item writing guidelines have been examined in many studies. However, the way of functioning and effect sizes of each guideline are still uncleared. If flaws in items affect item characteristics such as proportion of correct under the situation that examinees do not aware of the flaw, it will ruin the measurement. To clarify these problems, 52 paired multiple-choice questions’ items on English, mathematics and language are developed and administered to 477 university students. In each item pair, one follows an item writing guideline and the other does not follow the guideline. As the result, it is found that proportions of correct are different in several item pairs although the flaws are not recognized by examinees. Proportions of correct of items which depend on other item or contain negative- wording options become lower. And, these item flaws are rarely pointed out by examinees. Item writers should notice that item characteristics can be unintendedly affected by item writing. | |
| Keywords: multiple-choice question, item writing guideline, Item flaw | |
| ▶ General research | |
| An Empirical Analysis based on Surveys on the Answering Process of Multiple-Choice Items to Measure the Ability to Think | |
| Sayaka Arai | |
| National Center for University Entrance Examinations | |
| For multiple-choice items, it is required to measure not only knowledge but also the ability to think. Although it is evaluated that some items of the National Center Test measure the ability to think, do examinees actually answer them using the ability to think? In this study, I used some items of the National Center Test (World History), in order to investigate participants’ answering process of them. In [Survey 1], a questionnaire survey was conducted. In [Survey 2], descriptions of the actual answering process were collected and analyzed. Through [Survey 1] and [Survey 2], it is shown that the items which are evaluated as items to measure the ability to think are actually answered by inferring based on the contents of the items as being expected. It is also shown that even the items deemed as measuring knowledge were answered not only by knowledge but also by inference based on knowledge. |
|
| Keywords: multiple-choice items, ability to think, answering process, National Center Test, world history | |
| ▶ General research | |
| Compensatory cognitive diagnostic models with continuous attribute mastery parameters | |
| Yoshito Tan, Kensuke Okada | |
| Graduate School of Education, the University of Tokyo | |
| Cognitive diagnostic models (CDMs) are a class of statistical models that diagnose the mastery of respondents’ cognitive traits, which are called attributes or skills. In standard CDMs, mastery of attributes is represented by binary discrete parameters. However, attribute mastery can be better described by continuous parameter that represents the degree of mastery rather than binary mastery/nonmastery status. The probabilistic-input, noisy conjunctive (PINC) model, as well as its higher-order extension, were proposed as noncompensatory models that assumed continuous attribute mastery, relaxing the assumption of discreteness. In the present study, we propose the probabilistic-input, noisy disjunctive (PIND) model as well as its higher-order extension that apply the same idea, relaxation to continuous variables, to compensatory models. After showing parameter recovery performance by simulation study in several conditions, we report the estimation result of these models using real data. The paper concludes with a discussion of the applicability of the proposed models. | |
| Keywords: cognitive diagnostic model, PINC model, compensatory model | |
| ▶ General research | |
| A Note on “the Loss of Information” due to the Use of Graded Score | |
| Shin-ichi Mayekawa, Tatsuo Otsu | |
| Research Office on Admission Selection Studies, The National Center for University Entrance Examinations | |
| So called “the loss of information” due to the use of graded score was investigated using both theoretic approach and numerical simulation. In theoretic approach, the standard error of estimation of the true score from graded score was derived within the framework of classical test theory. It was found that, when the reliability of the test is not extremely high, the loss of information is negligible. The similar result was obtained as the result of simulation study. | |
| Keywords: graded score, test score, stanine, classical test theory, test information, standard error | |
| ▶ Case study | |
| Scores in the “Practical Reading” and “Mathematical Thinking” Tests at the University Entrance and Admission Category | |
| Kumiko Shiina, Hirohito Sakurai, Kei Ito, Sayaka Arai, Hisao Miyano | |
| National Center for University Entrance Examinations | |
| Differences in scores in the “Practical Reading” and “Mathematical Thinking” tests between admission categories are analyzed for freshmen in a university participated the monitoring survey. Although both of the tests are non-subject-based tests, the score distributions of the freshmen in admission category with a high weight of the academic achievement tests are higher than those of other admission categories. This suggests that the abilities cultivated based on the learning of subjects up to high school are reflected in the scores of the tests. The freshmen by admission on recommendation with basic test of Japanese language and English show a relatively high score distribution for the both tests. It is suggested that applicants’ preparation for the basic tests led to the development of the abilities to be measured by the “Practical Reading” and “Mathematical Thinking” tests. | |
| Keywords: “Practical Reading” test, “Mathematical Thinking” test, validity, admission on recommendation, Admissions office entrance examinations | |
| ▶ Case study | |
| High school opinion on the articulation reform policy and its change: From the view point of The Principle of Prospective Student Protection | |
| Naoki T. Kuramoto, Tomohiro Miyamoto, Hiroyuki Nagahama | |
| Tohoku University | |
| In the university entrance examination of the 2021 academic year, the system will be changed on a larger scale than ever before. Correspondingly, Tohoku University released “notices” twice in 2018. Results of a high school survey conducted the previous academic year became a valuable basis for making decisions regarding changes to the university admissions. Therefore, to create a reference to use for making decisions for the 2022 academic year exam, we surveyed opinions on the basic policy of Tohoku University. This paper mainly reports a summary of the results of that survey, focusing especially on changes in opinion between the two surveys regarding the English tests provided by private testing companies, the introduction of which was postponed by the Ministry of Education. As a result, high schools expressed strong support for the basic policy of Tohoku University. The result drew strong attention to the power and responsibility of individual universities related to university entrance examinations. | |
| Keywords: articulation reform policy, prospect student protection, high school, English tests provided by private testing companies, basic policy of Tohoku University | |