CASPIAN JOURNAL

MANAGEMENT AND HIGH TECHNOLOGIES

Combinatorial methods for IDENTIFYING the coherence of experts' JUDGMENTS with a numerical grading scale

Read

Golik Felix V. Combinatorial methods for IDENTIFYING the coherence of experts' JUDGMENTS with a numerical grading scale // Caspian journal : management and high technologies. — 2017. — №4. — pp. 34-50.

Golik Felix V. - Doct. Sci. (Engineering), Professor, Novgorod Branch of the Russian Presidential Academy of National Economy and Public Administration, 31 Germana Street, Veliky Novgorod, Novgorodskaya Oblast, 173003, Russian Federation, felix.golik@mail.ru

The article aims at elaborating procedures for identifying coherence of experts’ judgments (CEJ) in the form of expert estimates obtained from group assessment with numerical scales, as well as defining probabilistic characteristics of the proposed procedures and results of their assessment. The object of the study is experts' estimates obtained from group assessment with the use of a numerical scale. The subject is CEJ assessment. An average value, median and variation coefficient of experts' estimates were used as measures of CEJ estimates with the use of a numerical scale. However, it is not possible to verify the adequacy of these indicators correctly since the law of probability distribution of estimates is unknown as there is a link between the degree of CEJ and the distribution law type. With the complete CEJ the distribution is degenerate, and in its absence the distribution is flat. In this regard, it is proposed to verify the fact of coherence presence instead of its quantitative assessment. The procedure is reduced to testing the hypothesis about the law of distribution of experts' estimates. If the distribution differs insignificantly from the flat one, then a decision of CEJ lack is made. Otherwise, it is considered that the experts' opinions are coordinated. In order to solve the problem, the author used the system of combinatorial placement theory, probability theory and mathematical statistics. Estimates of the probability characteristics of CEJ identification procedures are obtained through computer simulation. The research results include the elaboration of constructive methods of CEJ identification with the use of a numerical scale for assessment; procedures of identifying CEJ in relation to specific objects and group assessment in general; estimates of the probability characteristics of assessment procedures through computer simulation. The results can be applied in management decision-making, forecasting and long-term planning in different fields.

Key words: экспертная оценка, шкала оценивания, согласованность экспертных оценок, критерий «пустых ящиков», статистическая гипотеза, уровень значимости, доверительная вероятность, кривые Пирсона, расстояние полной вариации, компьютерное моделирование, принятие реше