Contents
- Index
Tools>Consensus Analysis
PURPOSE Performs a concensus anlysis on a number of different types of respondent data matrices.
DESCRIPTION Consensus analysis examines a respondent matrix and determines the most likely "correct answer" amongst the respondents and simultaneously assesses the competence of each of the respondents. Allowable data formats include assessing each other, answering multiple choice questions, constructing proximity matrices and constructing non-square data matrices. The routine incorporates techniques for True/False, multiple choice and interval type data.
PARAMETERS
Input datasets
Name of file containing respondent data. Data valued multilevel matrix
Output competencies (Default=competence)
Name of file containing competence scores of each repsondent.
Output agreement matrix (Default = agreement)
Name of file giving amount of agreement between each pair of respondents.
Output answer key (Default = answerkey
Name of file containing answer key
Type of Data
Radio buttons which can be selected as follows:
Resp-by-resp similarity matrix: each respondent gives a similarity score for each other repsondent.
Profiles: A row of data for each resondent: a matrix in which row i col j gives respondent i's answer to question j.
Proximities: Square symmetric matrix for each resp. A multilevel data set in which the j,k entries at level i give repsondent i's similarity score in comparing j to k. Note that this must be the same as the k,j entry in matrix i.
Networks: Square non-symmetric matrix for each resp. A multilevel data set in which the j,k entries at level i give repsondent i's score in comparing j to k.
Matrices: A rectangular matrix for each respondent. A multilevel data set in which the j,k entries at level i give repsondent i's score in comparing j to k where j and k are from different modes.
Analytic Model
Radio buttons which can be selected as follows:
Covarience Model. When respondents are only answering true or false.
Multiple choice. Classic consensus model in which respondents are making a choice from a specified set of alternatives.
Interval (including ordinal) Respondents are giving a value as an answer.
Prop. True (Default=0.5) For covarience model only the proportion of 1s in the answer key. This is usually not known and the default of 0.5 gives the most conservative estimate.
Details for covarience and mutiple choice additional analysis is provided as specified in the log file when checked.
LOG FILE The agreement matrix adjusted for chance agreements.
The number of negative competencies ie respondents who are worse than chance, the two largest eigenvalues of the agreement matrix and the ratio of the largest to the second largest. If this is larger than 3 then this indicates the data fits the consensus model.
The competence scores of each respondent.
The answer key which gives the consensus amongst the respondents.
For the covarience and the multiple choice model ticking the details box means the output log gives a detailed frequency breakdown of each item.
TIMING O(N^3)
COMMENTS The method makes some important assumptions about the data namely that there is a common truth, that the answers given are independent of each other and that the questions are drawn randomly from a universe of possible questions on a given topic.
REFERENCES Batchelder, W.H., E. Kumbasar and J.P. Boyd. 1997 Concensus analysis of three way social network data. Journal of Mathematical Sociology 22 29-58.
Borgatti, S. P. and D. Haglin 2011 Concensus Analysis in Kronefeld et al (eds) Blackwells Companion to Cognitive Anthropology.