You are on page 1of 16

Coding Tutorial Online at: https://v-dem.

net/coding%20instructions Please read before you begin coding Thank you for agreeing to serve as a country expert for the Varieties of Democracy project! The information you provide will be of great benefit to social scientists, development agencies, and other users around the world. Confidentiality Your identity will remain confidential to the general public unless you choose otherwise. The first question in your survey will allow you to indicate your preferred level of confidentiality. You may choose not to be publicly identified as a coder for this project, to be identified without revealing which scores are yours, or to be identified as the source of all the scores you submit. Your choice will be respected. Below are you will find brief tutorials on all aspects of serving as a country expert for this project. We urge you to read them all before beginning and to refer back to them if you encounter questions as you work. Additional information -- FAQs, definitions of concepts, and the complete text of all the survey questions -- is available at the Reference tab in the website. Contents 1. Logging in and finding your survey questions (p. 1) 2. Navigating through the survey (p. 3) 3. Entering, submitting, and revising your responses (p. 4) 4. Basic coding rules (p. 14) 5. Ending and resuming your work sessions (p. 16)

1. Logging in and finding your survey questions How to log in as a country expert Only those we invite are allowed to respond to our surveys. Before you can begin your work, we must assign you a user name and a password. As soon as we do that, you will receive an email message with your username and password. You must use the assigned username to log in (although you can change it afterwards). If you forget your password, the site will give you the option of having your password sent to the email address we have for you. If you encounter any problems, send a message to the site administrator at contact@v-dem.net. Log in using the "Log in" tab at the top-right corner of the homepage.

VDem Coding Tutorial p. 1

How to Begin If you have read all the tutorials, after logging in, click on the button marked "Begin or Resume Coding." This will take you to a menu of links to all the surveys that have been assigned to you. If you want to answer a part of the survey that does not appear here, please send your request to your designated contact at v-dem.net.

You may start with either the first question or the first question that you have not completely answered. Please be sure to answer the confidentiality question at the beginning and the Post-Survey Questionnaire at the end.

VDem Coding Tutorial p. 2

2. Navigating through the survey Because our complete questionnaire is very long more than 150 questions, usually covering more than 100 years we do not ask many country experts to answer all the questions. Therefore, we have divided the questions into more specialized "surveys." You have been assigned one or more surveys based on your expertise and the amount of work you are willing to do. Only the surveys that have been assigned to you will appear in your menu of surveys. If you would like to complete additional surveys, please tell your designated contact at vdem.net. When you begin a survey, the first screen will contain some instructions specific to that survey. When you are ready, click Next to go to the first question. Only a few questions at a time, most often one question, appear on your screen. If you want to see all the questions on the same page, go to the "Codebook" pages in the Reference menu. You cannot enter any responses there, but this is where you can see in advance how many questions there are in the survey and the range of topics each survey covers. All of the Reference materials--FAQs, text of questions, and the glossary--open in a new tab so that you can consult them while you are answering questions without navigating away from the survey. At the top of the screen you will see the title of each survey, and the country name in parentheses (in the example below, Elections, Sweden). Each question is identified by a unique title (in the example below: Disclosure of campaign donations) that will always be displayed in parentheses before the question itself. This title will help you identify different questions in a survey and move back and forth between them.

VDem Coding Tutorial p. 3

At the lower-left corner of each question window there is a small navigation tool:

This tool gives you two ways to move from question to question. If you wish to move in the sequence of questions, you may use the Previous or Next buttons. If you wish to skip directly to a previous or later question, you may use the "Jump To" drop-down menu. You will be instructed to skip questions when you select certain answers to some questions. Clicking on the arrow in the "Jump to Question" menu displays a list of question titles:

You can then select the question to which you wish to continue to. The titles makes it easy to navigate among the questions and you know that you have jumped to the correct question as the title always appears in parentheses at the beginning of the question. If you need further help to navigate among the titles and questions, you may consult the Codebook under the Reference menu. 3. Entering, submitting, and revising responses For each question you must enter both a response (which may be a code, a range, a date, a text answer, or a numeric answer) and a confidence rating. If you cannot see the whole page, shrink your browser view (ctrl-minus) until you can. The coding interface looks like this when you begin a question:

VDem Coding Tutorial p. 4

The question to be answered is displayed at the top left corner of the page, in bold. Right below the question there may be a hint that provides either a definition or clarification about the question or navigation instructions. 1. Immediately below the question you will see a list of answers to the question--some are multiple-choice questions as in this example, some are yes/no questions, others text boxes, some a date question with a pop-up calendar, and still others provide a slider to select a number. Under the answer options you will see a slider that you use to select your confidence rating for your answers. (The slide is omitted for a few questions that do not require a confidence rating.) To the right of the answers is a grid of years from 1900 to 2012. This grid simplifies the task of coding more than a century of history. Usually the best answer to our questions will not change every year, so the same score could be assigned to a range of years. Below the year grid you will find a box to enter specific dates (if the question calls for this, or if you wish to code specific dates). More about confidence ratings and specific dates later.

VDem Coding Tutorial p. 5

To the right is a grid showing the years to be coded. The blue years in the first column identify decades; the blue numbers in the top row of the grid identify years within decades. For example, the year 1913 is found in row "1910" and column "03." There are two situations in which your initial grid may look different from this. First, some of your years may be grayed out. This means that we want you to code only the years that have not been grayed out. This happens when we plan to treat your responses as applying to, for example, only election years that we have identified for you in advance. Second, some data have been precoded into the grid for you. In such situations, the precoded values (or an icon signifying coding) show up instead of years. We have done this only when the data are factual and available from secondary sources, such as election dates and names of legislatures and important officials. Precoding prevents you from having to look up this information yourself and ensures that experts are coding the same events, institutions, and officials. We ask you not to change these precoded values even though you will have to submit them as though they were your responses in order to proceed. If you feel strongly that the precoded values are incorrect, please contact your Regional Manager before your change any of these values. We want to ensure that all experts on that country are coding the same correct event, body, or official. 2. To enter your scores, you begin by clicking on the first year of a historical period. The selected year (1944 in this example) is outlined in red and highlighted in soft blue:

VDem Coding Tutorial p. 6

If you are coding a range of years simultaneously, use your mouse to drag the cursor to the last year of the range in which the score remained the same (the last year will be highlighted in red). The entire range of years between the first selected year and the last selected year is then highlighted in red and blue. In the example below, 1944 through 1999 have been selected:

3. If you made a mistake or have changed your mind and want to select a different range of years before you have assigned any ratings, just click and drag to select the years you want; or ctrl-click to unselect any years within a selected range. You may also ctrl-click to select multiple non-sequential years. A list of the years you have selected appears in the "Specific Dates" box underneath. When you are satisfied with the range of years you have selected, look at the answer options to your left and click the "Apply" button beside the best answer to choose a score from the response options. If it is not a multiple-choice question, use the range slider, pop-up calendar, or text box to enter your response. The response you choose will be assigned to all the years in the selected range automatically. The years in the cells are replaced with an orange triangle with an exclamation point until you provide a confidence rating for the years selected.

VDem Coding Tutorial p. 7

If you make a mistake or want to change your answers before you have submitted them, click "Undo" (upper right) and start over for that range of years. 4. Some pages may contain multiple questions. The Elections survey, for example, is designed so that you can answer a series of questions about just one election year on just one page, and then answer the same series of questions about the next election year on the next page. Some other survey pages contain multiple questions to be answered for multiple years. In either case, you still select the year or years to be coded before answering the question or questions. Then you can select answers to one or more questions (as many as have answers that do not change during the selected range of years), rate your confidence, and submit your ratings. For grouped questions, the same confidence score applies to all answers that were coded together. 4. Confidence Rating: While the coded years are still selected, choose a confidence rating for the answers you just entered by using the Confidence slider located below the answer options. As you move the slider, legends will appear to provide guidelines about which level to assign. In the example below we choose to assign a confidence rating of 80

VDem Coding Tutorial p. 8

You must use this slider to record the degree of confidence you have in the scores you just entered. The slider allows you to choose any value between 0 and 100. As you move the slider, legends providing benchmark interpretations will appear below. (More specifics on the meaning of these confidence ratings will be discussed later, in the Basic Coding Rules). Please record your confidence in all the scores you assign. Precoded scores should already have a confidence level that is based on our reading of the secondary source; we ask that you leave this unchanged as well. You will not be allowed to submit your answers until a confidence level is assigned to them. As with the answers to the question, you may code many years at once: the slider bar changes the degree of confidence in all the cells that have been selected. The historical periods for degrees of confidence do not have to coincide with periods you assigned for your answers to the question; just select one range of years and apply one confidence level, then select another range and apply a difference confidence level.

VDem Coding Tutorial p. 9

Once you choose a confidence rating a green check will appear in the cell and the confidence ratings inside the parentheses in the grid will automatically change accordingly for all selected years. Note that in this example, confidence is being rated for only some of the years that had been coded "1" (from 1965 to 1999):

If you would like to check the rating you assigned to a certain year, it will appear when you place your mouse over a date in the Specific Dates box at the bottom of the screen. Here, the respondent assigned a code of 2 to answer the question and a confidence rating of 80% for 1966.

VDem Coding Tutorial p. 10

Again, if you make a mistake or want to change your answers before you have submitted them, click "Undo" (upper right) and start over for that range of years. "Undo" returns the coding grid to a state reflecting only the values that have been submitted to the database. 5. Submit your ratings. Before you move on to coding a different range of years, submit your answers by clicking the "Submit" button, located right below the confidence slider.

VDem Coding Tutorial p. 11

6. Entering specific dates: One change that you may want to make is to change the date of the rating for the first year in the range. The default date is January 1 of that year, which means that the new score is assumed to start on January 1 and continue to the end of the last year in the range you have selected. If you have more precise information about when the score actually changed later in that year, we would be grateful if you would enter that information. You can do this easily by selecting the year in question and clicking on the "Edit" button below the Specific Dates box. A calendar will pop up. Use the left and right arrows at the top to move to the desired month in that year, then click on the exact date, then "Update," to change January 1 to the more specific date. If you know only the month of the change, select the first day of the month, which will tell us that the score changed sometime later in that month. In the example below we are changing the specific date to be March 15th, 1944:

VDem Coding Tutorial p. 12

You may also additional specific dates to a selected year if there was more than one event in the year that would change the score on the question. To do this, click inside the "New Date" text box. Navigate to the desired date on the pop-up calendar, then click the "Add" button. The added specific date will then appear in the New Date field. Assign a score and a confidence level to your answer. You can add as many specific dates in one year as are necessary. Simply click New Date again, and repeat the process outlined above.

If you make a mistake or change your mind, you can delete any of the added specific dates by clicking the "Del" button. You will not be able to delete the default date of January 1, but as explained above, you can change it to the specific date of the first score change for that year. Therefore, if you want to code just one specific date in a year, do it by editing January 1 rather than by adding the specific date. 7. You may either code all ranges and then Submit or submit each range as you code it, but be sure to submit your ratings before you try to move to a different question! When you click Submit, you will receive a confirmation message: "Data submitted successfully." You then repeat this for other ranges of years until all the years are coded. You will not be able to delete your answers after you submit them, but you will be able to change them. If you need to delete any responses, send a message to your designated contact at v-dem.net. We suggest that you submit your scores frequently as you code, especially if your Internet connection is frequently lost.

VDem Coding Tutorial p. 13

8. After you have finished coding, either click "Next" at the upper left of your screen to go to the next question, or select a different question from the "Jump To" drop-down menu:

The next question and its answers will appear in the next screen. Repeat the procedure above, submitting your responses as you go, until all questions are answered. The menu of question tags can be used to jump to any question out of sequence. This could be useful if, for example, you wish to answer a series of questions about one time period before answering the same questions about other time periods. Completing the questionnaire will probably require more than one session. You may interrupt your work at any time come back to it later and resubmit answers if needed. Any work you submitted before ending the session will be saved for your next session. 5. Basic Coding Rules

Interpretation. The questions and responses were written with care. They are our best assurance that scores will be comparable over time and across countries and regions. It is therefore extremely important that you read the questions and coding criteria carefully. If you have any questions, even seemingly trivial ones, about how to interpret the questions, please ask your designated contact at v-dem.net; our team will answer you and provide guidance to all the country experts in the Reference materials. Degrees of confidence should reflect your intuitive confidence in the scores. Use a 0100% scale, interpreted as follows:

0: I have no idea at all. [Any scores accompanied by a confidence level of zero will be treated as missing data.] 20: I know a little about this but there is only a small chance that my response is correct. 40: I think this response is close, but this is very hard to assess and I don't have all the relevant information. 60: I am fairly sure that this response is correct, but experts have significant disagreements
VDem Coding Tutorial p. 14

about this. 80: I feel confident that this response is correct but there is a small chance that I am wrong. 90: I feel very confident that this response is correct and there is very little chance that I am wrong. 100: I am completely certain that this response is correct. These interpretations pop up as you adjust the confidence slider. You will be able to enter intermediate values (in 5-percentage-point increments) as well, although we provide no explicit interpretations for intermediate values. Because it is easier to be confident of an approximate answer than of a precise one, your degree of confidence should vary somewhat depending on the precision of the scaled answers.

For categorical responses, yes/no responses, or responses with 3 to 5 ordered levels, estimate your confidence that your response is better than the other available responses. For example, on a zero-to-three scale, how confident are you that your response of 2 is better than 0, 1, or 3? Or on a yes/no question, how confident are you that the true answer is yes rather than no? For ordinal responses on scales with 6-10 levels, estimate your confidence that your response is within one point of the correct score. For example, if your answer on a zeroto-five scale is 4, how confident are you that the true answer is somewhere in the 3 to 5 range? If your answer on a zero-to-9 scale is 9, how confident are you that the true answer is somewhere in the 8 to 9 range? For percentages and other responses on a 100-point scale, estimate your confidence that your response is within 10 percentage points of the correct response. For example, if your answer on a 100-point scale is 36, how confident are you that the true answer is somewhere between 26 and 46? Dates. By default, any score that is different from the previous year's score is assumed to have changed at the beginning of that year. January 1 is therefore the default date, which we will interpret as indeterminate: "sometime during that year." However, you may specify a more precise date. When you select the beginning date of a range, you will have the option of entering a more specific date. If you are able to specify the day, please do so; if not, specifying just the month would still be useful. For example, if you specify that a new score begins on April 12, 1955 and the next score begins in August 1972, then the old score is cut off on April 11, 1955, the new score applies to the whole period April 12, 1955-July 31, 1972, and whatever score you assign to the next period begins on August 1, 1972. Semi-sovereign territories. You will notice that the coding grid often has space for coding years in which the country was not fully sovereign. We want you to code those years, too - years in which the territory in question may have been under foreign occupation, a colony, a protectorate, incorporated into an adjacent state (as Eritrea was
VDem Coding Tutorial p. 15

into Ethiopia) etc. We understand that this entails significantly more effort and that information is far more scarce; your confidence ratings should reflect this. We hope that this information will significantly extend our knowledge of democratization. Your scoring should be based on institutions and practices in that territory, not those in the colonial power or trustee that may have had jurisdiction over the dependent territory.

Shifting boundaries. The territorial borders of the country you are coding may have shifted over the years. If so, please assign your scores to whatever the territory/population was for the particular dates you are coding. In a few cases (Germany, Yemen, Korea, Czechoslovakia), states have split into two states. In those situations we consider one state the successor state and the other a new, separate state. We will usually want you to code both states, but this coding will be separate, i.e., there will be separate questionnaires for North and South Korea. In other situations separate states have merged, and we will want you to code both former states and the subsequent merged state. In any case, your invitation letter should clarify the situation, and if you have any doubts please consult Notes on territory included in states or email your designated contact at v-dem.net. Multiple coders. We will be assessing reliability by using multiple coders. Because you are only one of several country experts assigning scores, we can report some confidence in scores on which multiple coders agree (even if individually they lacked confidence in them) and less confidence in scores on which coders disagree (even if they each felt quite sure of their scores).

6. Ending and Resuming your work sessions You will probably need more than one session to complete your surveys -- perhaps because you do not have long blocks of time for it, perhaps because you need to be away from your computer or Internet connection to gather more information. This is not a problem for your data entry. All information that you submit during your sessions is automatically saved in the database. Just make sure that you submit your last set of ratings before you end the session! Then log out. When you start a new session, log in as before, click on the Begin or Resume Coding tab, and select the survey on which you wish to work. All the scores you submitted earlier should appear in the grid. (If they do not, please let your designated contact at v-dem.net know! You can use the Jump to Question menu to go directly to the place in the survey where you wish to resume your work. You may either add new scores or revise ones you entered in a previous session. You can modify any information you have entered into a survey as long as we have listed your status as "active." When you tell us you are finished, or when the pilot study ends, we will change your status to "inactive" and you will no longer be able to access your surveys.

VDem Coding Tutorial p. 16