The ability to ask questions is critical to learning. Well-framed questions elicit answers that further understanding and dialogue; however, learning to ask the right questions is a difficult skill to develop.

In this article, you will explore:

• General principles of identifying teaching points within publications.
• Methods for generating significant questions from short documents.
• Designing engaging Captivate questions

General Guidelines

• Each question set should address just one topic, guideline or technique and should not exceed 5-8 questions.
• If a guideline or opinion is short, two documents on the same subject should be combined to develop the required number of questions.
• Where possible support technical or clinical concepts using questions focusing on basics.
• Use the appropriate scoring options for the type of question being designed.
  • Points can be awarded when all the correct answers and none of the distractors are selected. In other words, the learner needs to get all the right answers to be awarded any points. This type of scoring should be used when it is important to evaluate that your learner knows all of the answers and not just some of the answers.
  • Partial points can be awarded for each correct answer and none of the distractors. In other words, each correct answer will add to a total score, however selecting a wrong answer has no impact on that score. You might use this type of scoring when the information is less critical and knowing some of the answers is still useful.
  • Partial points can be awarded for each correct answer and a penalty for selecting a distractor. In other words, each correct answer adds to the total and a distractor will subtract from that total. This is useful when knowing some of the information is valuable but accuracy is important. The education committee has recommended that this scoring type be used with multiple answer questions.Penalty points would be assigned by:
    

1. Checking “Partial Score” in the Quiz panel for the question
2. Next, select an answer
   
3. Click on the Properties panel
4. Then click the [Options] button
5. Enter a positive value for correct answers and negative value for distractors (incorrect answers)
   • Note: partial scoring for drag and drop questions can only be done with advanced action scripts

Providing Question Feedback

• Where appropriate, provide feedback that not only explains the answer, but builds upon that explanation.  For example, defining how to measure an increased abdominal circumference may be very helpful for the clinician.  I am also in favor of remarks that may increase retention e.g.: holding a breath does not give an accurate measurement.
• If narrative for question feedback is pulled directly from a published document it should be properly citied in the body of the feedback.
  • If the narrative provided in the question feedback contains reference numbers, carry the numbers forward in the feedback narrative.
  • If the reference document is a pdf or can be converted to a pdf, then question slides should provide an option for the learner to review feedback in context. Ideally, the learner should be taken directly to the page or paragraph in question. Note: links to bookmarks do not work from a URL, but links to specific pages by number or destination will always work if the learner’s browser uses Adobe Reader as plugin for viewing PDF files or the reader app is loaded on their device.
    • To add an external link:
      1. Select the text on the slide that will be used for the link.
         
      2. Click on the link icon in the Properties panel. The Link To popup window will be displayed.
         
      3. Assure that Web Page is selected in the type field.
      4. Paste the URL with the destination identifier associated with the question in the Location field.
         1. Accessing a PDF by a specific page number by using: http://www.domain.com/file.pdf#page=3
         2. Accessing a PDF by a destination identifier by using: http://www.domain.com/fileName.pdf#nameddest=destinationID
      5. Click on the drop-down arrow to the right of the Location field and select New. This selection will open the guideline PDF in a new browser window.
      6. Click the [OK] button to save the entries.
      7. Repeat steps 1-6 on each of the question slides.

Where possible provide URL for document referenced in the question feedback. This will allow the learner to explore topics in greater detail.

1. When possible, state the stem as a direct question rather than an incomplete statement.
2. Present a definite, explicit and singular question or problem in the stem.
3. Eliminate excessive verbiage or irrelevant information from the stem.
4. Include in the stem any word(s) that might otherwise be repeated in each alternative.
5. Use negatively stated stems sparingly. When used, bold and/or capitalize the negative word.
6. Make all alternatives plausible and attractive to the less knowledgeable learner.
7. Make the alternatives mutually exclusive.
8. Make alternatives approximately equal in length.

• Use at least four alternatives for each item to lower the probability of guessing.
• Randomly distribute the correct response among the alternative positions throughout the question set.
• Avoid irrelevant clues such as grammatical structure, well known verbal associations of simplistic connections between stem and answer.
• When possible, present alternatives in some logical order (e.g., chorological, most to least, alphabetical).
• Use the alternatives “none of the above” and “all of the above” sparingly. When used, such alternatives should occasionally be used as the correct response.
• Avoid “all of the above except”. The question type can be very confusing to the learner.
• List alternatives on separate lines (rather than including the options as part of the stem) so that all options can be clearly distinguished.
• Keep all alternatives in a similar format (i.e., all phrases, all sentences, etc.)
• Try to make distractors for a question approximately the same length as the answer.
• Multiple answer questions should have a maximum of three (3) correct responses. Use a matching, ordering or drag-and-drop question type if a question requires more than three correct selections.

1. Provide directions which clearly state the basis for matching the stimuli and responses
2. Use only items that share the same foundation of information.
3. Avoid grammatical or other clues to the correct response.
4. The column of stimuli should set the question clearly.

• Keep matching items brief, limiting the list of stimuli to 10 or less.
• When possible, reduce the amount of reading time by including only short phrases or single words in the response list.
• Use more involved expressions in the stem and keep the responses short and simple.
• Arrange the list of responses in some systematic order if possible (chronological, alphabetical, etc.)
• Make sure that there are never multiple correct responses for one stem (although a response may be used as the correct answer for more than one stem).

1. Provide a clear set of instructions.
   • Can each item in one column be matched with only one item from the other column? Can an item be matched with multiple items? Will all items by matched, or will some items go unmatched?
   •  Is there an order of operation to the items in the drag-and-drop activity?
2. Question stem should include all information needed to answer the question.
3. Question stems should only address one topic or concept.
4. Answers and distractor should be clearly right or wrong.
5. All alternatives should be related.
6. Use only items that share the same foundation of information.

• Keep matching items brief, limiting the list of stimuli to six or less.
• Use more involved expressions in the stem and keep the responses short and simple.
• Arrange the list of responses in some systematic order if possible (chronological, alphabetical, etc.)
• Try to make distractors for a question approximately the same length as the answer.

Where possible, more responses should be provided than premises, so the remaining responses don’t work as hints to the correct answer.

Good questions can identify areas of deficiency and areas in need of further learning.
Each item should

• focus on one important concept
• clearly map to a learning objective for a course or guideline
• assess higher order skills: comprehension, application, synthesis

You know it’s a good question if:

In general, a learner should be able to derive the correct answer by reading the stem and lead-in without seeing the option set.