How a Rosh Review Qbank is Made: Our Question Writing and Editing Process

You spend countless hours working through your Qbank to prep for your next big exam. As you read through questions, perhaps you’ve wondered how they were created. Who writes the content? How do they come up with the topics and scenarios? How do we ensure that information is up to date with the most current information? Creating informative, accurate information for these Qbanks is quite a process behind the scenes! These are the main steps involved.

---

How do we find question authors, and what are their qualifications?

Prospective authors are recruited mainly via our website or hiring boards when positions are available. Qualifications for prospective authors differ slightly based on the specialty and specific Qbank, but authors must have an appropriate level of training to write questions for the particular Qbank. Prospective authors submit a sample question that is reviewed by our medical editors (called “champions”) to assess their writing, ability to accurately convey medical information, and help determine whether they are a good fit for our author teams.

Having a diversity of author backgrounds on our teams means we have expertise and real-world experience while, at the same time, incorporating perspectives from clinicians with different but relevant levels of training.

For example, for creation of the IM ITE Qbank for residents, most authors have either completed residency, are in specialty fellowships, or are attending physicians in various IM fields. A small number of authors are current IM residents. All Qs are edited by a board-certified IM physician.

For USMLE, some of the content for Step 2 was written by fourth-year medical students who had already performed exceptionally well on Step 2. All USMLE content is reviewed and edited by a board-certified physician. For highly specialized projects, such as the Critical Care Qbank, the author team includes clinicians with expertise in the particular field.

---

How do we brainstorm question content and make sure coverage of content is comprehensive and includes a broad range of relevant topics?

We aim to create content that’s comparable with an exam’s content blueprint. To do this, we identify important topics and the percentage breakdown based on the exam’s blueprint. For example, for the ABIM initial certification exam Qbank, we want to mimic the rough amount of content from each category that the exam uses.

Using the exam blueprint topics, champions develop learning objectives about these topics based on high-yield and evidence-based teaching points broken down by tasks. Tasks may differ according to the specialty or exam but may include diagnosis, treatment, professional practice, and scientific concepts.

We pay attention to when exam blueprints are updated by their governing body (e.g., ABEM, ABP, ABPN, NCCPA) and regularly update our topics based on the most updated version of the blueprint. Here are some examples of a topic we might come up with based on an exam blueprint:

• Recognize that a new-onset headache over the past 2 weeks that is triggered by exertion or position change is an indication for imaging
• Recognize and treat dengue infection with supportive care and know that antiviral agents are not typically indicated

In addition to teaching content covered on exams, we also identify topics that are important to clinical practice and would be relevant and important to a practicing clinician in that field. By including questions on these topics, we’re doing our part to educate learners on important topics that make them good clinicians with a well-rounded knowledge base (not just high scorers on exams).

For example, in FM, all the topics covered by the most recent AAFP journal articles within the last 5–10 years are reviewed to ensure that these topics are covered in our FM Qbanks. We also produce a free resource each year called Quiz for a Cause that tackles important current health crises.

---

What is the editing process?

Once questions are written, they undergo a rigorous editing process. Champions or editors from each specialty carefully read through each question to be sure the question addresses the learning objective and is well crafted with challenging distractor answer choices. The correct and incorrect answer explanations are reviewed to make sure the information they contain is accurate, up to date, and full of high-yield teaching points. 

Each question also undergoes careful copyediting. See below for a breakdown of the copyediting process:

Our copyeditors are trained to pay attention to grammar, spelling, sentence flow, and inclusive language (among other things) to make sure the information is consistent, easy to understand, and respectful toward the hypothetical patient. For example, you shouldn’t see alternate spellings, like fundoscopic and funduscopic, occurring in the same explanation. 

Before the copyeditors fully read through the content, it’s run through a plagiarism checker. Why? Because the wording shouldn’t be exactly what you’d read from your textbooks or journal articles; rather, it should be in the author’s own words.

Next, the content is analyzed to help the team find wording to change based on our preferred spelling and usage. The copyeditors have developed this preferred wording list and a style guide over the years to track all of our wording, grammar, and format, which keeps content as consistent as possible, regardless of who edits it. 

Once all plagiarism, preferred wording, and format checks are complete, the copyeditor reads through the entire set of questions in a document and makes further changes to help with sentence flow, grammar, and ease of reading. This includes paying attention to inclusive language and updating wording as language continually evolves. We’re passionate about inclusive language because every person deserves to feel respected, whether they’re in a clinician’s office or reading about a hypothetical patient in a Qbank. The more we think about how to respectfully discuss people, the more it carries over in our day-to-day activities and interactions.

---

How do we choose and create our images?

During the medical editing process, after carefully reading the question and associated explanations, the champion includes an image from our large illustration library or requests a new image that they feel will best represent and solidify the question’s teaching point. 

For brand new images (or changes to an image that already exists, based on current guidelines or other information), the champion creates a mock-up based on a relevant source. The illustration team evaluates these mock-ups to figure out how to best present the information. They use a style guide to keep consistent with color palettes, labels, shapes, and templates and create the images with standard image-creation software. The illustrations are then reviewed by the champion and copyeditors and uploaded into the Qbank.

Some incorrect answer choice explanations also include links to images so learners get the additional benefit of seeing related information on a distractor answer choice. For example, if a question tests whether a learner can recognize clinical manifestations of systemic lupus erythematosus (SLE), the main image will link to information regarding diagnosis of SLE. If another condition (like rheumatoid arthritis) is included as an incorrect answer choice, there is often a link from the incorrect answer choice to an image containing more information regarding rheumatoid arthritis.

---

How often are questions updated?

When guideline updates that result in a clinical practice change are released by different societies, our images, questions, and Rapid Reviews related to the topic are reviewed and updated to reflect the most current guidelines. We update on a rolling basis, not just once per year or once every 6 months.

Qbanks are also regularly reviewed and updated to provide our learners with current standard of care information on each topic. Some updates are identified through our Qbank by our learners, as each question offers a “submit feedback” option that is reviewed by the specialty clinician (see below):

Each specialty has one or more content support agents (clinicians with expertise in the specialty) who are available to answer questions from learners regarding content questions or clarifications. The content support agents work with the champions to update questions based on learners’ feedback, if needed, after careful review of the content and literature. 

---

Does this process vary depending on the specialty within GME, medical school, or PA?

The process is similar overall. Topics will vary based on each exam’s blueprint.

Numbers of questions will depend on the Qbank, and this number is determined by various factors, such as feedback from learners and program directors regarding the optimal Qbank size. Our Qbanks aim to contain a generous amount of content so learners have enough study material to fully grasp the subject.

We continually strive to incorporate improvements to bring the best to our learners. We are always exploring ways to innovate and integrate new information and technologies into our processes in order to assist (but never replace) medical professionals in item development.

---

Now that you’ve gotten a peek behind the curtain of how we create our content, we hope you can appreciate all the work that goes into your Qbanks! Hopefully they help you pass your upcoming exam with confidence.

---

Rosh Review is a board review company providing Qbanks that boost your confidence for your boards and beyond. 

Get started with a Rosh Review free trial to the Qbank of your choice (no credit card required!) and gain access to board-style practice questions, detailed explanations, beautiful medical images, and more.