. . . . . . . . . .

American Board of Internal Medicine

Application for Maintenance of Certification Credit Lifelong Learning and Self-Assessment Products

The American Board of Internal Medicine sets the standards and certifies the knowledge, skills and attitudes of physicians who practice in Internal Medicine and its subspecialties.

ABIM Criteria for Lifelong Learning and Self-Assessment Products for the Maintenance of Certification Program
Lifelong Learning and Self-Assessment is synonymous with the Self-Evaluation of Medical Knowledge or Skills component of the ABIM Maintenance of Certification program. To help expedite the acceptance process for your self-assessment product, please complete and submit this application along with your product questions in accordance with the ABIM criteria for Lifelong Learning and Self-Assessment Product development.

Criteria
1. The product provides a learning and self-assessment activity that is relevant to practice and up to date in the given certification area. a. ABIM will follow a guideline of allotting one MOC point for every three clinical scenario multiple-choice questions (MCQs) or the equivalent. ABIM will not review products that are determined to be fewer than 8 MOC points. Content coverage should be sufficiently broad for the number of MOC points allotted. b. The product can be accepted for MOC credit for a maximum of three years. Content of the selfassessment component should be reviewed annually and updated as needed. Subsequent editions of the product should have an entirely new self-assessment component (i.e., no assessment questions can be repeated for MOC credit). c. ABIM diplomates must be able to access and complete the product without being required to join the product provider organization. 2. The product provides science-based, unbiased content. a. The product must meet ACCME standards for commercial support of CME products; sources, amounts, and uses of any commercial funds or services used to support the product must be disclosed to ABIM during the application process. ABIM has set a goal (timetable to be determined) of disallowing direct or indirect commercial support of products used in MOC. b. No third-party and/or proprietary advertising or promotional material can appear in or be part of the experience of completing the product. 3. The development/maintenance process incorporates adequate expert peer review. a. At least three content experts with valid ABIM certification in the content area or with other relevant expertise should review each question in the self-assessment component during the product development process. b. The following are strongly recommended: i. Pretesting (tested on prospective subscribers to evaluate self-assessment component quality); ii. Adherence to ABIM question construction standards or equivalent best-practice processes (see Appendix B); iii. Use of performance statistics to guide question revision/replacement. 4. The product provides appropriate performance feedback. a. Documentation of how a minimum performance passing standard was set should be provided to ABIM during the application process. 5. The product encourages use of the educational resources available in practice. a. Correct answers and explanations for the answers must be withheld until the subscriber has committed to a response or answered the question correctly. b. Only electronic (web- or CD-based) products will be accepted by ABIM for MOC credit. No paper-based product versions may be substituted for electronic versions.

-2-

6. The product provides objective evidence of successful completion by diplomates. a. The diplomates initial and/or final response to each question (or equivalent objective data verifying successful performance at or above a minimum performance level) must be included in the diplomates data submission to ABIM for MOC credit.

Applicant Information
New/Renewal Application
1. Name of Organization: 2. Is this your first product submission? Yes No

If No, are you submitting this product as a renewal of a previously accepted MOC product? Yes No

3. Please provide the product to ABIM in one of the following ways: a) Attach the product as a PDF file b) If it is web-based, please provide login information for 4 ABIM reviewers: 1. 2. 3. 4. c) Please indicate if your organization needs access to the ABIM secure FTP site to share the content due to large media files. Yes No

Product Description
4. Name of Lifelong Learning Product: a) Version Number: Year:

b) Date module will become available to your members: c) Date module will be discontinued: d) If this is a version change, percentage or number of questions replaced since last version: 5. General description of Product: a) Description of product for display on ABIM website (4,000 character limit):

-3-

b) Web link for use on ABIM website to direct interested physicians to your organization for product information an ordering: c) Describe nature of educational content; indicate specific content areas addressed:

d) Describe Self-Assessment Component (e.g. multiple-choice, short answer, etc):

e) Total number of questions and number of modules: f) Type of module: Web-based CD-ROM Web-based and Paper CD-ROM and Paper No

g) Will the product be updated during the period specified in 3b and 3c above? Yes If Yes, please describe nature, extent, and frequency of updating:

h) Will your organization provide CME credit for the completion of this module? Yes i. If Yes, how many CME credits? ii. Please describe any other pertinent CME credit information.

No

6. Financial Relationships with Commercial Medicine Entities: a) Has this product received financial support from any commercial entity(ies)? Yes If Yes, please specify sources, amounts, and uses of external funds received. No

b) Please be sure to complete and return Appendix A (attached), specifying additional information regarding the funding of this product. c) Please provide financial conflict of interest disclosures for all question authors.

Product Availability/Cost
7. Is the product available to all ABIM diplomates, regardless of membership in your society/organization or area of certification?

-4-

Yes

No

If No, describe limitations:

a) What is the product fee?

Product Development/Maintenance
8. Are the educational and self-assessment components written by physician content experts who are certified by ABIM in the discipline of the module or, if the topic does not correspond to an ABIM certification area, who are certified by ABIM in any discipline? Yes, entirely Both ABIM-certified physicians and other content experts*

* Physicians certified by other ABMS Boards, other types of medical care providers, PhDs, etc.

No ABIM-certified physicians developed these components process:

If No, describe the development

9. Are the educational and self-assessment components peer-reviewed by physician content experts who are certified by ABIM in the discipline of the module or, if the topic does not correspond to an ABIM certification area, who are certified by ABIM in any discipline? Yes, entirely Both ABIM-certified physicians and other content experts*

* Physicians certified by other ABMS Boards, other types of medical care providers, PhDs, etc.

No ABIM-certified physicians peer-reviewed these components process: a) How many content experts review each question?

If No, describe peer review

10. Is the self-assessment component pretested on potential subscribers to evaluate quality through No analysis of performance data? Yes a) If Yes, how many pretesters answer each question? b) Are performance statistics (e.g., item difficulty and discrimination) available for the self-assessment No component? Yes c) If Yes, what performance criteria are used to determine whether self-assessment questions should be retained/revised/replaced?

-5-

If the answers to 10 and 10b are No, please describe how the quality of the self-assessment component is evaluated:

User Performance Feedback

11. Can the learning component be accessed by users as they complete the self-assessment component? Yes No

12. Describe performance feedback to users (e.g. raw or standardized scores, comparisons with other user scores, qualitative feedback, educational feedback):

13. Does the self-assessment component include a performance standard reflecting a minimum No If yes, please answer questions a c below. recommended level of performance? Yes a.) Must the user achieve a passing score to receive CME credit? Yes b.) What is the passing score the user must attain? c.) Please describe how this passing score was derived: No

14. Is data available for the average amount of time it takes a user to complete the self-assessment component? Yes No If Yes, please describe the data:

Impact On Practice Performance

15. Is there data available on the impact of the product regarding a change in practice performance? Yes No If Yes, describe data:

-6-

IT Information
16. Will subscribers be able to submit documentation of successful completion of this product directly to No ABIM via the Web, as described in Appendix C (attached)? Yes a) If No and your organization will used file-based data transmission, how often will you provide subscriber completion files to ABIM? b) Will subscribers/organization submit subscribers first responses, final responses, or both? First Final Both

Contact Information
1. Name of authorized representative of applying organization: a. Title: b. Email: c. Phone: 2. Name of IT contact responsible for sending user completion reports: a. Title: b. Email: c. Phone: 3. Name of communications contact responsible for public announcements about the product:

a. Title: b. Email: c. Phone:

-7-

Appendix A: Funding for Development of Part 2 Modules Funding

1. Source of funding for Self-Assessment of Medical Knowledge module (check all that apply): Commercial support as defined by ACCME (if selected, complete question 2 below) Fee-based (enter participant fee): $ Foundation grant/other non-profit funding Government grant (HRSA, etc.) Institutional support Other, describe:

2. Commercial support (if applicable) a. Use the table to provide the following information for each organization providing commercial support: List of products made by the organization(s) that are relevant to the topics addressed in this module Location in the module where there is mention of the product(s)

Organization Name

Product(s)

Location(s)

b. Describe the process for ensuring compliance with the ACCME Standards for Commercial SupportSM:

c.

Describe the steps you have taken to ensure that this module presents a balanced presentation of diagnostic and therapeutic options and is free from bias:

d. Will any participant information be shared with the organization(s) providing commercial support of this module?* Yes No

If yes, please provide details:

e. Describe how the data collected will be used by your organization and by the organization(s) providing commercial support:

*Please note that if any participant information will be shared with the organization(s) providing commercial support, the details of this data sharing must be disclosed to participants before they begin the activity.

Appendix C

Specifications for providing lifelong learning MOC credit to non-ABIM products Revised 9/20/07

Verification Candidates MUST be enrolled in Maintenance of Certification Program in order to receive credit for these modules. The society module may include a button to "check eligibility". This button should access a page from the ABIM web site (https://www.abim.org/services/modulecheck.aspx). Passed from the society module to this page will be the following information: Module (cc=code assigned, b=book assigned, mmmm=major verison, vv=minor version) * ID (ABIM 6 digit candidate number) * lname, * fname, * dob (mmddyyyy format)
*

If these data elements are not provided, our system will prompt for them. We recommend societies collect these data elements and provide them so the user will not have to re-enter them at the data submission phase. The following checks will be performed: 1. The ABIM web site will then verify the ID, NAME and DOB are consistent with our records. If they do not match within an acceptable degree, an error message will be reported 2. The computer will determine if the person is registered for Maintenance of Certification Program. If not registered, a link to the "enroll in Maintenance of Certification Program" will be provided and an informational message advising the diplomate they must enroll to receive credit. 3. The computer will determine if the module has been already completed within the past 5 years. If the module had already been completed, an error message will be reported. ABIM module codes consist of an area code, book, major revision year, minor revision year. ABIM will provide a module code to the society for each revised release.

The ABIM currently requires that no module be repeated within a 5 year period, even if it is revised. Thus once someone completes the module, they cannot take another revised version of that particular module for 5 years. The ABIM is currently reviewing this interval for appropriateness, particularly related to society generated materials. 4. The computer will determine if the person enrolled in Maintenance of Certification Program via the component payment option. If so, a message describing the fees associated with this module will be displayed. 5. If the person is eligible to take the module, a confirmation message will be displayed.

Data Transfer Completion data may be electronically submitted to the Board. The societies software should access

Appendix C Specifications for providing lifelong learning MOC credit to non-ABIM products Revised 9/20/07

https://www.abim.org/services/moduledone.aspx. The following data elements should be passed to this page Module (cc=code assigned, b=book assigned, mmmm=major verison, vv=minor version) * ID (ABIM 6 digit candidate number) * lname, * fname, * dob (mmddyyyy format) + rspfirst (initial responses, eg. ABBBACBA) + rspfinal (final responses, eg. CBBBACBA) + z1first (initial zero/one for right/wrong 000100100100100100100100 -1st annswer wrong, rest right) + z1final (initial zero/one for right/wrong 100100100100100100100100 - all answers right)
*

If these data elements are not provided, our system will prompt for them. The board requests both initial and final responses. If a society is unable to provide both initial and final responses, they must provide either an initial response or a final response. The page will display a message advising the diplomate that credit has been issued. If credit cannot be issued for some reason (such as the person is not currently enrolled in the maintenance of certification program), an appropriate error message will be produced.

Alternate Data Transfer The Board is currently working with societies to develop a standard file format to be used to transmit data for those who are unable to implement a web based solution. The following data elements are required. They should be transmitted in a coma delimited file format Module (cc=code assigned, b=book assigned, mmmm=major verison, vv=minor version) ID (ABIM 6 digit candidate number) lname, fname, dob (mmddyyyy format) + rspfirst (initial responses, eg. ABBBACBA) + rspfinal (final responses, eg. CBBBACBA) + z1first (initial zero/one for right/wrong 000100100100100100100100 -1st annswer wrong, rest right) + z1final (initial zero/one for right/wrong 100100100100100100100100 - all answers right)
+

The board requests both initial and final responses. If a society is unable to provide both initial and final responses, they must provide either an initial response or a final response.

sample record: ASA200601,188481,Kooker,Donald,08121963,ABBACCEDECB,ABCACCEDCBA,100100000100100100100100100100100,1001001 00100100100100100100100100 Data submitted via file will be checked using the same criteria as the web based submission process. The society will be provided a log indicating any issues with the data submitted for which credit has not been issued. The society should contact their participant and resolve any underlying issues and may then resubmit a request for credit for the problem individual with corrected information.

Result letters/credit

Appendix C Specifications for providing lifelong learning MOC credit to non-ABIM products Revised 9/20/07

A standard SEP result letter will be sent upon submission of the data. This letter includes information regarding progress in the program and/or certificate information should this be the final required component completed.

CME CME for this module will be issued by the society. CME survey/request forms and fees will not be collected by ABIM for these modules.