1. Define the Outcome Determine what on the job attribute you need to measure (e.g. Job Performance). 2.

Define the Psychometric Construct Define and research the trait characteristic that will predict the desired business / on the job outcome. (e.g General Cognitive Ability). 3. Write Items to Test Design draft questions to measure the target psychometric attribute. Writing good test items is difficult and the guidelines used are based on published research. Initially, many questions are drafted. Only 10-30% of the initial items will typically pass the various quantitative and qualitative quality control processes and make it into the final test. 4. Choose the Response Format Response formats are chosen and designed on the basis of the test function, structure and delivery method. 5. Scoring & Reporting Scoring thresholds and reporting are determined. Drafts of both are created. 6. Administration Instructions and the administration process are determined. In some cases new technology to aid delivery is required. 7. Trial Population A group of people who are considered representative of the final population are selected to test the draft assessment. Consideration needs to be given to how you will access and incentivise this group, as well as the logistics of the administration process. 8. Conduct Trial The trial population completes the full set of draft questions in conditions designed to mirror the environment the final assessment will be delivered in (as closely as possible). 9. Question Analysis The results of the trial are analysed to demonstrate the psychometric properties of the questions. That is, how effective the questions are at measuring the desired attribute or trait. Stages 6 and 7 are repeated until a final shortlist of questions is produced. 10. Full Form Validation Once finalised, the resulting shortlist of items is now considered the final full form. The full form is tested on a new trial population to determine how effectively the questions work together to measure the desired outcome. This is done by comparing the results of the assessment to either an existing measure of the same construct or to job metrics such as existing performance data.

11. Full Form Review The effectiveness and order of questions is analysed to ensure that, in combination, they provide the most effective measure of the desired outcome. Stages 8 and 9 are repeated until the final set of questions meet necessary psychometric standards. 12. Normative Data Relevant normative data is collected for score comparison following assessment publication. 13. Publish The assessment is released and ready to use. Supporting technical reports, collateral, example reports and training resources are developed. 14. Ongoing Maintenance and Validation Item exposure and performance is monitored, normative data is reviewed, and the assessment and reporting is modified as required. Ongoing validation studies are conducted to ensure the assessment continues to measure the required trait. So, next time you are evaluating psychometric testing options, ask to review each tests development and validation documentation to ensure the test you choose actually p redicts real life outcomes and isnt just smoke and mirrors'.