Job Description
Position Overview
Assessment & Technology Solutions (ATS) is seeking a Psychometrician to support the operational psychometric work for a K-12 large scale assessment program, including a statewide general education summative assessment, interim assessment, and an English language proficiency assessment. Responsibilities include statistical tasks such as cleaning data, managing data sets, and running descriptive and inferential statistics; psychometric tasks such as item and test analysis, calibration, equating and scaling, differential item functioning, and standard setting; using classical and IRT psychometric models to analyze large-scale assessment data; performing quality assurance checks on analysis and reports; collecting reliability and validity evidence; preparing technical documents; presenting to the Technical Advisory Committees and other duties as assigned.
At least one year of full-time equivalent experience working on a large-scale operational assessment is required.
Job Description
55%Use various types of psychometric models to analyze large-scale assessment data. Conducts all aspects of operational work related to scoring and reporting including data cleaning, key validation, calibration, equating, scoring, quality control, and reporting. Produces item and test analyses and summary data tables for use in technical documentation. Performs other psychometric tasks including differential item function and data analysis for standard setting.
30%Prepare technical documents, reports, and manuals. Work under direction of senior staff to document technical qualities of various aspects of the testing program, including the annual technical manual and stand-alone study reports. The tasks include creating tables, charts and graphs to communicate technical information and writing narratives summarizing the findings.
5%Present to various audiences, including state education agency partners, technical advisory committees, and at professional conferences.
5%Propose detailed research designs and psychometric analyses for new projects. Conduct research under the supervision of senior staff.
5%Other duties to support the research and psychometrics team
Position Requirements
This position will require attendance at regularly scheduled meetings, onsite or through telecommunication platforms.
This position allows for a telework or hybrid work location for well-qualified candidates. A telework position may be required to travel up to four times per year.
Required Qualifications
Doctorate degree in educational measurement, psychometrics, or other closely related field.
Two years of experience with quantitative analysis and coding in R.
Two years of experience applying classical and IRT psychometric models such as item and test analysis, calibration, equating, scoring, and scaling.
One year of experience managing multiple priorities and deadlines, including adapting in response to changes in work priorities and processes, as evidenced in application.
Detail oriented with strong organizational skills.
At least 1 year of full-time equivalent experience working on a large-scale operational assessment program (e.g., state large-scale or district-level assessments).
Preferred Qualifications
Experience with quantitative analysis in multiple IRT and other statistical software (e.g., flexMIRT, ConQuest, R, SAS).
Extensive experience writing scripts in R as evidenced in application materials and interview (e.g., building packages, vignettes, etc).
Provide evidence such as sample technical documents to show experience in technical writing including results of quantitative analyses.
Effective oral and written communication skills for technical audience
