Assessment Settings
While in Pending status, associated terms and assessment dates can be edited. Once a Juried Assessment is in the In Progress or Pending Scores status, only the Scoring Due Date can be edited; this is when the assessment closes to assessors, and artifacts can no longer be scored. The selected term(s) (1) drive which submissions are in scope for the assessment. The following date fields (2) drive when assignment linking must be completed, and the window in which assessor scoring can occur.
-
Assignment Linking End Date: The deadline to finish linking LMS assignments/artifacts to the outcomes/rubrics used for the Juried Assessment. After this point, the artifact set is expected to be “ready,” so the assessment can proceed.
-
Scoring Start Date: This is when assessors can begin scoring; it effectively opens the scoring window.
-
Scoring Due Date: The deadline for scoring to be completed; this is the end of the scoring window.
AI Evaluation Settings
Access to AI functionality is controlled through Institution enablement and role-based permissions. To learn more and enable AI functionality, please contact Support.
These settings control whether AI assistance is available during Juried Assessment scoring, and how that assistance is presented to assessors. These settings are important because they let Institutions balance efficiency and consistency with governance and reviewer independence by enabling AI support where it adds value, while keeping final scoring decisions firmly in human hands.
These settings can help assessors move faster through high-volume scoring, provide more consistent support for rubric interpretation, and reduce the time spent hunting for evidence in lengthy artifacts. When enabled, assessors will see AI-generated analysis and/or AI-suggested scores as part of the scoring experience.
Enable AI Analysis
When enabled, AI-generated qualitative help for scoring will be available. This is a criterion-by-criterion analysis of the artifact against the rubric levels. When disabled, assessors score artifacts without AI-driven analysis. Once enabled, assessors can use AI analysis during scoring and toggle whether the AI-generated write-up appears in the panel (1); by default, this is unchecked. The AI provides a narrative summary of the submission (2). Bullet points hyperlink to specific evidence in the artifact, helping the assessor quickly find support for a criterion-level interpretation. A Copy to clipboard action enables the assessor to reuse the analysis text for notes, feedback, or discussion (depending on workflow norms).
Enable AI Suggested Scores
Product Tip
This setting can work in conjunction with the Display All Scores At Once setting.
When this setting is enabled, assessors will receive AI-generated score recommendations for each rubric criterion during scoring. Assessors still choose the score, but the AI will display a recommended level plus rationale to speed up scoring and support more consistent interpretation, especially across multiple assessors.
Once enabled, assessors can use AI score recommendations alongside rubric options and toggle whether the AI recommendation is displayed during scoring (1); by default, this is unchecked. Associated rubric levels are listed along with the AI recommendation, indicating the AI’s recommended level for the criterion (2). The recommendation includes calibration reasoning as a brief explanation for why the AI suggested that level.
Display All Scores At Once
When Enabled AI Suggested Scores is enabled, this setting controls when AI-suggested scores become visible to assessors during scoring. When this is enabled, assessors will see AI score suggestions by default, rather than seeing them one by one using the Reveal AI Scores option in the scoring drawer. This can speed up scoring and reduce extra clicks (especially in high‑volume evaluations). Still, it also means assessors may see AI recommendations earlier in the process, which can influence how they think about a criterion before recording their own score.
Interrater Reliability Settings
Inter‑Rater Reliability (IRR) in Juried Assessment helps institutions make multi‑assessor scoring more consistent, fair, and defensible. IRR refers to how consistently different assessors score the same artifact using the same rubric. When enabled, IRR can be configured from the Assessors & Artifacts page. When enabled, the Juried Assessment process is enhanced with additional functionality to support IRR:
-
Detection of discordant submissions (when assessors disagree beyond the agreed policy).
-
Adjudicator review phase/status (between Pending Scores and Completed status).
-
Optional manual adjudication configuration.
-
Reliability and operations monitoring/reporting.
Notification Settings
Invitation email notification and reminder templates are available and can be edited by clicking the Edit Notification button. Reminders will be sent until the Scoring Due Date is met or an assessor has completed scoring all artifacts. When editing notification templates, text within curly brackets ({{example}}) indicates dynamic text and allows custom information to be pulled from fields within the platform. Learn more about dynamic text.
