Once a Juried Assessment is scheduled and the assignment linking end date is met, assessors and artifacts can be configured by selecting the Assessors and Artifacts page from the left-hand menu of the Assessment Homepage. The Assessors and Artifacts page displays each key assignment for a program; key assignments include artifacts imported from Learning Management System (LMS) assignments.
The view of this page will vary based on the assessment status and the outcome grouping defined during the scheduling process:
-
In Progress: The Assessors and Artifacts Manager displays what is available for assessment. In this status, assessors and artifact groupings can be configured.
-
Pending Grades: The Assessors and Artifacts Manager displays the scoring progress. While in this status, assessors can review and score artifacts.
During scheduling of a Juried Assessment, the artifact grouping configuration affects how the Assessors and Artifacts manager is displayed.
|
If |
Then |
|---|---|
|
Artifacts are grouped by course and key assignment… |
The Course(s) column displays the associated course name. |
|
Artifacts are grouped by outcome and key assignment… |
The Course(s) column displays the number of courses associated with the outcome. |
Downstream & Cross-Platform Impacts
-
Artifact Availability: The artifacts displayed in the Assessors and Artifacts manager depend on key assignment linking, LMS imports, and any externally uploaded artifacts. Changes to linking or uploaded artifacts can affect the artifact pool available for review and scoring.
-
Assessment Status Behavior: Available actions change as the Juried Assessment moves from In Progress to Pending Grades. Configuration actions are available while the assessment is in progress, while scoring progress and assessor review options become the focus once grades are pending.
-
Grouping Visibility: The grouping method selected during scheduling affects how courses and outcomes display in the manager. Course-based groupings show the associated course name, while outcome-based groupings show the number of courses associated with the outcome.
-
Scoring Readiness: Assessor configuration, artifact sample size, and artifact availability determine whether assessors have the correct artifacts available to score. Incomplete configuration can limit review options even when LMS artifacts are available.
-
Review and Replacement Workflow: Configuring assessors and artifacts enables additional review functionality, including discarding and replacing unscorable artifacts. This affects which artifacts remain in the sample that assessors review and score.
-
Progress Transparency: In Pending Grades, the manager displays scoring progress, total assessors, and the number of discarded artifacts. These indicators help administrators monitor scoring completion and identify where follow-up may be needed.
Considerations
-
Artifact Source: Review whether artifacts are coming from LMS assignment linking, external upload, or both. The artifact pool reflects the artifacts pulled in or added through these sources.
-
Grouping Method: Confirm whether artifacts are grouped by course and key assignment or by outcome and key assignment. The selected grouping method affects what administrators see and which configuration options apply.
-
Sample Size: Review the sample size of artifacts selected for scoring before assessors begin scoring. The sample should align with the Institution’s assessment plan and scoring expectations.
-
Assessor Configuration: Confirm that the appropriate assessors are assigned before scoring begins. The total assessor count and available review actions depend on the configuration completed for each outcome or course.
Best Practices
-
Confirm the assignment linking end date has passed before configuring assessors and artifacts.
-
Configure assessors and artifact scoring before the assessment moves into active scoring whenever possible.
-
Review artifacts to confirm they are appropriate for scoring, and discard or replace unscorable artifacts when configuration allows. Validate the artifact sample size and assessor count before assessors begin scoring.
-
Monitor Pending Grades regularly to track scoring progress, assessor activity, and discarded artifact counts.
In Progress
💡 Product Tip: If assessors and artifact settings have not been configured, artifacts pulled from the LMS will still be available for review. Configuring assessors and artifacts enables additional functionality to discard and replace unscorable artifacts.
The Artifact Pool displays the number of artifacts imported from the LMS through assignment linking or added through external upload (1). The Artifacts to Score displays the number of artifacts selected for scoring, while Assessors displays the number of assessors configured to review and score the artifacts (2). AI Evaluation Progress displays how far AI evaluation has progressed for artifacts (3). Access options to manage the artifact pool and scoring workflow by expanding the Actions menu for an outcome or course (4) (depending on the grouping method). Settings can be selected to configure assessors and artifact scoring, while Review Artifacts can be selected to access and review collected artifacts. Additional artifacts beyond what was pulled in automatically via assignment linking can be added by selecting Add Artifacts, and Import Details is used to gain visibility into how artifacts got into the artifact pool and their import status.
Pending Scores
The Pending Scores view displays scoring progress after assessors and artifacts have been configured and scoring is underway. Completion Progress displays how many artifacts have been scored out of the total expected to be scored (1). The Artifacts to Score and Discarded Artifacts columns display how many artifacts are expected to be scored vs. how many have been removed from the artifact pool (2). AI Evaluation Progress displays how far AI evaluation has progressed for artifacts (3). Access options to manage the artifact pool and scoring workflow by expanding the Actions menu for an outcome or course (4) (depending on the grouping method). Assessors can be selected to review assessor configuration and scoring progress, while Review Artifacts can be selected to view the artifacts currently included in the scoring sample. The Reliability option allows administrators to review scoring consistency across assessors and identify where additional calibration may be needed.