Configure Assessors and Artifacts

Once a Juried Assessment is scheduled and the assignment linking end date has passed, the number of assessors required to score each artifact and the distribution of the sample size across assessors can be configured until the scoring start date is met. The platform supports the following user roles as Juried Assessment assessors:

  • College Admin

  • Course Coordinators

  • Course Section Director

  • Department Admins

  • Institution Admins

  • Instructor

  • Program Coordinators

  • Teaching Assistant

Juried Assessments do not use the organizational hierarchy; therefore, the college or department to which assessors are aligned will not affect an assessor's access.

Configuring Assessor and Artifact Scoring

  1. Navigate to the Assessors and Artifacts manager and select Settings from the expanded Actions menu.

    image-20261103-214429.png
  2. On the Artifact step, set how many artifacts will be assessed out of the total available. The sample size can be set as a Percentage of Total or a Number of Artifacts. Click the minus or plus icons next to either field to lower or raise the sample size, and the Artifacts Selected indicator updates automatically.

    1. In the example below, 10 artifacts are available. The sample size is set to 100%, so all 10 artifacts will be assessed.

      image-20261103-220410.png
    2. In the Assessors Per Artifact section, select how many assessors will score each artifact. The summary banner below the options updates to show how many artifact submissions will be scored and the total number of artifact scoring opportunities, which is the sample size multiplied by the number of assessors per artifact.

      1. In the example below, three assessors will score each artifact. Assessors will score 10 of 10 artifact submissions, for a total of 30 artifact scoring opportunities.

        image-20261103-220638.png
    3. On the Assessors step, add the assessors who will score the artifacts and set how the scoring load is split among them. Search for and select the users who will be assessors and choose how to split the scoring load using the distribution options (2) in the upper right. Enter each assessor's share in their Artifact Distribution field. In the example below, three assessors are selected. Two are set to 33.33% and one to 33.34%, for a total distribution of 100%.

      1. By clicking % or # or the clock icon (2), the sample size can be configured as a percentage of the total artifacts available for scoring, as a numerical value based on the total artifacts available for scoring, or based on each assessor's scoring hours.
        💡 Product Tip: At least 3 assessors are required, and the total distribution must equal 100% before you can continue to the next step. Clicking the balance icon (2) will automatically split the artifacts evenly across all selected assessors.

        image-20261104-011235.png
    4. On the Review and Save step, review the selections before saving. The Assessor Workload Distribution chart (1) shows how artifact assignments are spread across assessors. Below the chart, the summary tiles show quick glance information about artifact distribution (2).

      image-20261104-011851.png
    5. The review also includes the user(s) assigned as adjudicators (1), along with assessor and artifact distribution information (2). Each assessor’s share of the total artifact submissions to score appears next to their name (for example, 5 of 30). Click Save Settings to save the configuration and close the settings window.

      image-20261104-012202.png
  3. Once settings are configured and saved, review the randomly selected artifact samples (based on sample size) to ensure all artifacts are valid for assessor scoring.

Replacing An Assessor

Replacing an assessor may be necessary due to scheduling conflicts, changes in assigned responsibilities, or other unforeseen circumstances. Assessors can be replaced while Juried Assessments are in Pending Scores status. When replacing an assessor, scoring from the previous assessor can be either retained or erased. When an assessor is replaced, the platform will automatically regenerate any associated action items and assign them to the new assessor.

  1. To replace an assessor, navigate to the Assessors and Artifacts Manager and select Assessors from the expanded kebab menu.

    image-20261104-170756.png


  2. From the expanded assessor kebab menu, select Replace Assessor.

    image-20261104-170811.png
    1. Search for and select the new assessor, then decide how to handle the previous assessor’s progress (1); clicking the magnifying glass opens advanced search with filter options. Only one user may be selected as the new assessor. After clicking Replace Assessor (2) and confirming the replacement, the assessor will be replaced, and the assessor's progress will be retained or removed, as configured.

      • Retain Previous Work: Selecting this option will transfer all work (artifacts scored, un-scored, and in progress) to the new assessor. Saved work will be intact for the new assessor to either keep or modify.

      • Start Fresh: Selecting this option will transfer all work to the new assessor and erase any previously saved work and comments from the original assessor. This means the new assessor will start scoring from scratch.

        image-20261104-170838.png

Adding External Artifacts

Learning Management System (LMS) assignments that have been linked to outcomes are automatically imported from the LMS during the assignment linking period, when the Juried Assessment is in Pending status. Additionally, external artifacts can be uploaded for an outcome through the Assessors and Artifacts page. When adding external artifacts, the file must be a .zip file, be less than 1 GB per upload, and users can upload multiple files one at a time. Artifacts can be added when a Juried Assessment is in Pending or In Progress statuses.

info Juried assessments utilizing only externally uploaded artifacts will skip the Pending status and proceed directly to the In Progress status.

  1. To add external artifacts for assessors to review, navigate to the Juried Assessment Homepage and select the Assessors and Artifacts page from the left-hand menu. Expand the kebab menu and select Add Artifacts.

    image-20250612-172408.png
    1. Zip files can be added by clicking the Upload File section to open a file explorer or by dragging and dropping the file to the File Attachments section.

      image-20250613-222156.png
  2. Once uploaded, the Upload Artifacts drawer will automatically close, and the page will refresh to display a success message. The user will then receive an email once processing is complete.

    image-20250613-222620.png

Managing Artifacts

Artifacts should be reviewed to ensure they are valid for scoring by the assessor. Artifacts that aren’t valid for scoring can be removed and replaced with a new randomly selected artifact, given that additional artifacts are available in the sample pool. The total count of discarded artifacts will be displayed via the Discard Artifacts column of the Assessors and Artifacts Manager once the Juried Assessment transitions into the status of Pending Scores.

  1. To review artifacts, navigate to the Juried Assessment Homepage and select the Assessors and Artifacts page from the left-hand menu. Expand the kebab menu, then select Review Artifacts.

    image-20250613-223814.png
    1. Artifacts can be reviewed (1), and given that additional artifacts are available in the sample pool, artifacts that aren’t valid for scoring can be discarded and replaced (2) with a new randomly selected artifact. Discarding an artifact submission will add it to a list of non-usable artifact submissions for the assessment.

      image-20250613-225750.png
      1. View options and search functionality can be utilized for each artifact (1); the view functions can be used to open the thumbnail panel for an artifact, zoom, rotate, etc.; additionally, if an artifact contains multiple pages, page navigation can be used (2) to review all artifact contents.

        image-20250616-154515.png
      2. Click the artifact information option to display the assignment name, associated course section, and term information. Artifacts can be downloaded, and the last icon displays the count of pages within the sample.

        image-20250616-155931.png
    2. When artifact review is complete, click Exit to return to the Assessors & Artifacts page.

      image-20250616-160349.png