Reporting

The Reporting page provides a rollup view of Juried Assessment scoring results across the evidence hierarchy, helping interpret performance and reliability without pulling raw scoring exports. The controls are meant to help change the display (how detailed it is, what grouping, hierarchy level, etc.), while the table columns summarize how much evidence is included in each rollup and what that evidence suggests. By default, the view is configured to display the outcome level with all rows collapsed.

The Evaluation Hierarchy column names what each row is summarizing and provides the expand/collapse control for moving between broader outcomes and more detailed evidence levels (1). This benefits users by giving them a clear path to start with a high-level rollup and then drill into the specific areas that explain the results. Unique Artifacts shows how many distinct pieces of student evidence are included in the row, helping users judge whether the summary is based on a meaningful sample size (2). IRR Average Score provides a quick snapshot of overall performance for that row (3), where Evaluation Count shows the number of assessments included, which may be higher than the artifact count when multiple assessors score the same artifact (4); this gives important context for how much scoring activity contributed to the summary. The Kappa value is a reliability indicator that helps to understand how consistently assessors agreed (5). Higher values generally indicate stronger agreement, while lower values may signal disagreement, ambiguous rubric guidance, calibration needs, or other scoring complexities that warrant closer review.

image-20261104-175117.png

Changing The Display

The Depth selector (1) chooses how deep the hierarchy expands in the results table. Selecting a shallower depth reveals tiers beneath each row, from broad results down to the level where scoring patterns and rubric performance become clear. In practice, this means users can start with an overview and progressively increase detail to narrow in on where questions or issues arise. On the right, the outcome and organization toggles (2) change the orientation of the view, essentially deciding what appears at the top of the list. The underlying data is the same; the toggle simply provides the most natural navigation depending on how results need to be analyzed.

  • Outcome First: Starts from outcomes and nests the organizational and instructional levels underneath, which is useful when the primary question is “How did we do on each outcome?”

  • Organization First: Starts with the organizational structure and nests outcomes beneath it, which is useful when the primary question is “How is a specific unit performing, and on which outcomes?”

image-20261104-175818.png

Drilling Down

Regardless of the depth or orientation, expanding the levels allows for drill-down (1). When drilling down to view detailed data (especially at the score level), the table displays score distribution information (2) that breaks results into achievement levels (for example, showing how many ratings fell into each level, such as Advanced/Competent/Developing/Beginner/Not Present). This distribution view is important because two criteria can have the same average score yet very different underlying patterns—one might be consistently “middle,” while the other might be split between high and low ratings. A common workflow is to begin with a high-level view, scan for surprising averages or low Kappa, and then drill down until the specific course/criterion/score distribution explains what’s happening and where follow-up is needed.

image-20261104-175746.png

Exporting Scoring Ledger

info The reporting CSV is available only when Interrater Reliability (IRR) is enabled via the assessment Settings page.

The scoring ledger provides a comprehensive artifact-level view of all scoring and validation data, making it easier to analyze inter-rater agreement, verify adjudication outcomes, and support accreditation reporting without manual data compilation. The ledger can be accessed by expanding the More Actions menu and selecting Export CSV. The exported ledger contains one row per artifact-criteria pair. Each row reflects the current validation state of the scored artifact in the Status column:

  • Stable Consensus: When evaluators agreed on the performance level.

  • Discordant Value Identified: When the spread between assessed scores requires adjudication.

  • Confirmed Technical Failure: When a valid score could not be recorded.

image-20261104-200333.png