- The Reviewer Group Template: Attached Versus Detached
- The Details Tab
- The Reviewers Tab
- The Qualifications Tab
- The Questions Tab
- The Rubrics Tab
- The Opportunities Tab
- Cloning and Archiving Reviewer Groups
- Additional Info
Whenever you create a new reviewer group (Opportunity > Evaluators > Groups > Create Group), the group will automatically open up to allow you to set these configurations. Otherwise, you can configure other existing reviewer groups by selecting Opportunity > Evaluators > Groups, and clicking the name of a desired group.
The Reviewer Group Template: Attached Versus Detached
Whenever a new reviewer group is created in the system, it is always attached to the system's Reviewer Group Template by default. This template provides basic settings (set by System Administrators) to serve as a standard or starting point for all new reviewer groups. The first decision in customizing a reviewer group is to determine if it should be attached or detached from the template.
- ATTACHED: Receives default Reviewer Note, questions, and rubrics from the Reviewer Group Template.
- DETACHED: May be uniquely configured to display customized Reviewer Note, questions, and/or rubrics.
To detach a reviewer group from the Reviewer Group Template, select Opportunity>Evaluators>Groups, click the group, and select Details. Then click Detach Now. button. Note: Once a group is detached, it can never be re-attached to the Reviewer Group Template.
The Details Tab
This tab allows you to configure the most basic aspects of the reviewer group, including:
- Name (not seen by reviewers)
- Group Information (not seen by reviewers)
- Reviewer Note (instructions to reviewers in this group)
The Reviewer Note will be seen by your reviewers at the top of each scoring rubric while completing their evaluations. This may be a good location to provide instructions about proper evaluation of applications (ex: "Students who provide more than two letters of reference should automatically receive a 3 for the Commendations and References area.")
- Public - Selecting public will allow other administrators not in your scope to assign applications to this group.
- Scopes - If you do not make your reviewer group Public, it is recommended that you add one or more scopes so that other administrators can manage and assign applications to reviewers in the group.
- Assign Reviews - Determines whether or not the system will automatically generate application assignments and, if so, how many reviewers will be assigned to each application.
- Source - If your system uses a Conditional Application, the reviewer group's source determines whether or not the group will be able to evaluate applications associated with the Conditional Application.
- Only Qualified Applications - If selected, applications in an application category that is not visible to reviewers or are unqualified applications (with 0 qualification points) will not be available to assign to reviewers.
The Reviewers Tab
This tab allows you to select which individuals you would like to participate in this reviewer group.
Current Reviewers: Displays all users who are currently members of the reviewer group. Select Chair, Member, or None to designate them as a Chair, a member, or to remove them from the group. Click Update Reviewers when finished. You can also view information about their reviewer assignments, including:
Locked? - This is only available on reviewer groups that are set to have automatic assignments of "at least X reviews per application." Typically, this is used after reviewing has already begun; the setting allows administrators to prevent any additional applications from being assigned to the reviewer. Additionally, any reviews not yet submitted by the reviewer will be re-distributed evenly amongst all other remaining reviewers in the group (who are not also locked.)
Reviewed / Assigned - This Group - The number of reviewed applications the reviewer has submitted (finalized) / out of the number of applications they have been assigned. In other words, how much work they have completed out of the total amount of work visible to them through the reviewer group.
Reviewed / Assigned - All Groups - Similarly, the number of application reviews submitted / out of the total assigned to the reviewer through all of your system's reviewer groups (their grand total completed / total overall assignment count).
- All System Reviewers: Displays all other users who are designated as reviewers in your system, as well as their system-wide Reviewed / Eligible for Review amounts. To add a new reviewer to the reviewer group from this list, locate them in the list, and select Chair or Member to designate them as a Chair or a member of the group. Click Update Reviewers when finished.
- Adding an Individual Who is Not Yet a Reviewer: If you know of an individual you would like to add as a reviewer to this reviewer group, but they are not yet listed as a reviewer in your system (or perhaps are not yet a user in your system) you may also include these individuals from this page. Click Add Reviewers to get started.
The Qualifications Tab
Adding qualifications to a reviewer group is optional. This tab allows you to select specific qualifications for the applications that you would like this group of reviewers to interact with in your system. Much like qualifications criteria are established in your system's opportunities, qualifications of applications that should be directed toward this reviewer group can be established through a drop-down menu. For example, the picture below depicts setting a qualification criteria that the College of Education Committee only view applications from students who are Education majors.
Please Note: It is NOT necessary to re-create the qualifications from an opportunity in the reviewer group's qualifications tab!
This is merely provides you with an optional second layer of filtering. (For example, if you assigned the reviewer group to view applications from a specific opportunity -- but you wanted these reviewers to only view applications from students from a particular high school or major, then you could use the reviewer group's qualifications to say that within the applications received for an opportunity, this particular group should only view applications meeting an additional criteria.)
The Questions Tab
This tab allows you to determine any questions your reviewer group should encounter and/or respond to in order to evaluate an application. If your reviewer group is detached from the Reviewer Group Template (see "The Details Tab" section above), you may remove existing questions or add additional new questions using the Add Reviewer Question button. You may also choose to make questions required, or rearrange them as needed using the gray bars beside each question.
The Rubrics Tab
This tab allows you to set up a series of scoring rubrics that reviewers in your reviewer group will encounter when evaluating assigned applications. If your reviewer group is detached from the Reviewer Group Template (see The Details Tab section above), you may create new rubrics or edit existing rubrics. To create a new rubric, simply fill in the Name, Minimum and Maximum scores, the Offset (increments between possible scores), and the Weight (multiplication factor), and click Create Rubric.To disable or adjust an existing scoring rubric:
- Click Edit beside the desired rubric
- On the next screen, you may adjust the Name, Minimum and Maximum scores, the Offset (increments between possible scores), or the Weight (multiplication factor).
- You may also unmark Enabled to disable this rubric for reviewers in this reviewer group.
NOTE: Because it is possible that different Reviewer Groups may evaluate applications for the same opportunity using extremely different scoring rubrics, all reviewer scores in the system are converted to a 100-point scale when displayed in the Opportunity Administrator or Reviewer Chair portal.
The Opportunities Tab
This tab allows you to determine which applications you would like this reviewer group to encounter on an opportunity-by-opportunity basis. This page displays a list of all current opportunities in your system, as well as the Review Period, total number of applications received for the opportunity, and any additional assigned reviewer groups working on applications for that opportunity. (Hover over the number below Assigned Reviewer Groups to view this information.)
To select a desired opportunity, mark the box beside the opportunity's name. You may select as many opportunities as you wish.
NOTE: You must assign at least one opportunity (or the General Application or a Conditional Application) to begin matching applications to the reviewer group. If you do not select anything under the Opportunities tab, no applications will be directed toward the group, despite any qualifications you have established.
Cloning and Archiving Reviewer Groups
Reviewers will change from time to time, and this causes a need to update your reviewer group members. As it is not advised to simply remove members from a reviewer group if they have completed reviews in the past, you can clone and archive any reviewer group in order to update the members while leaving all completed reviews intact.
Note: It is important that you clone the reviewer group, and that all assigned opportunities have been archived before archiving a reviewer group. Also, Rubric Help Text does not clone over, so if these need to be copied, before archiving the old group, copy the Rubric Help Text and paste it into the newly-cloned group's Rubric Help Text section.
Once all assigned opportunities have been archived, to clone and archive a reviewer group:
- Navigate to the details tab of the reviewer group that you would like to update and scroll to the bottom of the page. Click Clone Now to clone the reviewer group.
- The newly cloned reviewer group will now display. This group is where you make any changes that are necessary for the new cycle. If you have not yet archived the opportunities that were being reviewed by the reviewer group, then the cloned group will still be assigned to those opportunities. You must unmark these opportunities from the last cycle and select the new cycle's opportunities to be reviewed on the Opportunities tab of the reviewer group, otherwise your group will be reviewing the last cycle's opportunities.
- Navigate back to the former reviewer group at Opportunity>Evaluators>Groups. Choose the reviewer group that does not include (cloned) in the title
- Once you open the correct group to be archived, click on the Reviewers tab. Be sure to check that the group does not have any completed reviews. Archiving a reviewer group that has completed reviews will remove those reviews.
- Once it is verified that there are no completed reviews, select the Details tab and click Archive.
- Once archived, the group will no longer be displayed on the All Reviewer Groups page's All tab, but instead will be displayed on the Archived tab.
- A single reviewer group can contain one reviewer or many reviewers.
- A single reviewer user can belong to more than one reviewer group, if they have multiple roles (evaluating multiple kinds of applications) within your evaluation process.
- Because a single reviewer user can belong to more than one reviewer group at a time, and because reviewer groups may have overlapping qualifications and opportunity assignments, it is possible that a single reviewer user may be assigned to review the same application twice.
- You may also assign applications to certain reviewer groups (or individual reviewer users) from the opportunity's Reviewers tab.
- See also reviewer assignment logic and other features