In Insights, you can get detailed information about the performance of your annotation team as well as your dataset's health.

llm insights

The Insights feature is a flexible and easy to use tool for measuring and reviewing your team’s performance and project status in an in-depth manner during project curation. You also have the ability to compare data through different charts to help you visualize the information easily.

Measure

Through this parameter, you can look into the different measurements of your items, such as count, time spent, and some component values.

Measure (y-axis)Aggregation methods
Items CountSum.
Annotation TimeSum, mean, median, min, and max.
QA TimeSum, mean, median, min, and max.
Admin TimeSum, mean, median, min, and max.
Total TimeSum, mean, median, min, and max.
Component Values- Count, Sum, mean, median, min, and max. - Component Values can only be viewed for the value of the Number, Slider, and Rating components, as well as the character length of the Text input, Text Area, Markdown, and Code components. This will display the data belonging to each individual component's ID.

Breakdown

Breakdown (x-axis)Parameter
None
StatusThe item's current status.
Folder nameThe item's current folder.
Category NameThe item’s assigned category.
Approval statusThe item's current approval status.
Annotator nameUsers with the Annotator role that have worked on the item in the editor.
QA nameUsers with the QA role that have worked on the item in the editor.
Admin nameUsers with the Admin role that have worked on the item in the editor.
Annotation dateA timestamp on when an Annotator last worked on the item in the editor.
QA dateA timestamp on when an QA last worked on the item in the editor.
Last action dateA timestamp on when the last action was taken on an item in the editor.
Component optionsBreak down the data by the Select, Checkbox, Radio, Voting, Rating, and Date component options for each individual component's ID.

Compare by

Insights gives you the ability to compare your metrics in a number of ways and display the differences clearly. The parameters you can select from are the same as the Breakdown list, excluding: Annotation date, QA date, and Last action date.

Chart types

You can display the information using a number of different chart types: Bar chart (default), Stacked bar chart (available only if comparison is selected), Horizontal bar chart, Stacked horizontal bar chart (available only if comparison is selected), and Line chart (available only when the breakdown selection is Annotation date, QA date, or Last action date).

Filter items

FilterDescription
Item → StatusUsers can select one or multiple statuses from the following: Not Started, In Progress, Quality Check, Returned, Completed, Skipped.
Item → FolderUsers can select one or multiple folders.
Item → CategoryUsers can select one or multiple item categories.
Item → SubsetUsers can select one or multiple subsets.
Item → Approval StatusUsers can select one or multiple approval statuses from the following: Approved, Disapproved, None.
Item → AnnotatorUsers can select one or multiple annotators.
Item → QAUsers can select one or multiple QAs.
Item → AdminUsers can select one or multiple Admins.
Item → Annotation DateIndicates the time when the last annotation was made on an item in the editor by an Annotator.

- Users can select a date range by providing From and To dates.
- In case only the From date is provided, then all the items that have been annotated since the From date should be shown.
- In case only the To date is provided, then all the items that have been annotated since the project start till the To date should be shown.
Item → QA DateIndicates the time when an item was last reviewed in the editor by a QA.

- Users can select a date range by providing From and To dates.
- In case only the From date is provided, then all the items that have been reviewed by the QA since the From date should be shown.
- In case only the To date is provided, then all the items that have been reviewed by the QA since the project start till the To date should be shown.
Item → Last Action DateIndicates the time when the last action was taken on an item.

- Users can select a date range by providing From and To dates.
- In case only the From date is provided, then all the items that have a last action date since the From date should be shown.
- In case only the To date is provided, then all the items that have a last action date since the project start till the To date should be shown.
Component → Created At- Users can select a date range by providing From and To dates.
- In case only the From date is provided, then all the items that have a last action date since the From date should be shown.
- In case only the To date is provided, then all the items that have a last action date since the project start till the To date should be shown.
Component → Created ByUsers can select one or multiple creators.
Component → Creator RoleUsers can select one or multiple creator roles from the following: Annotator, QA, Admin.
Component → Updated At- Users can select a date range by providing From and To dates.
- In case only the From date is provided, then all the items that have a last action date since the From date should be shown.
- In case only the To date is provided, then all the items that have a last action date since the project start till the To date should be shown.
Component → Updated ByUsers can select one or multiple updaters.
Component → Updater RoleUsers can select one or multiple updater roles from the following: Annotator, QA, Admin.
[COMPONENT_VALUE]The name of this filter and its data will be based on each individual component's ID.

- One or multiple options of the Select, Checkbox, Radio, Voting, and Rating components.
- Numerical value of the Number and Slider components, using the following operators: Equal, Not equal, Greater than, and Less than. For the Range slider component, users can make use of the following operators: Greater than and Less than.
- Date range of the Date component by providing the From and To parameters.