Hire WorkforcePython SDKRequest DemoDocumentation
Documentation
Start typing to search…

About Insights

In Insights, you can get detailed information about the performance of your annotation team as well as your dataset's health.

If you'd like to learn more about using the Insights tool with Multimodal projects, read here.

insights

The Insights feature is a flexible and easy to use tool for measuring and reviewing your team’s performance and project status in an in-depth manner during project curation. You also have the ability to compare data through different charts to help you visualize the information easily.

📘

The Insights tab is available for the following roles: Team Owners, Team Admins, and Project Admins.

🚧

In case of group projects, the Insights tab will only show information on the project visible within the group’s data scope.

Generate Insights

  1. Select the project you want to generate insights for.
  2. In the left panel, click on Explore.
  3. Go to the Insights tab in the top right.
  4. Click Generate.

📘

Refresh

Click Refresh to see the most up-to-date information.

Items

You can look into your project’s item status thoroughly using Insights. You can do so by setting a number of parameters to filter the status and information on all items within your project. These are the parameters available.

Measure

Through this parameter, you can look into the different measurements of your items, such as count, time spent, metadata, and the length or duration of item.

Measure (y-axis)

Image

Text

Video

Count (default parameter)

✅

✅

✅

Instance Count

✅

✅

✅

Video Duration

❌

❌

✅

Text Length

❌

✅

❌

Custom Metadata (number type)

✅

✅

✅

[Role_Name]

Time

✅

✅

✅

Admin Time

✅

✅

✅

Total Time

✅

✅

✅

The [Role_Name][Role_Name] Time refers to the amount of time spent on the project by a user with the role in question. In this case, [Role_Name] will be the name of any role available within your project’s workflow, and it’ll display their corresponding working time. You’ll see each individual role’s name in this drop-down list.

Breakdown

You can get a breakdown of your items by looking into the general information about them, including their current status, the folder they belong to, the contributors that worked on them, and more.

Breakdown (x-axis)

Image

Text

Video

None

✅

✅

✅

Status (default parameter)

✅

✅

✅

Folder

✅

✅

✅

Approval Status

✅

✅

✅

Custom Metadata (strings)

✅

✅

✅

Instance Count

❌

✅

✅

Keyframe Count

❌

❌

✅

[Role_Name]

Name (Assignees)

✅

✅

✅

[Role_Name]

Name (Updated by)

✅

✅

✅

Admin Name

✅

✅

✅

[Role_Name]

Name (Action date)

✅

✅

✅

Last Action Date

✅

✅

✅

📘

  • The [Role_Name][Role_Name] Name parameter refers to the name of any assignee with any role available in your project. For example, if you have three roles (Annotator, TextEditor, and Reviewer), you’ll see their options individually in this drop-down list as follows: Annotator Name, TextEditor Name, Reviewer Name.
  • The [Role_Name][Role_Name] Date refers to the timestamp of any person with the corresponding role that worked on the project in the editor.
  • The [Role_Name][Role_Name] Name, and Admin Name parameters will bring you results of any users with the corresponding roles that have worked on the items you're querying, regardless of their current assignment.

Sorting

When you’ve selected a Breakdown parameter, you can further sort your chart (all chart types except Line) by the following options:

  • Ascending by number value (1-100)
  • Descending by number value (100-1)
  • Ascending by text (A-Z)
  • Descending by text (Z-A)

You can only sort the chart in this way when selecting the following Breakdown parameters: Status, Folder name, Approval status, [Role] Names, Instance Count (number value only), and Keyframe Count (number value only).

Compare by

Insights gives you the ability to compare your metrics in a number of ways and display the differences clearly. The Breakdown parameters, excluding all date-related parameters, are available as Compare by parameters and are presented in alphabetical order in the graph.

Compare by

Image

Text

Video

Status

✅

✅

✅

Folder Name

✅

✅

✅

Approval Status

✅

✅

✅

Custom metadata

✅

✅

✅

Instance Count

❌

✅

✅

Keyframe Count

❌

❌

✅

[Role_Name]

Name (Assignees)

✅

✅

✅

Admin Name

✅

✅

✅

Instances

You can look into your project’s instance status thoroughly using Insights. You can do so by setting a number of parameters to filter the status and information on all instances within your project. These are the parameters available.

Measure

Through this parameter, you can look into the different measurements of your instances, such as count, probability, attribute count, polygon and point count, and length or duration of instance.

Measure (y-axis)

Image

Text

Video

Count (default parameter)

✅

✅

✅

Number of Attributes

✅

✅

❌

Number of Polygon Points (only for

Polygons

and

Polylines

)

✅

❌

❌

Probability

✅

❌

❌

Instance Duration

❌

❌

✅

Instance Length (text length)

❌

✅

❌

Keyframe count

❌

❌

✅

Breakdown

You can get a breakdown of your instances by looking into the general information about them, including their current status, their class name, the contributors that worked on them, and more.

Breakdown (x-axis)

Image

Text

Video

None

✅

✅

✅

Class Name (default parameter)

✅

✅

✅

Instance Type

✅

✅

✅

Approval Status

✅

❌

❌

Group ID

✅

❌

✅

Creator Name

✅

✅

✅

Creator Role

✅

✅

✅

Creation Type

✅

✅

❌

Created At

✅

✅

✅

Updater Name

✅

✅

✅

Updater Role

✅

✅

✅

Updated At

✅

✅

✅

Sorting

When you’ve selected a Breakdown parameter, you can further sort your chart (all chart types except Line) by the following options:

  • Ascending by number value (1-100)
  • Descending by number value (100-1)
  • Ascending by text (A-Z)
  • Descending by text (Z-A)

You can only sort the chart in this way when selecting the following Breakdown parameters: Instance Type, Approval Status, Class Name, Creation Type, Creator Name, Creator Role, Updater Name, Updater Role, and Group ID

Compare by

Insights gives you the ability to compare your metrics in a number of ways and display the differences clearly. The Breakdown parameters, excluding all date-related parameters, are available as Compare by parameters and are presented in alphabetical order in the graph.

Compare by

Image

Text

Video

Class Name

✅

❌

✅

Instance Type

✅

✅

✅

Approval Status

✅

❌

❌

Group ID

✅

❌

✅

Creator Name

✅

✅

✅

Creator Role

✅

✅

✅

Creation Type

✅

✅

❌

Updater Name

✅

✅

✅

Updater Role

✅

✅

✅

Filter

You can filter your items and instances thoroughly by their status and information. Each option has further selections to refine your filter to suit your needs.

Filter

Description

Image

Text

Video

Item → Status

Users can select one or multiple statuses from the following: Not Started, In Progress, Quality Check, Returned, Completed, Skipped.

✅

✅

✅

Item → Folder

Users can select one or multiple folders.

✅

✅

✅

Item → Subset

The user can select one or multiple item categories.

✅

✅

✅

Item → Approval Status

Users can select one or multiple of the approval statuses: Approved, Disapproved, None.

✅

✅

✅

Item → Instance Count

Users can select one of the logical operators (equal, not equal, greater than, less than) and provide a number.

✅

✅

✅

Item → Text Length

Users can select one of the logical operators (equal, not equal, greater than, less than) and provide a number.

❌

✅

❌

Item → Keyframe Count

Users should be able to select one of the logical operators (equal, not equal, greater than, less than) and provide a number.

❌

❌

✅

Item → Video Duration

Users should be able to select one of the logical operators (equal, not equal, greater than, less than) and provide a number.

❌

❌

✅

Item → Custom Metadata

Users can select one of the logical operators (equal, not equal, greater than, less than) and provide a number.

✅

✅

✅

Item → [Role_Name][Role_Name] (Assignees)

Users can select one or multiple user names under the selected role.

✅

✅

✅

Item → [Role_Name][Role_Name] (Updated by)

Users can select one or multiple user names under the selected role.

✅

✅

✅

Item → Admin

Users can select one or multiple Admins.

✅

✅

✅

Item → [Role_Name][Role_Name] Name (Action Date)

Indicates the time when the last time a user of any role interacted with the item.

  • Users can select a date range by providing From and To dates.
  • In case only the From date is provided, then all the items that have been interacted with since the From date should be shown.
  • In case only the To date is provided, then all the items that have been interacted with since the project start till the To date should be shown.

✅

✅

✅

Item → Last Action Date

Indicates the time when the last action was taken on an item.

  • Users can select a date range by providing From and To dates.
  • In case only the From date is provided, then all the items that have a last action date since the From date should be shown.
  • In case only the To date is provided, then all the items that have a last action date since the project start till the To date should be shown.

✅

✅

✅

Instance → Class Name

Users can select one or multiple classes in the project.

✅

✅

✅

Instance → Instance Type

Users can select one or multiple annotation types from the following: Polyline, Polygon, Bounding Box, Rotated Bounding Box, Ellipse, Point, Cuboid, Tag, and Relationship.

✅

✅

✅

Instance → Approval Status

Users can select one or multiple approval statuses from the following: Approved, Disapproved, None.

✅

❌

❌

Instance → Group ID

Users can provide Group IDs (separated by commas).

✅

❌

✅

Instance → Creator name

Users can select one or multiple names of the instance creators.

✅

✅

✅

Instance → Creator role

Users can select one or multiple roles of the instance creators (Annotator, QA, Admin).

✅

✅

✅

Instance → Creation type

Users can select one or multiple creation types from the following: Manual, Prediction, Preannotation.

✅

✅

❌

Instance → Created at

- Users can select a date range by providing From and To dates.

  • In case only the From date is provided, then all the items that have instances which were created since the From date should be shown.
  • In case only the To date is provided, then all the items that have instances which were created since the project start till the To date should be shown.

✅

✅

✅

Instance → Updater name

Users can select one or multiple names of instance updaters.

✅

✅

✅

Instance → Updater role

Users can select one or multiple roles of instance updaters from the following: Annotator, QA, Admin.

✅

✅

✅

Instance → Updated at

- Users can select a date range by providing the From and To dates.

  • In case only the From date is provided, then all the items that have instances which were updated since the From date should be shown.
  • In case only the To date is provided, then all the items that have instances which were updated since the project start till the To date should be shown.

✅

✅

✅

📘

Filters that include [Role_Name] will show every role within the project’s workflow individually.

Chart types

You can display the information using a number of different chart types: Bar chart (default), Stacked bar chart (available only if comparison is selected), Horizontal bar chart, Stacked horizontal bar chart (available only if comparison is selected), and Line chart.

Bin size

The bin size can be:

  1. Hour (available only for instance type graphs)
  2. Day (default bin size)
  3. Week
  4. Month

📘

This filter is only available if one of the following breakdown parameters is selected:

  • For items: [Role_Name] date, or Last action date.
  • For instances: Created At or Updated At.

Download data

To download your data, click the download button in the top right corner. The download file is a CSV file which consists of a table with 2 columns if no comparison is selected.

  • Column 1: Breakdown parameter
  • Column 2: Calculation parameter + calculation method

The number of rows is equal to the count of breakdown parameters for each item/instance.

If you have a comparison parameter selected, the downloaded CSV file is a matrix where the rows represent the breakdown parameters and the columns represent the comparison parameters.

The data in each cell of a matrix represents the calculation parameter + the calculation method for each item/instance group of the (breakdown + comparison) parameter.

Save reports

When filtering your project’s data through the Insights tab, you can save your filtration settings as a Report. When clicked, a report will apply all of its corresponding filters and preview parameters to save you time from doing it again manually.

To save a report:

  1. In the Insights tab, click on the star button.
  2. Type in a name for your report.
  3. Check Save as shared report if you want the report to be visible to other admins (optional).
  4. Click Save.

Edit report

  1. In the Insights tab, click on the star button.
  2. Find the report you want to edit and click the three dots ⋮.
  3. Click Edit.
  4. Make your changes to the report accordingly.
  5. Click Save.

🚧

You can’t edit reports made by other users.

Delete report

  1. In the Insights tab, click on the star button.
  2. Find the report you want to edit and click the three dots ⋮.
  3. Click Delete.

Updated about 1 month ago