To support comprehensive evaluation and performance reviews, SuperAnnotate has a native logging system that tracks the total duration of work done in a project editor. All annotation status changes made by users are also included. Data logged within this system are gathered in a set of reports, which can be found in the Telemetry Reports tab, provided the feature is enabled for the organization.

The logging system applies to Multimodal, Image, Video, and Text project types only.

In order to do evaluation and performance reviews, the following reports are available to be generated in the Telemetry section. You need to download the report as a CSV file to view the data.

There are three report types you can generate:

• Status flows
• Transitions made by users
• Time spent by users

Status flow

The report provides information on the number of items per annotation status by the end of each day, as well as the count of in-flows and out-flows per status. Folders can be selected as an additional breakdown level. This information is aggregated on a daily basis.

The report contains following fields:

The report data is aggregated and overwritten twice per day UTC 14:00 and UTC 00:00 for the current day. Any time the report is generated the latest refreshed data is available for the current date.

Transitions made by users

The report provides information on the number of transitions between specific annotation statuses within the selected period. The start and end dates are included in the report, unless the end date is the day the report is generated. Data aggregation is performed at the end of each day (UTC). The current day's data isn’t included in the report. The main breakdown level is the assignee of the destination status, based on the project workflow's designated roles.

Time spent by users

The report provides summarized information on the time worked by users and the number of items they worked on during the selected period. User roles and project folders can be selected as additional breakdown levels.

The report retains data from up to 6 months prior to the day it is generated.

Time tracking

The logging system tracks a user’s active state while working in the editor to calculate the total working time across any editor.

Whenever a user works on an item in the Editor, their activity is automatically logged. This activity refers to: typing, scrolling, clicking, any mouse movement, playing a video, or opening and closing items. The system checks for activity every 10 seconds, and if any of the aforementioned actions were taken, a new entry is made in the log. These actions are considered by default and don’t require any additional specification to be logged. Actions need to be specified only when setting up time tracking through a Web component.

Activity tracking is limited to the editor — no other sections of the SuperAnnotate platform are monitored. It applies to all user roles.

All the activity tracking logs are aggregated into a daily cumulative working duration for users based on time spent on each item in a project.

Aggregation calculation takes into account the Max inactivity duration, which sets the maximum number of seconds a user can remain inactive. Any inactivity that lasts longer than the maximum duration will be excluded from accumulation.

Time tracking example

1. Let’s assume that the Max inactivity duration is set to 60 seconds in the project’s Setup.
2. A user, who has never worked on that item before, opens it for the first time and the following actions take place:
   • After actively typing for 40 seconds, the user switches to a different tab and does something else for the next 59 seconds.
   • Then, the user returns to the item’s tab and continues typing for another 100 seconds. After which, he switches tabs again to do something else for 70 seconds.
3. The accumulated working duration for that user on that item, on that day, will be: 40 + 59 + 100. The 70 seconds won’t be counted because it exceeds the project’s maximum duration that a user can remain inactive (60 seconds, as mentioned in point 1).
4. Overall accumulated working duration is 199 seconds.

Integrating time tracking in Web components

Time tracking works differently with Web components, and requires additional setup to integrate properly. Using a simple JavaScript-based telemetry mechanism, you can decide when and how to log activity from inside your component.

To do this:

1. Open your Web component’s JavaScript code editor from the form builder by hovering over the component and clicking the pencil icon.
2. Place the code snippet for the actions you need to track for time tracking.

Simple event logging

Simple events examples are (clicking, typing, mouse movement etc). To consider and log an action, implement and call the SA.logActivity(EventTag) function, replacing EventTag with a relevant label for your activity (e.g. Click). Make sure to call the action only when your preferred activity is happening. This way, additional idle time isn’t logged into the overall activity count.

SA.logActivity("EventTag");

Periodic events logging

Periodic events are events that occur over a period of time and can be considered as ‘active’ without constant actions being done in an item’s tab by the user. A periodic event can be something like a video or audio file playing without any additional active input from the user.

Here is an example of how tracking should be done for this type of action.

Video playing event code snippet example

let intervalId = null;
const LogInterval = 5000; /* 5 seconds */
const EventTag = "VideoPlayback";

function periodicTracking(startTracking) {
  if (startTracking === true) {
     if (intervalId !== null) {
        clearInterval(intervalId);
        intervalId = null;
     }
     intervalId = setInterval(() => {
        SA.logActivity(VideoPlayback);
     }, LogInterval);
  } else {
     clearInterval(intervalId);
     intervalId = null;
  }                
}

/* Start periodic logging */
periodicTracking(true);

/* Stop periodic logging */
periodicTracking(false);

Call periodicTracking(true) when the user starts interacting (e.g., presses play). Call periodicTracking(false) when the interaction stops (e.g., presses pause).