Video Project - Export annotation format

When you export a Video Project, the download folder contains JSON files of your annotations. Each video has its own JSON file. You can export annotations by frame or by timeline (by second).

Annotation JSON (by timeline) in Video with attached URLs projects contains 3 main fields:

{
    "metadata": {},
    "instances": [],
    "tags": [],
}

Annotation JSON (by frame) in Video with attached URLs is a list of frame annotations. You need to export the annotations using the SDK function below to get the output in this format.

SDK function

frame_annotations = SAClient.get_annotations_per_frame(project = "Project Name",
                                                 video ="video.mp4", fps = 30)

Output

[
    {
        "frame": 1,
        "annotations": []
    },
    {
        "frame": 2,
        "annotations": []
    },
]

Metadata

{
    "metadata": {
        "name": "video_file_example_1",
        "width": 1920,
        "height": 1080,
        "status": "Completed",
        "url": "https://sa-public-files.s3.us-west-2.amazonaws.com/Video+project/video_file_example_1.mp4",
        "duration": 30526667,
        "projectId": 101067,
        "error": null,
        "annotatorEmail": "[email protected]",
        "qaEmail": "[email protected]",
        "lastAction": {
            "timestamp": 1628861289127,
            "email": "[email protected]"
        }
}

Description:

  • "name": string - Video name as stated in the CSV file or generated by SDK.
  • "width": integer - The video’s width. The field is null if the video was not opened in the editor.
  • "height": integer - The video’s height. The field is null if the video was not opened in the editor.
  • "status": string - The video’s status. It can be: NotStarted, InProgress, QualityCheck, Returned, Skipped, or Completed.
  • “url”: string -The URL from the attached CSV file.
  • “duration”: integer - The video duration in microseconds.
  • “projectId”: integer - The project ID.
  • "error": boolean - If it is true, the video is disapproved in the tool. If it is false, then the video is approved in the tool. The field is null if the video is neither approved nor disapproved.
  • "annotatorEmail": string - The annotator’s email address.
  • "qaEmail": string - The QA’s email address.
  • "lastAction": Information about the user who last edited the video. Edits include changing the class of an instance, adding or deleting attribute, pointlabels, or tags, and adding, deleting, moving, or scaling instances.
    • "email": string - The email address of the last user who added or edited an instance. Format example: "timestamp": “1626152220563”
    • "timestamp": integer - The date and time when the instance was last edited.

📘

Mandatory fields

  • The object name is mandatory.
  • If the object lastAction exists, then the email and timestamp are mandatory.

Instances

Bounding Box

The JSON of Bounding Boxes exported by timeline:

[{
            "meta": {
                "type": "bbox",
                "classId": 759338,
                "className": "Class 1",
                "pointLabels": {
                    "0": "Pointlabel_1"
                },
                "createdBy": {
                    "email": "[email protected]",
                    "role": "Annotator"
                },
                "createdAt": "2021-08-13T13:26:32.409Z",
                "updatedBy": {
                    "email": "[email protected]",
                    "role": "Admin"
                },
                "updatedAt": "2021-08-13T13:27:13.667Z",
                "start": 0,
                "end": 23332991
            },
            "parameters": [
                {
                    "start": 0,
                    "end": 23332991,
                    "timestamps": [
                        {
                            "points": {
                                "x1": 515.51,
                                "y1": 216.7,
                                "x2": 866.79,
                                "y2": 678.58
                            },
                            "timestamp": 0,
                            "attributes": []
                        },
                        {
                            "points": {
                                "x1": 515.51,
                                "y1": 216.7,
                                "x2": 866.79,
                                "y2": 678.58
                            },
                            "timestamp": 10617971,
                            "attributes": [
                                {
                                    "id": 1065772,
                                    "groupId": 291577,
                                    "name": "Attribute1",
                                    "groupName": "Group 1"
                                }
                            ]
                        },
                        {
                            "points": {
                                "x1": 1208.82,
                                "y1": 112.01,
                                "x2": 1560.1,
                                "y2": 573.89
                            },
                            "timestamp": 23332991,
                            "attributes": [
                                {
                                    "id": 1065772,
                                    "groupId": 291577,
                                    "name": "Attribute1",
                                    "groupName": "Group 1"
                                }]
                        }]
                }]
        }
]

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a Bounding Box.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”). If the instance has an undefined class, the value should be -1 (classId": -1)
    • “className”: string - Class name. If the instance has an undefined class, the key should not be in the JSON.
    • “pointLabels” : The keys are from “0” to “7” for the 8 vertices of the Bounding Box.
    • "createdBy": Information about the user who created the Bounding Box.
      • "email": string - The email address of the user who created the Bounding Box.
      • "role": string - The role of the user who created the Bounding Box.
    • "createdAt": string - The date and time when the instance was created.
    • "updatedBy": Information about the user who updated the Bounding Box.
      • "email": string - The email address of the user who updated the Bounding Box.
      • "role": string - The role of the user who updated the Bounding Box.
    • "updatedAt": string - The date and time when the Bounding Box was updated.
    • “start”: integer - The time that marks the beginning of aBounding Box. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Bounding Box. The timestamp is in microseconds.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of a Bounding Box. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Bounding Box. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about a Bounding Box at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the Bounding Box. The timestamp in microseconds.
      • “points”: objects - Points of the Bounding Box in this location on the timeline. The list of floats is:
      • "x1, y1" for the left upper corner, and "x2, y2" for the right lower corner.
      • “attributes”: array - List of attributes for the Bounding Box, in this location on a timeline.
        • “id”: integer - Attribute ID (must be in “classes.json”)
        • “groupId”: integer - Group ID (must be in “classes.json”)
        • "name" : string - Attribute name
        • "groupName" : string- Attributes group name

The JSON of Bounding Boxes exported by frame:

[
  {
     "frame": 15,
     "annotations": [
          {
              "type": "bbox",
              "className": "car",
              "points": {
                    "x1": 669.58,
                    "y1": 85.36,
                    "x2": 854.66,
                    "y2": 259.59
              },
              "attributes": [
              {
                "id": 1065772,
                "groupId": 291577,
                "name": "Attribute1",
                "groupName": "Group 1"
              }
              ],
              "keyframe": true
          },
     ]
  },
]

Description:

  • “type”: string - The dictionary is a Bounding Box.
  • “className”: string - Class name. If the Bounding Box has an undefined class, the key shouldn't be in the JSON.
    *“points”: objects - Points of the Bounding Box in this location on the timeline. The list of floats is: "x1, y1" for the upper left corner, and "x2, y2" for the lower right corner.
  • “attributes”: array - List of attributes for the *Bounding Box** in this location on the timeline.
  • "name" : string - Attribute name
  • "groupName" : string- Attribute group name
  • “keyframe”: boolean - Specifies whether the Bounding Box was annotated manually or interpolated by the algorithm. The value is true for manual annotation.

Polygon

[{
"meta": {
  "type": "polygon",
  "classId": 1374911,
  "className": "Class 1",
  "createdBy": {
    "email": "[email protected]",
    "role": "Admin"
  },
  "createdAt": "2022-07-04T05:41:49.406Z",
  "updatedBy": {
    "email": "[email protected]",
    "role": "Annotator"
  },
  "updatedAt": "2022-07-04T06:56:34.117Z",
  "start": 0,
  "end": 8466792
},
"parameters": [
  {
    "start": 0,
    "end": 8466792,
    "timestamps": [
      {
        "points": [
          2569.35,
          589.34,
          2239.38,
          722.35,
          2693.83,
          851.81
        ],
        "timestamp": 0,
        "attributes": []
      },
      {
        "points": [
          2908.24,
          644.52,
          2578.27,
          777.53,
          3032.72,
          906.99
        ],
        "timestamp": 331064,
        "attributes": [
             {
                    "id": 1065772,
                     "groupId": 291577,
                     "name": "Attribute1",
                      "groupName": "Group 1"
              }
                      ]
      },
      {
        "points": [
          2406.54,
          516.11,
          2076.57,
          649.12,
          2531.02,
          778.58
        ],
        "timestamp": 2385171,
        "attributes": [
             {
                     "id": 1065772,
                      "groupId": 291577,
                      "name": "Attribute1",
                       "groupName": "Group 1"
                }
              ]
      },
      {
        "points": [
          2406.54,
          516.11,
          2076.57,
          649.12,
          2531.02,
          778.58
        ],
        "timestamp": 8466792,
        "attributes": [
             {
                   "id": 1065772,
                    "groupId": 291577,
                     "name": "Attribute1",
                     "groupName": "Group 1"
              }]
      }]
  }],
}

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a Polygon.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”). If the instance has an undefined class, the value should be -1 (classId": -1)
    • “className”: string - Class name. If the instance has an undefined class, the key should not be in the JSON.
    • "createdBy": Information about the user who created the Polygon.
      • "email": string - The email address of the user who created the Polygon.
      • "role": string - The role of the user who created the Polygon.
    • "createdAt": string - The date and time when the instance was created.
    • "updatedBy": Information about the user who updated the Polygon.
      • "email": string - The email address of the user who updated the Polygon.
      • "role": string - The role of the user who updated the Polygon.
    • "updatedAt": string - The date and time when the Polygon was updated.
    • “start”: integer - The time that marks the beginning of aPolygon. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Polygon. The timestamp is in microseconds.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of a Polygon. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Polygon. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about a Polygon at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the Polygon. The timestamp in microseconds.
      • “points”: objects - Points of the Polygon in this location on the timeline.
      • “attributes”: array - List of attributes for the Polygon, in this location on a timeline.
        • “id”: integer - Attribute ID (must be in “classes.json”)
        • “groupId”: integer - Group ID (must be in “classes.json”)
        • "name" : string - Attribute name
        • "groupName" : string- Attributes group name

Polyline

{
  "meta": {
    "type": "polyline",
    "classId": 1374910,
    "className": "Traffic sign",
    "createdBy": {
      "email": "[email protected]",
      "role": "Admin"
    },
    "createdAt": "2022-07-04T06:25:38.013Z",
    "updatedBy": {
      "email": "[email protected]",
      "role": "Admin"
    },
    "updatedAt": "2022-07-04T06:56:37.852Z",
    "start": 1569598,
    "end": 3110554
  },
  "parameters": [
    {
      "start": 1569598,
      "end": 3110554,
      "timestamps": [
        {
          "points": [
            1209.22,
            550.59,
            1525.34,
            511.07,
            1527.81,
            423.2,
            1846.4,
            295.81,
            1848.73,
            295.81,
            1311.89,
            891.07,
            1315.63,
            880.51,
            1315.63,
            882.82
          ],
          "timestamp": 1569598,
          "attributes": [
            {
                   "id": 1065772,
                    "groupId": 291577,
                     "name": "Attribute 1",
                     "groupName": "Group 1"
              }
          ]
        },
        {
          "points": [
            444.76,
            925.27,
            760.88,
            885.75,
            763.35,
            797.88,
            1081.94,
            670.49,
            1084.27,
            670.49,
            547.43,
            1265.75,
            551.17,
            1255.19,
            551.17,
            1257.5
          ],
          "timestamp": 1858477,
          "attributes": [
            {
                   "id": 1065772,
                    "groupId": 291577,
                     "name": "Attribute 1",
                     "groupName": "Group 1"
              }
          ]
        },
        {
          "points": [
            444.76,
            925.27,
            763.35,
            797.88,
            1081.94,
            670.49,
            1084.27,
            670.49,
            547.43,
            1265.75,
            551.17,
            1255.19,
            551.17,
            1257.5
          ],
          "timestamp": 2039058,
          "attributes": [
            {
                   "id": 1065772,
                    "groupId": 291577,
                     "name": "Attribute 1",
                     "groupName": "Group 1"
              }
          ]
        },
        {
          "points": [
            444.76,
            925.27,
            852.42,
            1266.78,
            1081.94,
            670.49,
            1084.27,
            670.49,
            547.43,
            1265.75,
            551.17,
            1255.19,
            551.17,
            1257.5
          ],
          "timestamp": 2385171,
          "attributes": [
            {
                   "id": 1065772,
                    "groupId": 291577,
                     "name": "Attribute 1",
                     "groupName": "Group 1"
              }
          ]
        },
        {
          "points": [
            444.76,
            925.27,
            852.42,
            1266.78,
            1081.94,
            670.49,
            1084.27,
            670.49,
            547.43,
            1265.75,
            551.17,
            1255.19,
            551.17,
            1257.5
          ],
          "timestamp": 3110554,
          "attributes": [
            {
                   "id": 1065772,
                   "groupId": 291577,
                   "name": "Attribute 1",
                   "groupName": "Group 1"
              }
          ]
        }
      ]
    }
  ]
}

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a Polyline.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”). If the instance has an undefined class, the value should be -1 (classId": -1)
    • “className”: string - Class name. If the instance has an undefined class, the key should not be in the JSON.
    • "createdBy": Information about the user who created the Polyline.
      • "email": string - The email address of the user who created the Polyline.
      • "role": string - The role of the user who created the Polyline.
    • "createdAt": string - The date and time when the instance was created.
    • "updatedBy": Information about the user who updated the Polyline.
      • "email": string - The email address of the user who updated the Polyline.
      • "role": string - The role of the user who updated the Polyline.
    • "updatedAt": string - The date and time when the Polyline was updated.
    • “start”: integer - The time that marks the beginning of a Polyline. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Polyline. The timestamp is in microseconds.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of a Polyline. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Polyline. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about a Polyline at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the Polyline. The timestamp in microseconds.
      • “points”: objects - Points of the Polyline in this location on the timeline.
      • “attributes”: array - List of attributes for the Polyline, in this location on a timeline.
        • “id”: integer - Attribute ID (must be in “classes.json”)
        • “groupId”: integer - Group ID (must be in “classes.json”)
        • "name" : string - Attribute name
        • "groupName" : string- Attributes group name

Point

{
  "meta": {
    "type": "point",
    "className": "Animal",
    "start": 0,
    "end": 4250000
  },
  "parameters": [
    {
      "start": 0,
      "end": 4250000,
      "timestamps": [
        {
          "x": 6575.34,
          "y": 324.97,
          "timestamp": 3750000,
          "attributes": [
            {
              "name": "Running",
              "groupName": "Motion"
            }
          ]
        },
        {
          "x": 298.46,
          "y": 23.76,
          "timestamp": 4250000,
          "attributes": [
            {
              "name": "Jumping",
              "groupName": "Motion"
            }
          ]
        }
      ]
    }
  ]
}

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a Point.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”). If the instance has an undefined class, the value should be -1 (classId": -1)
    • “className”: string - Class name. If the instance has an undefined class, the key shouldn’t be in the JSON.
    • "createdBy": Information about the user who created the Point.
      • "email": string - The email address of the user who created the Point.
      • "role": string - The role of the user who created the Point.
    • "createdAt": string - The date and time when the instance was created.
    • "updatedBy": Information about the user who updated the Point.
      • "email": string - The email address of the user who updated the Point.
      • "role": string - The role of the user who updated the Point.
    • "updatedAt": string - The date and time when the Point was updated.
    • “start”: integer - The time that marks the beginning of Point. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Point. The timestamp is in microseconds.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of a Point. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a Point. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about a Point at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the Point. The timestamp in microseconds.
      • “x”: integer - The x-coordinate of the instance.
      • “y”: integer - The y-coordinate of the instance.
      • “attributes”: array - List of attributes for the Point, in this location on a timeline.
        • “id”: integer - Attribute ID (must be in “classes.json”)
        • “groupId”: integer - Group ID (must be in “classes.json”)
        • "name" : string - Attribute name
        • "groupName" : string- Attributes group name

Event

The JSON of Events exported by timeline:

[
         {
            "meta": {
                "type": "event",
                "classId": 813377,
                "className": "Car",
                "createdBy": {
                    "email": "[email protected]",
                    "role": "Annotator"
                },
                "createdAt": "2021-10-06T14:51:19.503Z",
                "updatedBy": {
                    "email": "[email protected]",
                    "role": "Admin"
                },
                "updatedAt": "2021-10-06T14:51:37.151Z",
                "start": 0,
                "end": 11546512},
            "parameters": [
                {
                    "start": 0,
                    "end": 11546512,
                    "timestamps": [
                        {
                            "timestamp": 0,
                            "attributes": [
                                {
                                    "id": 1176348,
                                    "groupId": 338632,
                                    "name": "Yellow",
                                    "groupName": "Color"
                                }
                            ]
                        },
                        {
                            "timestamp": 11546512,
                            "attributes": [
                                {
                                    "id": 1176348,
                                    "groupId": 338632,
                                    "name": "Yellow",
                                    "groupName": "Color"
                                }
                            ]
                        }]
                  }]
         }
]

Description:

  • “meta”: objects
    • “type”: The dictionary is an Event.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”). If the Event has an undefined class, the value is -1 (classId": -1).
    • “className”: string - Class name. If the Event has an undefined class, the key should not be in the JSON.
    • "createdBy": Information about the user who created the Event.
      • "email": string - The email address of the user who created the Event.
      • "role": string - The role of the user who created the Event.
    • "createdAt": string - The date and time when the Event was created.
    • "updatedBy": objects
      • "email": string - The email address of the user who updated the Event.
      • "role": string - The role of the user who updated the Event.
    • "updatedAt": string - The date and time when the Event was updated.
    • “start”: integer - The time that marks the beginning of the Event. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of the Event. The timestamp is in microseconds.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of the Event. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of the Event. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about the Event at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the Event. The timestamp in microseconds.
      • “attributes”: array of objects - List of attributes for the Event in this location on a timeline.
        • “id”: integer - Attribute ID (must be in “classes.json”)
        • “groupId”: integer - Group ID (must be in “classes.json”)
        • "name": string - Attribute name (must be in “classes.json”)
        • "groupName" : string - Attributes group name (must be in “classes.json”)

The JSON of Events exported by frame:

[
  {
     "frame": 15,
     "annotations": [
          {
              "type": "event",
              "className": "traffic",
              "attributes": [
              {
                "id": 1065772,
                "groupId": 291577,
                "name": "highway",
                "groupName": "Type"
              }
              ],
              "keyframe": true
          },
     ]
  },
]

Description:

  • “type”: string - The dictionary is an Event.
  • “className”: string - Class name. If the Event has an undefined class, the key shouldn't be in the JSON.
  • “attributes”: array - List of attributes for the Event in this location on the timeline.
  • "name" : string - Attribute name
  • "groupName" : string- Attribute group name
  • “keyframe”: boolean - Specifies whether the Event was annotated manually or interpolated by the algorithm. The value is true for manual annotation.

Tags

Tag

{
   "meta": {
     "type": "tag",
     "classId": 666131,
     "className": "Weather",
     "createdBy": {
       "email": "[email protected]",
       "role": "Annotator"
     },
     "createdAt": "2022-07-18T09:27:45.813Z",
     "updatedBy": {
       "email": "[email protected]",
       "role": "QA"
     },
     "updatedAt": "2022-07-18T09:40:44.867Z",
     "attributes": [
       {
         "id": 847267,
         "groupId": 215515,
         "name": "Yes",
         "groupName": "Rainy"
       }
     ]
   }
 }
}

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a tag.
    • “classId”: integer - Class ID (one of the class IDs in “classes.json”).
    • “className”: string - Class name (one of the class names in “classes.json”)
    • "createdBy": Information about the user who created the tag.
      • "email": string - The email address of the user who created the tag.
      • "role": string - The role of the user who created the tag.
    • "createdAt": string - The date and time when the instance was created.
    • "updatedBy": Information about the user who updated the tag.
      • "email": string - The email address of the user who updated the tag.
      • "role": string - The role of the user who updated the tag.
    • "updatedAt": string - The date and time when the tag was updated.
    • “attributes”: array - List of attributes for the tag.
      • “id”: integer - Attribute ID (must be in “classes.json”)
      • “groupId”: integer - Group ID (must be in “classes.json”)
      • "name" : string - Attribute name
      • "groupName" : string- Attribute group name

Free text tag

[
        "Tag3",
        "Tag2",
        "Tag1"
]

Description:

  • "Tag" - string: List of tag names added to the video.

Comments

{
   "meta": {
     "type": "comment",
     "start": 0,
     "end": 10000,
     "correspondence": [
       {
         "text": "Fix this",
         "email": "[email protected]",
         "role": "Admin"
       },
       {
         "text": "Sure.",
         "email": "[email protected]",
         "role": "Annotator"
       }
     ]
   },
   "parameters": [
     {
       "start": 0,
       "end": 10000,
       "timestamps": [
         {
           "timestamp": 0,
           "points": {
             "x1": 10,
             "x2": 10,
             "y1": 10,
             "y2": 20
           }
         },
         {
           "timestamp": 7000,
           "points": {
             "x1": 10,
             "x2": 10,
             "y1": 20,
             "y2": 20
           }
         },
         {
           "timestamp": 10000,
           "points": {
             "x1": 10,
             "x2": 10,
             "y1": 15,
             "y2": 15
           }
         }
       ]
     }
   ]
 }

Description:

  • “meta”: objects
    • “type”: string - The dictionary is a comment.
    • “start”: integer - The time that marks the beginning of the comment. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of the comment. The timestamp is in microseconds.
    • "correspondence": array of objects - List of messages in the comment.
      • "text": string - The comment.
      • "email": string - The email address of the user who wrote the comment.
      • “role”: string - The role of the user who last creates the comment.
  • “parameters”: array
    • “start”: integer - The time that marks the beginning of a comment. The timestamp is in microseconds.
    • “end”: integer - The time that marks the end of a comment. The timestamp is in microseconds.
    • “timestamps”: array - Shows information about a comment at specific timestamps (start, end, edit).
      • “timestamp”: integer - Timestamp that marks the change of the instance/comment. The timestamp is in microseconds.
      • “points”: objects - Points of the comment in this location on the timeline. The list of floats is: "x1, y1" for the left upper corner and "x2, y2" for the right lower corner.