Circle Dynamic Shape Specification

[dayo09]

Tip

Never use [] for shape_signature. Always use None for static shapes.
If you get typing errors from this, just add a # type: ignore comment to suppress mypy warnings.¹

Since there is no formal documentation for the circle dynamic shape specification, confusion and errors may occur during development and operation. Below is a summary of the spec, including potential error cases and important notes.

Basic

Dynamic Shape in Torch Exported Program

• torch tensor.shape: [s, 3, 3, 4]
  (where s represents a dynamic dimension. type: torch.SymInt)

Dynamic Shape Definition in Circle

Field	Example
circle::tensor.shape	[1,3,3,4]
circle::tensor.shape_signature	[-1, 3, 3, 4] (if dynamic shape)
circle::tensor.shape_signature	None (if no dynamic shape)

• tensor.shape_signature = None
  → Valid (OK)
• tensor.shape_signature = []
  → Invalid, causes errors in the ONE runtime!

Note: For static shapes, shape_signature must be set to None.
Using an empty list ([]) is not allowed because ONE runtime may mistakenly treat it as a shape, resulting in errors.

Related: ONE Runtime Behavior

• If the shape_signature field exists in the circle,
  → The runtime uses shape_signature.
• If shape_signature does not exist,
  → The runtime falls back to using shape.

Caution When Generating Circle Files (e.g., using TICO)

• Assigning an empty list ([]) to shape_signature causes the field to be created in the circle file.
• This leads to malfunctions in the ONE runtime.
• Therefore, for static shapes, the shape_signature field should not be created at all.

Invalid Typing in Circle Schema

• The circle_schema python package we are using has an significant typing error.
  • To set tensor.shapeSignature = None, we should force mypy assert ignore manually. Related Issue: https://github.sec.samsung.net/AIP/python-tools/issues/887

Footnotes

1. Error of Flatbuffers. It forced Optional[List[int]] -> List[int] while generating python circle_schema package. ↩

[dayo09]

What is different when using [] instead None?

You cannot see the difference in Netron.
However, when it's converted in json (flatc --json --raw-binary $CIRCLE_SCHEMA_FBS $CIRCLE_FILE), you can see the difference.
!! shape_signature field is created and its value is defined as [] !!

      tensors: [
        {
          shape: [
            8
          ],
          buffer: 1,
          name: "tico::p_conv2d_bias",
          shape_signature: [  # <--- This field is created!

          ]
        },

      tensors: [
        {
          shape: [
            8
          ],
          buffer: 1,
          name: "tico::p_conv2d_bias"
           # <--- No shape_signature field exist.
        },

[dayo09]

NOTE: Unknown shape vs Dynamic shape

Type	Meaning	Example shape	tensor shape	tensor shape_signature
Static shape	All dimensions are specified	[1,2,3,4]	[1,2,3,4]	[1,2,3,4] or None
Dynamic shape	Some of dimensions are unknown	[1,?,3,?]	[1,1,3,1]	[1,-1,3,-1]
Unknown shape (not official word)	Any information about shape itself does not exist	?	None (but exported as [])	None

Originally posted by @sk-namkoong in https://github.sec.samsung.net/AIP/python-tools/pull/941#discussion_r4983076