Slide Score Anno2

This format is the new iteration of Slide Score annotations, designed and tested for very high performance, and space-efficiency. This makes it specifically suited to visualize AI generated annotations on a Whole Slide Image.
It does this by converting an input dataset of annotations (TSV, GeoJSON, JSON), into a set of files that allow a viewer to smoothly draw the needed annotations, when they are in view. The relevant files are always stored in a ZIP file for easy handling. This format is referred to as a binned annotation file, or Anno2.

Table of Contents

  1. Conversion
  2. Annotation types
  3. Coloring and item metadata
  4. Help

Conversion

To generate an Anno2 ZIP for your dataset, a Command Line Interface (CLI) is provided. An alpha version is available on the Test Python Package Index. Since this CLI is still in active development, below documentation may be out of date.

The CLI can be accessed using: $ python -m slidescore.bin_data. The command line option hopefully are self-explanatory.

$ python -m slidescore.bin_data --help
usage: bin_data.py [-h] --items-path ITEMS_PATH [--output OUTPUT] [--metadata METADATA] [--points-type {mask,circles}]

This program converts a items TSV file (or slidescore_anno1.json) of either points in a mask, polygons or a heatmap, into a binned format for fast lookup. Author: Bart.

optional arguments:
  -h, --help            show this help message and exit
  --items-path ITEMS_PATH, -i ITEMS_PATH
                        Input file path, should be a TSV / GeoJSON / SlideScore JSON file
  --output OUTPUT, -o OUTPUT
                        Output path of the binned items file
  --metadata METADATA, -m METADATA
                        Path of JSON file containing user-specified metadata about the input file, will be encoded in the output file
  --points-type {mask,circles}, -pt {mask,circles}
                        Type of points that are provided in the TSV, either single pixels (mask), or center points of circles (default)

Metadata

The --metadata flag allows a user to include a fully custom JSON file in the Anno2 ZIP file. This can be useful for keeping track which tools were used to generate the annotations, and keep track of the provenance of the annotations. This JSON is stored as user_metadata.json in the Anno2 file.

Annotation types

Anno2 support several kinds of annotations, these are:

Other type of annotations, like rectangles, will need to be represented as one of the options above, like polygons.

Polygons

Arguably the most versatile annotation format available is the Polygons one. You can generate this format from all 3 input options; TSV, GeoJSON, and Slide Score anno v1 JSON. The input coordinates always refer to pixel offsets, in the highest resolution of the slide image. If you have offsets based on another coordinate system, like patch based offsets or micrometer offsets, you will have to convert them first.

TSV

The expected format for a polygon in a TSV file, is one polygon per line. With its vertices separated by spaces/tabs, like: 

x1p0 y1p0 x2p0 y2p0 x3p0 y3p0 (etc...)`
x1p1 y1p1 x2p1 y2p1 (etc.)`

This is the simplest format. All lines should contain an even number of elements, and more than 2 points should be encoded per polygon.

GeoJSON

The GeoJSON format is supported, but not all GeoJSON files are compatible. It is expected that all Polygons are encoded in a single FeatureCollection, with no other types of Features. The coordinates property is where the Polygons are extracted from, and these vertices are rounded to integers. Holes are currently not really supported, they are simply saved as another polygon.

Example of a compatible GeoJSON file: 
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [
          [
            [400, 150],
            [440, 150],
            [440, 200],
            [400, 200],
            [400, 150]
          ]
        ]
      },
      "properties": {
        "uuid": "3f084b4c-c7bf-41c3-94b2-8b8f9995e707",
        "cell_type": "Lymphocyte",
        "color": "#FF00FF"
      }
    }
  ]
}

Anno1 JSON

In order to provide backwards compatibility, Slide Score Anno1 JSON's are supported as well. Both the polygons and brush anno V1 are converted. Labels (floating text with explanations) are also supported.

Points

If your model outputs certain points of interest. Like the coordinates of a cancer cell, this type is best suited. These points are visualized using circles on the WSI. Whether the circles are a fixed size (like 5uM) or relative to the zoom can be configured in the Slide Score administration interface. Up to 10 million circles are supported, but if the density is very high, the front-end can experience some lag.

TSV

The expected format is 1 point per line. Each line should thus contain 2 tab separated values, like the following:

x1 y1
x2 y2
x3 y3
etc.

Be sure to set the --points-type circles CLI flag to make sure you are encoding the correct type.

Anno1 JSON

Again this type is a 1 to 1 analog to the Anno 1 version, so the simple Anno1 JSON with a simple list like [{x: 42, y: 12}, ...] is also supported.

Masks

Binary pixel level masks are supported as well. In order to to support as many possible inputs, coordinates need to be supplied in a TSV file, with an entry for all "active" pixels. Again the input coordinates always refer to pixel offsets in the highest resolution. If your model does not output such high resolution information, you can use a binary-heatmap. If you need represent non-binary information, a normal heatmap might be better suited.

TSV

The format of the mask TSV is the exact same as the Points TSV described above. But make sure to change the CLI flag to --points-type mask.

Heatmaps

The heatmap can be an excellent option if you have information regarding bigger regions on the WSI, with an additional intensity dimension. You can think of a heatmap as a low-resolution greyscale overlay on top of the WSI, with a specified offset and size.

TSV

Because a heatmap needs some additional information, the starting offset and size, these need to be added to the TSV file as a header. The first item on the first line should be Heatmap. Following it should be the x and y offset, of the top-left of the WSI image, in pixels. After that should be the size per pixel. For example, 1 heatmap pixel should cover 16x16 WSI image pixels, on the highest resolution. After this, the pixel data can be passed for the heatmap, with an x, y and an intensity value. The maximum x and y value determine the width and height, respectively.

Heatmap 100 100 16 # x_offset y_offset size_per_pixel
x1 y1 value1
x2 y2 value2
etc.

Binary Heatmap

In some cases you want to display information on a slide image that is binary, but covers more than a single pixel. In that case you can choose to use a binary-heatmap. It is like a cross-over between a Mask and a Heatmap. This annotation type is well suited for 2-type segmentations or other higher-level annotations.

The format too supply is the following in a TSV file:

binary-heatmap 100 100 16 # x_offset y_offset size_per_pixel
x1 y1
x2 y2
etc.

Anno1 JSON

Slide Score v1 annotations of type heatmap are also compatible, the size_per_pixel is dynamically calculated based on the entire heatmaps width and height.

Coloring and Item Metadata

To color Anno2 annotations, you have three options:

  1. The color set in the Score Editor (default option).
  2. A hex color string specified in the user_metadata.json under the root property color.
  3. Color assignments defined for individual polygons or points.

If you want all items in an annotation to be a single color, you can either allow the Study Admin to set it in the Score Editor or enforce a specific color by providing a JSON file to --metadata that looks like this:

{
  "version": "1.13",
  "color": "#00FF00"
}

For cases where you wish to assign specific colors to each item, such as to distinguish between different cell types. You can utilize item-level metadata.

Item-Level Metadata

In addition to the global Anno2 metadata support in the user_metadata.json, Anno2 annotations permit the inclusion of metadata specific to individual items, such as points or polygons. Currently, item-level metadata is supported only for the points and polygons Anno2 types and can only be used with the GeoJSON input format. If the GeoJSON includes the "properties" field, it will be saved as item-specific metadata and displayed to the user when they hover over the polygon. Note that points do not have an area and therefore do not support hover interactions and thus only the color property would be suitable.

Coloring Items

To assign a specific color to a point or polygon, you can define a "color" property within the GeoJSON "feature" field. This property should contain an RGB string, such as "#FF00FF". If specified, this color will be used to render the item; if not, the default coloring will apply.

For optimal performance, we recommend using a limited palette of colors, ideally no more than 20 distinct colors, when assigning colors to items.

Format schema

Format schema

Help

If you need help figuring it out, just send us an email.