DICOM Annotation: How to Label CT, MRI and NIfTI Medical Images

DICOM annotation means outlining organs, lesions or other structures on CT, MRI, PET or X-ray scans so a model can learn from them. Supervisely is a web-based platform that labels DICOM, NRRD and NIfTI files in 2D and 3D, with AI-assisted tools, team workflows, and cloud or on-premise deployment.

I developed several of these tools over a few months of work, and this guide shows how to use them on medical volumes. It also covers 2D images such as mammograms.

What are DICOM, NRRD and NIfTI?

DICOM (Digital Imaging and Communications in Medicine) is the standard that defines the formats for medical images exchanged for clinical use. It was first published in 1993 and is recognized by ISO as ISO 12052. A DICOM file holds the image and metadata such as StudyInstanceUID and SeriesInstanceUID.

NRRD (nearly raw raster data) stores one n-dimensional array per file, with a header that describes type, sizes, encoding and spatial orientation. NIfTI was designed for exchanging neuroimaging data between analysis packages and can be stored as a single .nii file.

Format	Typical files	What it is	Typical use
DICOM	.dcm	Clinical standard, image plus patient and study metadata	Scanner and PACS output
NRRD	.nrrd, .nhdr	One array per file with a text header	Research, 3D Slicer, segmentation masks
NIfTI	.nii, .nii.gz, .hdr + .img	Neuroimaging array format	Brain imaging, research pipelines

Supervisely imports DICOM and NRRD volumes as 3D projects. For 2D work, the Medical 2D importer accepts .dcm, .nrrd and .nii files and converts non-NRRD files to NRRD on import.

What is medical image annotation used for?

Labeled scans train models that find and measure structures. Common use cases:

• Cardiology: 2D images of the heart help assess cardiac function, detect valve problems and visualize blood vessels to find blockages.
• Oncology: regular imaging tracks tumor size and treatment response. Radiotherapy planning needs accurate outlines to spare healthy tissue.
• Orthopedics: fractures, dislocations and joint degeneration.
• Dentistry: dental X-rays for cavities and tooth root conditions.
• Radiology: X-ray, CT and MRI for fractures, tumors, internal bleeding and congenital abnormalities. Mammograms are low-dose X-ray images used to find early signs of breast cancer such as tumors, cysts and calcifications.

For surgical video instead of scans, see AI tools for surgical video annotation.

Which DICOM annotation tool should you use?

Medical annotation needs domain experts, and their time is expensive. The right tool depends on who labels and at what scale.

	Desktop research tools (3D Slicer, MITK)	Supervisely
Runs in	Desktop application	Web browser
Best for	Research, one or two experts, custom image processing	Teams of annotators, reviewers and managers
Collaboration	Files shared by hand	Labeling jobs and queues, roles, activity statistics
AI assistance	Extensions and modules	Smart Tool, interpolation, models from the ecosystem
Deployment	Local machine	Cloud or on-premise

3D Slicer is a modular application for viewing, processing and analyzing medical images, and it is a strong choice when you need its processing modules. You can also use both together, as shown in how to use 3D Slicer with Supervisely.

The toolbox:

1. Works directly with original DICOM files and 3D objects.
2. Is compatible with ITK-SNAP, 3D Slicer and MITK workflows.
3. Runs in the browser. You only need a browser.
4. Can be installed on your own servers. See the Enterprise Edition.

How to annotate DICOM images step by step

Step 1. How do you import DICOM, NRRD or NIfTI files?

For 3D volumes, the Import application creates a new project and adds the original DICOM volumes, and it allows the patient data to be anonymized.

Supervisely App

Import DICOM Volumes

Import volumes in DICOM and NRRD formats without annotations

Run in SuperviselyGitHub

For 2D images such as mammograms there are two options.

• Option 1: use the Import Wizard. It supports .nrrd, .dcm and .nii files. For DICOM images, the grouping tag is chosen automatically.

  Select the Medical 2D option in the Import Wizard so the data is imported correctly. The Auto Import application detects the format, creates a project with the right settings, and uploads the data.

  Steps for creating a new 2D DICOM project

• Option 2: use the Import DICOM studies app.

  

  Supervisely App

  Import DICOM studies

  Creates project with images grouped by selected metadata, converting DICOM data to NRRD format in the process.

  Run in SuperviselyGitHub

  The file structure of a project without annotations looks like this:

  📂 cardio
  ┗ 📂 research_1
    ┣ 🩻 IMG-0001-00001.dcm
    ┣ 🩻 IMG-0001-00002.dcm
    ┣ 🩻 IMG-0001-00003.dcm
    ┗ 🩻 IMG-0001-00004.dcm

  We've prepared 🔗demo data so you can follow along.

  In the modal window, select the grouping tag from the list of predefined tags, or choose the tag manually.

  How to run the "Import DICOM studies" application

Images can be grouped by a tag such as StudyInstanceUID or SeriesInstanceUID, so you can view and label several related images on one screen. This is a special case of multi-view image annotation.

Single-view and multi-view modes

💡 You can always ungroup the images by disabling Group Images mode in the project settings. See the multi-view section below.

Step 2. How does windowing work on CT and MRI?

Medical scans hold far more intensity values than a screen can show. A CT scan can use 12-bit values, which means 4096 levels from -1024 to 3071, where air is defined as -1000 HU and water as 0 HU. Windowing maps a chosen range of those values to the display, so the tissue you care about becomes visible.

Two parameters define a window, as Radiopaedia explains. Window width (W) is the range of values shown, and window level (C) is the midpoint of that range. For example, a brain window of W 80 and L 40 shows values from 0 to 80 HU: anything above is white and anything below is black.

Supervisely has presets for different tissues. You can also hold the left mouse button and drag to set width and level by hand.

Take the Mammary Gland W: 1500, C: 1700 preset as an example. Window width 1500 means a range of 1500 pixel values is used for display. Window level 1700 is the central value of that range.

Example of the windowing scheme

The Crosshair Tool jumps to any point of interest on all planes in one click. When you hold the point and move it, the other planes and the 3D view update in real time.

Step 3. How do you label a 3D volume?

The labeling interface shows the volume on three planes and in 3D.

DICOM labeling toolbox interface with axial, sagittal and coronal views

What you see in the image:

1. Panel with Interpolation, a screenshot button, the hotkeys map, layout settings and history of operations.
2. Toolbar for navigating planes and working with annotations.
3. Workspace with the volume on its planes and a 3D perspective.
4. Tabs that list objects, figures, volumes and their settings.
5. Additional display settings.

The default layout is a 2x2 grid, and you can rearrange the grid up to 4x4 to use several planes at different scales. You can also switch to a panel variant and rearrange the tabs.

What is Mask 3D?

The volumetric mask, Mask 3D, is a 3D figure rendered on all planes and editable on any of them. It is the main option for annotating objects in volumetric data. Three tools create and edit it.

The Pen Tool combines Polygon and Brush. It draws fine lines along the edge of an object and fills the shape when you close the outline. It can also cut a shape out of a figure.

The Brush Tool draws Mask 3D shapes and Bitmaps, and you can change its size on the fly. It includes an Eraser, and a Bucket Fill for filled contours.

The Smart Tool reduces manual labeling of each slice. See the next section.

How does AI-assisted labeling work on medical scans?

Segmenting an object over hundreds of slices is slow. The Smart Tool labels an object with a few clicks using pretrained neural networks. You frame the object with a bounding box, and you can run several models at once and switch between them.

How does interpolation speed up 3D labeling?

A 3D object can span hundreds of slices on different planes. You label only the key slices, and interpolation reconstructs the object on the others. According to the docs, it uses an ITK-based morphological contour interpolation method and needs the Volume Interpolation app running. The manual labels and the result form one 3D object, which you preview in 3D and edit on any plane.

If the result is not good, undo it, add more intermediate slices and run it again.

Combined with the Smart Tool, creating Mask 3D can be much faster.

Step 4. How do you label 2D medical images?

After windowing, the target structure is easy to find. Multi-view mode shows related images side by side, which speeds up labeling.

Available tools:

• Polygon, Brush and Mask Pen for shapes when you don't need much precision, for example the breast.
• Bounding Box (Rectangle) to mark an object without defining its outline.
• Polyline for the narrowest tissues.
• Smart Tool, which can run several pretrained models. You can also serve your own. See Smart Tool.

Bitmap Brush - Polygon - Rectangle

Step 5. How do you add tags and properties?

Annotation is more than outlines. A tagging system adds properties to objects and to whole volumes.

Tags and properties of objects in the DICOM labeling toolbox

In the data tabs you can change the properties of objects, shapes and volumes: change the shape type, add a tag, or change the layer order of overlapping objects. Read more in mastering image tagging.

You can group images by any tag, including custom ones. To create a tag, open the tags tab of the dataset page. You can attach it to a specific label or to the whole image.

Create new tag

Hotkeys switch tools, manipulate objects and adjust the display, and you can customize them.

Hotkeys of the DICOM labeling toolbox

Undo and Redo revert changes. Restore mode brings back deleted objects and figures from the project.

Step 6. How do you export medical annotations?

You can export results as standardized NRRD or STL and open them in MITK, 3D Slicer and similar software.

Supervisely App

Export Volumes with 3D Annotations

To Supervisely format, compatible with 3D Slicer, MITK

Run in SuperviselyGitHub

The volume project export app keeps the whole project usable in both the platform and medical tools. It is compatible with 3D Slicer and MITK.

For 2D images, export annotations in Supervisely JSON format.

You can reimport the data with the Import DICOM studies app. To keep the grouping, export and import in the Supervisely format. See the annotation format docs.

How do you turn on multi-view mode in an existing project?

To group images by tag in an existing project, activate Multiple Images Views Mode in the More menu. This groups images only for the current session.

Choose multi-view mode

To keep the grouping, open the project settings:

Settings ➭ Visuals ➭ Project labeling interface ➭ Medical 2D and click Save.

Choose Group Images sync mode to annotate several images at once.

Group settings

How do you upload DICOM images with the Python SDK?

Install the supervisely package to use the API:

pip install --upgrade supervisely

More examples are on the Developer Portal. This code uploads two DICOM files as a group:

images_paths = ['/path/to/IMG_001.dcm', '/path/to/IMG_002.dcm']
images_metas = [{'YourMetaData':'Val01'}, {'YourMetaData':'Val02'}]

# upload 2D DICOM images
api.image.upload_medical_images(
    dataset_id, images_paths, "GroupTagName", images_metas
)

The images_metas parameter is an optional list of dictionaries with metadata for each image. All DICOM metadata is preserved after upload and shown in the IMAGE METADATA section of the labeling tool.

How do teams annotate medical images together?

Distribute work between colleagues with labeling jobs. You can also build a hierarchy of roles: manager, annotator and reviewer. See labeling jobs and labeling queues for details.

1. Job management: describe what to annotate and how.
2. Progress monitoring: track annotation status and review submitted results.
3. Access permissions: limit access to specific datasets and classes within one job.
4. Screenshots: take a screenshot for an urgent task and share the link, with no extra app.

If you work with 3D Slicer, the Slicer integration lets annotators move between the web interface and Slicer inside one labeling job.

Is medical annotation in Supervisely HIPAA compliant, and can it run on-premise?

Medical data is protected health information, so privacy requirements are stricter. The DICOM toolbox page says the platform and team help you maintain HIPAA compliance, with:

• encryption in transfer and at rest,
• automatic logouts after inactivity,
• data and personal information anonymization,
• user activity monitoring and access alerts,
• security audits and vulnerability scanning.

See the security and permissions page for details. The Enterprise Edition is the on-premise version of the platform. It can be installed on your own bare-metal server or cloud instance, and it does not need an internet connection. Hospitals and research groups can keep DICOM studies on their own servers, which helps with HIPAA and GDPR requirements.

Summary

DICOM, NRRD and NIfTI volumes and 2D images can be annotated in the browser, with windowing, Mask 3D tools, Smart Tool, interpolation, tags, labeling jobs and exports to 3D Slicer and MITK. To try it, start with the medical labeling toolbox or the free Community Edition.