How To Use Model Assisted Labeling To Boost Annotation Performance With Supervisely
A Deep Dive into the Beloved Supervisely Smart Labeling Tools for AI-Assisted Interactive Object Segmentation
Table of Contents
Smart Tool: What is it?
Smart Tool is a high-powered interactive segmentation tool. Its functionality is based on the creation of a mask predicted by artificial intelligence (AI). This unique tool occupies a special place in the Supervisely arsenal, significantly accelerating the annotation process.
What makes the Smart Tool special? The Smart Tool provides users with access to a diverse range of neural network algorithms for object segmentation task integrated into Supervisely. These include robust models such as RITM, Segment Anything and others, and we are constantly expanding our Ecosystem by integrating new models. It's crucial to note that the quality, accuracy, and speed of segmentation heavily rely on the chosen model.
RITM interactive segmentation SmartTool
State-of-the art object segmentation model in Labeling Interface
Serve Segment Anything in High Quality
Run HQ-SAM and then use in labeling tool
Smart Tool is particularly useful for tasks that involve selecting areas with complex contours or a large number or variety of objects. Its ability to perform such tasks in a matter of seconds makes it highly effective. Where the model predictions shall be amended, combination with such complementary tools as Brush and Pen is beneficial. These tools offer the ability to correct imperfections in the masks created by the neural network. The synergy between the Smart Tool and other Annotation instruments enhances the overall versatility and robustness of the annotation process.
This makes Smart Tool one of the most popular annotation tools in Supervisely. Smart Tool may be utilised in almost any domain, including:
- Urban Planning and Infrastructure: the Smart Tool is great for marking up urban scenes and infrastructure designs. Whether you're involved in city planning or site development, it is excellent at accurately annotating structures, roads, and utilities.
- Medical: the Smart Tool facilitates the precise delineation of organs, tissues, and anomalies, thereby contributing to accurate diagnostics and comprehensive treatment planning and also helps to accurately segment medical instruments, which contributes to the development of robotic surgery.
- Agriculture: farmers may rely on AI tools to assess crop health, plan harvests efficiently, and minimize waste through targeted interventions, enhancing overall agricultural productivity. The Smart Tool enables precise annotation of fruits, vegetables, weeds, crops, pests and much more, significantly improving the quality assessment of agricultural produce.
- Automotive: The Smart Tool is really helpful for showing and dividing car damage, like dents and chips. It's also good at finding spots where the tires are worn, giving a better idea of how the vehicle equipment is overall.
- Infrastructure Inspection: In infrastructure inspection, the Smart Tool can be employed to detect damage and hazards on roads, such as potholes or obstructions. This is particularly critical for the safety of autonomous vehicles, which leverage this information to navigate safely and avoid potentially dangerous areas.
Video Tutorial for Smart Tool in the New Labeling Toolbox
Master the efficient utilization of our Smart Tool through a concise 5-minute video tutorial, featuring explicit guidance from our Supervisely expert. Familiarize yourself with the updated interface and explore the functionalities within our annotation toolbox with practical, hands-on experience.
Smart Tool: key features
You can try out our legendary Smart Tool for free directly on the Supervisely Platform!
1. Interact with the Model for Precise Predictions
Creating a mask with the Smart Tool is simple:
Click anywhere in the scene to make the top point of the rectangle.
Then click again for the bottom point in the opposite direction.
The neural network will attempt to identify the main object within that rectangle.
To adjust the neural network's predictions, place and drag positive 🟢 and negative 🔴 points around the entire area inside the bounding boxes.
Click and put a positive point on the part of the object you want IN the final mask. For parts of the background you want to leave OUT of the final mask, put a negative point by clicking.
Finalize the process by pressing the
Pro tip: The bounding box around an object doesn’t need to be precisely close to the object’s boundaries. We recommend placing the initial bounding box with about a 10% padding from the object boundaries. This way, the model will have more context, and segmentation will be more accurate near the object boundaries.
2. Segmentation of Complex Objects Made Easy
Regardless of the number or complexity of objects in the image, Smart Tool can manage them with its neural network algorithms. The accuracy and satisfactory performance of the algorithm are significantly influenced by your involvement in editing the mask.
3. One-click Segmentation
Thanks to the accuracy provided by the Smart Tool, objects with simple shapes often require no further adjustments. Simply create bitmap masks and enjoy!
4. Smooth Manual Editing
Neural networks can sometimes struggle with objects they were not trained on or due to other factors. If you need to manually edit the Smart Tool predictions, you can use the Brush, Eraser and Pen tools to remove unnecessary parts or add new ones.
To add a part of the object, select the shape and draw the missing part. To delete a part of the object, press and hold the
SHIFT key while repeating the previous steps. The algorithm is the same for both tools.
5. Fast Right-click Object Class Changing
For a seamless transition between object classes, initiate a right-click on the selected item and choose the desired option from the pop-up list. Beyond class changes, this menu provides convenient options for deleting or hiding an object, further streamlining your annotation workflow.
6. Switching Between Models
Neural Networks have various designs and were trained on diverse data, leading to different performances with different types of data. In some cases, it might be helpful to use different neural network models for the Smart Tool within the same dataset.
Switching between interactive segmentation models for the Smart Tool is easy. You can use the
ID button in the left toolbar.
Switch between interactive object segmentation models
On our Community instance at Supervisely, we prefer to integrate ClickSEG by default as we see it as the best solution for many different tasks. At the same time our users can run any pretrained model from interactive segmentation category (like Segment Anything, RITM, etc) on their own computers with GPU or train and and deploy interactive segmentation model on custom training dataset, creating unique customisation opportunities. The whole process is done in a few simple steps, making it as convenient and accessible as possible.
Our enterprise customers have the privilege of selecting their own interactive segmentation models through a convenient administration panel and share it with all users on their private instance. To get the best experience of the Supervisely platform check out our blog post for more details about how to train custom interactive segmentation model (plants segmentation use case in agriculture) and deploy it on your own server.
How to run any serving App
Train models the way you need
Popular neural network models, such as Segment Anything, were trained on a large portion of data; however, these models are general-purpose and may face difficulties with specific tasks, such as segmenting agricultural images in low quality or detecting road cracks.
In Supervisely, you have the capability to train the Smart Tool specifically for your data directly within the platform, and the best part is, no coding is required! We're actively experimenting with this feature and are impressed by the results. Feel free to explore other blog posts dedicated to this topic.
Tailor your settings to suit your needs by adjusting opacity levels, selecting random colors for objects while annotating and and many other useful features.
Control the Smart tool
0 more efficiently with
|Detect object inside BBox -
CLICK LBM + CLICK LBM
|Add positive point -
|Add negative point -
SHIFT + CLICK LBM
|Remove feedback point -
ALT + CLICK LBM
|Drag rectangle -
ALT + HOLD LBM
|Drag rectangle -
ALT + ARROW KEYS
|Finish & start new -
LBM stands for Left Mouse Button
ID button to change model. You can deploy more interactive segmentation models in Ecosystem.
|Deployed model: Example - Serve ClickSEG#47962.
MOUSE WHEEL. Hold
RIGHT MOUSE BUTTON to move scene.
In summary, the Smart Tool is a fantastic AI tool for precise segmentation. It's versatile and easy to use, making it popular in different fields. Try it for free on the Supervisely Platform and see how it can upgrade your object segmentation tasks.
Supervisely for Computer Vision
Supervisely is online and on-premise platform that helps researchers and companies to build computer vision solutions. We cover the entire development pipeline: from data labeling of images, videos and 3D to model training.
The big difference from other products is that Supervisely is built like an OS with countless Supervisely Apps — interactive web-tools running in your browser, yet powered by Python. This allows to integrate all those awesome open-source machine learning tools and neural networks, enhance them with user interface and let everyone run them with a single click.