How To Tag Multi-Camera Video Segments For Action Recognition In Computer Vision | Full Tutorial
Guide on video annotation, tagging and management tool tailored for multi-camera videos in Supervisely
Table of Contents
The Role of Video Segment Management
When it comes to monitoring and managing data, especially in crowded and dynamic environments such as airports or traffic roads, the role of video management technology is crucial. Sometimes it is important to keep track of certain areas in videos to focus on critical moments or areas with high precision. This segmented approach not only streamlines the process of monitoring and analysis but also increases efficiency and accuracy.
A segment refers to a section of a video that is demarcated by its starting and ending frames, primarily utilized for recognizing actions within the video. A tag segment acts as an identifier for these segments, applicable either to an individual video or across multiple videos in a multiview setup, with each action commencing and concluding at designated timestamps.
In multiview video analysis, one of the key challenges is ensuring the synchronization of multiple camera feeds. This synchronization affects how action recognition is conducted, which can be on a single video stream or across multiple synchronized or unsynchronized streams. The Mark Attributed Segments on Multi-camera Videos application is designed to handle a variety of use cases, thereby enabling flexible and efficient action recognition across diverse video setups.
Check out this 3-minute video guide to learn how to use multi-camera action recognition app in Supervisely. It allows to work with both unsynchronised and synchronized video pairs, visualize spatial annotations on top of the video streams for highlighting the areas on videos from stationary cameras to simplify the work of labeling team.
Video Segments analysis is utilised in various industries, including:
Enhanced Vehicle Monitoring and Toll Evasion: Using video segment analysis, it is possible to identify and count vehicles entering and exiting toll roads. Custom attributes can be tagged to each vehicle, including the time of entry/exit, type of vehicle, and even specific details like license plate numbers if combined with recognition technologies.
It is also possible to monitor specific areas, such as the toll booth lanes for identifying toll evasions or security breaches, tagging these incidents for further action. Over time, the collected data can provide valuable insights into traffic patterns, peak hours, and usage trends on toll roads.
Retail Customer Behavior Analysis and Security: Understanding how customers move through the store, where they spend time, and what captures their interest is crucial for sales and layout optimization. By labeling different areas of the store and analyzing foot traffic, retailers can gain insights into popular products and design more effective store layouts. This can also aid in managing and distributing staff according to busy areas and times.
Video segments managing tool can enhance security measures by tagging and alerting staff to suspicious behavior or unauthorized access to restricted areas. Integrating facial recognition or other advanced features can help identify known shoplifters or alert staff to unattended bags or packages.
Airport Security and Queue Management: Airports are high-security zones. By tagging and monitoring video feeds, security personnel can be alerted to unattended baggage, unauthorized access in restricted areas, or other security breaches. The ability to draw increased attention regions allows for close monitoring of sensitive areas such as check-in counters, security checkpoints, and boarding gates.
Moreover, analysis of various video segments can be employed to monitor and analyze the flow of people through various checkpoints. By understanding traffic patterns and identifying bottlenecks, airport management can allocate resources more effectively, perhaps opening more counters or redirecting staff during peak times, thus improving the overall traveler experience.
To put it simply, video segment management technology, especially the kind that can handle lots of video from multiple cameras and division of their areas of attention, is really useful. Today, we're looking at Supervisely App designed specifically to address the task of tagging and managing video segments, Mark Attributed Segments on Multi-camera Videos, created by the Supervisely Development Team.
Developer Portal: Your Customization Hub
The Developer Portal is an important element of the Supervisely Ecosystem, providing comprehensive tutorials, documentation and insights on the process of automating and tailoring Supervisely for integration with your software. With the help of the Developer Portal, users can create their own applications or modify existing ones. You can learn how to automate and customize Supervisely Platform, smoothly integrate it with your software and build custom computer vision apps that perfectly fit your requirements. As a result, applications on the platform are constantly evolving and adapting to meet the ever-changing needs of industries around the world.
Mark Attributed Segments on Multi-camera Videos App
This App in fact represents a specialized labeling tool with a dual-panel interface for users to effectively tag, manage, and compare segments of paired videos in a side-by-side view. Designed to streamline video data analysis, this application is perfect for accurately segment labeling and management in research, surveillance, and complex video analysis tasks.
Mark attributed segments on multi-camera videos
Tag segments (begin and end) with custom attributes on single or multiple videos in dual-panel view
Recent enhancements of the App include information widgets, improved video controls, sorting capabilities, and the advanced feature for drawing masks with regions on videos.
The application operates in two modes:
Single Video Labeling for analyzing the same video across different times or conditions
Pair Video Labeling for comparing two distinct video segments
The Mark attributed segments on multi-camera videos application can go beyond conventional video tagging by integrating advanced labeling features such as bounding boxes, segmentation and various other functionalities. This labeling interface is designed for flexibility, allowing it to handle a wide range of tasks.
How to Use
Here's a guide on how to utilize the Mark attributed segments on multi-camera videos application. For deeper understanding of the application functionality, please see the full user manual available in the application description and try it for free! on the Supervisely Platform.
1. Prepare Tags
Start your project by setting up tags. These tags can vary in type (none, text, number, one of...) and are crucial for attaching attributes to segments. They enrich each segment with valuable context and specificity.
It is recommended not to change existing tags. To use the tags in the app correctly, you should prepare tags before launching the app. You can also add new tags later (don’t use existing or removed tag names). Just restart the app to use created tags.
2. Launch and Select
Run the application either from the Ecosystem or directly through your video project or dataset's context menu. Choose video datasets for labeling. The application's dual-panel view improves the labeling process by providing a comparative view. Simply navigate through the video table and select the left and right videos you aim to label by clicking on the corresponding buttons.
3. Create Regions
After selecting the videos, click on the
DRAW REGIONS ON VIDEO button to display a mask with figures.
Algorithm for creating masks with regions on a video:
Define the necessary number of classes that correspond to the required region names (Ex: Region 1; Region 2).
Select the class with required name and draw regions on the first frame of the video.
If you've introduced new classes, a quick restart of the app might be needed to reflect these changes.
Preview and adjust your segments at any time by clicking the
SHOW ALL SEGMENTS button.
4. Tag and Manage
Now it's time to add tags to your segments and manage them efficiently. Users can easily manage segments with existing tags or create new ones. Customize segments with unique identifiers, tags and timestamps for a thorough labeling process.
To start tagging, i.e. create or delete tags segments, use the
START SEGMENTS TAGGING button.
To manage selected segment attributes, simply click the
EDIT button on the segments table.
5. Results Storage
All your segmented data is efficiently stored in JSON format in Team Files. This format captures metadata details such as video IDs, names, timestamps, and user information for easy access and collaboration among team members, learn more in app readme.
Note: All segment attributes are created and added to TagCollection object using the meta tags of current project and stored in a file after serialization to json. All segment attributes are extracted from json files and serialized into TagCollection object to manage them.
Management of Segments on Video plays an important role in administrating and understanding different spaces.
We've built Supervisely platform to be customized easily by our users. This means that users can change the way they manage video segments, add special features for specific tasks, or even create something entirely new within the ecosystem. This customizable aspect is something we're proud of!
As we continue to improve and expand the capabilities of our technology, we remain focused on making it adaptable, powerful and easy for everyone to use. With these advanced features, particularly available in our Enterprise version, we aim to help our users not only meet their immediate needs, but also be ready to adapt and grow with whatever new challenges or changes their areas may bring.
Supervisely for Computer Vision
Supervisely is online and on-premise platform that helps researchers and companies to build computer vision solutions. We cover the entire development pipeline: from data labeling of images, videos and 3D to model training.
The big difference from other products is that Supervisely is built like an OS with countless Supervisely Apps — interactive web-tools running in your browser, yet powered by Python. This allows to integrate all those awesome open-source machine learning tools and neural networks, enhance them with user interface and let everyone run them with a single click.