How To Tag Multi-Camera Video Segments For Action Recognition In Computer Vision | Full Tutorial

The Role of Video Segment Management

When it comes to monitoring and managing data, especially in crowded and dynamic environments such as airports or traffic roads, the role of video management technology is crucial. Sometimes it is important to keep track of certain areas in videos to focus on critical moments or areas with high precision. This segmented approach not only streamlines the process of monitoring and analysis but also increases efficiency and accuracy.

A segment refers to a section of a video that is demarcated by its starting and ending frames, primarily utilized for recognizing actions within the video. A tag segment acts as an identifier for these segments, applicable either to an individual video or across multiple videos in a multiview setup, with each action commencing and concluding at designated timestamps.

In multiview video analysis, one of the key challenges is ensuring the synchronization of multiple camera feeds. This synchronization affects how action recognition is conducted, which can be on a single video stream or across multiple synchronized or unsynchronized streams. The Mark Attributed Segments on Multi-camera Videos application is designed to handle a variety of use cases, thereby enabling flexible and efficient action recognition across diverse video setups.

Video Tutorial

Check out this 3-minute video guide to learn how to use multi-camera action recognition app in Supervisely. It allows to work with both unsynchronised and synchronized video pairs, visualize spatial annotations on top of the video streams for highlighting the areas on videos from stationary cameras to simplify the work of labeling team.

Video Segments analysis is utilised in various industries, including:

• Enhanced Vehicle Monitoring and Toll Evasion: Using video segment analysis, it is possible to identify and count vehicles entering and exiting toll roads. Custom attributes can be tagged to each vehicle, including the time of entry/exit, type of vehicle, and even specific details like license plate numbers if combined with recognition technologies.

  It is also possible to monitor specific areas, such as the toll booth lanes for identifying toll evasions or security breaches, tagging these incidents for further action. Over time, the collected data can provide valuable insights into traffic patterns, peak hours, and usage trends on toll roads.

• Retail Customer Behavior Analysis and Security: Understanding how customers move through the store, where they spend time, and what captures their interest is crucial for sales and layout optimization. By labeling different areas of the store and analyzing foot traffic, retailers can gain insights into popular products and design more effective store layouts. This can also aid in managing and distributing staff according to busy areas and times.

  Video segments managing tool can enhance security measures by tagging and alerting staff to suspicious behavior or unauthorized access to restricted areas. Integrating facial recognition or other advanced features can help identify known shoplifters or alert staff to unattended bags or packages.

• Airport Security and Queue Management: Airports are high-security zones. By tagging and monitoring video feeds, security personnel can be alerted to unattended baggage, unauthorized access in restricted areas, or other security breaches. The ability to draw increased attention regions allows for close monitoring of sensitive areas such as check-in counters, security checkpoints, and boarding gates.

  Moreover, analysis of various video segments can be employed to monitor and analyze the flow of people through various checkpoints. By understanding traffic patterns and identifying bottlenecks, airport management can allocate resources more effectively, perhaps opening more counters or redirecting staff during peak times, thus improving the overall traveler experience.

To put it simply, video segment management technology, especially the kind that can handle lots of video from multiple cameras and division of their areas of attention, is really useful. Today, we're looking at Supervisely App designed specifically to address the task of tagging and managing video segments, Mark Attributed Segments on Multi-camera Videos, created by the Supervisely Development Team.

Mark Attributed Segments on Multi-camera Videos App

This App in fact represents a specialized labeling tool with a dual-panel interface for users to effectively tag, manage, and compare segments of paired videos in a side-by-side view. Designed to streamline video data analysis, this application is perfect for accurately segment labeling and management in research, surveillance, and complex video analysis tasks.

Supervisely App

Mark attributed segments on multi-camera videos

Tag segments (begin and end) with custom attributes on single or multiple videos in dual-panel view

Run in Supervisely GitHub

Recent enhancements of the App include information widgets, improved video controls, sorting capabilities, and the advanced feature for drawing masks with regions on videos.

The application operates in two modes:

Single Video Labeling for analyzing the same video across different times or conditions

Pair Video Labeling for comparing two distinct video segments

The Mark attributed segments on multi-camera videos application can go beyond conventional video tagging by integrating advanced labeling features such as bounding boxes, segmentation and various other functionalities. This labeling interface is designed for flexibility, allowing it to handle a wide range of tasks.

How to Use

Here's a guide on how to utilize the Mark attributed segments on multi-camera videos application. For deeper understanding of the application functionality, please see the full user manual available in the application description and try it for free! on the Supervisely Platform.

1. Prepare Tags

Start your project by setting up tags. These tags can vary in type (none, text, number, one of...) and are crucial for attaching attributes to segments. They enrich each segment with valuable context and specificity.

2. Launch and Select

Run the application either from the Ecosystem or directly through your video project or dataset's context menu. Choose video datasets for labeling. The application's dual-panel view improves the labeling process by providing a comparative view. Simply navigate through the video table and select the left and right videos you aim to label by clicking on the corresponding buttons.

3. Create Regions

After selecting the videos, click on the DRAW REGIONS ON VIDEO button to display a mask with figures.

Algorithm for creating masks with regions on a video:

1. Define the necessary number of classes that correspond to the required region names (Ex: Region 1; Region 2).

2. Open Supervisely Video Labeling Tool.

3. Select the class with required name and draw regions on the first frame of the video.

   

4. If you've introduced new classes, a quick restart of the app might be needed to reflect these changes.

Preview and adjust your segments at any time by clicking the SHOW ALL SEGMENTS button.

4. Tag and Manage

Now it's time to add tags to your segments and manage them efficiently. Users can easily manage segments with existing tags or create new ones. Customize segments with unique identifiers, tags and timestamps for a thorough labeling process.

To start tagging, i.e. create or delete tags segments, use the START SEGMENTS TAGGING button.

To manage selected segment attributes, simply click the EDIT button on the segments table.

5. Results Storage

All your segmented data is efficiently stored in JSON format in Team Files. This format captures metadata details such as video IDs, names, timestamps, and user information for easy access and collaboration among team members, learn more in app readme.

Conclusion

Management of Segments on Video plays an important role in administrating and understanding different spaces.

We've built Supervisely platform to be customized easily by our users. This means that users can change the way they manage video segments, add special features for specific tasks, or even create something entirely new within the ecosystem. This customizable aspect is something we're proud of!

As we continue to improve and expand the capabilities of our technology, we remain focused on making it adaptable, powerful and easy for everyone to use. With these advanced features, particularly available in our Enterprise version, we aim to help our users not only meet their immediate needs, but also be ready to adapt and grow with whatever new challenges or changes their areas may bring.