How to detect weapons with video analytics?

Hello and welcome to another episode of

video

analysis 101. We're doing this live today as we do every month, last month we were talking about AI Dark Silicon with AMD to talk about how GPUs are getting more powerful but less efficient and why this is a problem if you haven't seen this head over to youtube to the

video

analysis 101 youtube channel watch the video don't forget to subscribe while you're there so you don't miss anything else today we're talking about

weapons

Detection of how to

detect

weapons

with vegan analysis is definitely a controversial topic and also a very popular one and today we are trying to get to the bottom of it and I am happy that you are joining us here.

He is the CEO and co-founder of v sites. Welcome. Me, hello Florian, hello everyone, it's a pleasure to be here and share some of our knowledge and opinions on this very important topic, unfortunately it's become important in the last few months, yes, and I think it's even more important to let things go clear. what's possible what's not possible how it works and get some background there so maybe to start off can you give us an overview of what v sites do in general and why it's different than other video

analytics

? products, yes, this site is an almost 7 year old video analysis company that was established with the vision of developing a video analysis technology or technology that can not only classify objects, but basically our vision is that we will be able to understand live video content. very similar to how humans understand and interrupt visual content, primarily with a computer vision, with that vision, we actually went and founded the company and it took us several years until we managed to create that technology and that technology allowed us to develop and Introducing very advanced video analysis capabilities to the market and basically all the capabilities necessary to process video in real time and in an uncontrolled environment, that is, not in a closed room but outdoors, is a great challenge, even The fact to the point I regret that it has become a real commercial product, so our behavioral analysis engine uh is able to create an interior that eventually provides very high performance to our customers.

I hope that's clear, so thank you. What I took away and what I always found interesting is that you focus on behavior, recognition, using behavior for video analysis, and this is also why I thought it was interesting to talk about weapons

detect

ion because you have a different approach in all aspects. Other solutions I see out there generally try to detect weapons based on how they look in individual images on individual frames, which means you have to train them on basically every type of weapon there is and generally seems to me just like the technology. It's still in its infancy as far as gun detection goes, it's still not something that's commodity and very easy, so yeah, maybe we can tell each other how gun detection is done and how it works differently than other solutions exactly before that.

I'll just, you know, give a brief explanation that will really show what behavior recognition means how it's different from everything else out there because I don't want it to, it's still going to be a buzzword, so behavior recognition is the ability. of recognizing the behavior of the object and the context in which that behavior actually is, you know that the combination of recognizing the behavior of the object is taking place, so if we're talking about humans, okay, it's not just of being able to classify humans within the video stream or some attributes you know. the human, but basically I understand the behavior of the human, okay and this is how this is our engine, everything that you know we are doing, even recognizing a person, the color of clothing, we are seeing it as a behavioral characteristic , is not a person. with just uh we're not trying to figure out if that clause reads, but we're looking at it as the person who's like wearing a red shirt, now the gist here is we're looking at the sequence of frames uh. not just a single frame and also about weapons, so now let's talk about weapons, detecting a weapon is a very, very challenging task, a challenging task, because weapons are burst type, whether we are talking about guns, rifles or knives, They are the first.

They are very small, it's okay when a person is standing in front of the camera, although you know it takes a very small number of pixels and you can easily confuse them, for example you can easily confuse the gun with the mobile phone, now everyone is involved. The video

analytics

industry knows that, in addition to everything that is important to the customer, the system is not going to be noisy, so if you want to create a commercial product that is basically capable of detecting the weapon, you have to have it very, very accurate . which means especially in the false positive rate, otherwise you know customers won't use it, so originally when we started developing the gun detection capability we didn't originally think about the shooter use case active, which unfortunately again became very popular. but the main use case was robbery and what we found really works on a large scale and when I talk about robbery on each site we have at least a couple hundred cameras so in that type of composition what we really found that is working and has a very low false positive rate is that we are not actually looking for the gun, but rather the behavior and the behavior is a person actually holding a gun in a threatening position.

This was our initial implementation. before we actually upgraded it to an active shooter use case, uh, I can detail that later and when we really went in that direction, then the system became very, you know, accurate, well, the positive force number was really very low. Today we're talking about one false positive for every nine thousand detections of a person, which is really good, it means, you know, on a typical site, one first alert per week and that's how we actually went to market with this guy of capacity because we realized that if you try to search for the gun, it's out in the open again, it's becoming an impossible task, which creates a lot of noise and you can't.

You know, crying wolf every other day, not every hour, but even every other day, is unacceptable because, I can imagine, I can imagine that something is much more likely to look like a weapon than someone pretending to wield a weapon that barely happens. ever, so I guess it's a lot more robust if you actually detect the behavior of having a threatening pose or something like that and then detect a weapon, so it seems very logical to me that it's more robust, yeah, think of it like that, you know, in computer vision. We know that you eventually feed the algorithm with data, so first of all, the uniqueness of our system is that we train the system on video clips.

Now in the regular implementation, which is really common in the industry, when you want to detect a weapon, you go into the image and it actually marks where there is a weapon, we actually feed the algorithm a lot more data because we know how to tag the person. plus the gun, so there is another anchor for the algorithm to make more conscious decisions and eventually become much more accurate. and again, this is the name of the game in our industry, precision is more important than the type of features you have, but the feature you sell has to work very, very precisely, yes I agree, maybe let's start use it. cases in our preparation when we spoke before you, you talked about the different use cases between robbery and active shooter and to be honest I never thought about that, but you're right, it's a completely different use case and I think it's important to distinguish them .

Also when looking for solutions, maybe you can describe what the difference is between a robbery and an active shooter from a technical perspective and the whole setup etc. Yes, there are several differences, some of them are technical, some of them are operational, for example for Robbery: what we implement initially is only to search for a person who is holding a weapon in a threatening position because, for example, if you are searching in a bank, okay, a bank branch, so we didn't want to generate alerts about the gun. that the guard is fine, so even if he just knows, holding it in his hand for a second, then we train the algorithm to only alert when a person is holding a gun in a threatening position if he is not holding the gun. in a threatening position, the system should ignore it and again we did everything for the sake of accuracy because the system was implemented on 100s and then thousands of cameras, so you have to be very, very precise, also when you speak when you are. looking, you know when you're going to issue the alert in case of a robbery, you can even send the alert after 20 seconds and it'll still be fine because it's, you know, it's a long event, unfortunately, now the active shooter is completely You know, something different, first, what we really added to the system is that you know anyone who is holding a gun in any kind of position, you have to catch them even if the guy you know comes out. his skull at the moment the weapon that you know in his hands or the weapon is visible on the person, you have to detect it and second, this is the most challenging change that we added to the system, you have to do it as quickly as possible because every second counts well and also from the furthest distance you can, it will still be accurate because the robbery in many cases the person is close to the camera in most scenarios, although you would like to have an idle shooter. to capture the person in most cases when they are actually outdoors and as far away from the camera as possible, so if you are talking about the technicality, the size of the object is different, the behavior is different and the delay of the alert should be as small as possible, so this is almost as our product manager says, totally different product and in the last 12 months we have invested a lot to also respond to the active shooter scenario that has those unique characteristics. uh, features that are very, very different from how we initially designed the system.

To me, it sounds similar to facial recognition, where people just hear facial recognition and assume that if the system can do facial recognition, it can use it anywhere, but it's a completely different scenario. If you use facial recognition to unlock your phone or verify your passport, finding millions of people in the crowd of millions of other people are completely different use cases and just because one works doesn't mean the other works basically like you. What we're seeing is that just because a system or a big scan can detect weapons doesn't mean it's usable for an active shooter scenario exactly if it's going to take you, you know, I don't know, 40 seconds to alert about a weapon and you're just going to do it. when the person is facing very, very close to the camera.

I'm not saying it won't help, but you'd actually be much better off doing it when the person is away from the camera. camera, I don't know like 40 uh, 40 meters from the camera and you will send the alert after a few seconds, not 40 seconds, so yeah, again we would like to create a technology that helps as much as possible to, you know, prevent the damage that can occur with this kind of horrible, horrible scenario and of course it's even more important to understand what you are capable of and what you are not able to do so we make the right decisions, one thing I wanted to ask you: are you saying that is detecting or using the behavior or context.

What does it mean in terms of performance? It requires larger servers than other video analytics. Can you give us a typical idea of ​​how many cameras you can typically run? server, yes, again, when we actually come to design a product that can be sold in the mass market, of course we have to consider all aspects including the hardware footprint, which we invested a lot in and continue to invest in from day one , today in our system. has several types of configurations that we have even for the active shooter, we actually came up with a small server configuration, a server that can run four to eight streams because before that and that server, specially designed for a small educational center for for schools that have that many cameras looking at the surroundings, they don't need to connect to the system, the whole set of cameras and currently our Harvard fruit printing is about, you know, our system runs on nvidia gpu.

This is the technology we chose a few years ago and so far we are very happy also with all the support we receive from nvidia and we are able to run between 15 and 20 streams for one gpu and we are currently using the t4 gpu I know the new series exists which we are currently running in our lab, but it is not ready yet. I think it will be ready next quarter, but currently our recommended gpu is t4 and the basic server that is most popular isfingers. t4 that can run again between 30 and 40 40 streams and if you want to translate that to two dollars, we're talking about seven to eight dollars a month per camera, so this is the hardware footprint that we currently have, uh uh, that our The product It is currently consumed and as I understand it it is very good and we continue to invest in it and every half year we are managing to reduce it more and more.

Again, so that the total cost of ownership for our customers is as low as possible, okay, great , this is actually a pretty common number, I would say 15 to 20 streams per gpu and it can definitely be used for these types of use cases because you usually don't put it on every camera, I guess maybe to finish, Can you give me your opinion again on active shooter detection? Do you think that in the future video analysis will be good enough to use? For active shooter detection, do you think this could be the only technology or would you combine it with other technologies?

Are there any recommendations you would give to end users looking for solutions? So again, the active shooter use case is a really complex one, you know, on the first philosophical side, you know, the best solution would be, you know, gun control in education, but leave that aside considering that the event is happening, so in my opinion there is no single solution. eventually it's a combination of solutions, well, you know, some of them are technological and some of them are operational like what we have today in schools, all those drills where staff and students are being trained on the technological side.

It's going to be a combination, visual detection of a gun is just another type of safety net that we're going to add and protect our children, which is the most valuable thing that we would like to put in place for all of us. as much of a safety net as we can so that the strength of the visual recognition that you can know is applied when the person is approaching the site and before we even start to do anything in case we actually detect the weapon in case it is visible and combine it with smart access control and we have the active trigger system that is working on the audio.

All of these are just an extra layer of security that we will add to prevent this completely. or mitigate the damage, so I don't think there is a single, non-technological and definitely not, non-operational solution, but you know, if I'm thinking about the root cause, then it will probably also help to deal with the root cause of that phenomenon that it's much more complex from technology, yes, I think that's a very good final statement that as always in our space, it's really a combination of many things, technology is definitely not the only answer to all problems it's really a piece of the puzzle that needs to be combined with standard operating procedures with prevention with education with training and technology can only be a tool to help, but it needs to be integrated into an overall strategy and that's true for assets. shooters like with anything else in the security industry, really nice, closing statement, thank you, thank you for joining us, that's all for this moment, we're trying to keep these things short, so we're trying to compress the information like In as much as possible, if you have any other questions you can of course contact v sites and I guess also myself on LinkedIn or the website.

We have the website. I believe in comments too and subtitles. So take a look if you want more information. That's all for this month. We have something very good prepared for August for everyone who is not on vacation. Thanks for joining and until next time. Thanks Aaron. Thank you all. Bye bye. you