Intelligently verified, crowdsourced, terrorist searching & tracking.

Silicon Valley


Technologies Used

Amazon AWS
Apple Core Motion
Amazon Lambda
MapKit Directions
Advanced Python
amazon alexa
GovTech Alliance
Amazon API
Amazon Alexa

Project Team

Sohan Vichare

This team is looking for

Product Manager Investor


(Scroll all the way down for business model) Why? “According to the CIA World Factbook in 2014, Syria had 87 mobile phones per 100 of the population” ( That compares to about 123 per 100 in cities like London. The fact that camera-equipped smartphones were still this prevalent in a terror-stricken region, in addition to GovTech’s challenge, got us thinking - what if we could somehow use all of those cameras to track militants, helping out both intelligence agencies and civilians alike. The app we built would have to both incentivize civilians to participate, bring together camera data in a meaningful way, and be proof to fraud (so assailants could not publish misinformation). Thus, Sentry. What? Sentry is an app that finds, identifies, and tracks armed militants through crowdsourcing. Citizens will be attracted to it because it increases their public safety though its danger-avoiding pathfinding, and intelligence communities will be attracted to it because of the massive amount of crowdsourced data + intelligent arms verification using Keras ML + identification and tracking using SSIM. Through the iOS App, citizens can: 1. Point their camera at armed militants and tap the screen to report them. To prevent people from reporting falsely, the images are analyzed to look for 1) people and 2) firearms. This is done through a model trained through ImageNet. 2. Access militant data to know if they are in a high danger area. 3. Access a pathfinding tool to find the safest walking/driving path from their current location to another. The pathfinding algorithm avoids high-danger areas (areas with high concentrations of militants) within reasonable bounds. 4. Get a notification telling them to call an Uber to get out of there using the Button SDK if they are in a high danger area! Through the Web App, intelligence officials can: 1. View the aggregate crowdsourced map data, alongside the type of predicted weapon and danger rating. 2. Track individuals captured by different citizens. Militants are recognized as the same militant through the SSIN (Structural Similarity Index) algorithm, which allows tracking of the same person across multiple cameras. Through the Alexa Skill, citizens and intelligence officials can: 1. Vocally check on the safety of their area. 2. Call an Uber to get out of there if it is dangerous! 3. Works with the app to update your safety status How? Sentry verifies whether a reported militant is actually a militant by 1) making sure they are a real person and 2) checking for a firearm. This is done by running predictions through two separate convolutional neural network models trained on data from the ImageNet dataset. Sentry identifies two images of one person as the same person by using the SSIM algorithm (Structural Similarity Index). Amazon AWS Lambda was used to create and deploy the Alexa skill. Button’s iOS Swift SDK was used to enable functionality to escape quickly using an Uber if a civilian was in a high danger area. OpenCV’s HOG classifier was used to filter out people to be sent to the two server-side neural networks on iOS. What next? We’re proud to have all the basic functionality we wanted up and running! But there’s still a lot more to do. For starters, the person detection and weapon detection combined take between 7-10 seconds to run. This would need to be significantly cut down in the future by training smaller nets with more varied data and running them on an accelerated GPU. Second, we would like to improve the caliber of the identification algorithm (SSIM), which relies on features such as clothes, something which can be easily changed. This would be done by implementing facial-based tracking. Business Model: For civilians, the service is completely free. We have two revenue streams, one: subscription based API calls for intelligence agencies, and two: licensing to government agencies. Pricing for API calls is as follows: 0-5K Calls: Free 5K-15K Calls: $9.99 15K-30K Calls: $29.99 Unlimited Calls: $49.99 Our customer base consists of regular civilians, intelligence agencies, and government agencies. The app is highly scalable; as more people are actively tracking and updating statuses, the more accurate the location tracking and civilian data.