Violence detection has remarkable importance in developing advanced computerized video surveillance systems. There is an increasing demand for computerized video surveillance with increasing threats in society making it infeasible for manpower to monitor them constantly. Generally, detecting violence in a crowded locality is challenging because of the swift movement, overlapping features due to obstruction, and scattered backgrounds. Fortunately, the current Deep Learning techniques that are developed can detect the anomalies to an extent. Violence detection is very fast and can be used to filter the normal surveillance videos, spot or take note of the individual causing the abnormality, and then send the videos containing the anomalies to be scrutinized by officials. The goal is to come up with a better violence detection system that recognizes the violence and evokes an alarm so that immediate assistance can be provided. This paper proposes a new deep learning model that uses the concept of transfer learning for violence detection and also identifies the aggressive individuals by learning the detailed features in the videos. MobileNet model, in combination with LSTM, yielded an accuracy of 94.9% and thereby, proving its superiority over other experimented ImageNet models.