Object detection on images with high accuracy is an essential issue for many application areas, especially real-time applications. Convolutional neural networks, on the other hand, are deep learning-based methods that have been used in object detection applications in recent years and have achieved high accuracy. However, although classical convolutional neural networks can detect objects on original images with high accuracy, their performance may be insufficient on images where adversarial attacks such as FGSM, PGD, and APGD are applied. To overcome this problem, different models and pre-processes are developed for object detection on attacked images. However, the performance of the models may vary for attacked and non-attacked cases. Therefore, it is necessary to determine whether the attack occurred and select the most successful model according to the case. To solve the problem mentioned above, detect whether there is an adversarial attack on the images using convolutional neural networks has been performed in this study. Within the scope of the study, YOLO v5 and Faster R-CNN models were trained for the adversarial attack detection task with and without transfer learning. Experimental results show that the Faster R-CNN model with transfer learning achieved the most successful result among the four models with an f1 score of 0.971.