The short message service is one of the most commonly used communication channels. Increasing number of short messages that are used for many different purposes including personal, advertising, promotion, event notification, sales approval, etc. makes it difficult to keep track of messages and to find the target message quickly in the message box. On the other hand, another impotant problem is the spam messages filling the message box unneccasserily. In this study, a novel Multi-tier Filtering Architecture is proposed to classify Turkish Short Messages and to create a smart SMS box. This architecture is also implemented on an Android application. Received messages and old messages are divided into 5 different categories under personal, commercial, otp codes, reminders and spam messages with the help of the proposed architecture. This architecture consists of 4 tiers including Black List, Regex, Machine Learning, and White List filters. The performance of Naive Bayes, Bayes Net, J48 and Random Forest algorithms were examined and Random Forest was preferred for mobile application due to its success of 87%. Thanks to the proposed multi-tier system architecture, the classification success rate has been increased to 93% while the message classification time is reduced. In addition, the selected features, the use of the Zemberek library and the acquisition of the types and roots of the words within the message played an important role in the promotion of success.