Financial Fraud Detection: Machine Learning vs Traditional Rule-Based Systems – A Comparative Analysis of Model Performance

Abstract

With the rapid development and widespread use of electronic payments and digital financial transactions, financial fraud has become increasingly common. It has already caused significant threats and losses to customers and financial institutions. How to detect financial fraudulent activities in a timely and accurate manner has become crucial, as it can not only reduce financial losses but also improves customers’ trust in the financial system. Traditional rule-based systems rely manually defined thresholds and patterns, which can identify anomalous behaviors to a certain extent, but it is difficult to identify complex fraudulent behaviors in the face of continue changing fraud patterns. In contrast, machine learning provides a dynamic, data-driven approach that is more adaptive and accurate, and it can learn hidden patterns from large datasets.

This study uses a simulated credit card transaction dataset generated by the Sparkov tool, trained and tested under the same dataset conditions. Standard pre-processing steps were applied to compare a traditional rule-based system with four machine learning models, including logistic regression, decision trees, random forests and support vector machines SVM, and to use metrics such as accuracy, precision, recall, F1-score and AUC-ROC to evaluate the model performance. The experimental results show that among all the methods, the random forest model in machine learning performs the best in terms of overall evaluation, showing strong fraud detection ability and low false alarm rate. The decision tree model also performs relatively well with high recall. Although the SVM model has high recall, it falls behind the F1-score and precision. Logistic regression has relatively high recall but very low precision, meaning it produces many false positive. On the other hand, the traditional rule-based system is relatively highly accurate, but its precision and recall are poor, which means it has a high false alarm rate and the actual detection effect is limited.

Overall, the results show that machine learning models, especially Random Forests, perform well compared to the traditional rule-based system under the same data conditions. The findings indicate that machine learning models outperform traditional rule-based systems under the same conditions and are more suitable for financial fraud detection in practice.