The Use of Machine Learning to Detect Financial Transaction Fraud: Multiple Benford Law Model for Auditors

Background: Fraud in financial transaction is at the root of corruption issues recorded in organization. Detecting fraud practices has become increasingly complex and challenging. As a result, auditors require precise analytical tools for fraud detection. Grouping financial transaction data using K-Means Clustering algorithm can enhance the efficiency of applying Benford Law for optimal fraud detection. Objective: This study aimed to introduce Multiple Benford Law Model for the analysis of data to show potential concealed fraud in the audited organization financial transaction. The data was categorized into low, medium, and high transaction values using K-Means Clustering algorithm. Subsequently, it was reanalyzed through Multiple Benford Law Model in a specialized fraud analysis tool. Methods: In this study, the experimental procedures of Multiple Benford Law Model designed for public sector organizations were applied. The analysis of suspected fraud generated by the toolkit was compared with the actual conditions reported in audit report. The financial transaction dataset was prepared and grouped into three distinct clusters using the Euclidean distance equation. Data in these clusters was analyzed using Benford Law, comparing the frequency of the first digit’s occurrence to the expected frequency based on Benford Law. Significant deviations exceeding ±5% were considered potential areas for further scrutiny in audit. Furthermore, the analysis were validated by cross-referencing the result with the findings presented in the authorized audit organization report. Results: Multiple Benford Law Model developed was incorporated into an audit toolkit to automated calculations based on Benford Law. Furthermore, the datasets were categorized using K-Means Clustering algorithm into three clusters representing low, medium, and high-value transaction data. Results from the application of Benford Law showed a 40.00% potential for fraud detection. However, when using Multiple Benford Law Model and dividing the data into three clusters, fraud detection accuracy increased to 93.33%. The comparative results in audit report indicated a 75.00% consistency with the actual events or facts discovered. Conclusion: The use of Multiple Benford Law Model in audit toolkit substantially improved the accuracy of detecting potential fraud in financial transaction. Validation through audit report showed the conformity between the identified fraud practices and the detected financial transaction.


I. INTRODUCTION
The field of audit is experiencing a significant shift toward digital transformation through information technology.Traditional audit practices are evolving, moving towards continuous audit through the automation of accounting and audit procedures with the aid of support systems [1], [2].In certain situations, auditors are required to use an audit support toolkit when evaluating a subject to ensure compliance with standards, expedite processes, and facilitate decision-making.This support toolkit comprises various models and information technology used by organizations A. Benford Law Theory Benford Law defines how digits are distributed from the first to fourth positions of a dataset (from the left).The first digit can only be one of nine possibilities, namely 1, 2, ..., 9, while the second to fourth digits can be 0. According to Benford Law [7], [9], and various study references [15], [16], the Benford Law is formulated using (1) and the distribution of numners in Benford Law can be seen in Table 1.
where Prob = probability or expected frequency, D1 = d = sequence of first digits.The logic behind using Benford Law is to identify unusual patterns in financial transaction activity [10], [10].When individuals engage in fraud practices, they may repeatedly use the same numbers or values.In this scenario, variations in the first and second digits, as predicted by Benford Law, could help auditors uncover intentional fraud transaction [12], [17].
The application of Benford Law consists of calculating the frequency of the first digit in the dataset and performing a z-statistical parameter test to check its compatibility with Benford expected frequencies.The results are then visualized using tables and graphs with a significant deviation of more than ± 5% strongly indicating potential fraud practices.The z-value is calculated using (2).
Where AP is the actual probability value of a digit occurrence, EP is the expected probability according to Benford Law, and N is the amount of data.
When the z-test shows a significant deviation of more than ±5%, audit process proceeds.However, in case there are no significant deviations, the dataset is divided into three data groups using K-Means Clustering algorithm for further analysis with Benford Law.

B. K-Means Clustering Algorithm
The next stage is to apply K-Means Clustering algorithm to the dataset when significant frequency deviations of more than ±5% are absent or to conduct more in-depth data analysis.The Euclidean Distance formula in (3) is used in K-Means algorithm [18]- [20].
Where dEuclidean (x,y) is the distance between data at points x and y, x represents centroid data points (x1, x2, . . ., xn), y denotes object data points (y = y1, y2, . . ., yn).k is the number of data clippers, n is the number of data attributes, and i=1 refers to an index of the data cluster.
Euclidean distance measures the similarity between data points and is calculated swiftly and efficiently.It is a heuristic function based on obstacle-free distances, such as the length of the diagonal line in a triangle.This formula assesses the distance between two points in Euclidean space, with each group or centroid marked by the center of gravity of the points.
K-Means Clustering algorithm categorizes financial transaction data into three clusters based on the similarities, with randomly determined cluster centers representing low, medium, and high-value financial transaction.These divided data clusters are then recalculated using Benford Law equation [21], [22], as shown in Fig. 1.

A. Data Understanding and Preparations
Data understanding and preparation were conducted to gather information, focusing on attribute completeness and financial transaction data accuracy.The financial transaction data from audited entities was retrieved from the official financial reporting system in Indonesia, referencing the SAKTI application (Sistem Aplikasi Keuangan Tingkat Instansi).This dataset comprised detailed cash flow reports in PDF format, which had to be converted into Excel format.Furthermore, it consisted of various financial transaction details, including transaction codes, dates, debit and credit transaction names, and financial transaction values.
The converted transaction dataset (PDF to Excel) was then prepared for analysis, and in this paper, the data preparation process was carried out using Python.The purpose of data preparation was to separate shopping (expense) transaction data from data irrelevant to fraud analysis.Excluded data types comprised bank transaction fees, fund withdrawal transaction, direct bank transfers, and budget refunds.The data preparation process included the following stages.
1) Dataset Fig. 3 showed the format of the financial transaction dataset slated for analysis.

3) Cleaning Data Transaction:
The cleaning process comprised eliminating rows of data that were not relevant to the analysis.For example, data such as bank administration fees, bank withdrawal transaction, bank transfer payments, and budget refunds were excluded, as they were not the focus of the suspected fraud analysis.4) Output Dataset for Analysis: Table 2 showed an example of the results of the data preparation process.The data was exported in Excel format and used in audit toolkit for fraud detection using Multiple Benford Law Model.

B. Implementation of Multiple Benford Law and K-Means Clustering Model
The application of Benford Law and K-Means Clustering to dataset included two distinct models.In this study, the law and the model were interconnected sequentially to improve the level of analysis.

1) The Application of Benford Law First Digit
This process began by determining the frequency of the first digit occurrence in the entire dataset.Subsequently, a suitability test was conducted to ensure that the dataset variables were applicable to Benford equation.The conformity test was carried out to assess the difference between the theoretical distribution assumed in Benford Law and the observed distribution of financial transaction values.When the Law did not yield a significant suitability value during implementation, the analysis was continued by categorizing financial transaction data into clusters using k-means clustering algorithm.Subsequently, data analysis was performed on the predefined data groups using Benford Law.2) The Application of K-Means Clustering Algorithm K-Means Clustering was used to group financial transaction data into three clusters representing low, medium, and high-value transaction.The number of clusters was determined based on the distribution of transaction values, taking into account the first digits from 1 to 9. Grouping the data into three clusters was essential to adjust the design of audit toolkit, which had been designed to accommodate three clusters.When there were fewer or more than three clusters, adjustments were made to audit toolkit.Consequently, this study aimed to evaluate the extent to which Multiple Benford Law Model, using three data clusters, could improve the accuracy of fraud detection based on Benford Law.

C. Evaluation and Validation Model
The evaluation and validation of the model comprised applying procedures to assess its accuracy and performance.The evaluation process was used to examine the calculation of the toolkit and analysis results, while model validation was applied to compare the outcomes of suspected fraud generated by the toolkit with the facts documented in audit report of the organization sampled in this study.

D. Deployment
The results were disseminated through a website and an audit toolkit equipped with analytical instructions for users.Additionally, information dissemination could be carried out on technology forums such as GitHub and through Indonesian audit association media.

IV. RESULTS
To obtain the results of suspected fraud analysis, the financial transaction dataset, prepared as previously described, was used in audit toolkit.The analysis followed these stages:

A. Application of Benford Law in Data Transaction
This study provided an example of applying Benford Law to five auditee organization units.The analysis showed that most of the data had similarities between the frequency distribution of the first digit according to Benford Law (benfordset) and the frequency distribution of numbers in the actual financial transaction data in the dataset.This implied that three of these units (Units 1, 3, and 4) did not show significant deviations ≥ ±5%.However, the remaining two units (Units 2 and 5) had a one in nine chance of potential fraud, namely transaction starting with 1 (Unit 2) and those starting with 3 (Unit 5).As shown in Fig. 4, analyzing the frequency of first-digit appearances offered insights into the potential for manipulation or fraud from Benford Law perspective.Under the circumstances, it was clear that auditors needed to exercise heightened caution when scrutinizing the data.Additional models needed to be explored in depth because the majority of the analyzed data did not show detected cases of fraud.
Based on the conditions stated above, the procedure adopted in this study guided auditors to classify financial transaction data using an audit toolkit that applied K-Means Clustering algorithm.Audit toolkit provided instructions for dividing data into three clusters with small, medium, and high transaction categories, ensuring a balanced distribution of data.This balanced distribution across digits 1-9 enabled subsequent analysis using Benford Law.Clustering data using k-means algorithm included the auditor repeatedly determining the centroids to achieve an optimal distribution of numbers 1-9.  3 showed the results of experiments on various auditee units across different audit periods.It was worth stating that audit unit was a public sector organization in Indonesia.The analysis results showed the continued relevance of Benford Law in the context of financial transaction in Indonesia, further supporting the need for ongoing evaluation and investigation.This paper provided a detailed case of fraud detection analysis using Multiple Benford Law Model, focusing on sample unit 1 for clarity.The procedures and results for units 2-5 were fundamentally the same and were not extensively discussed in this paper.

B. Grouping Dataset Using K-Means Clustering Algorithm
K-means clustering is an algorithm that requires k input parameters to divide a set of n objects into k clusters, ensuring high similarity in each cluster and low similarity with members of other clusters.The primary objective of K-Means is to minimize the total distance between elements in clusters and the respective centroids.K-Means algorithm can be implemented using these stages (a) selecting the desired number of k clusters, (b) initializing k cluster centers (centroids) randomly, (c) assigning each data or object to the nearest cluster based on Euclidean distance, (d) recalculating cluster centers with current cluster membership, and (e) defining cluster centers as the mean of all data or objects in a particular cluster.
The number of clusters was determined based on the frequency distribution of nominal amounts of financial transaction.Data clusters were categorized into small, medium, and large-value transaction.Cluster center points were determined using a random method based on the range of values in each transaction category, namely small (IDR 100,000.00 to IDR 500,000.00),medium (IDR 1,000,000.00to IDR 5,000,000.00),and large (IDR 6,000,000.00to IDR 20,000,000.00).Criteria for determining data cluster categories included (a) the dominance of small transaction in the organization financial transaction, (b) large value transaction for significant projects such as contracts, and (c) the distribution proportion of the first digits 1-9 in each cluster.After conducting several experiments to determine the best-balanced centroids for Benford Law analysis as shown in Table 4, the centroid values were established as k1 = IDR 285,000.00,k2 = IDR 1,448,000.00,and k3 = IDR 12,000,000.00.Audit toolkit, when used to classify data based on the calculation of K-Means Clustering algorithm, automatically computed the shortest distance to the centers of clusters k1, k2, and k3.The determination of the number of clusters within audit toolkit resulted in three clusters, categorizing the data into small, medium, and large transaction.

C. Application of Multiple Benford Law Model to Groups of Data
The application of Benford Law to three groups of clustered data using K-Means Clustering algorithm followed the same procedure as in previous iterations.This implied that auditors had three separate opportunities to analyze the distribution of digits 1-9 in the data groups.Based on this study, the Law was applied to Cluster 1, Cluster 2, and Cluster 3, as shown in Table 5.In the data groups, it was evident that the frequency distribution of digits 1-9 deviated significantly by  5% from the expected frequency.This deviation allowed auditors to make a strong allegation of fraud practices in financial transaction.It also provided substantial material for further review and testing during the audit.
Table 5 showed a deviation of  5% in Cluster 1, Cluster 2, and Cluster 3.This information explained the analysis of groups of financial transaction data in a company or organization during the preliminary audit stage.In Cluster 1, there was a possibility of manipulation of values starting from the number 1.In Cluster 2, it was suspected that numbers starting from 1, 2, 3, 4, 5, and 9 were included, and in Cluster 3, this was most likely with numbers starting from 1 and 6.Auditors could then focus on conducting a more in-depth examination of the financial transaction indicated by Benford Law.
In the result column of Table 5, audit toolkit provided potential fraud indications, implying that auditors needed to prioritize the presumption of innocence.Specifically, auditors needed to conduct substantive testing to examine transaction evidence during audit examinations and determine whether the data from the analysis truly contained fraud practices

D. Evaluate and Validation Model Result
Evaluation and validation are carried out on several important aspects including:

1) Evaluation of the Detection Using Multiple Benford Law Model
The evaluation model applied a comprehensive data analysis, focusing on cases identified through Multiple Benford Law analysis.In this study, the analysis performed through Benford Law alone detected 40.00% potential fraud.However, when Multiple Benford Law Model was used, 93.33% of the analyses indicated suspected fraud, as seen in Table 6.2) Evaluation Performance of Audit Toolkit Evaluation was carried out by summing the frequency of first digits in dataset with transaction values beginning with 1, 2, 3, …9 in the toolkit shown in Table 7.This process also tested the suitability of Benford Law by comparing the expected frequency with the actual frequency in the dataset, considering the optimal toolkit calculation results.

3) Model Validation by Comparing the Case Results to Audit Results
The subsequent stage focused on model validation, aimed at determining the accuracy of the toolkit audit analysis concerning the factual conditions encountered during the audit.This validation test was conducted using audit report of the XYZ organization, and the results were shown in After validating the application of Multiple Benford Law Model in eight groups of detected cases, it was established that six cases (75.00%) were proven to be genuine.Even though the validation data could not entirely represent the facts because audit results demanded verification by a professional auditor, this process also consisted of a communication stage between the concerned parties for approval of audit results.The results showed that the application of Multiple Benford Law Model had both advantages and disadvantages.The model could enhance the accuracy of data analysis in the list of financial transaction.However, it was essential to know that the data resulting from this analysis were still probable and required verification by a professional auditor.Auditors also faced limitations in terms of scope and time available for examining all detected transaction evidence.

E. Deployment Model
Multiple Benford Law Model was designed as a worksheet-based application (toolkit) that could be directly downloaded and used by professionals in audit field [23].Users could access the usage guide on the website, namely https://msi-program.info.To apply the model, users could download the toolkit from the website by logging in with the following credentials, including username -auditor_me@gmail.com and password -123456.In the future, it is expected that academics will further develop this model in a more comprehensive and automated manner to make it easily accessible and usable by audit organizations worldwide.

V. DISCUSSION
The analysis results showed that Multiple Benford Law Model effectively guided auditors to concentrate the examinations on potential areas where fraud might occur.This was evident from the case issues identified in accordance with the analysis results, which were consistent with several theories explored previously.These theories suggested that Benford Law was an effective tool for auditors to detect fraud transaction [11], and it was considered an efficient and straightforward model for targeting areas of concern in accounting data [5].However, there were still some interesting points to discuss in this study, as follows:

A. The analytical results shows the possibility of experiencing similarities with the expected frequency based on Benford Law
With the continuous growth of digital technology, the number distribution pattern based on Benford Law, as originally proposed by its creators, remained unchanged in the ages and remained relevant today [7]- [10].The results from cash flow reports in various Indonesian agencies indicated the enduring relevance of Benford Law to frequency distribution.This observation showed the importance of considering Benford Law in the context of financial management practices, particularly in Indonesia.
In various previous studies, Benford Law served as a tool for detecting fraud, with a prevailing trend in the field of audit to show fraud financial transaction.Benford Law had been used to identify fraud in various contexts, including online marketplace shopping transaction [14], tax payments [12], and financial transaction in the public sector [13].Setyawan [13] showed that applying Benford Law to detect fraud in financial transaction resulted in a detection probability ranging from 20% to 50%.This traditional application of Benford Law alone was not entirely effective in fraud detection, implying that transaction data might appear free from manipulation.In this situations, some experts suggested halting the analysis process due to the absence of potential irregularities [14].However, Prasetyo [12] argued that conformity did not necessarily indicate the absence of data manipulation, rather it encouraged auditors to exercise greater care when handling financial transaction data during audit.
Concerning the evaluation of models used in detection analysis, some studies omitted the assessments and relied solely on analyzing the conformity of Benford Law frequency with data.Meanwhile, the evaluation model of the Hybrid Model Algorithm Framework, as proposed by Kaithekuzhical et al. [14], included performance testing of the model based on three machine learning algorithms, namely Random Forest, Logistic Regression, and XG Boost.This model served as an evaluation criterion, comparing the use of standard Benford Law with Multiple Benford Law Model.
The experimentation with Multiple Benford Law Model in the XYZ organization showed a consistency between the frequency of numbers in the dataset and the expected frequency according to Benford Law.This suggested that concealed manipulation practices in the data remained undetected or the audited organization might be free from manipulation practices.Similar circumstances were recorded by several previous studies, as shown by Kaithekuzhical et al. [14], suggesting that the frequency matches might lead to a potential halt in the analysis process (exiting the algorithm).On the other hand, Prasetyo et al. [12] argued that a match between Benford Law frequency and the data did not guarantee the absence of manipulation.Compliance with Benford Law frequency should prompt auditors to exercise greater caution when interpreting financial transaction data during audit assignments.To address this situation, K-Means Clustering algorithm and Benford Law were incorporated into further analyses.This model aimed to break down the data, providing more analytical opportunities for the formed data groups.This was in line with the theory articulated by Sekar [6], indicating that the application of the K-Means Clustering algorithm could be valuable for revealing concealed information in extensive and random dataset.
The rationale in this context showed that when Benford Law was used to analyze the frequency of digits 1-9 appearing only once in the entire transaction data, dividing the data into three clusters representing transaction with small, medium, and large values expanded the analysis potential.This division increased the chances of digit 1 appearing three times in each data cluster.This study clearly showed that one-time analysis yielded different results

Fig 1
Fig 1 Application of Multiple Benford Law and K-Means Clustering Model

Fig 2
Fig 2 Multiple Benford Law and K-Means clustering algorithm model for fraud detection

Fig. 3
Fig. 3 Preprocessing-1 data preparation using the Python 2) Feature Selection Naturally, irrelevant features in several columns were removed, retaining only essential columns such as transaction date, transaction number, codification, transaction identity, and transaction value.

Fig. 4
Fig. 4 Test the suitability of the first-digit frequency to the sample unit data

TABLE 5 APPLICATIONS
OF MULTIPLE BENFORD LAW ANALYSIS FOR CLUSTER 1-2-3

TABLE 6 EVALUATION
AND COMPARISON OF MULTIPLE BENFORD LAW WITH BENFORD LAW STANDARDS