The demand for reliable and intelligent solutions to identify and prevent financial fraud is growing in proportion to the complexity and sophistication of this crime. This article outlines a comprehensive plan to improve financial security via the use of state-of-the-art AI techniques. The suggested architecture consists of three main parts: a K-Means clustering for dataset preparation, an extended deep Q network (EDQN) for analysis and prediction, and a principal component analysis (PCA) for feature selection. An enhanced K-Means clustering method that can manage complex and large-scale financial data is our primary contribution. A key motivation for this strategy is the hope that it would facilitate the detection of legitimate and fraudulent activities by making it simpler to see clear patterns in the data. By streamlining the grouping process and producing more accurate clusters, a more efficient clustering technique improves the overall effectiveness of fraud detection. In the aftermath of getting the dataset ready, PCA feature selection follows. By lowering the dataset's dimensionality without sacrificing any of the valuable information, principal component analysis (PCA) finds and retains the most relevant characteristics. This guarantees that the chosen qualities greatly aid in the identification and prevention of fraudulent transactions, while also improving the computational efficiency of future activities. Lastly, an EDQN is used in a system for classification and prediction. By including characteristics that enable the model to comprehend more complex links in financial data, the EDQN enhances previous Deep Q Networks.