Elliptic curve cryptography (ECC) is a public key cryptosystem which is widely used for different real world applications. With the introduction of side-channel attacks, there is a growing concern regarding the security of such implementations. Indeed, side-channel attacks have been reported to break even the theoretically secure ciphers due to the exploit in the physical leakage. The non-profiled side-channel attacks, especially are considered more serious than the profiled counterpart, as the former can work in almost black box setting. Several attacks have been proposed, however, one of the main issue normally encountered is regarding the selection of relevant features from the side-channel signal. For ECC implementation, normally the side-channel measurements will contain lots of irrelevant points which could hinder the effectiveness of the attack. For profiling scenario, these features can be determined, since the attacker has full knowledge, however, for black box non-profiled setting, this might pose an issue. In this work, we investigate different feature selection approaches to improve the accuracy for non-profiled attacks on ECC. We demonstrate the effectiveness of proposed methods on real measurements from FPGA and microcontroller targets, achieving accuracy comparable to profiled case (88.6% and 98.4% respectively).