Hi Oipjuss,
you can start e.g. here: http://sitecheck.sucuri.net/
And yes, having such folder and files inside of that particular folder is really suspicious. My guess is some kind of spam infection but without more details, I'm unfortunately not able to say more.
You can also check this link where you can get more useful information:
http://codex.wordpress.org/FAQ_My_site_was_hacked
If none of these helps, just try to give us more detailed description, e.g. what's your site address, are there any other problems or suspicious behavior on your site? What WP version do you use? Etc.. Generally, the more details you provide, the better the community will be able to help.