In my experience, as an English teacher with over 10 years' experience, Spanish students learning English generally say the hardest aspects of English are prepositions, phrasal verbs, exceptions to grammar rules and the phonetics (including consonant clusters and th 'th' sound). I would think students from other nationalities would give similar answers.
On a similar note, generally my students indicate that speaking is more difficult than reading, listening and writing. Maybe this is the same for you.