What's harder? English speaker learning French or French speaker learning English?

Just wondering which might have the hardest time knowing enough to survive on the streets using typical/modern versions of the Lang. I'm not asking for fluency but overall who would end up struggling more?