My website is mostly in English but my articles regularly contain words and phrases in French.
Here is an example:
Should I use the lang attribute for each of these words and phrases? I have never done so and I wonder if it could be hurting me in term of SEO.
It won't make a difference - Google doesn't take note of it within HTML at all:
That said, I'd keep it as a tag in the head of the page. (In your case, it'd be English.) Yes, it's been deprecated by Google, but can help with web standards, particularly for certain other search engines and screen readers.