That's correct, but to answer the direct question, no.
To the OP though, do you mean your gross income was below the filing threshold, or below the foreign income exclusion, or below the minimum income required to sponsor? Minimum threshold for filing is very low. Please clarify.
Either way, no, you do not need notarized letter, but you will still need to show concrete steps to indicate your intention to re-establish US Domicile at interview.