As others have said, the earliest you can file for removal of conditions is 90 days prior to the expiration date printed on your green card. The date of your wedding anniversary has nothing to do with immigration now.
As for filing for citizenship, see the box at the bottom of page 22 of the M-476.
QUOTE
If you are applying based on 5 years as a Permanent Resident or 3 years as a Permanent Resident married to a U.S. citizen, you may file for naturalization up to 90 days before you meet the continuous residence requirement. For example, if you are applying based on 3 years of continuous residence as a Permanent Resident married to a U.S. citizen, you can apply any time after you have been a Permanent Resident in continuous residence for 3 years minus 90 days. You may send your application before you have met the requirement for continuous residence only. Therefore, you must still have been married to and living with your U.S. citizen spouse for 3 years before you may file your application. You must also meet all the other eligibility requirements when you file your application with USCIS.
I don't know enough about your living arrangements to say when you'll be eligible, but I'll note that it's not uncommon for folks who come on a CR-1 visa to start living together on the date LPR status starts. You must have been married and living together for three full years before you can send the paperwork in.