I'm attaching the front cover of the Apostille so you can see what it looks like. I also thought it would have to go to some higher up federal office but apparently that's not the case, it's the local court since the judge is the one who officiated the name change. We also obtained a copy of the N-662 via the local court here, not via USCIS or any federal office.
Note: We only got an Apostille on the N-662, not on the Naturalization Certificate itself. Perhaps this is why we didn't need a federal apostille?