So you mean post office in your current/home country? Yes, you can do a change of address for forwarding with them, but that won't be relevant to your SSN or GC as neither will be delivered outside the US anyway. So for those you just need to give your new US address when you enter. I typed ours up on a piece of paper just to make sure it was correctly entered when I handed it to the border officer.
The fee is for the green card production and delivery. Assume you've already paid it? If not, then note the price goes up next month.