that has long been an issue, covered in the book:
"The Deporter: One Agent's Struggle Against the U.S. Government's Refusal to Expel Criminal Aliens" by Ames Holbrook
it's well woth the read, and amazing how the bureaucracy within the usa is often our own worst enemy when it comes to removing illegals.
those swept up give false names, false countries of citizenship, so it's a wild goose chase to figure out who they really are.
even if the illegal gives correct information, the country can deny accepting them back and we're stuck with them. due to
crowding and length of time the government can keep them, often these same illegals are released. the book cited above deals
with new orleans - which is one reason new orleans has a crime problem.
my answer - pay a country in an economically depressed country that can use manual labor to build roads to accept them at a hard labor prison
until their country of origin takes them back. and if their home country does not want them, then it's cheaper to house them in a prison there.
additionally, deny all visas from that country until said country cleans up their act and cut foreign aid to zero to any country that will not take back their citizens.