The governors of South Carolina and Kansas are threatening to sue the Obama administration if detainees from Guantanamo Bay, Cuba, are brought to military installations in either state.

In a letter Tuesday, Govs. Nikki Haley and Sam Brownback told Defense Secretary Ash Carter they are prepared to take “any action within our power” to stop the transfers.

Last week, Haley said Defense Department officials were “wasting their time” in evaluating the Naval Brig near Charleston as a potential site to house detainees. Fort Leavenworth, Kansas, is also being considered.

The closure of the Guantanamo Bay detention center has been a top priority for President Barack Obama. But that effort has faced persistent hurdles, including staunch opposition from congressional Republicans and some Democrats.