Taking back sunstones from shipyard

[Ralp]

I built the armored ship before the altar, but my worker took all 5 sunstones instead of only one. Now the remaining 4 are stuck in the shipyard and my builders refuse to take them from there to build the altar. Destroying the shipyard also destroys the stones instead of moving them to the treasury. Is there anything else I can do, other than grind dungeons for more?

Playing on android.