You can only banish players who are not in your alliance. To do this, a banish task has to be assigned first. Tap on the player's castle, then on the tasks icon located at the bottom-left corner of the pop-up and choose Banish Player. After the player is defeated, their castle will be automatically teleported from the alliance territory (banished). Note that you have to be at least a Task Master in the Alliance in order to be able to set such a task.