The delete button on the toolbar deletes emails from the archive permanently.
By default, end-users are not permitted to delete emails from the archive. However, depending on administrator preferences, the option can be enabled.