Archive keeps the mail in Archive Folder. The only exception is when Gmail is your provider as Gmail handles the Archive folder differently. Delete puts the email in a Trash folder that is periodically cleaned, and can also be manually deleted forever on demand. You can access both of those folders from the Folders picker.

You can View your Folders using: Swipe from left edge or by tapping the top left corner logo, and choosing the desired Account. Choose Trash/Archive.