Working with media in WordPress is, for the most part, a fairly simple task; especially with the more recent improvements to the media manager. Deleting media items, however, can sometimes be a somewhat risky process as you are only presented with the option to Delete Permanently. This behaviour differs from working with posts and pages, as you are, by default, given the ability to Move to Trash before deciding whether or not you want to permanently delete the item.
Fortunately, there is a trash feature for the WordPress media manager and it can be enabled through one simple line of code;
// In wp-config.php define('MEDIA_TRASH', true);
Enabling this feature will replace the Delete Permanently links with an aptly named Trash link. Any 'trashed' items will then appear in the media trash where you can later delete them permanently, should you wish to.