Delete files without holding Shift key

By default, Windows moves files to the Recycle Bin when you delete something in your computer.

If you want to delete files without moving to Recycle Bin Hold the Shift key and then the delete button. If you don't want to hold shift key but still delete the file permanently follow these steps:

1. Go to Start -> Run then type gpedit.msc and press Enter
2. Go to User Configuration
3. Then Administrative Templates -> Windows Components -> Windows Explorer 
4. And locate "Do not move deleted files to the Recycle Bin" setting and change it.