You can delete the items in the folder, by right clicking. The delete option is there.
If you want to delete the Storage" folder item itself, go here:
C:\Users\username\AppData\Local\Microsoft\Windows Live Mail
There you can right click the folder and delete it. I would not advise it, as Windows LIve mail uses the folder periodically to store your files when they become excessive.