The downloader makes always a backup of the image file even though MC creates and uses an embedded image if possible.
You can change the Track Images Location option to "In specific folder" and delete the files from there. This affects only newly added images. Your old image links (if you have any) will remain unchanged.