linux bash - remove all files which are in one directory from another directory

Question

I want to remove all files that exist in folder new-files from another folder in linux using bash commands.

I need this for two things:

• I got some setup scripts which copy some pre-configured config files over. I would like to have the option to remove those files again
• Sometimes it happens that archives get unpacked into the root of your downloads directory and not into a subdir because the person packing the file put everything to the archives root

What's the best way to do that?

Edit, to clarify:

1. I got a folder with files called new-files.
2. Now I execute cp -r new-files/* other-directory/.
3. Lets say other-directory is not the directory I wanted to copy them to but it already contains other files so I can't just do rm other-directory/*.
4. I need to delete all folders which I accidently copied. How do I do that?