Follow the below steps:
1. Install R studio from here, https://www.rstudio.com/products/rstudio/download/
2. Click next to all and Click install.

3. Run R Studio.

4. Now type command and run.
install.packages("Rserve")
5. Type below command and run
library(Rserve)
6. Type below function and run.
Rserve()

7. Now Go to Tableau and Open Manage External Service Connection.

8. A window opens, Select Rserve and default port is set as 6311.

9. To check connection, Click on test connection and click ok.

10. Now tableau can use R to perform operations.
Hope it helps!