When using traefik configured for swarm mode, set the label on the service instaed of the container. You do that by moving the labels inside of a deploy block in the compose file:
services:
site:
ports:
- 4000:4000
image: mywebsite:latest
deploy:
labels:
- traefik.site.port=4000
- traefik.enable=true
- traefik.frontend.rule=Host:mydomain.com
networks:
- traefik-net