There are many reasons for an TCP RST to be sent. Some are not normal, meaning errors, and some are normal connection cleanups that the TCP/IP stack or application performs.
An example of a normal TCP RST would be a long lived connection that exceeds some time limit imposed by one side or the other. Once the time limit is exceeded the connection can be "forceably" closed which will generate the RST.
An example of a not normal TCP RST would be an application that abruptly disconnected due to an internal error.
A poorly written application can also cause TCP RST when it does not perform graceful shutdowns on the TCP socket before closing the connection.
I will guess that the behavior you are seeing is not a problem. However, to really know, you will need to do a wire trace and protocol analysis on each connection to determine exactly what is happening.
To know more about Migrating to AWS, join our AWS Cloud Migration Course. Also, If you wish to master the principles in AWS and work on step-by-step tasks that have significant relevance for the world of business, industry professionals developed the AWS syllabus.