What techniques do you use to cache or pre-compute frequently generated responses to reduce model load

Question

Can you tell me the names of the techniques use to cache or pre-compute frequently generated responses to reduce model load?

Answer 1

Most efficient techniques to cache or pre-compute frequently generated response are as follows:

• Response Caching
• Memoization
• Embeddings Caching
• Indexing
• Pre-Training with Fixed Responses

Note that these techniques will help in reducing model load and improving efficiency also.