In layman's terms, here's how SQL Server processes queries, etc.
A query is run (SELECT, UPDATE, DELETE, INSERT),
SQL Server processes the query,
SQL Server pulls the data into memory (RAM),
it then holds that data in memory for a period of time (no, I don't know how long) in case you run the same or similar query.
Eventually it releases the data, for example if a different query is run.
In your case, I think you are running the INSERT. The information is being held in RAM. You DELETE the data. Again held in RAM. For a short while that information is available so that a subsequent INSERT will be quicker.
But then the data 'goes away' and other data is placed in RAM so the next 'repeat INSERTs' take longer.
I know it's not a technical explanation and I hope I got it all correct.
-SQLBill