this is important when running a test with --loop (cherry picked from commit e3e69c25fd05b608f5ea8d612bc0e377922a6115)