Preheat your oven to 375°F (190°C).
In a large oven-safe skillet or Dutch oven, heat the olive oil over medium-high heat. Add the ground venison and cook, breaking it up with a spoon, until browned. Drain off any excess fat.
Add the chopped onion, diced carrots, and diced celery to the skillet. Cook until the vegetables begin to soften, about 5-7 minutes.
Stir in the minced garlic, dried thyme, and crushed rosemary. Cook for another minute until fragrant.
Season the venison and vegetable mixture with salt and pepper.
Sprinkle the flour over the mixture and stir well to coat. Cook for 1-2 minutes, stirring constantly, to cook out the raw flour taste.
Gradually pour in the beef broth and red wine (if using), stirring continuously to prevent lumps. Bring the mixture to a simmer.
Stir in the Worcestershire sauce and tomato paste. Reduce the heat to low, cover, and let it simmer gently for 15-20 minutes, allowing the flavors to meld and the sauce to thicken.
Meanwhile, place the peeled and quartered potatoes in a large pot and cover with cold water. Add a generous pinch of salt. Bring to a boil and cook until fork-tender, about 15-20 minutes.
Drain the potatoes thoroughly and return them to the pot. Add the warm milk and butter. Mash the potatoes until smooth and creamy. Season with salt and pepper to taste.
Stir the frozen peas (and corn, if using) into the venison mixture in the skillet. Ensure the filling is evenly distributed.
Spoon the mashed potatoes evenly over the venison filling, spreading to cover the entire surface. You can create decorative peaks with a fork if desired.
Place the skillet in the preheated oven and bake for 20-25 minutes, or until the mashed potato topping is lightly golden brown and the filling is bubbling around the edges.
Let the Shepherd's Pie rest for 5-10 minutes before serving.