Warm the whole milk in a saucepan, then add the Earl Grey tea leaves. Let it steep for 10 minutes until aromatic, then cool completely.

In a mixing bowl, beat the softened unsalted butter and granulated sugar together until light and fluffy, about 3-4 minutes.

Add the large eggs, cooled tea mixture, and vanilla extract to the butter-sugar mixture, mixing until smooth and well incorporated.

In a separate bowl, whisk together the all-purpose flour, baking powder, and salt until evenly combined.

Gradually fold the dry ingredients into the wet batter, being careful not to overmix.

Divide the batter evenly between greased cake pans and bake in a preheated oven at 350°F (175°C) for 25-30 minutes, until a toothpick inserted comes out clean. Let the cakes cool completely in the pans.

In a mixing bowl, beat the powdered sugar, heavy cream, and lemon zest until fluffy and spreadable, adjusting the consistency with more cream or sugar as needed.

Once the cakes are cooled, layer them with frosting in between, and generously frost the top and sides.