Preheat oven to 350F. Butter and line two 9″ cake pans with parchment. Set aside.
In the bowl of a stand mixer, combine the lemon zest and sugar. Rub together with your fingers for 1 minute to release the lemon essential oils. Add the softened butter. Using the paddle attachment, mix on LOW speed until incorporated then increase to beat on HIGH for 2 minutes.
Switch to the whisk attachment, then add the eggs to beat for 2 more minutes, until doubled in volume.
Add the lemon juice, olive oil, and vanilla. Whisk vigorously (or beat on HIGH speed) for 2 minutes, until doubled in volume and pale in color. Mix in the milk.
Add the flour, baking powder, baking soda, and salt to a separate bowl. Mix very well.
Add the flour mixture to the olive oil mixture. Mix just until combined and smooth. Don't over mix the batter once the flour is added.
Divide the cake batter between the two prepared baking pans.
Bake for 25-30 minutes, until a toothpick inserted into the center comes out clean and tops are slightly golden.
Let cool 10 minutes in the pan, then remove from pan to cool completely.
Once completely cooled, slice each cake layer in half to create four thin cake layers.