Step 1In a wide bowl, whisk together the yolks, 1 cup of heavy cream, 1 teaspoon vanilla and a pinch of kosher salt. Place bread rounds into the custard, coating the slices. Let stand 30 minutes, turning over occasionally.
Step 2Meanwhile, make the Chantilly cream. In a standing mixer or with a whisk in a large bowl, whip the remaining cream (make sure it is cold), the remaining one-fourth teaspoon vanilla and powdered sugar until soft peaks form. Refrigerate until needed.
Step 3In a large saute pan, melt 1 tablespoon of butter over medium-high heat. When it foams up in a few minutes, add the apples, sugar, lemon juice and a pinch of kosher salt. Stir to coat the apples and cook until the apples are soft and beginning to caramelize, about 3 minutes, stirring frequently. Remove from heat and reserve, keeping warm.
Step 4In a clean, large saute pan, melt the remaining 1 tablespoon butter over medium-high heat. Wipe any excess custard from the rosemary bread and cook until golden on both sides, 2 to 3 minutes per side. You can do all rounds at once or cook them in batches, depending on the size of your saute pan. Remove and blot on a paper towel.
Step 5Place one round of pain perdu on a plate, arrange one-fourth of the sauteed apples on top of the round, then add about a tablespoon of Chantilly cream on top. Spoon a tablespoon of caramel sauce around the pain perdu and sprinkle with rosemary leaves. Repeat for the other plates and serve immediately.