Step 1Arrange a rack in the middle of the oven and put a large baking sheet on the rack below it, to catch any drips later. Heat the oven to 350 degrees.
Step 2In a large bowl, toss together the bread, buttermilk, heavy cream, milk, raisins and nutmeg and set aside to soak for 20 minutes. Meanwhile, in a medium bowl, toss together one-fourth cup of the granulated sugar, lemon juice, cinnamon and peaches, to coat; set aside.
Step 3Grease a 9-by-13-inch casserole dish with butter; set aside. Whisk together the eggs, vanilla, salt and remaining 1 1/4 cups of the granulated sugar in a medium bowl, then add to bowl with the bread mixture. Add the macerated peaches and stir well to combine. Transfer to the prepared dish and spread out evenly.
Step 4Sprinkle the turbinado sugar over the top and bake until cooked through and deep golden brown, covering with a sheet of foil if parts of the top get too dark, about 1 1/4 hours. Serve warm or at room temperature. Serve with a drizzle of cream, if desired.