Step 1Heat the oven to 400 degrees.
Step 2Prepare the duck: Poke the skin all over the duck with a fork. Season the duck all over with 1 1/2 teaspoons salt and one-fourth teaspoon pepper, rubbing the seasoning over the skin.
Step 3In a small bowl, stir together the celery, onion, cinnamon sticks, star anise and 1 tablespoon olive oil. Stuff the mixture into the cavity of the duck and tie the legs together with the tail to prevent the stuffing from falling out.
Step 4Heat a large saute pan over medium-high heat and add the remaining tablespoon oil. Brown the duck, turning every few minutes to color each side evenly and well, about 20 minutes total.
Step 5Place the duck and any juices in a baking pan and roast, basting every 15 minutes, until the juices run clear when you prick the thigh, about 2 hours.
Step 6Remove the duck and set aside to rest 20 to 30 minutes before carving.
Step 7While the duck is resting, make the kumquat sauce. Place the honey in a 2-quart heavy-bottom saucepan over high heat. Bring the honey to a boil and cook just until it begins to darken and caramelize. Immediately remove the pan from heat and add the vinegar and orange juice, stirring to combine. Cook over medium heat until the liquid is reduced by three-fourths, 20 to 25 minutes. Stir in the butter and kumquat slices and simmer gently for 5 minutes, stirring occasionally to allow the flavors to marry. Remove from heat and serve with the duck.