Step 1Make a small horizontal slit in the top of each kumquat (over the stem end), no more than one-fourth-inch deep. This will allow the marinade to permeate the entire fruit.
Step 2Place the fruit along with 3 quarts water in a large soup or stock pot over medium-high heat. Gently simmer until tender, about 30 minutes, stirring occasionally.
Step 3Drain the water out of the pot, and add 3 cups fresh water. Stir in a generous 1 3/4 cups sugar and heat the mixture over medium heat. Cook until a thermometer inserted reads 200 degrees, or until the mixture barely begins to simmer, stirring occasionally. Be careful that the mixture does not come to a boil. Remove from the heat and set aside overnight.
Step 4Repeat the process two more times, draining the syrup and adding another generous 1 3/4 cups of sugar with 3 cups fresh water.
Step 5Make the spice infusion: In a medium saucepan, combine one-half quart of water with the cinnamon, allspice and cloves over high heat. Bring to a boil, then reduce to a simmer and cook for 30 minutes. Strain and set aside.
Step 6Drain the kumquats and add the remaining sugar, with 3 cups water, one-fourth cup of the spice mix, and the vinegar. Heat the mixture until a thermometer inserted reads 200 degrees; remove from the heat. At this point, the kumquats can be refrigerated, covered, for several weeks, or canned according to the canning-product manufacturer's instructions.