Step 1Whisk together the sugar, tapioca, milk, egg and salt in a saucepan and set aside for 5 minutes.
Step 2Heat the tapioca mixture to a rolling boil over medium heat, stirring frequently. Split the vanilla bean and scrape the seeds into the mixture.
Step 3Remove the tapioca from the heat and pour it into a bowl to chill (the mixture will be somewhat liquid; it will thicken as it cools). Refrigerate until set, at least 1 hour.
Step 4Heat the oven to 350 degrees. Spread the almonds on an ungreased jelly roll pan and, shaking the pan occasionally to prevent scorching, bake until they begin to become fragrant and turn golden, about 7 to 10 minutes. Remove them from the oven and immediately transfer them to a bowl to stop the cooking. Set them aside.
Step 5Peel the mandarins and carefully remove as much of the white pith and strings from the fruit as you can. Break the mandarins into 2-segment sections and cut each section in half crosswise. Collect the half-sections in a bowl and set aside.
Step 6Sprinkle the minced ginger over the chilled tapioca and stir roughly to mix it in thoroughly. After mixing, the tapioca should be somewhat creamy in texture, rather than set firm.
Step 7Spoon 2 to 3 tablespoons of tapioca into a small glass (a martini glass is perfect; a wineglass or flat Champagne coupe will work well too). Spoon 3 (2-segment) half-sections of mandarins on top, then scatter a tablespoon of sliced almonds over that. Repeat twice more so you have 3 succeeding layers of tapioca, mandarins and almonds, ending with almonds. Chill for up to an hour before serving.