Combine the balsamic vinegar, honey, brown sugar, soy sauce, rosemary sprigs, and garlic in a bowl and pour into a large resealable plastic bag. Add the drumsticks to the bag and seal, removing as much air as possible. Turn to coat drumsticks and place in the refrigerator to marinate for 2 hours.
Preheat oven to 450°F. Place the chicken on a foil-lined baking sheet. Bake until the skin is caramelized and dark in spots, about 30-35 minutes.
Meanwhile, place the marinade in a small saucepan. Bring the marinade to a boil and simmer, over low, until thick (about 15 minutes). Use a pastry brush to brush some of the cooked marinade on the cooked chicken. Place the chicken on a serving platter and sprinkle with toasted sesame seeds.