In a large saucepan set over medium heat, combine 2 quarts of milk with the almonds, almond extract, zest and sugar. Bring the mixture to a gentle boil, between 200° and 212°, stirring regularly, and cook for about 5 minutes. Strain the milk, discarding the solids. Keep the liquid warm.
Whisk the egg whites with the remaining 2 tablespoons of milk (chilled, if possible) and stir them into the warm milk mixture. While stirring, add the brandy and rum. Beat the mixture vigorously until it has a light froth on top. Pour into a punch bowl and serve immediately.