Combine the cloves, celery, bay leaves, thyme, onions, garlic, lemons, salt, coriander, dill, mustard seeds, red pepper flakes, cayenne, allspice, peppercorns, paprika and water in a large pot and bring to a boil. Reduce the heat to medium and simmer, stirring occasionally, until the onions and celery are soft and the flavors have melded, about 40 minutes.
Add the potatoes to the broth and cook until almost tender, about 15 minutes. Add the corn and cook for another 6 minutes. Add the shrimp and cook, stirring occasionally, until the shrimp are just cooked through, about 4 minutes more.
Using a large slotted spoon, transfer the shrimp and vegetables to a large paper-lined surface. Serve with bowls of melted butter and cocktail sauce, lemon wedges and the reserved cooking liquid.