Add the peppers, white vinegar, sugar, and salt to a medium pot. Bring this to a boil and cook on medium-high heat for 10 minutes. Drain the peppers over a bowl to reserve the vinegar pickling liquid.
While the peppers are cooking, cut the cauliflower into small ½-inch florets and cut the red pepper into ½-inch pieces. Slice the carrots, celery, and garlic into thin slices.
When the hot peppers are cool enough to handle, slice them into ¼-inch pieces. In a large bowl, combine the peppers with the cauliflower, red pepper, carrots, celery, garlic, olive oil, celery seeds, and oregano. Then pour in the reserved pickling liquid.
Transfer the mixture into jars with lids and put into the fridge to marinate for 24 hours. The giardiniera will last in the fridge for 2 weeks.