I would definitely consider the bracket replacement and advise against running without it.
Aside from supporting the weight, it also minimizes flex.
Under hard cornering and elevation changes, the continual flex at the connection points could break bolts and spread debris across the track (i.e. your exhaust falls off).
Worst case scenario, you lose the whole system from the cat back and shred a rear tire, the long tube becomes a leverage point and causes the car to flip or roll.
Nevermind the obstacle to trailing drivers....