"A" is the frame, "B" is the outer right bracket; left side is similar. Also, there are short brackets at the very front of both frame rails, for a total of four brackets - one inner and one outer on each side, two bolts per bracket. Leave all of the brackets bolted to the vehicle and remove just the carriage head bumper bolts.
With careful observation you should be able to determine whether the bumper and/or the brackets are the cause of the misalignment. But, generally, the bumpers easily distort to absorb the energy of impacts.