boxers fracture on left hand

HI its been about 5 weeks sice I broke my hand I got a cast and I took it off my self 3 day early. I wanted to know if 3 days mattered that much? after all of the swelling went down ,its like my pinkey nuckle isnt there anymore. I cannot bend it fully and my finger is up too far. its been 5 weeks and I just wanted ot know if it would get better,and if I could punch fully with it again.
thank you