Its not that hard to figure out. Basically, he was an abusive drunk to them and his wife growing up. Eventually, when the boys were teenagers, his wife finally found the courage to leave him. She took Tommy with her and moved out of the city but Brennan stayed. Not so much out of loyalty to his father moreso than it was for his love for the woman that would eventually become his wife.
Tommy took that move as Brennen choosing Paddy over him and their mother. That's where his hatred comes from.
In addition to his abuse, Brennen's hatred towards Paddy also comes from his jealously over his perceived favoritism towards Tommy growing up.
Then as adults, Paddy finally found the strength to get sober and is trying to redeem himself to his boys
something they clearly refuse to be a part of given the years of abuse and pain he'd caused them and their mother.
Its left to you to interpret what happens. It wouldn't be a stretch to assume that Brennan offers some of his winnings to the Marine widow. At the end of the day, the focus of the movie is the relationship between the boys and their father. The widow being there is more so to show that Tommy is more than some tough guy looking for the next person to beat down
there's more to him than he tries to let on.