Why did Gideon tell the men of Penuel that he would pull down the tower?

Gideon told the men of Penuel that he would pull down the tower because they would not give his army bread, just as the men of Sukkoth refused.