the city

This refers to Jerusalem.

he wept over it

The word "it" refers to the city of Jerusalem, but it represents the people who lived in that city. (See: rc://en/ta/man/translate/figs-metonymy)