These are people who tend grape vines and grow grapes. It can be translated as "grape farmers."
## of the fruit of the vineyard ##
"some of the grapes" or "some of what they produced in the vineyard." It could also refer to things made from grapes or the money they earned by selling the grapes.
## sent him away empty-handed ##
This can be translated as "sent him away without paying him" or "sent him away without the grapes."