Army deputy chief of staff Major General Ahmad Saif Al Yafii was killed by a heat-seeking missile on the outskirts of the coastal town of Mokha, the source told AFP.
The rebels and their allies had pushed back south and reached the eastern outskirts of Mokha, which the army had overrun on February 10.
It was a major setback for an offensive launched by government forces in January to try to recapture Yemen’s 450 kilometre (280 miles) Red Sea coastline, which had previously been almost entirely in rebel hands.
Government commanders had talked confidently of pushing north towards the rebel-held port city of Hodeida, a vital conduit for UN-supervised aid deliveries to rebel-held areas.
The loyalists’ capture of Mokha was their biggest success in months.
Despite nearly two years of military support from a Saudi-led coalition, government forces are still largely restricted to the south and areas along the Saudi border.
The rebels still hold the capital Sanaa and much of the central and northern highlands as well as the coast around Hodeida.
Before the 19th Century, Mokha was Yemen’s main port and export hub for coffee grown in the highlands and its historical symbolism meant it was fiercely fought over.
Its role was overtaken by Hodeida and second city Aden, where the government is based.
The conflict on Yemen has shown no let-up despite UN warnings of looming famine.
UN humanitarian coordinator for Yemen Jamie McGoldrick warned on Tuesday that seven million Yemenis are close to starvation.
More than 7,400 people have been killed since the coalition intervention began in March 2015, including around 1,400 children, according to World Health Organization figures.
McGoldrick said the death toll was closer to 10,000.