"We have full control of the port and the town, along with its historic castle," said Fifth Military Region commander General Adel al-Qumairi.
Intense clashes had raged in the area since mid-December, when government forces trained in nearby Saudi Arabia crossed the border and seized the town of Haradh.
The rebels have reportedly used Midi's port to bring weapons into their traditional stronghold in the north.
Despite losing Midi, they and their allies of renegade troops loyal to former president Ali Abdullah Saleh continue to control a long stretch of the coastline.
Government forces have enjoyed air support from the Saudi-led coalition, which launched a campaign against rebels in March after they advanced on the southern city of Aden where Hadi had taken refuge before fleeing to Riyadh.
The rebels continue to control the capital, which they overran in 2014, but loyalists have captured areas east and northeast of the capital recently.
On Wednesday, coalition warplanes bombed rebel positions in Sanaa, including the presidential complex, witnesses said.