According to the agreement, Israel will send at least 16,250 African migrants to countries in the West after reaching an "unprecedented understanding" with the UNHCR, Israeli Prime Minister Benjamin Netanyahu said Monday.
Thousands of other African migrants will be able to stay in Israel.
Speaking to reporters, Netanyahu said that many of those leaving the country will be moved to Canada, Italy and Germany. He said that 6,000 migrants will depart within the next 18 months.
There are approximately 37,000 illegal immigrants in Israel, the majority from Eritrea or Sudan, according to the Population and Immigration Authority.
Critics were enraged by the Israeli government's previous plan, in which migrants would be offered $3,500 and an airplane ticket to leave for a sub-Saharan African country.
Uganda and Rwanda were widely reported by the Israeli media as potential host countries.
The plan was successfully challenged by human rights groups at Israel's High Court on March 15, where a temporary order blocked its implementation.
Many of the migrants from Sudan fled from war and poverty. In Eritrea, they escaped a brutal dictatorship that conscripts men and women into the military for life.
Lots of those seeking refuge made their way to Israel. But in 2013, Israel completed a fence running the length of the Sinai border, halting the flow of illegal migrants there almost immediately.
At its highest point, there were some 65,000 illegal immigrants in Israel. Over the past decade, the Population and Immigration Authority says it has received 54,600 requests for asylum. Only 33 have been accepted. Tens of thousands remain mired in the bureaucratic process, though Israeli leaders say they have added staff to clear the backlog.
That makes Israel's rate of granting asylum among the lowest in the Western world.
In comparison, through the first three quarters of 2017, nearly 90% of asylum seekers in the European Union were granted refugee status, according to a compilation of data from Eurostat, the EU's data compilation site. More than 60% of asylum seekers from Sudan were granted similar status.