Reynders cited systemic problems with the rule of law in Hungary, adding that the Commission was ready to use all tools to protect democracy, including the suspension of EU funds
Budapest has long been at loggerheads with Brussels over democracy issues like judicial independence and press freedom, and its hardline anti-immigration policies.
The decision to quit came immediately after the EPP voted for a change in its rules that was billed as a way to expel Orban's hardline nationalist party from their ranks.
Orban also said all the 2.5 million to 2.6 million Hungarians who have registered for COVID-19 vaccinations so far would receive at least one dose by Easter, in early April.
Orban, speaking on state radio, said he hoped to get vaccinated with a shot developed by China's Sinopharm early next week.
Hungary's economy took a big hit in the pandemic, and the government expects gross domestic output to shrink by about 6% in 2020 followed by a modest rebound in 2021.
Leaders were looking to salvage the EU's 1.8-trillion-euro ($2.1-trillion) budget and coronavirus recovery package, seen as vital for the continent's battered economies, after it was blocked by Hungary and Poland over provisions linking it to the respect for rule of law.
The assessment prepared by the European Commission, the bloc's executive, voices "serious concern" about judicial independence in both Hungary and Poland.
EU leaders meet later this week to agree on a 750-billion-euro ($850 billion) coronavirus aid package aimed at helping countries hit hardest by the virus.