"The losses would deal a major blow to the UK's economy, as the EU market accounts for 46 percent of the UK's exports," said the study by the Geneva-based agency.
It also found a negative impact for some EU countries.
The most affected would be Ireland, which could see a 10-percent cut in its exports under the no-deal scenario.
Britain left the European Union last month and has vowed to strike a deal on new trading relations with the bloc by the end of the year, saying that it will adopt much looser economic ties if there is no agreement by then.
But the UNCTAD study found that even a "standard" trade deal, such as one modelled on the EU-Canada deal advocated by British Prime Minister Boris Johnson, would still see Britain's exports to the EU fall by 9 percent.
This is because standard trade deals normally concentrate more on reducing or eliminating tariffs rather than non-tariff measures and Britain has already indicated it will diverge from the EU in terms of regulation.
UNCTAD said this divergence means British producers would incur costs to meet standards when selling to the EU and there would be further costs because of customs checks.
A no-deal scenario could, however, create "some opportunities" for developing countries exporting to Britain and, to a lesser extent, to the EU, the study said.
"Trade barriers between the UK and the EU would benefit suppliers from third countries. By contrast, a deal between them would preclude the incentive to turn to third countries," it said.
It found that exports from developing countries to Britain could rise by up to 4.0 percent, with the strongest positive impact predicted for the agriculture, food and beverages and wood and paper sectors.