Parts of Scotland would have special access to EU workers under a proposal mooted by the Scottish Secretary.
David Mundell said the UK Government is “not minded” for immigration to be devolved to Holyrood – a key plank of Nicola Sturgeon’s demands over Brexit.
But the Conservative MP suggested geographical areas affected by depopulation and labour shortages will need to have unique arrangements.
Rural parts of Tayside and Fife rely on seasonal workers and many are seeing a population decline of younger people.
Mr Mundell told MSPs on Wednesday his government was looking at a UK-wide system which protects the supply of EU workers.
“Clearly we need to address issues around depopulation and the provision of services in those areas,” he said.
“I am not minded to a view that immigration should be devolved. The Scottish Government have clearly made that case.
“But going forward we want to have an immigration system that allows for those jobs, which are necessary in our economy, to be filled.”
In November, Angus Soft Fruits, which employs 4,000 seasonal workers from the EU, threatened to move abroad if post-Brexit immigration policy stops it from recruiting from the bloc.
Scotland voted with a 62% majority to Remain in the EU, but Leave votes elsewhere in the UK threaten to over-rule that.
Ms Sturgeon has put forward proposals for Scotland to stay in the European single market while part of a UK that leaves.
The First Minister says an independence referendum is “highly likely” if Scotland’s will is ignored.
Mr Mundell told journalists after his appearance at Holyrood’s EU committee that the way to attract workers to Scotland is not by making the country the highest taxed part of the UK.
Scots earning at least £43,000 will pay up to £400 more a year than their English counterparts after Holyrood set income tax under major new powers on Tuesday.
Mr Mundell also said the triggering of Article 50, which starts the two-year Brexit process, is not a deadline to negotiations with the Scottish Government.
A Scottish Government spokesman said that goes against assurances previously made by UK ministers. “Our position remains that it is essential UK ministers establish a position that properly reflects all parts of the UK ahead of Article 50 being triggered,” he added.