The policies promised to protect customers against losses if thieves used their cards, but such losses were likely to have been covered by the bank anyway.
Customers spent an average of £25 a year on policies including Card Protection, Sentinel and Safe and Secure Plus.
The period covered by the compensation is between January 2005 and August 2013, so it could stretch to nearly 9 years.
The refunds could add up to hundreds of pounds in some cases, including interest at 8 per cent.
The letters are being sent by a special company set up by banks, credit card companies and a US firm called Affinion which provided the insurance.
The Financial Conduct Authority is not accusing them of mis-selling. It says some features of the policies could be useful.
Customers will have to vote on the scheme which will then need High Court approval before payments are sent out in the autumn.