The voices of millions of taxpayers have been analysed and stored by HM Revenue and Customs (HMRC) without consent, privacy campaigners say.

Big Brother Watch says HMRC’s Voice ID system has collected 5.1 million audio signatures and accuses the department of creating “biometric ID cards by the back door”.

The Information Commissioner’s Office (ICO) has launched an investigation.

HMRC says the service is secure and callers can choose not to use it.

The Voice ID scheme, which was launched last year, asks callers to repeat the phrase “my voice is my password” to register. Once this task is complete, they can use the phrase to confirm their identity when managing their taxes.

HMRC says this process helps speed up security procedures and improves access to its digital services.

But Big Brother Watch said taxpayers were being “railroaded into a mass ID scheme”, as they were not given the choice to opt out. “These voice IDs could allow ordinary citizens to be identified by government agencies across other areas of their private lives,” said Silkie Carlo, director of Big Brother Watch.

She also called on HMRC to “delete the five million voiceprints they’ve taken in this shady scheme”.

In response, an HMRC spokesperson said the Voice ID system was “very popular with customers”.

HMRC sources also said that identifying details were stored separately from the voice recordings.