The group have instructed lawyers to attempt to begin proceedings in England's High Court, British newspapers said, in a move timed to coincide with Indian Prime Minister Narendra Modi's visit to Britain this week.
Modi heads to Britain on Thursday and will have lunch with Queen Elizabeth II during his trip, as well as hold talks with his British counterpart David Cameron.
Satish Jakhu, of the Rubric Lois King law firm, was quoted as saying they would claim under the common law doctrine of "trespass to goods", arguing that Britain had stolen the stone, one of the world's biggest diamonds.
They are also seeking to take the case to the International Court of Justice in The Hague, he added.
Rubric Lois King did not respond to requests to comment when contacted by AFP.
The stone is "one of the many artefacts taken from India under dubious circumstances", David de Souza, from the Indian leisure group Tito's, was quoted as saying by The Independent newspaper. He is partly funding the action.
Bollywood actress Bhumicka Singh said the stone was "part of our history and culture and should undoubtedly be returned".
- 'Mountain of Light' -
The diamond was presented to queen Victoria in 1850 after the Anglo-Sikh wars in which Britain gained control of the Sikh empire of the Punjab, which is now split between Pakistan and India.
Newspapers said the precious stone was worth £100 million ($150 million, 140 million euros). Historically it had been in Mughal and Afghan hands.
The 105-carat diamond is set in the crown made for the queen Elizabeth, the wife of Queen Elizabeth II's father king George VI, for their 1937 coronation.
Its name translates as "Mountain of Light". Traditionally the diamond is only worn by a queen and it is said to bring bad luck to any man who wears it.
The crown jewels are held by Queen Elizabeth II as the sovereign head of state rather than in a personal capacity.
This means that if ever negotiations were to take place over the diamond, Queen Elizabeth II would only ever act on the advice of her government.
Cameron is against returning the diamond.
"If you say yes to one you suddenly find the British Museum would be empty," he told NDTV television in 2010.
"It is going to have to stay put."
And on a 2013 visit to India he said: "I certainly don't believe in 'returnism'."
In 1976, prime minister Jim Callaghan refused a request to concede it, citing the terms of the Anglo-Sikh peace treaty and concluding: "I could not advise Her Majesty the Queen that it should be surrendered."
Historian Andrew Roberts told The Mail on Sunday newspaper that the prospective lawsuit was "ludicrous", saying Britain had a "watertight" case for keeping the stone, based on a legally-binding treaty.
When contacted by AFP, Buckingham Palace declined to comment on what would or would not be discussed at Friday's lunch between Queen Elizabeth and Modi.
