"The Prime Minister has a conscience. It will be enough for us if he acts according to his conscience," Karunanidhi said.
"Also I would like to clarify that I did not threaten the United Progressive Alliance (UPA) on withdrawing support on the issue," he said.
The Congress Core Group on Friday met here to take a final decision on Singh's visit to Sri Lanka.
The meeting was headed by party chief Sonia Gandhi.
The decision is likely to be announced shortly.
Congress leader and Union Environment Minister Jayanthi Natarajan on Monday reportedly said she is opposed to Singh's visit to Sri Lanka.
"I have requested a meeting with the Prime Minister. I want to convey the strong sentiments of people of Tamil Nadu on his Sri Lanka visit," said Natarajan.
"I will request him to factor sentiments against his visit when he considers his final decision," said the Tamil Nadu MP.
Union Finance Minister P Chidambaram is also reportedly opposed to Singh's visit to Colombo.
CNN-IBN channel quoting sources reported that Chidambaram as a Tamil MP feels that the government must factor in state interests.
Speaking to media, Chidambaram said the government will take decision and the process of decision making is on.
"It's the decision that the government will take. There is still time to take that decision," he said.
"I think MEA spokesperson has already said that the process of decision-making is on and the decision will be taken," Chidambaram said.
"I am aware that some ministers were opposed to Prime Minister's participating in CHOGM. We are aware of that," the Finance Minister said even as media reports suggested he is opposed to PM's Colombo visit.
According to reports, the Tamil Nadu state assembly had adopted a resolution urging that the PM skip the event.
Union Shipping Minister G K Vasan, who belongs to the Congress party, met Singh on Oct 31 to urge him to miss the Colombo session.
--IBNS (Posted on 08-11-2013)