Sadhu last week said there is no one better than the BJP strongman in the PM race.
"There is no one who is better than Narendra Modi. Congress Vice President Rahul Gandhi is not even in the race for it, he is far behind," Yadav told CNN-IBN.
"Let the Congress take action against me, who is stopping them," he said.
Yadav met Modi in Gujarat's Gandhinagar on Friday on his way to Dwarka city to visit a famous temple dedicated to Lord Krishna.
Yadav, who is Rashtriya Janata Dal (RJD) chief Lalu Prasad's brother-in-law, had a 45-minute meeting with Modi.
After the meeting, he said Modi has better qualities to become the Prime Minister of India than Rahul Gandhi.
"I firmly believe that Modi has better prime ministerial traits than Rahul Gandhi," Yadav had told reporters.
"Narendra Modi is a powerful leader and we need a powerful leader like Modi as the PM in today's scenario," he said.
Comparing the two leaders, Yadav said: "One reason why Modiji is better than Rahul Gandhi is, he will immediately come to your help when you ask him for it. But you have to wait for three years for Rahul Gandhi's appointment."
--IBNS (Posted on 20-08-2013)