Congress leader Mallikarjun Kharge on Wednesday asked why Prime Minister Narendra Modi did not approach security agencies if he felt that former Prime Minister Manmohan Singh was hatching a conspiracy against him with Pakistan's support.

Talking to reporters after the Congress walked out of the Lok Sabha, Kharge said if Modi believed that Manmohan Singh was part of a conspiracy, he should have taken the help of the intelligence agencies, filed an FIR and got Manmohan Singh arrested.

"By taking the matter to public, you have breached the oath of secrecy," he said.

Kharge said it was wrong on the part of Modi to cast aspersions on the patriotism of a former Prime Minister.

Asked if the Congress, which has been protesting in both the Rajya Sabha and Lok Sabha on the issue, wanted an apology from Modi, he said: "Let him come to the House and explain it first."

Modi, during the Gujarat poll campaign, claimed that at a dinner meeting at Congress leader Mani Shankar Aiyar's residence, Manmohan Singh and other guests discussed the Gujarat election Pakistani diplomats.