Few people are buying Devin Nunes' explanation for why he was at the White House last week

The revelation that the chairman of the House Intelligence Committee, Devin Nunes, was on White House grounds the day before he briefed President Trump on executive branch documents related to surveillance of his transition team has fueled speculation that the White House itself was Nunes' source.

Nunes, who was a member of President Donald Trump's transition team, said on Monday that he went to the White House to view the classified documents Congress doesn't have "networked access" to those reports, and insisted that no one from the White House knew he was there.

Nunes' spokesman said later that the chairman was there to view the documents in a Secure Compartmented Information Facility (SCIF) at the White House in order to "safeguard the proper chain of custody and classification of these documents."

National security experts, Republican and Democratic congressmen, and White House reporters have argued that it is highly unlikely Nunes could have obtained access to the White House grounds unnoticed. Many have wondered why Nunes couldn't have viewed the documents inside an SCIF at the Capitol after consulting with his fellow committee members.

"This is insane," Juliette Kayyem, a CNN national security analyst and former official at the Department of Homeland Security, told Business Insider on Monday. "You only go to the White House to get briefed by the White House. It's not a meeting point, like a McDonald's off the Turnpike."

Democratic Rep. Eric Swalwell, a member of the House Intelligence Committee, echoed Kayyem's assessment on Tuesday morning: The White House "is not an internet café," Swalwell told MSNBC. "You can't just walk in and receive classified information."

"If this was done the proper way, they could have brought it over, shared it with both parties of the committee," he said, referencing what he said was the bipartisan nature by which intelligence-committee investigations are typically conducted.

"Everyone in the building knows that you're there in the building" when you're a member of Congress and you visit the White House, Swalwell said, adding that "this is what a cover-up to a crime looks like."

"To come onto these grounds at the White House you must be authorized," CNN's Senior White House correspondent Jeff Zeleny said on Tuesday. "Someone invited him in, cleared him in, escorted him in. This White House has made the decision to not say who that was."

Republican Sen. John McCain agreed that Nunes' move was unusual: "I've been around for quite a while and I've never heard of any such thing," he said Tuesday morning on CBS.

White House press secretary Sean Spicer said on Tuesday that while he doesn't know who Nunes spoke to, any conversations the Republican congressman had with his source - assuming that person had security clearance - were "100% legal" and "completely appropriate."

The day after he went to the White House, Nunes, a member of Trump's transition team, announced in an unexpected press conference that the president and his advisers may have had their communications "incidentally collected" by the intelligence community during the transition period.

He said that the reports he had seen had nothing to do with Russia, which is why he felt comfortable briefing Trump on the matter without informing his intel committee vice-chair, Democratic Rep. Adam Schiff, beforehand.

Schiff joined a growing chorus of Democrats calling for Nunes to recuse himself from the investigation into possible collusion between the Trump campaign and Russia, saying in a statement that "none of the committee members - Democrats or Republicans - have seen the relevant documents, now almost a week later."

"There was no legitimate justification for bringing that information to the White House instead of the committee," Schiff added. "That it also obtained at the White House makes this departure all the more concerning."