Mr Kelly said his defection came down to “freedom of speech” as he felt restrained about speaking his mind on coronavirus treatments.
“I won’t hide my disappointment in Craig’s announcement yesterday,” Mr Frydenberg told Today.
“It’s very disappointing. He was elected under the Liberal Party’s banner in his electorate and he was a member of our team.”
Mr Frydenberg said the former Liberal MP had, however, “made it clear” he would continue to provide “confidence and supply to the government”.
“He supports Prime Minister Scott Morrison,” he said.
“I owed it to my mates to look them in the eye and explain my decision face-to-face,” he said.
He reiterated he hoped Mr Morrison went on to be “one of our longest serving and greatest prime ministers”.
Mr Kelly ruled out joining the Nationals or One Nation, saying it would be best for him to run independently.
“I believe I am best to speak independently, so I can speak my mind fearlessly and freely.”
Mr Kelly today again defended his appearance on the podcast, insisting he was a victim of “cancel culture”.
“Pete Evans is not a criminal,” he said.
“If I can’t speak to someone like that … there’s something wrong with our country.”