The store owner's move shouldn't stop Niagara police from finding out who's distributing the CDs, said Bernie Farber of the Canadian Jewish Congress.
"How many other record stores are buying it?" Mr. Farber said. "This is the tip of the iceberg."
The owner of Niagara Falls' Sloth record store said he didn't mean to offend anyone.
"Believe me, I'm not a racist", said Frank Florio. "I only brought the stuff in to make a few extra dollars. I wish I never would have."
Mr. Florio faced more than possible criminal charges. A local rabbi and a group called Anti-Racist Action Niagara had threatened protests and boycotts.
"I guess he's gotten scared at the last minute", Mr. Farber said. "It's nice he's pulling the merchandise but it's certainly not because of any changed moral attitude."
Mr. Farber contacted police after learning a newspaper reporter from St. Catharines, who was investigating an outlawed CD title Under The Gods by a band called No Remorse.
"The lyrics, in my opinion, are as hateful as they get", Mr. Farber said.
One song, called White Country, includes the lines: "Too many immigrants, junkies, and queers, the rot is increasing every year ... you've got to fight for a white country."
Another song is called Final Solution, an apparent reference to the Nazi extermination of six million Jews during the Second World War.
Anti-Racist Action Niagara said the store also sold editions of Resistance magazine that used racist names for blacks and immigrants.
Several issues of Resistance have been banned as hare propaganda by Revenue Canada. the magazine is published by a Detroit record label that distributes music by several hate-rock groups.
Mr. Florio's decision to stop selling the material is a change from Thursday, when he told the St. Catharines Standard that people should be able to express any opinion they wish.
"There's a demand for it, so I supply it", he said at the time.