The Rev. Al Sharpton said Wednesday he refused to accept an apology from actor Michael Richards, who spouted a racist rant at a comedy club last week.

But Sharpton said he did agree to meet with Richards to try to start a process to address racism in America.

"I [told Richards] you need to sit down and deal with this," Sharpton told CNN. "This is not about accepting an apology, this is about starting a process to really deal with the continual problem of racism in this country." (Watch Richards' meltdown on stage -- 1:39 Video)

Sharpton said he got the call from Richards earlier Wednesday.

"I think that what he did was so injurious that he has to sit down with a group and decide how he tries to ... deal with healing the obvious problem he's got in his own mind and his own heart, because it couldn't come out of you if it wasn't in you," Sharpton said.

The incident took place Friday night at the Laugh Factory in Los Angeles, California.

In a video clip posted on the celebrity gossip Web site TMZ.com, obtained from an audience member, Richards unleashed a nearly three-minute tirade punctuated by racial slurs at a group of African-American hecklers.

In it, he made a reference to lynching and added shortly before walking offstage, "That's what happens when you interrupt a white man."

TMZ.com is a subsidiary of Time Warner, as is CNN.

Richards apologized Monday on CBS's "Late Show" with David Letterman, saying he is not a racist.

"I was at a comedy club trying to do my act and I got heckled, and I took it badly and went into a rage," said Richards, who appeared on the show via satellite.

"For me to be in a comedy club and flip out and say this crap, you know, I'm deeply, deeply sorry." (Watch Richards apologize -- 2:28 Video)

Sharpton said Letterman's show was not the appropriate venue for such an apology.

Richards played comedian Jerry Seinfeld's oddball neighbor Kramer on the NBC comedy "Seinfeld," which ran from 1990 to 1998.

Sharpton said he also chastised Richards, telling him, "Here you are, somebody we had in our living room, you were Kramer. We were used to you and to come and see you say this is frightening to many Americans."

The use of the "n-word" by both blacks and whites needs to be stopped, Sharpton said.

Richards has hired New York public relations expert Howard Rubenstein to help deal with the fallout from the incident. (Watch a history of Hollywood's bigoted outbursts -- 1:55 Video)

In an interview with CNN, Rubenstein said one of first pieces of advice he gave to Richards was to call Sharpton.

"I knew that would be a good place to start," said Rubenstein, who is a longtime friend of Sharpton.

Rubenstein said Richards also called and spoke with civil rights leader the Rev. Jesse Jackson, and Jackson accepted his apology.

"He will be doing a good deal of apologies over the next few months," Rubenstein said.