Well, there is no need to get all bent out of shape. But, I think the point people were trying to make is that the joke is implied to be from only one race and not any other. Obviously it is asian. Many asian people already have issues to how the world sees them as not being able to say each word clearly or have slanted eyes or many other things that distinguish them from the rest of the other races. I see the humor behind it as so do many others. However, many asians would not feel the same way and therefor would be offended. A perfect example is on On December 5, 2006, Rosie ODonnell made a comment in reference to the November 29, 2006 appearance on The View by Danny DeVito who had been up and celebrating most of the night. ODonnell was totally blown away that the controversy had become such an international media story, and joked that it was being talked about as far away as China. "You know, you can imagine in China it's like, 'Ching-chong, ching-chong. Danny DeVito. Ching-chong, ching-chong-chong. Drunk. The View. Ching-chong.'" Some, including The Asian American Journalists Association, interpreted her comments as a "mockery of the Chinese language" to which ODonnell responded it was simply part of her sense of humor and not meant to make fun of the asian culture. She later apologized on air saying, "To say ching chong to someone is very offensive, and some Asian people have told me it's as bad as the n-word.
When it comes to comedians, there is an expectation that they are going to say something funny/degrading or what have you. But when a person who is not in the industry says it or even if they are, the timing can become everything.