Kids not understanding why I cant attend their church.?

By this I mean no harm to anyone. There are a couple of really nice little girls that will come to my house and give me pamphlets about their church. I give them the old ones back so their church don't have to spend extra money, try to conserve. The little girls are black, very sweet. I am white. They are wanting me to join their church. The church is a black church. I don't want to be the token white person there. I don't know how to tell them nicely I cant go there. I prefer at this time not to attend any church but don't mind talking to them about beliefs. I live across the street from the church I was baptized at. I'm not anti black, but I would be stared at, talked about and just uncomfortable anyway. I need a nice way to say thanks but no thanks.