Where and when I grew up, I knew racism existed, but I didn't live with it. My friends were of different colors and as you said we would "deal with folks BASED on them as a person and NOT skin color." If anything we were more aware of people's religions but only because we had to work around religious obligations in order to schedule things on weekends. Race didn't require any thought, so we didn't think about it.
And although my immediate family was not mixed, I spent my summers where my grandmother and great aunt both had their extended families visiting regularly, and two of the "gang of cousins" were mixed race. Again, it wasn't an issue.
Now I'm confused. Before the goal was a color-blind world. Now it seems that race should be taken into account. I don't understand how viewing the world through racially divided lenses will end racism. I do understand frustration that racism, while less than in the past, isn't dead. I get that. But I'm not sure how SOME (and I know it's not everyone) thinks focusing on race is the answer. It seems to be pushing people apart, not together.