Taylor represented New Zealand in 112 Tests, 236 ODIs and 102 T20Is.
Taylor retired from the game earlier this year, having represented the BlackCaps in 112 Tests, 236 ODIs and 102 T20Is.
Taylor, who has Samoan heritage on his mother's side, narrated in his book "Ross Taylor Black and White" how he and other teammates had to deal with insensitive "banter" from white players.
"In many ways, dressing room banter is the barometer," wrote Taylor, who played his last international match in April.
"A team mate used to tell me, 'You're half a good guy, Ross, but which half is good? You don’t know what I’m referring to.'
"I was pretty sure I did. Other players also had to put up with comments that dwelt on their ethnicity. In all probability, a Pakeha (white New Zealander) listening to those sorts of comments would think, 'Oh, that’s okay, it’s just a bit of banter'.
"But he’s hearing it as white person, and it’s not directed at people like him. So, there’s no pushback; no one corrects them."
New Zealand's most prolific Test batter with 7,683 runs, Ross Taylor also admitted that these episodes had a negative impact on him.
"You wonder if you should pull them up but worry that you’ll create a bigger problem or be accused of playing the race card by inflating harmless banter into racism," he said. "It’s easier to develop a thick skin and let it slide, but is that the right thing to do?"
He said former staff members had made uncomfortable comments that "landed with a thud".
"Let me be clear: I don’t think for one minute that they were coming from a racist perspective," he said.
"I think they were insensitive and lacked the imagination and empathy to put themselves in the other person’s shoes.
"Instead of the message being, 'You’re one of us, mate,' it is, in effect, 'You're one of them'."
The 38-year-old also urged New Zealand Cricket (NZC) to do more to bring Polynesian talent into the sport.
"New Zealand Cricket should be putting more resources into the Polynesian community because there must be more where I came from," he wrote.
(With Reuters Inputs)