On Monday, the Derbyshire Dales District Council stated that the 18th Century iron and wood sign in Ashbourne would be "removed with immediate effect."
Before the council could act, about 150 residents showed up to remove the sign, and vowed to return it.
On Friday, a petition was launched asking for the removal of the sign. It garnered over 40,000 signatures.
According to Ashbourne resident Mark Redfern, the head is “part of the Grade II*-listed sign for The Green Man & Black's Head Royal Hotel in St John Street,” and "will be restored while on the ground and returned to its position at a later date."
He stated that local councilors were present during the removal.
Redfern stated that the head would get "a lick of black paint" to help save it from vandalism.
His son, 17-year-old Shaun Redfern, began a petition asking that the head remain for historical reasons. It has received over 4,000 signatures.
Resident Darren Waring stated that he had been involved in the removal of the sign, and maintained: "People are making these racist overtures without knowing the facts." He explained that the head was that of a Turkish man who had traveled to Ashbourne hundreds of years ago and was involved in the coffee trade.
"He brought prosperity. He was a well-respected and well-loved Turkish man so in his honor they put his head up. It has nothing to do with slavery," he said.
He added that he had helped restore the sign with his father in the 1980s.
"The sign was actually dark brown," he said. "My dad explained to me this is a Turkish man, not a black man. It was painted brown then something has gone wrong in the restoration."
The council's decision to take down the head comes after the statue of a slave trader in Bristol was removed during demonstrations on Sunday.
A council spokesman stated that a councilor had approached the group and explained that the council needed to temporarily remove the sign for the sake of public safety.
He said: "The group, who had ladders, then decided they would remove the figure themselves for safe keeping and, not wanting to create a confrontation, in the circumstances we did not object. We expect to have possession of the head figure later today."
A 20-year-old anthropology student who wished to remain anonymous stated that the sign resembled a gollywog – the 19th Century rag doll largely considered racist.
"I think people are ashamed of it," she said.
Council leader Barry Lewis stated that while the figure was "clearly culturally insensitive and racist," it should not be taken down.
He stated: "Cultural heritage is there to challenge us sometimes, to make us uncomfortable."
Sources: America Now