This week the conservative Charlotte Republican got a letter from the Virginia-based Tea Party Leadership Fund PAC expressing its dismay over his vote last month to end the government shutdown.
Pittenger was one of 87 House Republicans (including two others from North Carolina) who joined House Democrats in voting for the measure that ended the shutdown. Tea party allies in Congress opposed the measure, holding out for changes in Obamacare and concessions in exchange for raising the debt ceiling.
"You have failed to honor your commitment to your constituents and the values they entrusted you to uphold," wrote Dan Backer, the group's treasurer and general counsel. "All leaders must face and accept accountability for their choices. You chose to disregard your pledge to these American voters and now must be held accountable."
Pittenger included the message in a fund-raising letter to supporters.
"I understand and appreciate their concerns regarding Obamacare, I voted against it 15 times," Pittenger told the Observer. "The Democrats just didn't want to cooperate. I just didn't feel like what they (the tea party) were pursuing made sense. I'm fully committed to dismantling Obamacare ...
"What I think they didn't understand, maybe still don't, is the political process. They only had 14 votes in the Senate. There's no way it would pass."
But Pittenger said he's still concerned about a tea party challenge in an election when turnout is generally down. "Any off-year election is volatile," he said.
Though Pittenger is a first-term incumbent, any challenger would face an uphill battle. In 2012, Pittenger, a real estate investor, spent $2.3 million of his own money.
Barker said his organization could muster a national fund-raising base for the right candidate. But he said the election's not about money.
"At the end of the day, (Pittenger) voted to continue funding the government at these unsustainable current levels, to increase the debt limit without a single concession, and to fund Obamacare," Barker said.
"Pittenger and the surrender caucus folded like a cheap deck of cards."