WASHINGTON (AP) — On the campaign trail, Democratic presidential contender Pete Buttigieg blasts Vice President Mike Pence’s cultural and religious conservatism. But as the mayor of Indiana’s fourth largest city, his tone toward the state’s Republican former governor was more muted.

During the four years in which they overlapped in Indiana politics, Buttigieg, the South Bend mayor, had a cordial relationship with Pence. The two collaborated on economic development issues . Buttigieg presented Pence with a South Bend promotional T-shirt that said “I (heart) SB.” And at ceremonial events, Pence would lavish Buttigieg with praise.

The relationship between the two men has come under scrutiny as Buttigieg’s campaign becomes a surprise hit, raking in $7 million during the first quarter. As he formally launches his White House campaign on Sunday, the gay mayor has emerged as a celebrated voice for LGBT equality and religious tolerance. And Buttigieg has hardened his rhetoric toward Pence, using President Donald Trump’s vice president as a foil representing an oppressive opposition.

At a recent LGBT event, Buttigieg spoke of the importance of his marriage to his husband, Chasten, and framed his sexuality in religious terms.

“If me being gay was a choice, it was a choice that was made far, far above my pay grade,” Buttigieg said. “And that’s the thing I wish the Mike Pences of the world would understand: that if you’ve got a problem with who I am, your problem is not with me. Your quarrel, sir, is with my creator.”

He has previously called Pence’s religious conservatism a “fanatical” ideology.

By Buttigieg’s own admission in his campaign memoir, his relationship with Pence is “complicated.” In an interview with CNBC that aired on Thursday, Pence said they had a “great working relationship” and criticized the mayor’s characterization of his religious beliefs.

“He knows better,” Pence said. “He knows me.”

The complex dynamic between Buttigieg and Pence reflects a connection born of political necessity between two men on the cusp of unlikely political and personal paths. And to some fellow Indiana Democrats, it was frustrating.

In a state with few influential Democrats, some hoped to tap into Buttigieg’s burgeoning political celebrity to help them rebut Pence and his policies, according to two party strategists who spoke on the condition of anonymity to discuss a sensitive internal debate. But they found he was often reluctant to take on the then-governor directly or forcefully.

In early 2015, Pence was grappling with an HIV outbreak in southern Indiana and facing calls from political leaders, including some fellow Republicans, to respond by approving a needle exchange program. Buttigieg, who was months away from publicly coming out as gay, was largely absent from that debate.

Pence caused a national uproar that year by signing a “religious freedom” law criticized as anti-LGBT. Republicans including the mayor of Indianapolis demanded that the governor and the GOP legislature “fix this law” and “do so immediately.” Buttigieg criticized Statehouse politicians for the “most embarrassing incident” and tweeted he was “disappointed” Pence signed the bill.

But several weeks later, as the controversy raged, Buttigieg attended a Pence event in South Bend. The mayor told the South Bend Tribune, “I’m focused on how we can work together across city limits and across the aisle.”

“With respect to Pence’s worst blunder, his most difficult controversy, Mayor Pete was not leading the charge against him,” said Robert Dion, a political science professor at the University of Evansville in southern Indiana. “If you’re a Democratic mayor in a Republican state and you rely on friendly relations with the General Assembly, you don’t necessarily want to be lobbing bombs all the time. But that should not preclude someone from speaking out forcefully on a pressing civil rights issue.”

Buttigieg publicly came out as gay about three months after Pence signed the “religious freedom” law. In his book, Buttigieg acknowledged that he passed up the chance to urge Pence in person to veto the measure.

“I wish I could say I made a good effort to talk him out of it, but it was clear from the look in his eyes that he had made up his mind,” he wrote, referring to a meeting in which Pence told Buttigieg and other mayors he planned to sign the legislation.

As mayor, Buttigieg’s allies argue he had to maintain a cordial relationship with a governor who held sway over money sorely needed in South Bend.

“It doesn’t help your city if you have an adversarial relationship with the sitting governor, and Mayor Buttigieg recognized that,” said Greg Goodnight, the Democratic mayor of Kokomo, Indiana, who supports Buttigieg’s White House ambitions.

For example, one of Pence’s last major policy achievements as governor was a program that awarded funding to several regions for redevelopment, including $42 million for the greater South Bend area . During a ceremonial 2016 groundbreaking at a long-shuttered Studebaker factory, Buttigieg and Pence stood side by side and shook hands after scooping dirt with their shovels.

“Where they agreed on issues, like economic development, the mayor worked with then-Gov. Pence for the good of his constituents and his city,” Buttigieg spokeswoman Lis Smith said. “Where they disagreed, the mayor stood up to him, like when Pence embarrassed the state of Indiana with anti-LGBTQ laws, or when he wanted to turn away refugees from the state, or when Pence’s policies hurt South Bend’s workers.”

Still, some wish Buttigieg would play it less safe, particularly now that he’s on a national stage. Tony Flora, president of the North Central Indiana AFL-CIO chapter, said the mayor says the right things but should expend more political capital.

“I would look to Pete Buttigieg to be a louder voice and be more forceful,” Flora said. “He came to our rallies, signed petitions and did make public remarks, but he could have been a bit more substantive instead of just making speeches — taking some action and being more supportive.”

Former Houston Mayor Annise Parker, who is a lesbian, said she sympathized with the complicated situation Buttigieg was in as mayor, especially when he dealt with Pence.

“The answer that I came to, and I’m assuming Pete did as well, is that first and foremost, you are the mayor,” said Parker, who is now the president and CEO of Victory Fund, a group that works to elect LGBT candidates. “You are not an activist. You are the mayor. And every time you speak, it has an impact on your city — for good or ill. And it’s something you have to navigate.”