Before I visited Rome I assumed I would not like it. I had been there briefly as a teenager and practically the only thing I remember was being unpleasantly pursued by Italian men.
Before I visited Rome as an adult I had visited or lived in several large cities that I very much did not like: Chicago, New York City, Paris. I now live in Saint Paul, Minnesota, and am not even fond of downtown Minneapolis. So, yeah, I don't like cities. Traffic, noise, dirt, garbage, getting around, crowds.
Before I visited Rome I had spent a total of 5 weeks in Italy --- Venice, Florence, and several towns throughout Italy including Sicily --- so I loved Italy, but still was rather dreading Rome. Even in the low season. But there were museums and buildings and churches I wanted to see, so I figured I could put up with it.
Instead, Rome surprised me. The first day, on leaving our B&B, I happened to look down a side street and there was the Colosseum. Just sitting there. Huge. Like it was an ordinary, everyday thing. Everywhere we walked we saw such amazing sights. And little things surprised me, too, such as that the traffic stopped when we crossed the street. In the old center of the city, there are no skyscrapers --- it actually feels very human-scale and often quite open. There are many different neighborhoods, each with its own character, and the city is very, very walkable.
There is an infinity of things to see and do --- we have returned to Rome several times and still have a long list of what we want to do next time.
If you do decide to go to Rome, make plans that will minimize the aspects you think you won't like and choose carefully a limited number of sites to visit. Personally, I would avoid the Vatican during its regular hours, I would stay either in a quieter and cheaper neighborhood OR one that was very close to the places I wanted to visit, I would do tons of research about where to eat and where to sleep, I would not try to do too much because that will give you that frazzled feeling, and if possible I would avoid the tourist season or at least the hottest part of the summer.