6 weeks in Europe/N. Africa w/ a 4-month-old—recommendations?

I can't speak to having done anything like that, but I think you are on the right track in where you are looking. I would definitely consider anything in Scandinavia. Even in small towns in Denmark, for example, most people would speak English. You could also consider smaller but fairly international cities like Billund, where there is a huge expat community for such a small town. It's walkable, with great family infrastructure and some local day hiking areas--easy to do with a kid on your back since it's Denmark and therefore flat. The downside is that you would definitely need a car.

You might also consider towns near big cities--there are probably a lot of cool places near Amsterdam or Brussels. The closer to water you are, the better at that time of year. However, the cooler the destination, the less likely you are to find AC (you generally only need it for a few weeks a year at most, but of course August is that time). If you are willing to be in a place where English is spoken, but perhaps not as reliably, I can HIGHLY recommend Glückstadt near Hamburg. Very cool little town center, very walkable, when it gets warm you can walk along the dike on the Elbe, look at the sheep, or just ride the ferry back and forth, and the community pool is fantastic. It's a great place for families, and it has pretty regular train and bus service, with quick access to Hamburg. It's a cute town with old buildings, only about 10K people, and a good balance of town, nature, and city access. You can bike (town offers bike rentals) to the beach along the Elbe at Bielenberg, canoe along the little river through town (the Rhin), there are farms and ponds and parks, it has a market twice a week (IIRC), proper grocery stores and bakeries and pharmacies, and a harbor that looks almost exactly like Copenhagen's famed Nyhavn--just without the crowds.

https://glueckstadt-tourismus.de/en/

My niece (with two small children) lives in Zurich and has never needed (or even has a car). It is clean, safe and English is spoken by nearly everyone. The Cotswolds is also beautiful, but you would probably need a car. Anywhere in North Africa in July would be too hot

1. Most expensive places, Switzerland and Norway are very expensive. Northern Europe tends to be more expensive than Southern Europe. Western Europe tends to be more expensive than Eastern Europe.

2. North Africa includes most third world counties that are significantly poorer than Europe. Morocco and Egypt are great to explore, but not sure I would advise taking a baby there.

3. Consider moving your base city every month, that way you can visit more places. Basing has the benefit of staying in one place, but you waste resources on excessive travel cost and wasting time as well.

4. Italy is loaded with history, art and great dining, I recommend you base there, either in Tuscany or Umbria. Siena, Florence (more expensive), Spoleto. Also, consider Verona in Northern Italy. Italy's trains are great. You don't want to drive with an infant. Other places to consider are Budapest, Hungary or Augsburg, Germany. Also, Strasbourg, France is great.

5. Are you planning to take your infant with you when you tour? Without a known respected child care, I expect so.

One consideration ought to be access to medical care, particularly for the baby. Travel medical insurance, obviously, but I would invest a little bit of time researching access and cost and who takes what insurance. And I would prioritize being within 1-2 hours of an airport with a direct flight home in case of emergency. So, not the Azores or other remote islands.
FWIW, I don't think you need a car in the Cotswolds. This is an area where you can set the map wide on AirBnB to see what fits your needs and budget (and has availability). You want to be in a market town, IMO, rather than a village, preferably one with a train station. Make sure the area has all the amenities you're looking for.