The centre of Basel is pedestrian only, apart from one street through the middle where the trams run.
Hence very stroller-friendly. Perfectly normal to push your stroller up the middle of Freiestrasse (the "main street"). Your two are probably too young, but toddlers are not held onto as much as they would be with cars present. Letting them waddle off a few metres is normal.
Click for photo of Freiestrasse. Notice walking in middle of road, because cars are banned. The red tower is the Rathaus (town hall). In front of this is the Marktplatz with the Markt (market). Click for photo
And click for photo of Münsterplatz (cathedral square). Again no cars, just a bicycle. Same applies to all the streets linking these places.
Most important Basel is a tram city. It has a dense network of trams so you can get from anywhere in the city to anywhere else. Each route has 1 tram every 7½ minutes. If you stay in a hotel in Basel you get a transport pass for the duration of your stay.
Most trams are low-floor, i.e. level access from street into tram, and they have areas set aside for strollers (and wheelchairs). I couldn't find a photo. There is no charge for strollers, or under-6. They really try hard to make it a family friendly place where you can get anywhere and do anything without a car, and in many cases cannot do it with a car.
Supermarkets in the city do not have car parks. They do have bicycle parking, and are located at prime spots near tram stops.
Colmar by train is a good idea. Also Freiburg im Breisgau and Luzern, both by train. Luzern is on a lake, wih boats you can take trips on.
Trains are also toddler-friendly.
Edit: P.S. Basel zoo all the usual animals (elephants, monkeys, lions, tigers etc etc).
Plus "Lange Erlen" a public park (no entry fee) which has easy-care animals in to. Wild Boar, donkeys, goats, and some European Buffaloes. By October the Storks will probably have flown south for Winter.
Website: https://www.erlen-verein.ch/
