Going into this you should know that the three most historically important cities in Andalucia (Seville, Cordoba and Granada) are not on the coast and will in most cases require at least 2 hours' travel time by public transportation from any small town on the coast. Driving to any of those cities and looking for a place to park a car wouldn't be fun, and in some cases it would take a lot longer than a train. An exception would be a place near Malaga, from which you might get to Cordoba in about 1-1/2 hours by a combination of local train and express train.
Tickets on the fast trains often needed to make day trips to Seville, Cordoba and Granada practical can be very expensive if not purchased well in advance, likely making spur-of-the-moment trips very costly--especially considering you'd have to pay for a round trip and wouldn't even have a full day at the destination. There's also the fact that all three cities would deserve more than one day-trip.
Edited to add: I'd position Barcelona and especially Madrid at the end of this trip, rather than the beginning, for odds of warmer weather--especially in Madrid. Madrid's average low temperature in February is 38.7 F, and it's 43.2 F in March. That's not a trivial consideration, given that Spain operates on a very late schedule, with the locals heading out for dinner at 10 PM.
Is there a reason you want to spend multiple winter/early-spring months in a single location on the coast? It won't be warm enough to swim, and the coast won't be appreciably (if at all) warmer than Cordoba and Seville. If the coast is a must, choose your location very carefully. Much of the coast has been turned into low-budget holiday spots for Northern Europeans seeking sunshine. As a result, thay are full of high rises, British pubs and German sausage joints. It can be hard to find any remaining Spanish character. That said, we have a few posters who know that area well (I do not) and can suggest some options to you.
I do know there are regional rail lines fanning out from Malaga (a very attractive city), so I think a smaller place with a train station in Malaga's orbit might be a fairly convenient choice for making some side trips.
Cadiz is an historic city on the coast (good access to Jerez, which has some horse shows the kids might like), but I'm less sure about easy transportation into and out of Cadiz from surrounding small towns.
These links may help you investigate transportation options:
Main Renfe website for regular trains: https://www.renfe.com/es/en
Commuter rail around Malaga: https://www.renfe.com/es/en/suburban/suburban-malaga/timetables << link corrected
Commuter rail around Cadiz: https://www.renfe.com/es/en/suburban/suburban-cadiz/timetables
ViaMichelin for routes and driving-time estimates: https://www.viamichelin.com/
For buses, I just Google something like bus Malaga to Granada. Not every town has a train station.