4 days to kill in Andalusia

Jerez would be a good choice, also Ronda. Arcos de la Frontera is another, a good overnight if you can get the car through the narrow streets up the hill. El Convento is a nice hotel in Arcos, and there's also a Parador on the plaza, both with outstanding views.

Easter 2022 is on April 17, so Holy Week--a very, very big deal in many places in Andalucia--begins April 10. It sounds as if you'll still be in Spain for part if not all of Holy Week. It will significantly affect lodging availability and costs in Andalucia; I found that to be especially the case in Seville and Cadiz. You may find more-stringent-than-usual cancelation policies during Holy Week, which is something to consider, given COVID-19 uncertainties. Your planned routing, ending in Madrid, will minimize your exposure to Holy Week prices but also limit your ability to observe or take part in the festivities. You may want to adjust your itinerary to experience Holy Week in the area that takes it most seriously.

Four nights is the minimum amount of time I'd recommend in Seville, but if it's your first stop in Europe, I'd add an additional night since your first day will probably be jetlagged and sleep-deprived. If you end up going to Seville during Holy Week, it's worth considering adding a sixth night there, because you just can't be as efficient a sightseer during Holy Week, especially in Seville. It's a large city with scattered sights. The processions block even pedestrian traffic in different areas at different times, so it takes longer to get from sight to sight. Because of the processions, some sights close for part of the day, resulting in different schedules for sights you'd normally try to see one after the other.

For the Seville-Granada-Cordoba part of your trip, a car is not going to save any time; trains would be about as fast, and you wouldn't have to locate or pay for parking; be sure you consider the parking situation in choosing your hotels. The car could be useful for the additional 4 nights you're trying to fill and certainly would be for the white villages.

I'd prioritize Ubeda and Baeza over Jaen, though Jaen is worth a stop as well. Ubeda and Baeza don't have train stations right in town, so they are visited by bus or car. Ubeda, incidentally, is known for pottery.

You could probably fit in Jerez along with the white villages--the quickest approach to Arcos de la Frontera is from Jerez, or alternatively you could combine Jerez with historic Cadiz. I haven't been to Jerez; all I know about it is that there are horse shows (not sure about in March/April) and sherry houses. Jerez and Cadiz have train service that would be about as fast as driving.

There are some nice day-trips should you seek a change of scene, though I think your time allowances will be easily filled without leaving your base cities even aside from the possibility of Holy Week activities: Carmona from Seville and Priego de Cordoba from Granada have decent public-transportation connections. Vejer de la Frontera is a stretch from Jerez, and only recommendable with a car; it's a bit closer to Cadiz but still takes nearly 2 hours by public transportation.

Thanks for the interesting suggestions. We are well aware of the Holy Week situation and will leave long before Easter. We usually have a car for at least part of our trips. Trains work well but we love the spontaneity of stopping when something interesting appears. The narrow streets in some villages is a concern so we’ll keep that in mind.
Will look at some of the towns you suggested, as well as the hotels. Very excited to be traveling again - finger crossed.

Second the suggestion above about Ronda. We spend 3 nights there about 10 years ago. Beautiful scenery and nice town for relaxing and wandering.

From Seville you can drive to Arcos de la Frontera (1h 30m) and sleep there. Ronda would make a nice day trip from Arcos (1h 45m). The drive from Arcos de la Frontera to Cordoba takes 2h 45m.

Driving to Cordoba is a bit quicker from Ronda than from Arcos, so I wouldn't day-trip from Arcos to Ronda and then drive to Cordoba the next day. That would be a lot of extra driving. I'd also probably prefer to overnight in Ronda if it worked with my schedule, because Ronda's larger and there are a lot more sight there. I guess the practicality of doing that would depend on what time you started the drive to Arcos.

Great title for the next James Bond Movie!

I am going to agree with the previous suggestions for Baeza and Úbeda. Very picturesque towns, so pretty. Great places to wander and explore. Lots of winding narrow streets and beautiful architecture. They are very close to each other surrounded by thousands of olive trees. Baeza had great lookout points over the Guadalquivir River Valley full of olive groves. We also visited a nearby olive farm that produces olive oil and did taste testing. Stay in either town and make sure to visit the other. We stayed in Baeza for three nights. Travelled to Úbeda by taxi one day and returned by bus. All very easy. Even easier if you have a car. We did visit by public transportation but the nearest train station is Linares-Baeza. We took a train from Madrid and arranged to take a taxi into Baeza. From Baeza and Úbeda we took a bus to Jaen. Like others have suggested, I would leave Jaen for another time.

I also liked Jerez de la Frontera. Very Andalucian. I didn't get to nearby Cádiz but if I have the chance to go I definitely want to visit there. Cadiz could be done as a daytrip from Jerez.

One other town that I suggest you look into as it is directly between Sevilla and Córdoba is Écija. I have not yet been there but it is on my wish list of places to visit after seeing many beautiful photos from the town online. I always like to include smaller lesser known towns in addition to the major tourist destinations.

Not necessary for ksw, but for others: There is bus service from Madrid to Ubeda.

Ecija made my list of places to visit, too, but I didn't have time for it.

We’ve just been in Cadiz and were really happy with the experience - we spent 4 nights. It was an easy train trip from Cordoba and an easy day trip to Jerez de la Frontera The walkways along the sea in Cadiz are lovely, the seafood delicious, and the people very kind. We had an apartment so we could buy things at the Central Market for meals or snacks. It’s all good!

I’m not sure what the weather will be like in March/April.