I will tell you my experience last year during Semana Santa. I was in Malaga and then Sevilla and ended in Cordoba for Easter Sunday.
Semana Santa (from Palm Sunday through Easter Sunday) is the biggest holiday period in Andalucia. That means the highest rates for rooms and the most crowds. There are long religious processions that wind through virtually all the streets in the historic centers. Each procession can last from 4 to 10 hours, include usually 2 floats and several marching bands, and lots and lots of penitents in colorful costumes. Locals and visitors line up along the routes (there are several) as much as an hour or more in advance of the beginning of the procession. In Sevilla, it is possible tthat there will be 2-4 processions at any one time, from afternoon until the wee hours of the night. Some streets are closed off for the entire week, others are closed off just when a procession is scheduled to pass by. I didn't see processions in Cordoba but I believe they are more solemn, without music, but just as long and just as popular. Several times I felt like a salmon swimming upstream when I encountered a procession while trying to get from point A to point B. I found it beautiful, fascinating, and at times very moving. Trying to keep 4 people together, especially kids, can be difficult.
If you are set on Andalucia and want to experience Semana Santa, I suggest starting in Cordoba and day-tripping to Sevilla. Then go to Granada and end in Malaga. Thursday night is the highlight for processions, and Sevilla gets more crowded and more crazy leading up to it, so it's best to visit there at the beginning of the week, there will still be crowds and closed-off streets but fewer processions. Malaga is smaller and from what I saw, they take the processions more seriously and there is more order. It's also easier to get flights out of Malaga.
If you want to go to Spain, a much better choice for that week would be Barcelona - there's plenty to see and do there to fill up a week, including great day trips, and Semana Santa is pretty much a non-event in Catalonia. Alternately you could split up the time and spend 2-3 nights in Valencia.