No, there is nothing direct from Barcelona to Merida, and in Spain, it seems you need to do everything in segments, for train for example, you may need to buy a ticket from Barcelona to Madrid, and then a second ticket from Madrid to Merida. Madrid is also the primary fast train hub, so going through Madrid really can not be avoided for train.
You could piece together a bus journey, avoiding Madrid, going closer to the coast (Valencia, through Andalusia, and on to Merida, but I think you would want a stop or two along the way, and schedules just may not align to do it in one day. Again, in the Alsa app, you just can't put in Barcelona and Merida, but must break it up into segments. There are also numerous bus companies, so someone else may run a route.
Once you get beyond Merida, bus is the most likely option into Portugal, train connections are almost non-existent.
I have had decent luck just using the public transport option in Google Maps, mainly to get names of train and bus companies and potential schedules, then going to the companies site to get the best information and other options.
Many like a car, probably not as cost effective if you are by yourself, with others it can be. You definitely do not want to look at picking up in Barcelona and dropping the car off in Lisbon, the fees are very high for that, if they even allow it. You need an IDP, I would consider insurance from the rental agency, or verify that your credit card has very good insurance. I have not driven in Spain, but have in Portugal, it really was not too bad, and outside of major cities, I do not think Spain would be an issue.