Is there a castle in Germany or Austria that we may stay in that is afordable?
There is a wonderful hostel castle that you can stay at in Bacharach. Jugendherberge Stahleck. There is also a castle hotel that is reasonable in St. Goar called the Schlosshotel Rheinfels. Both of these are great options along the Rhine.
What part of each country? I can think of at least 7 scattered throughout Germany.
Schlosshotel Rheinfels is a hotel built into/next to the castle ruins. I've stayed there, and I would not classify it as staying in a castle. Look at Castle Colmberg or Castle Veldenstein for actually staying in a castle experience...