Large Luggage on the Local Bus / ICE Trains

It might help if you told us the dimensions of your 'large' bag. Large to me would be bigger than a roll-aboard.

I think the OP means this: https://kernowfurniture.co.uk/cdn/shop/files/16223_DSC_6099_101769.jpg?v=1747321738

To help with useful advice, could you please specify the size of these large and smaller pieces of luggage. Its very vague. To me a large piece would be a 28" suitcase, while a small one would be my crossbody purse. Others would define them differently.

It sounds like this may be your first time travelling in Europe. A useful reference for train travel is the Man in Seat 61 website. This section deals with luggage: https://www.seat61.com/luggage-on-european-trains.htm#top Cross country buses (coaches) normally have luggage stored under the bus, with personal items such as a day pack kept with you at your feet or in a rack overhead. City buses and metros- you keep them with you, and with your day pack on your front, rather than on your back.
There is also an entire section of this site devoted to packing tips:
https://www.ricksteves.com/travel-tips/packing-light as well as a forum section:
https://community.ricksteves.com/travel-forum/packing

I highly recommend reading them. Most experienced travellers who use public transportation know the wisdom of packing lightly and using the smallest luggage that meets their needs. For example a one week trip could easily be managed with a carryon size suitcase and a day pack. Or if you have specific needs, perhaps one size up to a 24" suitcase. Neither of which would be called a large piece. And it goes without saying that nothing of value (passport, prescription medications, wallet, money, tickets, electronics) should ever be put in your suitcase. Keep them securely on your person or in your day bag which you always keep with you.

Most experienced travellers who use public transportation know the wisdom of packing approprately for their trip and thats generally less than many would presume.

i was wondering what the rules are on travelling with these items. Are there specific places I need to put my luggage on the ICE train/buses?

There are no special rules. You just need to be able to handle your luggage yourself; there are no porters. You can store your luggage in the luggage racks (often at the ends of the carriages), but you don't have to. It often fits between two seats facing each other. There are no luggage racks on public buses. On long-distance buses such as Flixbus, luggage is stowed away and is inaccessible during the journey.

I often travel with a case that is maximum check in size (and consistenly gets delivered on the oversized belt at Schiphol...)

On German trains it has never been a problem to stow it. I either put it in the overhead racks (which are remarkably large) or in the luggage rack at the end.
On buses you can just put it in the space reserved for large items. Buses in Europe have low entry, and space for things like prams and even bicycles. A suitcase is not a problem.

My suggestion however is not to go for a squat spinner, but rather for a more oblong rolling duffel style luggage. These fit better on trains, and having two fixed wheels, is better suited for the kind of abuse your luggage will get while traveling. I drag mine up and down stairs, in and out of trains, over cobblestones and even through the snow, and it takes it all without issue. And when you set it down it does not roll away...