Lodging near the Louvre Museum and the Musee D'Orsay in Paris

September is a very busy season in Paris and peak season room rates are in effect. I think a budget in euros or USD would be a good addition to your request because nothing too expensive is pretty subjective. For example for Paris, for my sister, that would be $250 to $300 per night but for my brother that would be $150 to $175. Also are you looking for a room to share or two single rooms?

First, the Louvre and d'Orsay are not really that near to each other; within walking distance, about 15 minutes, but not really close. The Louvre is in the 4th Arrondissement, and the Orsay is across the river in the 7th; look at a map.

Paris is a compact and very safe city, and has marvelous public transit, so visiting the sites should be quite easy no matter where your hotel is located. My suggestion would be to look for a hotel in either the 5th or 6th Arrondissements, where there are lots of both hotels and restaurants. Also, you should start stating your budget in Euros, rather than dollars, as a number of countries use the dollar, like Australia.

d'Orsay does not have many hotels or restaurants in its immediate area. The Louvre, and commerce devoted to it, takes up a large area but there are hotels within reach. Work your way east on Rivoli towards the Marais neighbourhood, or cross Pont Neuf bridge to the Left Bank. Google Maps lets you search; call up the map for the Louvre and click on the Nearby menu.
It is easy and safe (with due precaution to pickpockets) to ride the Underground around any district t.

Booking.com allows you to search using a map. You won't find inexpensive hotels near those museums, those are high-rent neighborhoods. But a hotel near a Metro station will allow you to get anywhere quickly. Personal safety won't be an issue in central Paris, but look out for your possessions on the Metro and in crowded areas where tourists congregate.

Look at the Grand Hotel des Balcons https://www.balcons.com/en/ and the Hotel de Suez https://www.hoteldesuezparis.com/en/hotel.html, and see if they are suitable for you in terms of price and amenities.

They are near each other, very convenient to the metro, surrounded by loads of restaurants, and as said above, safety is not an issue at these or most other Paris hotels. Whether they are "not too expensive" will depend on what you want to spend. And they are older hotels with small and quirky rooms; that's exciting for some, acceptable for others, and a nightmare for still others. I just stayed at the Hotel de Suez in May 2018, and the only real problem I encountered was that there were only two outlets in the room, neither one near enough to my bed for me to use my phone in bed while charging it (even with a 10 foot cord). Another possible issue is that neither one have air conditioning; in late September, it's probably not needed, but I definitely wouldn't stay in these hotels in summer.

Also, depending upon your budget and need for room service, etc. An AirBnB type thing might work. You can use their map search function to narrow down your area to be convenient to both the Orsay and the Louvre, look at the pictures and read the reviews. Make sure the rental has a tax number! But, as others have said, your timing will limit your selection. (Earlier in September we are staying in an AirBnB not far from the Orsay, probably within 750 feet of there and about 1000 to the Louvre, but made our reservation months ago.)
Good luck!

Maybe you can get a room for two at the Mije? It's a hostel so reasonably priced (includes a breakfast) and in the Marais so close to sights. It's one of my favorite places to stay in Paris because of cost and location.