Yes it's a steep hill from the Harbour up and then a bit of an incline of of town. Isle of Mull Cheese is a farm so you will be wandering down narrow country roads. We also had a car and so I don't know how much space there would be between you and a car passing, however you can go for a long time without seeing a car so it may not be an issue.
**check in advance for when it's open. We tried to go there a 2nd time in our 3 night stay and it was closed on a Friday.
For reference here's what I write in a Trip Report in 2022:
If you go, check out Isle of Mull Cheese https://www.sgriobruadh.co.uk/. It’s a family-owned farm that produces it’s own cheeses. No tours but they have a small restaurant that is only open from about noon to 3 each day…sometimes. We were expecting some small hole in the wall place, but it was beautiful, and packed with locals. To keep it quaint, there are a couple of dogs that roam the restaurant hoping for handouts. One is huge-it may be a wolfhound and he’s big enough to look down on your plate on the table. If you’re not a dog lover you may not appreciate it, but we loved it.
And in another day;
we drove back to Tobermory and decided to go for lunch at the Mull Cheese Farm again. But there was a handwritten sign at the entrance of the road saying they were not going to open today.