I spend three nights in Ljubljana last May. I had a rental car. On one day I drove from Ljubljana to Bohinj and Bled and back, and I stopped at Skofja Loka on the way, where I spent maybe an hour or 90 minutes wandering around. Went up to the castle, where the views are impressive.
In retrospect, I would say it was not worth the time, because the drive from there to Bohinj was absolutely stunning, and I had to cut my time short in Bohinj in order to make it to Bled with time to walk around the lake, and then get back to Ljubljana at a reasonable hour. I would have rather spent more time enjoying Bohinj and the scenery closer to there.
I'd say stop in Skofja Loka if time permits, but don't consider it a must-see. It's a charming town, but not so charming that you won't see other places equally or more beautiful.