I know exactly what you mean about being scared to snap something coz plastic trim is so fragile. Usually, there are little clips or tabs holding them in, and they can be tricky to find. Try gently prying around the edges with a thin plastic tool (like a trim removal tool, or even a credit card if you're careful). Sometimes they slide in a certain direction too, so try wiggling it a bit while you're prying. Don't force it! If it's really stuck, maybe check your owner's manual as sometimes they have diagrams. And if all else fails, a quick search on YouTube might turn up a video of someone doing it on a Kona or similar Hyundai. Good luck, and let us know if you crack the code!
