There’s a good post by Mike Brotherton at Science Fiction and Fantasy Novelists arguing against the idea of a dichotomy between fantasy and science fiction, in which fantasy supposedly is (or should) be about things that are not amenable to rational analysis. I’ve never agreed with the idea, which I hear from time to time, that fantasy and magic become less fantastical or magical if they follow rationally explicable rules. Aside from the fact that a truly unscientific world would probably be incomprehensible to minds built for a causal, rule-governed universe, it’s bad for the narrative- you can’t have drama without constraints on what the characters can do, and the characters can’t have constraints if just anything can happen. (This is known in literary theory as the “Superman Just Invents New Powers Out of Nowhere Every Five Minutes” problem.)
A rather large caveat: just because phenomena follow rules doesn’t mean they need to be explained, necessarily. H.P. Lovecraft was a materialist who expressed that worldview in his fiction- his monsters and “gods” were generally beings with unimaginable technology or subject to alternate physical laws, but not “supernatural” in the sense of being apart from the material universe. Nevertheless, Lovecraft did not go into much detail- we don’t learn just what the Colour Out of Space was, or how Cthulu’s biology allowed him to survive being rammed by a boat, or why the Great Old Ones can only awake when the stars are right, and we’d probably sink into gibbering lunacy if we tried to fit that knowledge into our brains. There’s nothing wrong with leaving things unexplained, or even saying that some things in the story are beyond the capability of humans to ever understand. But they should be in principle understandable, even if not from a human perspective. This approach also calls for some caution, since “this fictional world is a place where some things are beyond our comprehension” can degenerate into “stuff happens for no reason aside form the author writing himself into a corner” if care is not taken.
Even a supernatural, animistic universe is potentially amenable to scientific analysis, though perhaps without the rigor of physics. If earthquakes are what happens when the land spirits quarrel with each other, a knowledge of plate tectonics won’t help you predict earthquakes- but studying how the land spirits think and interact might. That would quite an interesting setting, actually- a world where humans understand and control nature not through the hard physical sciences, but through psychology and sociology. If someone wants to go to the trouble of writing an entire book about it and then give me half the money for the thirty seconds of work I contributed, knock yourself out. And, more conventionally, fantasy magic is rule-based- say certain words or perform a certain ritual or whatever and get a certain effect, even if the actual mechanism is not understood.
More subjectively and personally, with a very few exceptions (such as some horror stories, where the unknown nature of the horror makes it scarier) I’ve never understood the idea that explaining something makes it less interesting or less impressive. The little glowy thingamabobs in the night sky are pretty, but knowing that they’re colossal nuclear reactors putting out enough energy to be seen trillions of miles away makes them much cooler, and in general the night sky is more sublime when you have a sense of the true scale of it. I’ve spent over a year childishly infatuated with a woman working at my neighborhood bar, which is pretty damn stupid, but my excitement when she stops to talk with me and my inane attempts to amuse her with my horrible jokes and patented Guinness Mustache embarrass me slightly less when I think of how the flood of phenylethylamine in my brain that turns me into a mumbling idiot around her is an adaptation forged over millennia of evolution that promotes the survival of the human species. (I acknowledge that this is not a terribly romantic sentiment.)
Brotherton also hits one of my pet peeves- stories where the skeptical “scientist” character keeps insisting that blatantly supernatural events must have a mundane explanation, no matter how untenable the notion becomes in the face of mounting evidence. To be fair, a certain amount of this is probably realistic- people don’t give up a strongly held view of the world easily. It often gets taken to ridiculous lengths, though.
It’s especially annoying if the character lives in a fantastical world and has been frequently exposed to supernatural events in the past, yet once again becomes a dogmatic materialist every time another clearly supernatural phenomenon comes along. “Look, I know we’ve previously encountered vampires, werewolves, nymphs, Goetic demons, leprechauns, the Wandering Jew, valkyries, the prophet Elijah, tengu, nephilim, the Spear of Longinus, Satan, our own time-traveling past life incarnations, and the entire Aztec pantheon. But the idea that ghosts might be real is just absurd!"
Update: Edited to fix a typo.