There's an old law amongst writers of speculative fiction: the best writing has speculative or fantastic elements which, if removed, would ruin the story. You will frequently see writers say that if a tale is merely set in an SF universe, it is not a good story, because it could be mainstream ("realistic") writing, and is only posing as SF.
While I have great respect for the many people who believe this, I disagree, because such an idea gives in to the concept of the mainstream, contemporary, "realistic" story as the basis of all fiction, and much as I like much mainstream writing, I don't want to give it so much power over the imagination.
Certainly, writers have written stories which are not particularly interesting or compelling, and some of these stories suffer from having the only SF element be the setting (usually, the setting is not well realized or original, merely a few mentions of spaceships and nanotech machines). Stories which don't work should be criticized and analyzed, but they should be looked at on their own, and we must guard against using bad stories to create grand generalizations about how SF should be written. All formulas, no matter how valuable they seem, are pernicious.
Not all stories are about their setting, though SF stories tend to highlight setting more than mainstream stories, because SF stories seek to create an entire, often unfamiliar, world in the reader's mind, while mainstream writers can more or less assume that when they set a story in suburbia, their readers will understand the culture and details of suburban life.
But -- to make a completely uneducated guess -- 85% of mainstream stories have setting descriptions which are necessary and central to the story, either to highlight themes, give information about character, or take advantage of imagery. Think of some great mainstream writers -- any mainstream writer you consider good -- and more often than not you will have a sense of the milieu they wrote about in your favorite works. Favorite writers of mine, from Hawthorne to Faulkner to J.M. Coetzee and Paul Bowles, all have used setting to their advantage. (Other favorites, such as Donald Barthelme and Carole Maso, have often had other concerns, often using the text itself as a landscape.)
The hardliners say that if a story's SF elements could be removed without injury to the story, then it is not an SF story, no matter how brilliantly it is told. What sort of argument is this, though? A brilliant story is a brilliant story.
From Edgar Allen Poe on, critics have maintained that short stories should have no unnecessary elements. What readers perceive as unnecessary differs, but certainly, it seems to me, an author should know why she or he wrote whatever words were written, and should omit any which seem needless.
Novels are a different matter altogether. Many types of novels thrive on not-completely-necessary detail, though often the detail actually is necessary for giving the reader a full experience. While War & Peace is easy enough to abridge, any abridgement is a different work, a different experience, from the whole huge, ragged original.
If a writer wants to set a story in an SF setting, even though the story could be told in a contemporary setting, or even a historical one, what is the problem? Fiction should be trying to illuminate the experience of being human, of living life, of acting and being acted upon. Why do we assume that the contemporary, "realistic" setting is the basic one, the one that any piece of writing should employ unless it has specific reasons for doing otherwise?
Samuel R. Delany once said that the difference between SF and mundane (from "mundus", of the world) writing is that any sentence from mundane writing could fit into an SF story, but the opposite is not true. Both a mundane writer and an SF writer could write, "The streetlights turned on," but only the SF writer could write, "The streetlights turned around and greeted the woman with a smile." (Unless, of course, the mundane writer was chronicling a dream.)
Hence, when a writer sits down to write, why shouldn't he or she be able to choose to begin from the broadest possible imaginative starting gate? The only way to do that is to give up all assumptions about setting, to build the setting from imagination. It might correspond to what is generally agreed is "reality", or it might not. It might need its imaginative setting to justify fantastic or improbable events, or it might not. It might be a great piece of work, or it might not.
The fear of "non-SF in an SF setting" comes from a fear of diluting the genre of SF. But what if we give up on the idea of SF as a genre at all? What if we assume from the moment we begin that all writers are imaginative writers, and that they chose the setting they did because it seemed the one most appropriate for the story, or even for the writer's mood on the day composition began?
I don't think it's necessarily a good thing to get rid of all genre identifiers -- it's nice when you're in an SF mood to be able to go to the bookstore and find some books which will fit your mood -- but I do worry that it can be a limit on our imaginations, and that too much genre identification can cause writers and critics to seize on questionable propositions in order to keep the genre pure. The urge to purify genre is a fascistic urge, and fascism is the death of imagination, the quantification of creativity into an X/Y graph with parameters around which The Writer Should Not Travel.
Let's judge stories by their merits as stories, not by abstract concepts of what a story "should be". There are exceptions to all such strictures, and the exceptions should be embraced. If a story is a good story, if it works for the reader, if it excites the imagination, then it should be celebrated, regardless of genre, regardless of assumptions, regardless of empty critical pronouncements from on high.
SF writers chart the geography of the imagination, and readers and critics should challenge them to keep that geography boundless.