Science fiction imagines the world as it isn't. The best of the genre anticipates challenges that humanity will face, or casts new light on things that we take for granted.
Dune, Dune Messiah, Children of Dune, God emperor of Dune, and others. Dune hit me like a sledgehammer when I was at the right age to suffer from messianic delusions. I still consider it one of the greatest stories I've ever read. It has often been compared to the Lord of the Rings, although they are very different. In the forward to the Lord of the Rings, Tolkien wrote “I cordially dislike allegory in all its manifestations, and always have done so since I grew old and wary enough to detect its presence. I much prefer history – true or feigned– with its varied applicability to the thought and experience of readers. I think that many confuse applicability with allegory, but the one resides in the freedom of the reader, and the other in the purposed domination of the author.”
With Tolkien, Frank Herbert was one of the few authors who have ever managed to create a fictional universe with an imagined history so rich that it comes to life. Herbert was a brilliant student of human behavior and institutions (particularly religion, leadership and government), and was fascinated by evolution, ecology and possible variations of human potential. The first book, Dune, has a plot that moves like a freight train through a flawlessly perfect story arc while somehow managing to take the time to introduce a different universe and to meditate intelligently on it; it is his masterpiece. The later books in the Dune series get progressively stranger and less satisfying. Herbert's many other books, including The Dosadi Experiment and Destination: Void, generally make for difficult reading, but can be worth the journey if you share his curiosity about how cultures might develop under different conditions.
Stephenson has a fertile imagination, an engineer's love of mechanism and precision and the tech world, and a great sense of humor. Yet his relationships are usually lightly sketched and his books sometimes lack satisfying female characters. I can imagine that not everyone will be a fan, but if you do like his books, you will probably devour them.
Seveneves This is possibly his best book. It's written in two halves that could hardly be more different. The moon is hit by a high-speed object in the first sentence, and the next several hundred pages chronicle a desperate struggle for survival. The second half of the novel occurs 5000 years later; I won't give away more but it's entertaining and interesting.
Cryptonomicon is a rolicking story with two timelines, one during WWII and one in the present, involving Nazi gold, code-breaking, and two generations of the same families (which is initially confusing, since they have similar names). Stephenson balances his love of the nerdy code-breakers with stories about a tough sergeant who always gets the short end of the stick as he fights his way across the Pacific. The depth is limited but it is one of his most entertaining books.
Anathem A very unusual story, and one I loved, about a society where a caste of monks who hold most of society's scientific knowlege wall themselves off in castles for periods ranging from 10 to 1000 years. The longer the period, the stranger the results when they emerge. The story moves at a soporific pace for nearly half of the book (you begin to wonder why you are bothering), then suddenly accelerates to wild action and a mysterious ending.
Quicksilver, The Confusion, The System of the World This trilogy, also known as the Baroque Cycle, is an incredibly ambitous story about the period from roughly 1650 to 1750 when science, medicine, mathematics, banking, and many other institutions emerged from the oppressive religious states of Europe and began to flower in Europe and the American colonies. The cast is so gigantic that one needs to refer constantly to the Dramatis Personae to remember who is who. The novels are roughly the length of War and Peace, but they give you a vivid feeling for the times, and especially for the "natural philosophers" who were feverishly figuring out how the world worked, for the first time in human history. In one scene, Isaac Newton's college roomate comes back to his dorm room to find that Newton has inserted a spoon under his eyeball and is moving it around in an attempt to understand how his vision works. That seems to be a pretty good analogy for Stephenson's own level of curiosity.
Dodge in Hell. A dying rich man uploads a very high-resolution scan of his brain into the cloud, almost neuron for neuron, and everyone is surprised when it begins to show patterns that look like thought. Other people follow his lead and the book bifurcates into a digital afterlife with strange rules, and a real world where people have an oddly ambivalent view of the hordes of server farms and the vast energy budget that is required to run the rapidly-growing afterlife server cloud. They even begin to change their attitudes about living, since they start to anticipate some version of themselves living forever in the digital world. I didn't like this book as much as some of his previous books, but the ideas aren't as crazy as they sound; people are already trying several of them.
Kim Stanley Robinson
Red Mars, Green Mars, Blue Mars This trilogy tells the story of human colonization of Mars. The story winds slowly through the early days when colonists worked together to survive, into periods of serious conflict, and eventually to the gradual awakening of a new culture and human biology that is truly distinct from that of earth. The geology and ecology of Mars is imagined in tremendous and convincing detail, to the point where the action sometimes drags (one description of a man walking around on a glacier looking at lichens lasts about 25 pages), but as an ecologist I enjoyed it all. Relationships are deep, long-lasting and beautifully drawn; the last book is an extended meditation on old friendships.
The reader comes to understand with a sense of grief and ominous certainty that humans are going to screw it up when we make the jump to another planet (which is no longer as speculative as it was when Robinson wrote the trilogy). One realizes that it's nearly impossible to leave cultural baggage behind, and that even if colonists somehow succeeded in doing so, they would never be completely free from the influence of people on Earth. In some ways the books deeply saddened me, but they were also beautiful and optimistic about people's potential, and the ceaseless evolution of the planet and its people was fascinating to contemplate. Although the plot sometimes moved slowly, the journey was a very rich one.
Banks doesn't bother with realistic science fiction; he just leaps straight into the very, very distant future where nearly anything you can imagine is possible. In fact, he often outflanks the reader, making your own imagination seem rather meager in comparison. His best books are known as the Culture novels. They imagine a future in which benign artificial intelligences ("Minds") run the universe. Although inscrutable and occasionally engaging in violence, the Minds spend most of their time supporting and satisfying the whims of an enormous, lazy, liberal, and largely contented human population who have the resources, freedom and time to do whatever they want. It's slightly less far-fetched than one might initially assume, given human willingness to turn over any boring job to machines and the startling rise of capabilibies in AI. The series' optimism about what an AI-dominated world would be like is unique in my experience, and a refreshing break from the post-apocalyptic genre that dominates fiction today.
Most of the stories involve a sort of secret service called "Special Circumstances," which spends its time trying to inconspicuously nudge uncivilized planets towards more enlightened behavior. Amusingly, the human agents are never really sure if they succeeded or not, in part because they are not sure about the Minds' ultimate intentions. Banks's plots move fast and the action in his books is often intense and entertaining, but the plots are stupendously complex and readers should not expect endings that wrap up all of the threads. He is often more violent than I like (torture appears from time to time), but there is plenty of humor, too.
The real stars of his stories are the artificial intelligences, although they are rarely protagonists. From small drones to Minds that are housed in gigantic spaceships, they are quirky and unpredictable, and their wry sense of humor is displayed in the names of the ships: "Don't Try This At Home," "Unreliable Witness," "No Fixed Abode," "You Would If You Really Loved Me," "Size Isn't Everything," "Death and Gravity," "Xenophobe," "Boo!", "God told me to do it," "Bad for business," "No More Mr. Nice Guy," "Lightly Seared on the Reality Grill," "Experiencing a Significant Gravitas Shortfall," "Lasting Damage", "Hand Me The Gun and Ask Me Again," "Just Read the Instructions," "Of course I still love you," and many, many more. Recently, SpaceX named two of their spaceships after the latter two.
Consider Phlebas Lots of fun from the first paragraph, set during a war between the Culture and the Idiran race; the protagonist is on the Idiran side and the ending is surprising.
The Player of Games A bored, champion board-game player is blackmailed by the Culture into going to an uncivilized planet where a board game is played for life-and-death stakes. An entertaining story that includes a wonderful drone character (as do many of the other books).
The Use of Weapons A mercenary is hired by the Culture to nudge a backwards planet towards a better future. The mercenary suffers tremendously, and has doubts about whether the Culture is supporting both sides. The book features two confusingly interwoven stories moving in opposite directions through time. Neither the mercenary nor the reader is sure who is the good guy.
The State of the Art. A collection of stories, some involving the Culture.
The Excession The Culture encounters an "Outside Context Problem," meaning they encounter an advanced civilization that they hadn't imagined could exist. The plot is wildly complex and far-out, and Minds are the stars of the story. Some of the less social Minds have isolated themselves for long periods and are revealed to have become very odd, and the plot (and the reader's enjoyment) hinge on the fascinating dialogue between them and other Minds.
Inversions This book appears to be a story about intrigue in the court of a medieval king, with hardly a whiff of science fiction. Yet one of the characters is suspiciously like a Culture agent, and the central dilemma appears to be the same that Special Circumstances face; namely whether, how, or when an advanced culture should interfere with a less-advanced culture.
Look to Windward An intervention by the Culture into a civilization went bad in the distant past, and now that civilization looks to get revenge.
Matter Banks introduces fascinating "shell worlds," with spheres enclosing spheres, and a tangled plot involving people from about 6 different civilizations with powers that range from primitive to nearly godlike.
Surface Detail. I found this book hard to love because it features a digital Hell and some odious people who maintain it. But the conjecture about whether a digital avatar could experience reality just as intensely as a flesh-and-blood being is interesting.
The Hydrogen Sonata Banks imagines that entire advanced cultures could get tired of existence and "sublime," meaning to move to a higher plane of existence outside of the 4-dimensional spacetime that we inhabit. In this book, a culture is preparing to sublime, when an urgent question arises about their history. A person's mind state can be digitally stored and transmitted, and intense action unfolds around an effort to recover the mind-state of a key witness, to figure out what really happened.
Ancillary Justice; Ancillary Mercy, and Ancillary Sword, by Ann Leckie. The protagonist of this trilogy is a zombie-like human soldier whose mind is partly run by an AI which controls a spaceship and a lot of other human soldiers; in other words, she is part of a distributed brain. The plot picks up steam when questions arise about how much autonomy the brain's parts have, and who is really in control. She struggles to figure out if the mind is more like a centralized bureaucracy, warring factions, or perhaps even an ant colony. While the details are unrealistic, Leckie's stories point out that if or when artificial intelligence arrives, AIs will have a very different experience than humans do, since they will be able to tap into sensors and processors all over the world; they may operate more like a distributed brain. Even if you reject the idea that AI will ever be intelligent, the questions have resonance with current research which shows that human brains operate more like a collection of semi-independent modules than we would like to admit. Who is really in charge here, anyway?