Because a lot of publishers are fucking stupid.
This may not be true, but I read a while ago that while loads of budget is spent on modellers, mappers, artists, etc, etc, etc, hardly any is spent on finding quality writers who can produce a seriously compelling storyline that isn't full of cliches and is generally an uninspired bag of shit.
I'm inclined to believe it because most games I've played recently are indeed uninspired bags of shit, and I don't think I've actually genuinely laughed at a game in years. It's so rarely that you get a game with a rich storyline like MGS that's got twists a ten-year-old couldn't conjure up that I want to break down into tears.
I personally believe that a good story, setting, and characters are just as vital to the enjoyment of a game as gameplay as far as leaving an impression on you goes, which is evidenced by games like Grim Fandango and Monkey Island that admittedly haven't got the most exciting gameplay ever (basically just a long string of "Use X with Y, and if that fails try A, B, C, D, E, F, and G with Y" puzzles), but because the story, settings, and characters are so incredibly realised they leave a massive impression on most who play them and indeed make people often quote the as amongst the best games ever.
It's strange that so many games are still failing to hit the mark regarding balance of the various elements. It's always either too much graphics, too much gameplay (feels sterile), or a massively uninspiring world/story/etc. Then again, the film industry piles out no end of shit too so I guess it's not so surprising after all.