Patch-working multiple ideas together to create a map is a legit concern. It rarely turns out well, to be honest. It looks like a fair amount of thought has gone into this already though. And anyway, there's nothing to lose. It's not like someone is going to send you a bill if you end up making a bad level. That's never happened to me and I've made my fair share of bad levels. ? Plus, I really just enjoy watching people stretch their comfort zone, and try out unconventional map styles and design approaches. All of that makes for a better designer (assuming there's already a basic level of understanding in place).
The worry with the colors is obviously orientation. There are plenty of ways to assure that won't be an issue though. One of them is what you've already suggested - sticking to a limited number of colors. This allows you to then have a small number of buildings that are colored differently so that they really stand out. Or, you can go the route of using many, many colors, and then use other tools to aid with orientation. The sun or moon, mountain ranges, ocean, distinct architecture or shapes, etc.
Overall, this looks like a very cool concept. I'm curious to see how it turns out.