It is less of a stretch to envision a population of Homo erectus evolving into the Hobbits. After all, its brain has the strongest overall resemblance to that of Homo floresiensis. And Homo erectus have been in southeast Asia for at least 1.8 million years. Perhaps a few Homo erectus individuals were swept onto Flores hundreds of thousands of years ago and gradually evolved smaller brains.
Blimey, Gandalf, the little buggers are everywhere. Hide the textbooks.