Before explaining the difference, it might be useful to understand what blog means to many people. In common term, a blog is merely a site to store a chronological timeline of entry/article submitted (I like to use article rather than Post, because Post is more verb than noun eg Post entry/article means submit entry/article) and that what make a blog site different from a website. And it is exactly this common definition of blog that the Page feature is underused or even ignored.
Support for Internal and External Page
Thankfully, WordPress includes support for both external page and internal page. So what is the difference between them ?
The external page is actually a Page in a external website that is referenced explicitly using its URL (eg. http://geekycoder.byethost13.com/geezer.html) and is not stored as part of WordPress repository.