Website speed targets
Page loading time (rendered page):
Less than 1 second = perfect
1-3 seconds = good
3-7 seconds = average
7+ seconds = poor
How fast is fast enough? Great Paul Irish comment
What is fast enough? Well, speed index must be below 1000, and to achieve that you must deliver above the fold items in the first 14kb of the page.
What is important for Google?
Page speed does affects Google Page Rank. Time to First Byte is what affects page rank; directly only that, and not Pagespeed Insight Score. Pagespeed Insight Score influences indirectly.
Loading cleverly?
SEO problems?