An issue recently introduced in Joomla! 3.3.4 (pull request here) makes navigating back to the first page impossible when using SEF. When you click to go to the second page and then attempt to go back to the first page, you'll actually stay on the same page. This is because the "limitstart" parameter is missing from the URL due to this cosmetic change (which attempts to avoid duplicate URLs). They're fixing it in the upcoming 3.3.7 release (pull request here).
Read on for instructions on how to fix this.