how to implement infinite scrolling using native javascript and yui3 ?

Comments

Hi all, recently I had an opportunity to solve an interesting problem of implementing "infinite scrolling" or "continuous scrolling" or "endless scrolling" using native javascript and yui3. While I thought there was an existing solution for this problem, all of the solutions were pointing to jquery and none were using native javascript/yui3. Hence, I implemented a solution to solve the problem stated using native javascript and yui3.

Notes

As of Jan 2019, there are better ways of implementing infinite scrolling feature. Please refer to https://infinite-scroll.com or https://jscroll.com if you want updated code. The following still holds good conceptually.

Fundamental concept

I will use the below diagram to explain you certain basics about various heights within a webpage and after this solid foundation we will see how to use this in achieving infinite scrolling.

diagram explaining various heights within a webpage

  • pageHeight - This is the overall page height. This is the max height you can scroll. This can be accessed using document.documentElement.scrollHeight.

  • scrollPos - This is the current scroll bar position. This is accessed using document.documentElement.scrollTop in Internet Explorer and window.pageYOffset in Firefox, Chrome, Opera, Apple and other browsers.

  • clientHeight - This is the remaining height that the scroll bar can be scrolled to reach its maximum position, i.e, pageHeight. It can be accessed via document.documentElement.clientHeight.

Now that you know the above 3 heights, let us chalk the criteria for infinite scrolling. We want to initiate an update to fetch more items that will be concatenated to the page when the scrollbar is 50px above its max height. If we write a small algorithm for the same, it will be as below.

Algorithm

  • Check if (pageHeight - (scrollPos + clientHeight) < 50) is true whenever onscroll event is generated on the window.

  • If the above is true, initiate an update and prevent requests for further updates until the response arrives.

  • Update the page if the response is positive else handle error scenarios.

Fetching top RSS stories from YQL

To supply input to the page, I will use YQL to fetch top RSS stories. It can be accessed directly using YQL RSS Query.

https://query.yahooapis.com/v1/public/yql?q=select title,link,pubDate from rss where url='http://rss.news.yahoo.com/rss/topstories'

Since I want to paginate the results, I will pass two additional parameters namely "limit" i.e., number of articles to fetch at a time and "offset" that indicates the index offset to user when fetching further items.

Without digging into details, I directly curl the above api endpoint, parse the result and then generate markup. I also added some CSS to make it look prettier.

Curl the YQL API endpoint

<?php
     /**
     * Function to get RSS Feed Items
     *
     * @param String $user, Integer $page
     * @return object response
     */
     function getRSSFeedItems($limit, $offset) {
         $baseURL = "http://query.yahooapis.com/v1/public/yql?";
         $query = "select title,link,pubDate from rss where url='http://rss.news.yahoo.com/rss/topstories' limit ${limit} offset {$offset}";
         $url = $baseURL . "format=json&q=" . urlencode($query);
         $session = curl_init($url);
         curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
         $json = curl_exec($session);
         curl_close($session);
         return json_decode($json, true);
     }
?>

Markup generation

<?php
     /**
     * Function to build RSS item markup
     *
     * @param Array $data, Boolean $xhr
     * @return String $html
     */
     function displayHTML($data, $xhr) {
         if(!$xhr){
             $html = <<<HTML
     <!DOCTYPE html>
     <html>
         <head><title>Infinite Scrolling using native Javascript and YUI3</title>
         <link rel="stylesheet" href="infinite_scroll.css" type="text/css">
         <body>
         <div class="tip">Scroll to the bottom of the page to see the infinite scrolling concept working</div>
             <div class="stream-container">
                 <div class="stream-items">
     HTML;
         }else {
             $html = "";
         }
         if(!empty($data) && !empty($data['error'])){
             $errorDesc = "";
             if(!empty($data['error']['description'])) {
                 $errorDesc = "<p>Error: " . $data['error']['description'] . "</p>";
             }
             $html .= <<<HTML
         <div class="error">
             <p>Sorry, error encountered, please try again after sometime.</p>
             {$errorDesc}
         </div>
     HTML;
         }else{
             //Extract the results
             if(!empty($data) && !empty($data['query']) && !empty($data['query']['results']) && !empty($data['query']['results']['item'])){
                 $data = $data['query']['results']['item'];
                 foreach($data as $item){
                     if(empty($item['title']) || empty($item['link']) || empty($item['pubDate'])) {
                         return;
                     }
                     $title = $item['title'];
                     $link = $item['link'];
                     $pubDate = $item['pubDate'];
                     $html .= <<<HTML
     <div class="stream-item">
         <div class="stream-item-content rss-headline">
             <div class="rss-content">
                 <img class="rss-image" alt="rss icon" src="rss.png" />
                 <div class="rss-row">
                     <a href="${link}" target="_blank">$title</a>
                 </div>
                 <div class="rss-row">
                     <div class="rss-timestamp">
                         {$pubDate}
                     </div>
                 </div>
             </div>
         </div>
     </div>
     HTML;
                 }
             }else{
                 $html .= "No more top stories";
             }
         }
         if(!$xhr) {
         $html .= <<<HTML
             </div>
             </div>
             <div id="loading-gif">
                 <img src="loading.gif"></img>
             </div>
             <div id="maxitems">Maximum (200) no. of items reached, please refresh the page</div>
             <div id="no_more_rss">No more rss headlines left to fetch</div>
         <!-- JS -->
         <script type="text/javascript" src="http://yui.yahooapis.com/combo?3.3.0/build/yui/yui-min.js"></script>
         <script src="infinite_scroll.js"></script>
         </body>
     </html>
     HTML;
         }
         return $html;
     }
?>

Finishing touch

<?php
     function init(){
         $offset = 0;
         $xhr = false;
         if(!empty($_GET['offset'])){
             $offset = $_GET['offset'];
             $xhr = true;
         }
         $limit = 15;
         $data = getRSSFeedItems($limit, $offset);
         $markup = displayHTML($data, $xhr);
         echo $markup;
     }
     //Call init
     init();
?>

The heart of the hack - Javascript part

I will use YUI3 library to make the ajax part easy. Let's instantiate a YUI3 instance in which we will use "node", "event" and "io-base" modules. We also need to call function "handleScroll" whenever an onscroll event is generated. I assign a local variable self to this in order to avoid scope issues. I'll need to write a detailed post on how we bypass scoping issues by assigning "this" to "self, hence I advise you to look up good javascript books (Definitive Guide or Good Parts) to understand this concept. You can also achieve similar functionality by using anonymous/self-referencing functions.

Instantiating YUI3 and calling handleScroll

YUI().use('node', 'event', 'io-base', function(Y){
var updateInitiated;
var page;
var retries;
var maxRetries;
var allStoriesFetched;
function init(){
     var self = this;
     //Initialize updateInitiated flag and pagination unit
     this.updateInitiated = false;
     this.offset = 15;
     this.retries = 0;
     this.maxRetries = 3;
     this.allStoriesFetched = false;
     window.onscroll = handleScroll;
}

Please note that we also initialized the pagination unit (this.offset) and updateInitiated flag (to disable further requests until response arrives). I also use retries, maxRetries and allStoriesFetched flag for error handling scenarios.

Checking the condition and initiating an AJAX request followed by update

function handleScroll(){
     var self = this;
     var scrollPos;
     if(this.updateInitiated){
         return;
     }
     //Find the pageHeight and clientHeight(the no. of pixels to scroll to make the scrollbar reach max pos)
     var pageHeight = document.documentElement.scrollHeight;
     var clientHeight = document.documentElement.clientHeight;
     //Handle scroll position in case of IE differently
     if(Y.UA.ie){
         scrollPos = document.documentElement.scrollTop;
     }else{
         scrollPos = window.pageYOffset;
     }
     //Check if scroll bar position is just 50px above the max, if yes, initiate an update
     if(pageHeight - (scrollPos + clientHeight) < 50 && this.retries < this.maxRetries && !this.allStoriesFetched){
         this.updateInitiated = true;
         var offset = Y.all(".stream-container .stream-items .stream-item").size();
         //Stop updating once 200 items are reached
         if(parseInt(offset, 10) >= 200){
             Y.one("#maxitems").setStyle('display', 'block');
             return;
         }
         //Show loading gif
         Y.one("#loading-gif").setStyle("display", "block");
         document.documentElement.scrollTop += 60;
         //var url = "http://ravikiranj.net/drupal/sites/all/hacks/infinite-scroll/infinite_scroll.php?offset="+this.offset;
         var url = "http://localhost/infinite-scroll.php?offset="+this.offset;
         var oConn = Y.io(url, {
             on:{
                 success: function(id, o, args){
                     //Update pagination unit
                     args.self.offset += 15;
                     args.self.retries = 0;
                     var resp = o.responseText;
                     var regex = /No more top stories/;
                     if(resp.match(regex)){
                         args.self.allStoriesFetched = true;
                         Y.one("#no_more_rss").setStyle('display', 'block');
                     }else{
                         var list = Y.one(".stream-container .stream-items");
                         list.set('innerHTML', list.get('innerHTML')+resp);
                     }
                     Y.one("#loading-gif").setStyle("display", "none");
                 },
                 failure: function(id, o, args){
                     args.self.retries += 1;
                     Y.one("#loading-gif").setStyle("display", "none");
                     alert('Failed to get data from YQL :(');
                 },
                 complete: function(id, o, args){
                     args.self.updateInitiated = false;
                 }
             },
             arguments: {
                 self: this
             }
         });
     }
}

Notes

  • Here we check the condition (pageHeight - (scrollPos + clientHeight) < 50) is true or not. If yes, we check that there are less than 200 items on the page and no previous update has already been initiated.

  • If all the conditions are satisfied, we initiate an AJAX request via Y.io. You can read up more details about how to achieve AJAX via YUI3 at http://yuilibrary.com/yui/docs/io/.

  • We reset the updateInitiated flag once we receive the response and we also update the pagination unit i.e., offset by 10 if the response received is a set of items and concatenate it to the existing list of items. (Pagination Unit (offset) is not updated in case of failure).

  • Once we reach 200 items, we stop the infinite scrolling... i.e, we make it finite :) ! You can continue forever though :D !

Putting it to together

If we integrate the above mentioned PHP script and Javascript with some CSS and loading gif images, we achieve the Infinite Scrolling. You can find the complete source code @ Infinte-Scroll-SourceCode.

Comments

Comments powered by Disqus