{"id":14,"date":"2017-01-06T10:56:11","date_gmt":"2017-01-06T10:56:11","guid":{"rendered":"http:\/\/www.zombiesoftwares.com\/blog\/?p=14"},"modified":"2017-01-13T13:58:37","modified_gmt":"2017-01-13T13:58:37","slug":"site-search-bench-marking-via-crowd-sourcing","status":"publish","type":"post","link":"http:\/\/www.zombiesoftwares.com\/blog\/site-search-bench-marking-via-crowd-sourcing\/","title":{"rendered":"Site Search Bench-marking via Crowd Sourcing"},"content":{"rendered":"<h2><b>Objective:<\/b><\/h2>\n<p>Our objective is to device crowdsourcing platform find the relevancy of SRP and benchmark the with competitor. Apart from competitor we could also gauge search result relevancy with production and new releases planned.<\/p>\n<p>Since Site search contributes t<a href=\"http:\/\/www.zombiesoftwares.com\/blog\/the-importance-of-site-search-for-ecommerce\/\">o 60%+ revenue<\/a>s, It is essential to gauge search relevance and device a systematic approach for the same. The crowdsourcing platform could provide us optimal result in a very short span of time,<\/p>\n<h2><b>Query Set Preparation:<\/b><\/h2>\n<p>A comprehensive\u00a0query\u00a0set (2500 in number) , which will be a representative of overall search\u00a0queries\u00a0on our website (this set will form a significant %age of our key metrics \u2013 visits and revenue). It will consist of:<\/p>\n<ul>\n<li>Top 450-500 search\u00a0queries\u00a0based on visits, since they represent 20-25% of the total visit count, and are quite important<\/li>\n<li>From the next 27k\u00a0queries\u00a0(which represent 40% of the search visits), we will randomly\u00a0choose samples from all deciles\/quartiles<\/li>\n<li>Randomly select\u00a0queries\u00a0from the long tail<\/li>\n<\/ul>\n<p>Now, we will categorize this\u00a0query\u00a0set in the\u00a0below mentioned buckets (and all possible combinations). The key objective here is that QUL should be\u00a0able to correctly label these params within a search\u00a0query.<br \/>\n<b>Technical Buckets<\/b><\/p>\n<p>1\u00a0\u00a0\u00a0 Product name\/PRODUCT<\/p>\n<p>2\u00a0\u00a0\u00a0 Brand<\/p>\n<p>3\u00a0\u00a0\u00a0 Cat\/ Sub-Cat<\/p>\n<p>4\u00a0\u00a0\u00a0 Brand + Cat\/Sub-Cat<\/p>\n<p>5\u00a0\u00a0\u00a0 PRODUCT+ Cat\/Sub-Cat<\/p>\n<p>6\u00a0\u00a0\u00a0 Cat\/ Sub-Cat+ Highlights\/Attributes\/Filter values<\/p>\n<p>7\u00a0\u00a0\u00a0 PRODUCT+ Cat\/Sub-Cat+ Highlights\/Attributes\/Filter values<\/p>\n<p>8\u00a0\u00a0\u00a0 Brand + Cat\/Sub-Cat+ Highlights\/Attributes\/Filter values<\/p>\n<p>9\u00a0\u00a0\u00a0 Brand+ Highlights\/Attributes\/Filter values<\/p>\n<p>10\u00a0\u00a0\u00a0 Highlights\/Attributes\/Filter values<\/p>\n<p>Each Buckets will be represented with at least 150 queries<br \/>\n<b>Category \/ Sub \u00a0Category Buckets(Top interms of revenues\/orders)<\/b><\/p>\n<table>\n<tbody>\n<tr>\n<td>\n<ol>\n<li>Mobiles &amp; Tablets<\/li>\n<\/ol>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Computers &amp; Peripherals<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Appliances<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>TVs, Audio &amp; Video<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Men\u2019s Fashion<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Men\u2019s Footwear<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Women\u2019s Clothing<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Watches<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Bags &amp; Luggage<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Automotive<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Kitchenware<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Home Furnishing<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Toys &amp; Games<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<ul>\n<li>Sports &amp; Fitness<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1 class=\"entry-title\"><\/h1>\n<p>Each Buckets will be represented with at least 100 queries,<\/p>\n<p><a href=\"http:\/\/www.zombiesoftwares.com\/blog\/super-best-practices-list-for-ecommerce-site-search\/\">Find out best practices if site Search<\/a><\/p>\n<h2><b>Input to Crowdsourcing Platform<\/b><\/h2>\n<p>The test user will be provided with UI where query and respective top 10 -20 results (?) is shown.<\/p>\n<p>The results will be shown as per our business objective:<\/p>\n<ol>\n<li>New Release Plan<br \/>\nProduct Result and Test avatar results will be shown<\/li>\n<li>Competitor Benchmarking<\/li>\n<\/ol>\n<p>Product Results for host site\u00a0&amp; similar sites like flipkart, eBay and amazon will be displayed.<\/p>\n<p>Product and Competitor results will be shown<\/p>\n<p><b>User Interface<\/b><\/p>\n<p><img data-recalc-dims=\"1\" decoding=\"async\" data-attachment-id=\"420\" data-permalink=\"http:\/\/www.zombiesoftwares.com\/blog\/super-best-practices-list-for-ecommerce-site-search\/job-searching-online-8-best-practices-you-need-to-know\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/job-searching-online-8-best-practices-you-need-to-know.jpg?fit=450%2C450\" data-orig-size=\"450,450\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;DenisIsmagilov&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;Rear view of young professional who is drawing a recruitment flowchart on the whiteboard. A concept of recruitment and job searching processes. Wooden floor, concrete wall and three ceiling lights.&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;DenisIsmagilov&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;1&quot;}\" data-image-title=\"job-searching-online-8-best-practices-you-need-to-know\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;Rear view of young professional who is drawing a recruitment flowchart on the whiteboard. A concept of recruitment and job searching processes. Wooden floor, concrete wall and three ceiling lights.&lt;\/p&gt;\n\" data-medium-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/job-searching-online-8-best-practices-you-need-to-know.jpg?fit=300%2C300\" data-large-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/job-searching-online-8-best-practices-you-need-to-know.jpg?fit=450%2C450\" class=\"alignnone size-full wp-image-420\" src=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=840\" sizes=\"(max-width: 526px) 100vw, 526px\" srcset=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png 526w, https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=130 130w, https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=261 261w\" alt=\"111.png\" data-attachment-id=\"420\" data-permalink=\"https:\/\/producta2zblog.wordpress.com\/2016\/12\/20\/site-search-bench-marking-via-crowd-sourcing\/attachment\/111\/\" data-orig-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=723\" data-orig-size=\"526,605\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"111\" data-image-description=\"\" data-medium-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=723?w=261\" data-large-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/111.png?w=723?w=526\" \/><\/p>\n<ol>\n<li>Most Relevant , Best Result : Totally relevant<\/li>\n<\/ol>\n<p>: The document completely answers the question.<\/p>\n<ol>\n<li>Relevant , Good Result : Partly relevant<\/li>\n<\/ol>\n<p>: The information in the document is relevant to the question but not complete.<\/p>\n<ol>\n<li>Irrelevant , Somewhere Close: Related<\/li>\n<\/ol>\n<p>: The document mentions the subject or holds potentially good hyperlinks to relevant pages, but does not contain any actual information regarding the query itself<\/p>\n<ol>\n<li>Completely Irrelevant, Useless: Not relevant\/Spam<\/li>\n<\/ol>\n<p>: The document is off topic or spam, not giving information about the subject.<\/p>\n<h1 class=\"entry-title\"><\/h1>\n<table>\n<tbody>\n<tr>\n<td>Serial No.<\/td>\n<td>Subject<\/td>\n<td>Question Asked<\/td>\n<td>Scores Assigned<\/td>\n<\/tr>\n<tr>\n<td>1<\/td>\n<td>Most Relevant<\/td>\n<td>Best Result<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<td>2<\/td>\n<td>Relevant<\/td>\n<td>Good Result<\/td>\n<td>2<\/td>\n<\/tr>\n<tr>\n<td>3<\/td>\n<td>Irrelevant<\/td>\n<td>Somewhere Close<\/td>\n<td>1<\/td>\n<\/tr>\n<tr>\n<td>4<\/td>\n<td>Completely Irrelevant<\/td>\n<td>Irrelevant, Useless<\/td>\n<td>0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1 class=\"entry-title\"><\/h1>\n<h2><b>Output from CrowdSourcing Platform<\/b><\/h2>\n<p>The user will be asked to mark each PRODUCTs wrt to query provided, and the score will be stored in following format a query:<\/p>\n<table>\n<tbody>\n<tr>\n<td><b>Query<\/b><\/td>\n<td><b>PRODUCTS<\/b><\/td>\n<td><b>Rank<\/b><\/td>\n<td><b>User Scoring<\/b><\/td>\n<\/tr>\n<tr>\n<td rowspan=\"3\"><b>Q1<\/b><\/td>\n<td>PRODUCT1<\/td>\n<td>1<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<td>PRODUCT2<\/td>\n<td>2<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>PRODUCTn<\/td>\n<td>4<\/td>\n<td>2<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"4\"><b>Q2<\/b><\/td>\n<td>PRODUCT4<\/td>\n<td>1<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<td>PRODUCT9<\/td>\n<td>2<\/td>\n<td>0<\/td>\n<\/tr>\n<tr>\n<td>PRODUCT11<\/td>\n<td>4<\/td>\n<td>2<\/td>\n<\/tr>\n<tr>\n<td>PRODUCTz<\/td>\n<td>1<\/td>\n<td>3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p class=\"entry-title\">For multiple user feedback for same query weighted average rank and weighted average user scoring will be considered.<\/p>\n<h2><b>Post Analysis of Scores<\/b><\/h2>\n<p>We ll be calculating Normalized Discounted Cumulative Gain for each query(explained below), once we have computed NDCG values for each query, we can average them across thousands of queries. We can now compare two algorithms: we take the mean average NDCG values for each, and check using a statistical test (such as a two sided t-test) whether one algorithm is better than the other, and with what confidence.<\/p>\n<h3><b>Calculation of Cumulative Gains<\/b><\/h3>\n<p>Cumulative Gain (CG) is does not include the position of a result in the consideration of the usefulness of a result set. In this way, it is the sum of score values of all PRODUCTs for aquery. The CG at a particular rank position p is defined as:<\/p>\n<h1 class=\"entry-title\"><img data-recalc-dims=\"1\" decoding=\"async\" data-attachment-id=\"422\" data-permalink=\"http:\/\/www.zombiesoftwares.com\/blog\/super-best-practices-list-for-ecommerce-site-search\/attachment\/422\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/Zara_site_search-1.png?fit=2444%2C1114\" data-orig-size=\"2444,1114\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/Zara_site_search-1.png?fit=300%2C137\" data-large-file=\"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/Zara_site_search-1.png?fit=840%2C383\" class=\"alignnone size-full wp-image-422\" src=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/222.png?w=840\" alt=\"222.png\" data-attachment-id=\"422\" data-permalink=\"https:\/\/producta2zblog.wordpress.com\/2016\/12\/20\/site-search-bench-marking-via-crowd-sourcing\/attachment\/222\/\" data-orig-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/222.png?w=723\" data-orig-size=\"125,56\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"222\" data-image-description=\"\" data-medium-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/222.png?w=723?w=125\" data-large-file=\"https:\/\/producta2zblog.files.wordpress.com\/2016\/12\/222.png?w=723?w=125\" \/><\/h1>\n<p>Calculation for provided data<\/p>\n<p>After Computing weighted average scores for rank and scores, the CGs score as follows<\/p>\n<table>\n<tbody>\n<tr>\n<td><b>PRODUCTS<\/b><\/td>\n<td><b>Rank<\/b><\/td>\n<td><b>User Scoring<\/b><\/td>\n<td><b>CG<\/b><\/td>\n<\/tr>\n<tr>\n<td><b>PRODUCT1<\/b><\/td>\n<td>1<\/td>\n<td>3<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<td><b>PRODUCT2<\/b><\/td>\n<td>2<\/td>\n<td>0<\/td>\n<td>3<\/td>\n<\/tr>\n<tr>\n<td><b>PRODUCT3<\/b><\/td>\n<td>3<\/td>\n<td>2<\/td>\n<td>5<\/td>\n<\/tr>\n<tr>\n<td><b>PRODUCT4<\/b><\/td>\n<td>4<\/td>\n<td>3<\/td>\n<td>8<\/td>\n<\/tr>\n<tr>\n<td><b>PRODUCTn<\/b><\/td>\n<td>5<\/td>\n<td>1<\/td>\n<td>9<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1 class=\"entry-title\"><\/h1>\n<h3><b>Calculation of Discounted Cumulative Gains<\/b><\/h3>\n<p>The premise of DCG is that highly relevant documents appearing lower in a search result list should be penalized as the score value is reduced logarithmically proportional to the position of the result. The discounted CG accumulated at a particular rank position is given by:<\/p>\n<p>Rel1-&gt; Score for top position PRODUCT,<\/p>\n<p>Reli -&gt; for any position PRODUCT<\/p>\n<p>Calculation for provided data<\/p>\n<table>\n<tbody>\n<tr>\n<td>\n<h5><b>PRODUCTS<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>Rank(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>User Scoring(rel)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>CG<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>Log<\/b><b>2<\/b><b>(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>rel\/ Log<\/b><b>2<\/b><b>(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>DCG<\/b><\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT1<\/b><\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>N\/A<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT2<\/b><\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT3<\/b><\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>8<\/h5>\n<\/td>\n<td>\n<h5>1.585<\/h5>\n<\/td>\n<td>\n<h5>1.892<\/h5>\n<\/td>\n<td>\n<h5>6.892<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT4<\/b><\/h5>\n<\/td>\n<td>\n<h5>4<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>8<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>6.892<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCTn<\/b><\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>9<\/h5>\n<\/td>\n<td>\n<h5>2.322<\/h5>\n<\/td>\n<td>\n<h5>0.431<\/h5>\n<\/td>\n<td>\n<h5>7.323<\/h5>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h3 class=\"entry-title\"><b><br \/>\nCalculation of Normalized Discounted Cumulative Gains<\/b><\/h3>\n<p>The Normalized part in NDCG allows us to compare DCG values between different queries.<\/p>\n<p>Search result lists vary in performance depending upon query \u00a0. It\u2019s not fair to compare DCG values across queries because some queries are easier than others: for example, maybe it\u2019s easy to get four perfect results for the query samsung s4, and much harder to get four perfect results for short micro usb cable .<\/p>\n<p>This done normalizing DCG wrt to ideal discounted cumulative gains (IDCG), which is the best possible score is given the results we\u2019ve seen so far.<\/p>\n<p>Example : \u00a0the best scores for query with increasing order of rank are<\/p>\n<p>3 3 2 2 0, then <b>IDCG = 8.01<\/b><\/p>\n<p>Our NDCG is the score we \u00a0for given set DCG divided by the ideal DCG Now we can compare scores across queries, since we\u2019re comparing percentages of the best possible arrangements and not the raw scores<\/p>\n<table>\n<tbody>\n<tr>\n<td>\n<h5><b>PRODUCTS<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>Rank(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>User Scoring(rel)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>CG<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>Log<\/b><b>2<\/b><b>(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>rel\/ Log<\/b><b>2<\/b><b>(i)<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>DCG<\/b><\/h5>\n<\/td>\n<td>\n<h5><b>NDCG<\/b><\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT1<\/b><\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>N\/A<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>0.37<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT2<\/b><\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<td>\n<h5>0.62<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT3<\/b><\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>3<\/h5>\n<\/td>\n<td>\n<h5>8<\/h5>\n<\/td>\n<td>\n<h5>1.585<\/h5>\n<\/td>\n<td>\n<h5>1.892<\/h5>\n<\/td>\n<td>\n<h5>6.892<\/h5>\n<\/td>\n<td>\n<h5>0.86<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCT4<\/b><\/h5>\n<\/td>\n<td>\n<h5>4<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>8<\/h5>\n<\/td>\n<td>\n<h5>2<\/h5>\n<\/td>\n<td>\n<h5>0<\/h5>\n<\/td>\n<td>\n<h5>6.892<\/h5>\n<\/td>\n<td>\n<h5>0.86<\/h5>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h5><b>PRODUCTn<\/b><\/h5>\n<\/td>\n<td>\n<h5>5<\/h5>\n<\/td>\n<td>\n<h5>1<\/h5>\n<\/td>\n<td>\n<h5>9<\/h5>\n<\/td>\n<td>\n<h5>2.322<\/h5>\n<\/td>\n<td>\n<h5>0.431<\/h5>\n<\/td>\n<td>\n<h5>7.323<\/h5>\n<\/td>\n<td>\n<h5>0.91<\/h5>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h1 class=\"entry-title\"><\/h1>\n<p>NDCG = 0.91<\/p>\n<h3><b>Final Score Calculation For All set of queries<\/b><\/h3>\n<p>Once we\u2019ve computed NDCG values for each query, we can average them across thousands of queries.<\/p>\n<h1 class=\"entry-title\"><\/h1>\n<h3><b>Testing Across various setup<\/b><\/h3>\n<p>Once the score is calculated of each setup (production, avatar, \u00a0competitor) . The two setup \u00a0algorithm will be compared using a statistical test (such as a two sided t-test) whether one algorithm is better than the other, and with what confidence.<\/p>\n<div>by <strong><a href=\"https:\/\/www.linkedin.com\/in\/saugata-halder-95135355\" target=\"_blank\">Saugata Halder<\/a><\/strong><\/div>\n<div>\n<div>Saugata\u00a0is Senior Product Manager at a leading retail\u00a0eccomerce\u00a0\u00a0player and consultant with Zombie Software.\u00a0You can connect on\u00a0<a href=\"https:\/\/www.linkedin.com\/in\/saugata-halder-95135355?trk=hp-identity-name\" target=\"_blank\" rel=\"nofollow\">LinkedIn<\/a>.<\/div>\n<div><\/div>\n<div><b>&#8220;<\/b>For a <em>free consultation<\/em> with a member of our team call us now on +971-544177921\u00a0 or\u00a0 send query via this <a href=\"http:\/\/www.zombiesoftwares.com\/contact.php\" target=\"_blank\">link<\/a> \/ <a href=\"mailto:info@zombiesoftwares.com\">email<\/a> . \u201c<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Objective: Our objective is to device crowdsourcing platform find the relevancy of SRP and benchmark the with competitor. Apart from <span class=\"more-text\">&hellip;<\/span><\/p>\n","protected":false},"author":1,"featured_media":432,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[24],"tags":[62,72],"class_list":["post-14","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-site-search","tag-site-search","tag-user-experience-customer-experience"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.zombiesoftwares.com\/blog\/wp-content\/uploads\/2017\/01\/433942-636016874483880029-16x9.jpg?fit=450%2C450","jetpack_shortlink":"https:\/\/wp.me\/p8i7fD-e","jetpack-related-posts":[],"_links":{"self":[{"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/posts\/14","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/comments?post=14"}],"version-history":[{"count":6,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/posts\/14\/revisions"}],"predecessor-version":[{"id":726,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/posts\/14\/revisions\/726"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/media\/432"}],"wp:attachment":[{"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/media?parent=14"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/categories?post=14"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.zombiesoftwares.com\/blog\/wp-json\/wp\/v2\/tags?post=14"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}