Lean Analytics Cycle

Data & Analytics

hiten-shah
The present document can't read!
Please download to view
96
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Description
Text
  • Lean Analytics Lean Product & Lean UX Silicon Valley - July 2014 Hiten Shah [email protected] Download the slides: h!p://kiss.ly/leanprod2014 mailto:[email protected] http://kiss.ly/leanprod2014
  • Continuous Learning & Improvement
  • Don’t know where to start?
  • Wondering if you’re working on the right things?
  • Or, you just want to hear me babble!
  • HTTP://KISS.LY/LEANLYTICSBOOK http://kiss.ly/leanlyticsbook
  • Lean Analytics helps you Optimize for Learning
  • Lean Analytics Cycle AVINASH KAUSHIK HTTP://KISS.LY/LEANAC http://kiss.ly/leanac
  • Metrics  →  Hypothesis  →  Experiment  →  Act
  • HTTP://HELLOBAR.COM http://hellobar.com
  • What is Hello Bar?
  •  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  • Our funnel showed a low installation rate Hello Bar Funnel Metrics Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 15068 1868 716 688 398 % prev step 12.40% 38.33% 96.09% 57.85% % drop off -87.60% -61.67% -3.91% -42.15% % of visitors 100% 12.40% 4.75% 4.57% 2.64%
  • We  achieved  a  40%  increase  for  our   installa2on  rate  with  just  this  one   first  experiment. The Lean Analytics Cycle At Work
  • Original Hello Bar installation screen
  •  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  • Find out why people don’t install Hello Bar
  • Didn’t Meet My Expectations
  • I Had Trouble Installing It
  • I Don’t Have A Website
  • Not Ready To Install
  • Other
  • Hello Bar Experiment Hypothesis If  more  installa2on  op2ons  are  given,   then  more  people  will  add  Hello  Bar   to  their  website  due  to  the  need  for  a   WordPress  plugin  and  ability  to  email   the  code  to  someone  else.
  •  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  • Created a new flow to test
  • I can install code myself.
  • 41.67% responded with “WordPress Plugin”
  • 25% responded with “email my developer / web designer”
  • I need help.
  •  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  • Full funnel metrics for the experiment Variation Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7456 906 342 329 276 % prev step 12.15% 37.75% 96.20% 83.89% % drop off -87.85% -62.25% -3.80% -16.11% % of visitors 100% 12.15% 4.59% 4.41% 3.70% Control Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7469 925 353 341 197 % prev step 12.38% 38.16% 96.60% 57.77% % drop off -87.62% -61.84% -3.40% -42.23% % of visitors 100% 12.38% 4.73% 4.57% 2.64%
  • Resulted in a 30% installation rate increase HTTP://ISVALID.ORG Experiment resulted in a 40% improvement http://isvalid.org
  • ACer  a  dozen  more  experiments,  our   installa2on  rate  is  now  hovering   between  5%  and  7%   That’s  an  addi;onal  89%  increase!!! The Lean Analytics Cycle At Work
  • Metrics  →  Hypothesis  →  Experiment  →  Act How You Can Put It Into Practice
  • Metrics   Figure  out  what  to  improve
  • “What gets measured, gets managed.” Peter Drucker
  • Baselines DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA http://kiss.ly/optdatadna
  • Funnels KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL http://kiss.ly/kmfunnel
  • SaaS Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL http://kiss.ly/kmfunnel
  • eCommerce Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL http://kiss.ly/kmfunnel
  • Revenue BUFFER BAREMETRICS HTTP://KISS.LY/BUFFERBARE http://kiss.ly/bufferbare
  • Spreadsheets and Calculations HELLOBAR HTTP://HELLOBAR.COM Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Created Second Bar Totals 15068 1868 716 688 398 193 % prev step 12.40% 38.33% 96.09% 57.85% 48.49% % drop off -87.60% -61.67% -3.91% -42.15% -51.51% % of visitors 100% 12.40% 4.75% 4.57% 2.64% 1.28% http://hellobar.com
  • Hypothesis   Make  an  educated  guess
  • “A solid test hypothesis is an informed solution to a real problem – not an arbitrary guess. The more research and data you have to base your hypothesis on, the be!er it will be.”
 — Michael Aagaard, h!p://kiss.ly/abthypo http://kiss.ly/abthypo
  • Research   Tools  &  Tac;cs
  • Customer Development HTTP://WWW.CUSTDEV.COM http://www.custdev.com
  • Jobs To Be Done #JTBD HTTP://JOBSTOBEDONE.ORG http://jobstobedone.org
  • Usability Testing HTTP://USERTESTING.COM http://usertesting.com
  • Live Chat HTTP://OLARK.COM http://olark.com
  • Mouse Tracking HTTP://CRAZYEGG.COM http://crazyegg.com
  • Long Form Surveying HTTP://SURVEYMONKEY.COM http://surveymonkey.com
  • On-Page Surveying HTTP://QUALAROO.COM http://qualaroo.com
  • Personalized Messaging HTTP://INTERCOM.IO http://intercom.io
  • Email a survey to people HTTP://SURVEYMONKEY.COM http://surveymonkey.com
  • Ask people why they canceled HTTP://SURVEYMONKEY.COM http://surveymonkey.com
  • Get feedback right when they cancel
  • Exit Surveys HTTP://QUALAROO.COM http://qualaroo.com
  • Discover what’s missing HTTP://QUALAROO.COM http://qualaroo.com
  • Find out what words people use to describe your product HTTP://QUALAROO.COM http://qualaroo.com
  • Learn about people’s expectations HTTP://INTERCOM.IO http://intercom.io
  • Learn what frustrates people HTTP://INTERCOM.IO http://intercom.io
  • More actionable reading… QUALITATIVE DATA HTTP://KISS.LY/QUALABT http://kiss.ly/qualabt
  • Experiment Hypothesis Formula DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA http://kiss.ly/optdatadna
  • Experiment   Test  your  guess
  • Only  1  out  of  5  tests  win. 4  out  of  5  tests   cost  you  money. $
  • 1)  Which  top  metric  are  we  trying  to  improve?   2)  What  can  you  anchor  your  metric  against?   3)  What  is  your  hypothesis  about  why  this  metric  is  at  x%?   4)  What  do  we  expect  from  this  experiment?   5)  What  was  the  result  and  what  did  we  learn  from  it? HTTP://KISS.LY/6QSAB Answer key questions for each experiment http://kiss.ly/6qsab
  • Prioritize your experiments HTTP://KISS.LY/ABPIE http://kiss.ly/abpie
  • Don’t copy “best practices” blindly
  • Learn what works best for you
  • Act   Decide  what’s  next
  • Explore  the  results  and  make   data  informed  decisions. THE ANALYTICS THAT MATTER TO FACEBOOK HTTP://KISS.LY/FBINFORM http://kiss.ly/fbinform
  • KISSmetrics A/B Testing Report KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST http://kiss.ly/kmabtest
  • We know that people love Google Analytics
  • KISSmetrics Experiment Hypothesis If  people  authen2cate  with  Google,   then  more  people  will  sign  up  due  to   their  exis2ng  familiarity  with  Google   Analy2cs  and  alignment  with  our   winning  headline.
  • KISSmetrics Homepage Test Control
  • KISSmetrics Homepage Test Variation
  • KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST http://kiss.ly/kmabtest
  • KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST http://kiss.ly/kmabtest
  • KISSmetrics Homepage New Control
  • Inspiration
  • KISSmetrics Homepage New Test Variation
  • KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST http://kiss.ly/kmabtest
  • KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST http://kiss.ly/kmabtest
  • Growth! HTTP://KISSMETRICS.COM http://kissmetrics.com
  •  Metrics    Figure  out  what  to  improve    Hypothesis    Make  an  educated  guess        Experiment    Test  your  guess    Act    Decide  what’s  next
  • Hiten Shah [email protected] @hnshah You will get all you want in life if you help enough other people get what they want. Zig Ziglar Download the slides: h!p://kiss.ly/leanprod2014 mailto:[email protected] http://kiss.ly/leanprod2014
Comments
Top