Successful Web Analytics Approaches by Avinash Kaushik

My name is Avinash. And I am the author of the book
that you’ve just seen, and also the analytics
evangelist for Google, which is a fun position where you get
to do whatever you want, which is sweet. So thank you, Brett. What we want to do today is– at
least, I want to do today– is talk to you about successful
strategies and mindset around web analytics,
specifically. Later in the day, you’re going
to hear from other people who are going to talk and touch on
various other facets, but I want to talk about web
analytics, specifically– now starring Angelina Jolie. The first time, I saw version
two for Google, I used it for a few hours and I went over to
Brett, and I said, this is like Angelina Jolie. It’s undeniably sexy. You cannot argue. Both men and women think that. It does kick some major ass. It’s a really nice tool. It does some really
amazing things. And she works as the UN HCR
ambassador, and we give our tool for free. But all kidding aside, many of
you who use Google Analytics, and others amongst you don’t use
Google Analytics, my goal for today or for the
entire day is that you learn something. Independent of the tool
that you use will be– [SHRILL BEEP] Good morning. What we’re going to do is
actually share some of our passion and some of our
learnings in the space of web analytics, so all of the
sessions are geared towards teaching you something. You’ll see lots of these
mindsets and ideas showcased using Google Analytics, but we
should be able to apply them within too. We want you to get lots of
actionable insights at the end of the day today. So in my mind, the challenge
with web analytics is the very first time somebody plugged in
a web server into a wire, it turns out that it
spews out data. It’s a magical thing
that happened. You plug in the ethernet cable
and lots of data comes out. And the approach for the last
10 odd years with web analytics has been that we have
all this data, why don’t we add, subtract, and multiply
it, and puke it over the fence and the people will figure
out what to do with it. And that model worked
for a few years. But it does not work anymore. Web analytics is incredibly
complex, and simply puking data out is not a good way
to solve the problem. Because what happens is that if
you sit in the eyes of the customers– and I’ve spent sort of ten
odd years sitting on the practitioner side of things– all of these data just
simply provides questions and numbers. It does not provide
you with answers. And it rarely provides you with
any insights that you can take action on. And so the model that existed
thus far with web analytics is broken, because data puking
is no longer an option. And so, I think that we’re
living in this not-so-new paradigm that I definitely want
all of you to take away from today, is to live in a
world where data is at the service of driving action. And in the book, I have this
three layers of so-what test. And essentially, it is that
if you look at a metric, anywhere, on the dashboard or in
the report, and you ask it so-what three times, and at the
end of the third time, it doesn’t give you an action,
you’re wasting your time, no matter what guru or thought
leader pontificated about the use of the metric. Data should drive action. If it doesn’t drive action,
you’re wasting your time. And the interesting thing with
web analytics, unlike other things in the world is
there’s a lot of data to waste time on. So what I want to do
today is talk. There are two parts to
my representation. The first thing I want to talk
about are the six odd principles about how I think
Google Analytics helps you understand your data much more
efficiently, and drive action. And the second part of
my presentation– I want to talk about this
thing I call rules for revolutionaries. If you want to go back and
revolutionize your approach, your implementation of data, I
have five, six, principles, that I have learned through my
painful experience that want I to share with you today. So the first thing is Google
Analytics does a fantastic job of creating a data democracy. And unlike our adventure in
Iraq, this one actually works, and is a good thing. I walk into lots of different
companies– I do consulting for some
companies– and I walk in, and they’ll say here is our
dashboard, and it has 62 tabs. This is not a dashboard. This is a fricking report. With Google Analytics, it’s
actually pretty easy for you to go and create your
dashboards. It’s nice, it’s pretty, you can
look at your data, you can look at your performance. And the nice thing
about it is– Brett touched upon this– that
you can schedule and email this, which is wonderful. You can actually
go in and say– you can send a data
waku-up call. You can say hey, every Monday,
you’re going to get your data. At certain times, you’re
going to get your data. What a lot of people don’t
realize is you can escape from the 62 tab world. Because you don’t have to accept
what is by default available for you in
Google Analytics. You can go in and say you know,
I have somebody who is purely responsible doing all
merchandising on my website. And it’s very important for her
to know all of the data around visitors. OK? We can send her this
specific a report. Or you have somebody who is
responsible for all your affiliate marketing, all a
particular website, all your relationships somewhere. Just send Laura exactly what
she needs, not 62 tabs– the thing that she needs
to do her job. It’s easy for you to do that
in Google Analytics. And a lot of people don’t
realize that it’s that easy. You can send reports
for people– if Andy is responsible for
one particular page, just send him that. Why do you want to
do death by data? And so there are lots of
different options and lots of great ways in which you can
look at data in Google Analytics, but also you can
share it with people who need to take action on it. If somebody is responsible for
the checkout process on your website, somebody’s
responsible– in this case– user activation,
just send him this little snippet of data. Every time that they need
something, you can schedule it, you can email it, and they
can get on with their life without ever having to
come and bother you. A lot of organizations have
relied on very few people to actually look at the data,
analyze it, find insights, and then take action. It’s really difficult for you
to sit in a 10,000 people organization or even a 500
people organization and have that model scale. It does not scale. Because the people who are
running the business have the tribal knowledge that
can drive action. But they don’t have
access to data. In more than one way, Google
Analytics makes it easier for you to get data in the right
hands of the people at the right time. And they don’t even have
to log in to the tool. The other thing that I think the
current version of Google Analytics does a magnificent
job at is data discovery. Because the model thus far has
been we’ll give you lots of tables, we’ll give you
graphs, and they’ll show up at your desk. Actually it’s kind of funny. I was at a Fortune 20
company last month. And there, the person in the
company who makes decisions about the website has their
secretary print their reports and put it on his desk. Every week. I could not believe it. But the interesting thing is,
it’s really hard for you to make decisions based on a
printout, or a spreadsheet that’s sitting on your desk. Because every time you look at
a piece of data, you want to ask it questions. And you want it easy to get the answers back really quickly. So what GA does is you can
say, here is nice. OK. I know page views are
dipping over there. At least, I’m getting all
the context of my data, not just one. Now I know exactly how
everything else is going, but I wonder where all these
people are coming from. So 500,000 people came
to my website– where are all they
coming from? So it’s very easy. You click on a button– boom. There’s your visualization. It’s really easy for you to
understand, then you can say oh, I wonder what’s happening
in Africa. Maybe I want to run AdWord
campaigns in Africa. Now you can do that. Here’s another great one. You want to make the
most optimal content for your websites. So boom, there’s your report. This is exactly what’s going on
when it comes to content. But notice it’s really easy for
you to have all of these other options on the right, that
help you figure out what should I do next? What is it that is interesting
to me? So it’s like, all these people
came to my website. I wonder– oh. We do have good editors. I actually got this
the other day. It’s very cute. It’s nice. Please try again. Thank you for your patience. It’s very cute. Let’s get back to the story. But the nice thing is I can
say oh, I wonder where do people come from to a website. One of the things that a lot
of people don’t realize– I really love this one. A lot of people obsess and have
their egos tied in home pages of the website. And the thing that people don’t
realize is that you actually don’t have control
anymore about what the home page of your websites is. 80% of the people start surfing
at a search engine– an astonishing number
by the way. It scared me. But 80% of the people start
surfing at it– the internet. So Yahoo, MSN, and Google decide
what the home page of your website is, not you. Based on what people are typing
into a search engine, the search engine needs to do
its best to figure out what’s the right page on your
website to show them. And that is the home page
of your website. So as I browse around and look
at content reports– I’m on the route directory,
I’m over there– I don’t know how many people
enter my web at any given page, and that’s the first
page they see. It’s a great number to know. And it’s very easy. You go in, you say you know,
I want to know all of the entrance sources. I want entrance paths,
entrance key words– I get all this data within
easy reach that helps me understand where are people
coming from, what is driving them to my website, and do I
have relevant content on that page for these people? Don’t let your egos be tied
up to the home page. No one cares. So you can see where are these
people coming from? 83% actually entered the
site on this page. Its’s very important. I wonder what keywords are
driving them to my page– or actually what sources
they’re coming from. Sure enough, Google,
indeed, Yahoo– these are all the places. Oh nice, I wonder what keywords
are driving them. So I can make sure that this is
the right page for them to be landing on. All of these pages– and there’s
a by diversity added– I’m going to go back and say
whoa, should this be the case? Should I have a different
strategy? But to do all of this– I dive really deep into the
data and I’ve only pressed three buttons. And that’s the interesting
thing. It’s much easier for you to just
start at any level you want of your data, and
simply explore the data and find insights. This is really hard for you to
do with Excel spreadsheets. They might come from any tool. It doesn’t matter what
tool you’re using. The other one that I think is
really interesting is a lot of people cannot make decisions
based on the data, because they don’t have context. One of the most wonderful things
about the internet is that if you’re half-competent at
your business, all your web analytics numbers go up and
to the right over time. Even server errors go up into
the right over time. If that’s the reality
of the world, how do you make decisions? And I think people can’t make
good decisions on their data, because they don’t
have context. And context is extremely
important. I think one of the
things that– the team that builds GA is
sitting in the back over there, so I am simply showcasing
their work, so big round of applause for the
team over there– So one of the things that I
think they’ve done a really great job of is provide your
context as you as you look at your data. So, for example, rather than
looking at one solitary number, you can actually get a
nice performance overview. So you can see how many pages
are there, how long are people spending, what’s the exit rate,
get a dollar index– sweet. But what is actually better is
no matter where you go in Google Analytics, by default,
the team is trying to give you context from your data. In this case, if you look at
this strip, it’s actually telling you– it’s comparing the
numbers to site averages and actually giving you a hint
of where you should be focusing on. And that’s very nice. You get this by default. In most reports in Google
Analytics, you get context. So you no longer have to worry
about is that good or bad? It’s giving you an initial
indication that you’re doing kind of OK over there. But the nice thing is it’s
extremely easy for you to go in and say I want to compare
two different time periods. I want to compare June in this
year to June in last year. I want to compare this week to
last week, last three months to three months before. And that gives you even more
specific context for yourself. In this case, I’m comparing
May and June simple. It’s the dumbest thing to do. Compare two different months. Am I doing better or worse? In this particular case, well,
that’s kind of nice. I got 300% more page views on
the website, and look I kind of did not suck. By the way, this kind
of growth is really hard to manage. You get so many more new people
coming and consuming content, it’s really hard for it
not to actually impact your other numbers adversely. You’ll see this more
later in the day. So this website’s a spectacular
success. Context is the thing. Very quickly, you open
your report. You look at it. You have an idea if you’re
doing well or not. But there is also one other
subtle difference between these three things. When you show most senior
executives the first strip of data, just this data, with no
context, the question that they ask you is what the
definition of time on page. Because they have no idea if
that number is good or bad. They say tell me how you
captured the data. A useless conversation,
by the way. If you give them these two
pieces of data, nobody cares about what the definition is. You know where you’re good,
and where you’re not good. And it moves the conversation
along farther quickly, and that’s what you want to do. You don’t want to argue about
things that are not important. You can do this in
[UNINTELLIGIBLE]. You can do this in Excel
if you want. Right? Very simple strategies in which
you can give context to your data so you can actually
make decisions from it, rather than arguing about what
the definition is. Here’s another way. This is nice. Everybody wants to make money. You can set up your goals. If you don’t have goals in
Google Analytics, that’s very suboptimal. I’m a huge fan– there’s like
a whole chapter in the book that talks about how outcomes
are very important. If you don’t know what the
outcomes of your website are, no amount of analysis
will tell you what you should be doing. So goals are good. Sometimes you mess up and you
create a goal that you don’t want to track anymore. That’s fine. But create goals. And the nice thing is you can
say oh, I see conversions. it’s kind of sort of looks
like it’s going up. But I do the same exact thing
compared to different time periods, I get a much better
read on performance– not doing so well over
here, doing a little bit better over here. Very, very simple trick,
it give you context for your data. Exact same thing here. Isn’t it more fun to know that
you’re up 7%, rather than knowing you got 1,000,000
million visits on your website? Who cares about the
absolute number? What’s more important is to move
the conversation along. Same thing here–
search keywords. Where are the people
coming from? What traffic sources are
working better for me? Very, very easy for
you to understand. And the next thing I wanted to
share is Data in Service of Action, just to reiterate the
theme of what I’ve been talking about so far. And it’s nice. You go to your content. This exact same screenshot
I had a few slides ago. And it’s nice. I know the overall story–
nice page view trend. I have my strip. I have my context. And now I’m ready to move beyond
the summary level data to more of the nitty-gritty
level of what’s going on. And when you go to that level,
it’s kind of overwhelming to say this is cute and– dare I say– sexy, but
it’s a lot of data. It’s a lot of data. It’s nice, page views look good,
but where should I look? How should I think? In GA, you press one button– that button up there– and actually allows
you to compare two different metrics quickly. So not only do I know what pages
are more popular on my website, but I can actually
compute how well are those pages performing in terms of
sucking people into my website and getting them to spend more
time, so I can show them my banner ads. Quickly, notice some really
wonderful things here happening over here compared
to over there. One of my new principles– you know PETA, People for
Ethical Treatment of Animals? That’s a great organization. And so I came up with
a new acronym, and it’s called PALM– P-A-L-M. And it stands for
People Against Lonely Metrics. And the reason I say that is
because lots of people look at one number by itself. We all want a husband and wife
and a girlfriend and boyfriend, maybe all of
them at the same time. By the way, that’s
this screenshot. But if you want to make
effective decisions with data, always find a best friend
for whatever metric you trying to track. If page views is where you’re
trying to drive at, find the best possible metric that would
give you context to the data that allows you
to take action. In this case, for me personally,
I want to know what pages are important. Actually, this is much
more important. So I know what stories are more
important to people, what people react to, so
I can write more. If I did not look
at this metric– the boyfriend metric– the girlfriend metric might be
telling me something’s wrong. So for your metrics, as you
analyze the data, never, ever, on any dashboard or report, look
at one metric by itself. It will not give you the kind
of insight you need to take action that will impact
your bottom line. And the nice thing
is there a couple different ways to do it. Is this good or bad? I can make some judgments. But if you want to short circuit
the time it takes to make the decision, click one
more button and you get this. Now this is a wonderful view–
exactly the same data, exact same metrics. But now it’s comparing
performance of every page, in time at site, to average amount
of time people spend on your website. And this is very cool,
because now I don’t even have to think. I can say oh, cool,
cool, cool– suck, suck, suck. Very, very easy– you can
do this anywhere. I’m not going to impress you
with animations, right? But you can do this
anywhere you want. It’s very easy for you to find
a best friend for your metric that allows you to find
different performance. And then it’s very easy for you
to identify where you need to be paying attention. All in two clicks. Exact same thing, right? Everybody here loves Search. It’s going to look at all your
search data, for goals, outcomes, exactly
the same thing. Which search engine
is more important? In this case, if I want to drive
conversion, actually Yahoo seems to be more important
than Google. But Google brings in a stunning more amount of traffic. So you can balance
your strategy. That gives you insight. Exact same deal, but now
comparing to site average drives action. The wonderful thing about– the difference between this
graph and this is the same one that I’ve mentioned before. In this case, people tried
to understand data. And some analysts amongst you
want to try and spend time understanding data. This is what you send to
your senior management. They should not spend time
understanding data. They should be making
decisions. And it’s easy if you look at
this view for it to be accretive to driving
decisions quickly. So one of the more popular posts
I had written on my blog recently was that on
non e-commerce. So I’ve become a big fan
of not making money. And I’m sure there are many of
you here today who might not actually have e-commerce
websites. And the nice thing is that there
is a really wonderful area in Google Analytics that
you can use to track effectiveness of your non
e-commerce websites. So I’m a huge fan of the BBC. And that’s probably my primary
source of information. And they don’t sell anything. They just have a ton of content,
and so how should they measure success? Typically, what happens in this
case is people use these metrics at an aggregate level
to measure success. They’ll say oh, I want to know
the amount of page views and I want to know visitors
and time on sight. And I will use this. 10 minutes is a fantastic
number by the way. These numbers are
not for the BBC. Because they don’t use GA. They should, but they don’t. That’s a great time on site. So like oh, I feel good. 10 minutes is awesome. When was the last time you spent
10 minutes at a site? The interesting thing is– interesting about averages is
they hide the truth very effectively. So the average earnings per year
between me and Bill Gates is $20,000,000,000. I contribute $20,000 of that. He contributes $100,000,000
a year. And that’s the important
thing to remember. Averages often hide the truth. So, for example, rather than
measuring the unique visitors that come to a content site, in
this case, you would assume that approximately each
person comes twice. Like 500,000, 1,000,000,
that’s kind of what it looks like. But these are all
real numbers, by the way, for a site. The reality actually
looks like this. The averages hide the
truth effectively. Then you do distribution. You get a much better
understanding of what is actually going on the website. So what if most people actually
just come once to this hardcore content site
that’s starting to spam people with banner ads– God bless them. The distribution paints a
radically different story. You have a big head, then it
shrinks really quickly. And look what happens here. Whoa. There are these loyal, crazy
people like me, who are actually coming to visit
100, 200 times. If you actually understood that
you have this swath of traffic– about 30 odd percent
of the traffic right here– that is spending between 14 to
200 times they’re coming to your website, wouldn’t you
design the site differently? Wouldn’t you react to
them differently? Wouldn’t your content strategy
be different? This is a very, very powerful
way of looking at data by simply using distribution,
rather than using an average. So this is great metric. If you are a content site, you
want to know if people are loyal to your website. Right? And then you compare
over time. And you say am I getting
better or worse? I’m actually getting slightly
better in July. I got 63% over there. Then this recency. Again purely for non e-commerce
websites, it’s important to know– if that’s
your business– do people come again and again? Or when was the last time
I actually saw Jim? It gives me an indicator
of success. If I have good content, and
we’re providing value to you, you would probably come
more frequently. On this hardcore non e-commerce
website, 66% of the traffic was brand new. That’s, by the way, a stunning
number for them to realize. Because they had designed all
of their content navigation structure, based on the fact
that you knew them really, really well. They had no pages defining the
value that their site was bringing to them. And yet, there was this massive
amount of traffic that was brand new to
the franchise. So then you can go and figure
out, hey, what’s going on with recency. So this goes back literally– this website has had
Google Analytics for more than a year. It will go back. Take the people who you have
seen in this time period that you’re interested in, it will go
back all the way in history and say hey, when was the last
thing you saw this person? Very powerful report. If you’re doing a content site,
you probably want people to be over there. Not over here. Another average one is people
use time on site. 11 minutes over there, and
look at the distribution. See you have lots of people who
stay very little amount of time, and then you have a chunk
of people here stay really, really long
amounts of time. This is very effective in you
making decisions and measuring success of your website. Same thing with depth
of visit. How much content do
people consume? The wonderful thing about these
distributions is that it truly helps you understand the
value of a non e-commerce website from your web
analytics data. Are people loyal to you? Do they come back again? How long are they spending? And how much content
do they consume? And if you’re or you’re, you’re any website that does not do
e-commerce, if you want to measure success of your website,
it’s a powerful way for you to get started. The next thing you would
probably do is you’ll say OK, let me segment this data. How do the numbers
look from Google? How do the numbers look from
my marketing campaigns? How do the numbers look for
my important key words? You will segment this data, and
you’ll begin to understand a lot more effectively what’s
actually going on on the website and what action
you should be taking. And my recommendation for non
e-commerce sites, always, is throw a survey out. Throw a simple little survey,
three or four questions, and ask people what they thought
of the content. This piece of data
still does not– it’s kinda’ sorta’ the
value of content and other fuzzy metrics. So if you want to do fuzzy
metrics, throw up a quick survey, and I can say hey, as a
result of my experience, is my brand getting better? Is the likelihood that I’m
driving people to offline sales better? Or nine months from here, are
you going to buy a big caterpillar system for me? Because you’re not ready
to buy it now. So in combining a quick little
survey methodology with what you saw in GA with these
metrics, you can truly begin to understand if your content
website– if your non e-commerce website–
is performing as well as it should. And stay away from averages,
at least in this particular case. So I am a huge fan of the
metric bounce rate. So as I was putting this deck
together, I did not want to recommend a metric because I
fundamentally believe that there’s no such thing as oh,
here are God’s KPIs and if you measure them, you’re fine. That’s like 10-year-old
methodology– KPI, KPI, KPI. Every business is different. So my mindset is here’s
the data. Here’s an effective way for
you to navigate the data. You figure out what’s best
for you, rather than me pontificating on it. This is the one exception
I will make. The only exception today is a
metric called bounce rate. And I love it. I call it the sexiest
metric ever. The reason I like bounce rate is
because it is so powerful. I call it brilliantly dumb. Because it’s really easy
to understand what a bounce rate is. There’s no need for
definition. Bounce rate defines this
experience with a customer. I came, I puked, I left. Literally, that’s
the definition. The other thing is you all spend
tons of money on AdWords and direct marketing and
affiliate marketing and all of those things, and what this
metric can help you understand is where are you acquiring
crappy traffic from? Very quickly. So it’s easy for you to go in
and say oh, what’s the bounce rate of my site? 70% in this case. So a lot of people wonder why
their conversion rate is 1%. Actually, for most of you if you
do e-commerce, I bet you your conversion rate is 2%. That’s the average conversion
rate in the United States of America, whether you’re selling
elephants or iPods. 2%. If you’re really good at
direct marketing, your conversion is probably
closer to 30%, 35%. 2%. This is why your conversion
rate is 2%. Because most traffic comes
and leaves instantly. And yet when I do seminars, I
go telling companies, I’m stunned at how few people
actually know what the bounce rate of their website is. So I encourage you to do this. Now it’s possible that your
bounce rate is more like this. In which case, it needs
a love sign. This is a great bounce rate. On average, you’ll find the
bounce rate for your site will fall 40% to 60%. It should probably be closer
to this number. It’s a great bounce rate. You cannot convince
all the traffic in your website to stay. You can’t. Everybody has ADD. If you don’t have what they
want, they’ll leave instantly. You can’t win them all, right? But that’s a great
bounce rate. It’s a fabulous bounce rate. Outstanding [UNINTELLIGIBLE]. The next thing you want to do
is you say ah, I have all these sites. They’re sending me traffic. And I think, if you look at the
graph, you’ll notice it actually is trending up
and to the right. Slowly, but it is trending up. This is good to know. This is nice. But then you can say where is
all my traffic coming from? OK. Sweet. [UNINTELLIGIBLE] and– working great. I wonder what the bounce
rate performance is. And they all are sending quickly
and qualified traffic. So it could be I’m running
the wrong campaign. Maybe this is not a
great place to go. Bounce rate is a great
qualifying metric. It’s a great metric that helps
you ask the right questions. Exact same thing here
for search– my keywords? What’s my bounce rate? I can actually look at
my landing pages, and the bounce rate. I love this one, by the way. Definitely look at the top 20,
30 entry pages to your website, where people are
entering, and look at the bounce rate for those pages,
because remember your home page is not the home page
of your website. This is where people
are entering. Are these pages doing as good
job on your website as your home page might be– where you
spend a lot of love and attention, making the perfect
golden homepage in the world? But what about these
pages where all these people are entering? Measure bounce rate
at a site level. Measure bounce rate for your
co-acquisition strategies. Measure bounce rate for
your top entry pages. If you’re spending money on
AdWords, measure bounce rate for your landing pages. It will not give you all of the
answers, but will help you quickly distill down where
things are not going right. Then you can say is it
that I’m bidding on the wrong key word? Is it that the creative on my
landing pages is wrong? What’s going on here? It’s a quick qualifying
metric. You’re going to hear more today
about topics that I am going to skip. You’re going to have Stephanie
and Alex talk about segmentation. I’m a huge fan of
segmentation. At an aggregate level, it’s hard
to find tons of insights quickly– segment, segment,
segments, segments– that’s the religion. 10 times a day, you say
segmentation, segmentation, segmentation. And they’re going to
talk about it. Stephanie’s also going to
talk a lot about search. And at the end of the day,
Tom’s going to talk about customer experience
optimization– another thing I’m very passionate about. So this is the last part
of my presentation– Rules for Revolutionaries. These are sort of my key
learnings, from having done this for a long time in
companies big and small. So the first one is a rule I had
created three years ago. Until recently, I was the
director for Research and Analytics at Intuit. And so I’d done a presentation
in e-metrics some three years ago, and I created this
rule– the 10/90 Rule. And the goal for every one–
every single one of you in the room, if you’re here and paying
attention to me and not sleeping, the goal is you
actually want to get lots of value from the implementation
of analytics. And the rule that I had was if
you had $100, spend $10 on the cost of the tool and
the professional services for the tool. And spend $90 of that
on the people. The web is an extremely complex
beast. There are more people on your website trying
to do more weird things that will cause your data
to look funny. It’s really, really, really
important, that if you want to get serious about analytics,
that you don’t simply throw a tag on your website– most tools use tags now–
that you throw a tag and forget about it. You will need somebody
intelligent, who’s going to analyze the data, who’s going
to understand your business, your goals, who can actually
extract value. Because at the end of the day,
tools are only a way of getting data, and I find that a
lot of people underestimate the value of business acumen and
common sense that you need to attack the data with. So no matter what web analytics
tool you use, the differentiate for you between
success and failure will actually be– well, maybe all of you. It’s the investment that
a company makes. I’m stunned that companies
invest $1,000,000 in a tool, and they give it to the admin. It’s not going to work. And it’s not the fault
of the tool. It’s a damn good $1,000,000
tool. I would love to have it. It’s the fault of the
company for not investing in the people. The web is very complex and
it will remain so for now. It’s very important that you
actually invest in people who understand your businesses
and can analyze the data. The next rule I have is that
reporting is not analysis. A lot of people confuse
these two things. And the interesting thing I find
in my experience is that both of these activities
take exactly the same amount of time. You could spend all your
life doing reporting. You could spend all your
life doing analysis. And you have to make a conscious
choice about what you want to do. These are dead trees arriving
at your desk. I love this animation. These are dead trees arriving
at your desk, right? This is an actual Excel report
that I was getting at that time, a couple years ago. This is not even the
whole of it. There is more. These are prettier dead trees. if you actually want to get
value from your data, and you have somebody in your team whose
title says analyst. This is what they should be doing. Only 20% of the time spent doing
reporting, because you can’t get away from it. But you saw so many different
ways today, early in my presentation, in which you can
do data discovery, you can get context, you can segment. 80% of somebody’s time– well,
70%, because I am very generous person, I give 10%
time for bathroom breaks– but the rest of the time for
somebody whose title says analyst should actually be
spent in doing analysis, unstructured analysis. You cannot expect to get massive
insights from any analytics tool simply by
implementing the tool. That is the start of your
pain, not the end of it. The other one is to avoid
the data quality trap. The original title of this slide
was Data Quality Sucks, Get Over It, but I thought
it would be rude. One of the more interesting
things about the web is it changes massively. The web is changing at the speed
of light or even faster than that, if you
can imagine it. And it is probably the most
perfect medium to collect imperfect data. It has been deliberately built
to screw with you. And the interesting thing is
there are many, many reasons why your data is not perfect,
and it’s not going to tie. If you spend time trying to
figure that out, you will be a very unhappy person. But the interesting thing is
the web still provides probably more data and more
valuable data than any other medium on the face
of this planet. Now I am biased. A– I’m a web person. B– I’m standing at Google. But I can’t believe that people take out ads in magazines. Again, remember I am biased. Because ads in magazines to me
is a faith-based initiative. Thanks to President Bush
for that term. Because you say well, Fortune
magazine gets 1,000,000 subscriptions a year. And I will put my ad in it
because I have a faith-based hunch that it will
drive sales. Or I will do a quick market
survey and ask my mom if she saw the ad. That’s a faith-based
initiative. On the web, you can do better. I’m stunned that people hold the
web to a standard so much more higher than they hold
the magazine ad, which is simply insane. so my recommendation
to you– remember I want you to take action– my recommendation to you is
when you look at the data, assume a level of
comfort with it. When I analyze data– and I know the web
decently well– so I’ll say, you know what,
I have approximately 80% confidence in data. And I can go present to John. And John is a new person, he has
always deprinted catalogs, and he knows nothing
about the web. I’m not saying that. But John could look at the same
piece of data and he can say I have 40% confidence
in the data. It’s OK. Don’t argue with him. That’s the first thing. No matter what your level of
comfort with the data, the nice thing is you can
take a decision. Make a decision. Don’t argue about confidence. I was giving a speech at a very
big company last month and I said look, if I have 80%
of the confidence in data, and I come speak to Rob– Rob is the CEO of this
Fortune 10 company– Rob would say whoa, let’s
go buy Google. 80% confidence in the data. But if he only had 40%
confidence in the data, he might say let’s go buy Yahoo! Let’s play with this
for a little while. Let’s make a less
risky decision. The important thing is
to make a decision. You might not want to send a
rocket to the moon right away, but start assembling
the engine. What I find most people do is
they bicker and argue about the quality of the data. Remember even with 10%
confidence in a piece of data, you can make a decision. Trust me, you can. Then what you want to do over
time is you want to take micro-segments in your data. So you can say I want everybody
who comes to my website on the keyword
Avinash sucks. And I will take that
keyword traffic– just that– and over time, I
will try to understand why quality might be different. And what will happen is over
time, your confidence in the data will go up. It will move from 30% to 40%,
50%, 60%, 70%, 80%– you’re never going to get to 100%
quality, because it can’t. That’s the problem. And that’s OK. At some point we will all have
our RFID chips in our brains that will communicate with our
web servers and give us great quality data. Until that day, make sure that
you make decisions, rather than worry about quality
too much. A good friend of mine gave
me this wonderful quote that I love– an educated mistake is better
than no action at all. I am amazed at how often
I forget this. I’m amazed at how often other
people forget this. The next one that’s very
important– especially, I understand that a lot of you
come from large companies– it’s very important if you want
to create data driven organizations, that you actually
have a process and a structure around data,
around analysis, around taking action. Because most people say– and
this actually happened to me when I first got to
my prior company– people said we have a report
publishing schedule. And that’s a process. And that’s not a process. This is actually a graphic and
it represents the six sigma [UNINTELLIGIBLE] process– very simple. Define, measure, analyze,
improve, control. Simple process, right. It’s very important that for
your web businesses, you actually figure out
what a process is. In this case, an example is
I want to improve the merchandising capabilities
of my website. A lot of people think when you
talk about process, it’s flow charts and complex things that
take a billion years. It takes 15 minutes
to do this slide. It is a PowerPoint slide. It tells the entire organization
exactly how we behave in each step, who is
responsible for doing what in every single step. It levels the playing field. It creates a repeatable
process. As you go back and look at your
analysis, you look at your report, you look at your
dashboards, you look at what needs to happen in order to take
a decision, to create a test, make sure that you put
some level of process in place, because process scales. Ad hoc does not. This is the last rule I
think I have, which is experiment or go home. And this is sort of a painful
lesson I have learned over the last few years. And the lesson is that
most bad experiences are created by HiPPOs. And HiPPO is an acronym that
stands for the Highest Paid Person’s Opinion. Obviously, I have worked
in a lot of companies. We take all this data you have,
all these brilliant insights, and all these great
understanding, and you walk into a room and the person who
gets the highest salary– no. No, no, no, no. I want a woman running in slow
motion against a blue background. It doesn’t matter what
your data says. Since in my case, my HiPPO is
worth $2,000,000,000 and pays my salary, the woman
goes on the site. And it has been really hard, at
least until the last set of 18 months to two years to
actually figure out how to get around the HiPPO problem. How do you actually make
decisions based on what your data is telling you, what your
customers are telling you. And the interesting thing is
HiPPOs come under all ways, shapes, and forms. Some
HiPPOs are like that. Other HiPPOs are cuter. it is very, very common for me
to interact with people in black turtlenecks
with torn jeans. They are called creative
directors. Are there any here? I hope not. But they puke over
all your ideas. The nice thing about testing
and experimentation– this is why I’m a
huge fan of it– is it creates a democracy. You can now get a multi-varied
tool, and a b testing tool, and you can say if we have eight
great ideas about what this page should look like or
what kinds of promotions we should run or what is the best
creative for an AdWords ad– you get four lines and
that’s all you get– test it. Why should you let your HiPPO
decide what the customer experience, the promotion,
the content, creative, bullets, font– what should it be? Why don’t you simply test it? You can test it. Testing is faster. It’s cheaper. And it helps your customer
tell you what you should be doing. The more important thing I have
learned about testing is that in most companies, 80% of
the time, you are wrong about what your customers want. And the reason is
really simple. Just because I work at Google
and I use the Google search engine does not mean that I’m
a customer of Google. I’m too close to the company. You’re too close to
your company. Even if you use the software
services of your company, even if you work at American Express
and use the Americans Express bank, it does not make
you a customer representative of the company. You’re too close
to the company. The nice thing is you’re
going to be wrong. The wonderful thing about
testing experimentation is you can figure out how to be wrong
quickly and efficiently. And that is an awesome thing. And you’re going to hear later
at the end of the day from Tom what all of the wonderful
things you have at your disposal to do this but when you
go back to your offices, every single person, every one
of you, no matter what tool you use, should figure out what
your testing strategy is. Because it will be the
difference from you swimming in data and taking no action
to simply throwing up experimentations and hypotheses
on the website, and let the action be taken for
you by your customers. That’s it. Thank you. [APPLAUSE] I’ll be happy to answer a few
questions, and then we have a break at 10:30. And then at 11, we’ll have
the wonderful Stephanie. Yes? AUDIENCE: About time on page,
how does Google measure that with regard to tabs, people
being on your page, where does the browser opens the page. How is that measured, and how
does that impact that measurement? AVINASH KAUSHIK: Paul can
correct me if I’m wrong, but Google measures exactly the
way every other tool does. And the way all tools will
measure time on site is most tools now use cookies. And hence, when you first come
to a website, they will start a session for you. And if it detects 29 minutes
of inactivity, it will automatically terminate
your session. It assumes that you have gone
now to sleep, or to meet with your boss. The time on site for the last
page is the tricky problem for every single web analytics tool,
because the way a web analytics tool will measure
how long you have spent on this page is it’ll look for the
time stamp when you saw this page, it will look at
the time stamp on the next page you saw. and it will compute the delta. And it will say you were here
in minute one, you were in minute two, so you spent one
minute on this page. This is literally how
most analytics tools compute time on a page. Then time on a page translates
into time on a site, plus, plus, plus. Now the challenge of time on a
page– this especially affects blogs, for example– is if you only see one page on
a website, the tool actually has no idea how long you have
spent on this page. Because you load a page
and you leave. So a great example of
this is my blog. Because on my blog, all my
latest entries are on the homepage of the blog. So almost everybody who comes
to the blog sees one page, gets every single thing they
want, and then they leave. Because essentially, it’s
a one page website. In that case, it’s much harder
for me to compute what time on page is, how long people
are spending. Now they are there are hacks and
tricks and dom onload, and pinging things you can do, that
will allow you to compute how long people spend
on every single page including the last one. But most tools by default
don’t do that. So the answer is most
tools use cookies. They create a session when
customers come to the website. They use the delta to figure
out how long you spend on each page. And the time spent on the
last page is a tricky proposition for now. Make sense? Paul, did I do it OK? Thanks. Yes. AUDIENCE: So how does Google
differentiate, measure, and report human versus
nonhuman clicks? [UNINTELLIGIBLE]. AVINASH KAUSHIK: Where? On your AdWords campaigns
or on your site? AUDIENCE: Yes, just in
Google Analytics. AVINASH KAUSHIK: Oh, in
Google Analytics? Most nonhumans, our beloved
friend robots and crawlers, actually mercifully do not
execute Javascript tags. So if you use Google Analytics
or many other tools that are now standardized on using
Javascript tags to collect data, the robots– so the nonhumans– are actually not executing
them, so they are automatically excluded. Nonhuman traffic was more of a
problem during the web blog days, because when crawlers
crawl your website, they do leave all the entries behind
your web logs. So if you use web log files as
the source of your data, you have to manually filter
them out. And I think there is a website or something– do a quick search in Google, and
you’ll find a list of all the robot user IDs. So if you want to filter out,
you can do it, but with most tag-based analytics solutions,
this is not that much of an issue. There are a few robots, some
obscure ones that are smart and execute Javascript tags, but
it’s very, very rare that that happens. Most robots– 99.99% would not execute
Javascript. But this brings up
a great question. I’m a huge fan of SEO. So if your website, on your home
page, all the links or most of the important links
are wrapped in Javascript tags, because you click on the
link and it pops up a window. You click on a link and
it does some magic. With Javascript, remember the
crawler from Google is not executing those Javascript
calls, and it will not index that content. So it’s very important to know
robots don’t execute Javascript tags or Javascript. AUDIENCE: How does Google
Analytics and other analytic tools differentiate
natural traffic from page search traffic? AVINASH KAUSHIK: Paul? PAUL MURET: [UNINTELLIGIBLE]– AVINASH KAUSHIK: Paul is the
director for engineering for Google Analytics, and is a very
smart person who saves me all the time. PAUL MURET: [UNINTELLIGIBLE] or your campaigns are coming
from other searches, as long as you have campaign tags
on your landing pages, [UNINTELLIGIBLE] will
pick those up. And we’d be able to determine
that that’s paid traffic. If we don’t see any marketing
campaign information associated with the inbound
traffic that’s coming to your site, then we will then assume
that this is an organic search or natural search, and we’ll
put in that category. AUDIENCE: You mean by tags,
tags and then the URL? AVINASH KAUSHIK: Yes. PAUL MURET: Yes. correct. AVINASH KAUSHIK: And later
today, I know Alex is going to show you how you can split
organic traffic and paid– let’s say for Yahoo or for
other search engines. Because you can filter out. Every paid click comes with a
piece of data that says it’s a paid click. Now the interesting thing is
most web analytics tools do tell you that we will
automatically, brilliantly, geniously figure out what the
difference is, but most of the time, no matter what tool you
use, ask that question. Because if you’re using some
agency in New York who is playing with things and
redirecting traffic through their servers and sending it
over here, there’s a bunch of stripping going on and adding. In that process, it actually
gets pretty hard for any analytics tool to actually
figure out the split between organic and paid. Some most tools by default
will do their best job to figure out what the split
is, but for your tools, ask the question. Make sure it’s working right. AUDIENCE: On the visitor loyalty
and recency slides, is there a way that you can
see what keywords those users came in on? So, for example, if I wanted to
look at 30% where the curve got a little fatter, is there a
way I can tie it right back to my AdWords and figure out
which keywords resulted in the most repeated traffic? AVINASH KAUSHIK: You can. You can. So Alex? Dude, you have a
hard job today. He’s going to show you how to
segment and filter your data. But this is exactly what
you would use. And I encourage you to do
that– remember, data in aggregate is really hard for you
to understand what’s going on, so especially wherever
you’re spending money, segment that data. So you do affiliate,
AdWords, Overture– it’s called something
else, YSM– whatever you spend money, make
sure that you segment the data out so that you can truly
understand what populations make up the kind of
traffic you want. So you should filter–
you should use profiles and filters. And Alex is a great example
later– at 1:00. So attend his session. AUDIENCE: [UNINTELLIGIBLE] problems. For example, I have
a bonus raise sort of 7%. AVINASH KAUSHIK:
That’s not bad. Borderline bad. AUDIENCE: [UNINTELLIGIBLE]. AVINASH KAUSHIK: So the
interesting thing is you– AUDIENCE: [UNINTELLIGIBLE]. You have AdWords data. Is there a way to show the
AdWords data all on the site? Other sites? AVINASH KAUSHIK: At the moment,
I believe the true– according to terms and
conditions for Google Analytics, it does not mix and
merge data across sites, and things like that. So that is not the
option right now. But what I had advised people
is every website is unique. Your business of selling Sony
Viaos is very unique. So index yourself. Index over time. Rather than looking at bounce
rate for this month, why don’t you compare it over the
last 12 months? And see if the line– you’re
getting better or worse? Because under Visitors, I
think there’s a button. You press on it. It will give you that exactly
for bounce rate. So index against yourself to
see hey, over time, is it going up or down? Because there is no ambiguity. It has to go down. That’s the clean thing. The other thing I would do is
I would benchmark against other things that we’re doing,
so I can say, oh, the site average is 70% bounce rate,
but for AdWords it’s 12%. Good. Because it’s good to know
what is good and evil. So use your own data for now. Also sometimes, bounce
rate is a tricky one. I don’t know of a source where
somebody’s publishing bounce rate, but if you sign up for, it publishes a study every year, a couple times
a year, that creates and provides standards
for conversion and things like that. I’m not aware for bounce
rate though. AUDIENCE: [UNINTELLIGIBLE] average data, holes
in marketing. AVINASH KAUSHIK: So
at the moment, that data is not available. Yes. There are lawyers at the back
of the room watching me. AUDIENCE: What would
you say was a– AVINASH KAUSHIK:
–I’m kidding. I’m kidding. AUDIENCE: –large company that’s
purchased at least one of every enterprise solution
that’s out there and uses lots of reports, uses Omniture,
has a large partially optimized website? And give me a picture as to
where does Google Web Analytics fit in that picture. AVINASH KAUSHIK: It
actually fits perfectly into that picture. Because it’s actually
not uncommon. I was talking to someone
earlier today. And I said, I’m sometimes
surprised that people keep switching between
the top tools. They’re kind of the same. There’s no point in
you switching from one to the other. If you don’t like it, just
stick with the enterprise class tool that you have.
Because it is as good as the other one that’s punted– you know somebody’s annointed
it is as an enterprise class two. I think every tool has its
unique value proposition. Omniture has its own
proposition. HPX its own, WebTrends it’s
own, GA it’s own. One of the things that is
really hard to do, but I encourage you to do is figure
out what is the core strength of the tool that you have, and
does it match with what you actually need? In my mind, the core strengths
of Google Analytics are some of the things I’ve
covered today. It’s extremely easy to use, it’s
extremely efficient at helping you discover data,
and trends and insight. And it has many, many things
built in to make sure that even the most expert person or
the most simple person can understand the data
that’s there, and take action from it. It’s very, very good at it. I especially love the AdWords
section of the Google analytics tool. So I wish I was doing
Stephanie’s section, because I would like to be waving my
arms all over the place. Because I’m surprised. I should not have been
obviously, but I was surprised at how good the AdWords
reporting was. It’s a core strength. The loyalty metric
you saw today is a really good strength. So each tool has its strength. I wrote a really long post last
month, I think, on my blog, and it says what is an
enterprise class tool? Who defines it? Who gets to define it? And I think the days of
somebody, some analyst or author or random guru telling
you what tool you need– I think the nice thing is that
you have choices, and you should figure out what you need,
because every tool has its strength. The nice thing about GA is you
can get it for free, and you can install it on your website,
you can try it, and you can actually
learn by doing. Because the one core difference
is that if you use any other tool, you will learn
what it does by a bunch of marketing slides. It’s exactly how
you do an RFP. And you can actually use it. Put in your site, learn, use the
data– because after you implement the tag, two
and a half hours later, you have data. All data, all standard reports, any tool will provide. Put it up there. Try it. And through the learning
experience, you might figure out that Google is the
right tool for you. Or you’ll figure out it’s not
the right tool for you. But the interesting thing
is if you do arrive at a conclusion that it’s not the
right tool for you, when you go to the next vendor,
you will ask them intelligent questions. Because you would have figured
out what you need. Best case scenario–
you use it. Worst case scenario–
you get smart. Win-win. AUDIENCE: Do you have the
ability to import any data into analytics to measure some
other offline activity, like phone calls for example? AVINASH KAUSHIK: You cannot
import data into Google Analytics at the moment. Nick’s here. Nick, where are you? I saw him before. Nick will be here later today. And you can talk to him about
tracking online, offline. There are many different things
you can actually do to track online and offline. There are lots of case studies
and real examples– and we’ll make sure you get
to talk to Nick today– so like click to call, or ads in
magazine, or flyers in your Office Depot store. All of these things actually can
be tracked by doing a few different things. It won’t give you perfect,
perfect data, but it will give you a very good, solid,
indicat– better than faith-based initiatives. So it is totally possible. I think Jeff, at the end of the
day might be touching on audio ads for example. That to me is a great example. How the hell do you track
the audio ad? So you can do online, offline
tracking with web analytics tools now. We’ll make sure– there Nick. That’s the guy standing
over there. And go find him, and he has some
real examples and case studies where he can show
you exactly how this tracking can be done. Even though you can’t import
the data, you can track and measure success. Yes? AUDIENCE: How do you convince
your customers– AVINASH KAUSHIK: You’re
cutting into your own eating time. AUDIENCE: How do you convince–
how do you get past the whole data quality
argument? Because that’s probably one of
the largest ones we face where someone has an Omniture
[UNINTELLIGIBLE] say, well, Google Analytics shows this, our
tool shows this, tell us why Google is wrong or
Google is right. AVINASH KAUSHIK: The interesting
thing is neither one of them is right or wrong. The beautiful thing about web
analytics is everybody can create their own version
of history. No, but the interesting thing is
every single tool actually collects data differently. And every single tool– well, slightly differently at
least, even if they use the same mechanisms, they will
collect the data slightly differently– so the interesting thing is the
data is not going to tie. It simply won’t. So usually, usually,
the differences fall into these buckets. Either they’re using first
or third party cookies. That explains a bunch
of stuff. The way that they actually
start and end sessions– most tools differ in when they
terminate a session. Like I mentioned, 29 minutes,
the [INAUDIBLE] session timeout– some people actually have
other tools, have other intelligence. Intelligence that terminates
a session. So normally that’s another
one that’s a difference. And another one– sadly, but very common– is that the two tools won’t be
all sitting on the same pages. So the tagging is not right. Usually, usually, most of the
things will fall into these three categories. The differences- why the
numbers are different. But they will be different. So if you do end up in this
situation, the question to ask is not what your country–
no, I’m just kidding– but the question is not which
is better or worse. I have seven tools on my site. I need more. I simply compare the trends. And what you will notice is if
you have a couple different tools, and you churn the data
for a few months, you’ll notice that the delta stays the
same, the delta actually stays the same. It is very difficult for you to
say my numbers from tool a will match tool b. Because they fundamentally are
doing different things. If you do have a paid vendor,
stress them and say why are your numbers different to GA? That is a cute question
to ask. But mostly, they want tie–
trend it over time, make sure that’s good. If you notice something funny
like this month, the delta is 20% and next month
the delta is 2%. Something funny is going on. You’ll want to investigate that,
but you’ll notice the trend is the same. Yes. Last question, he said. AUDIENCE: GA has three rules,
three different rules. [INAUDIBLE]. So I’m going to add one. [INAUDIBLE]. AVINASH KAUSHIK: You can. You can. The most– and that’s true for most
analytics tools. So the question was you can
have four goals in GA. What if you run, create a site,
put the tag, and you’re running it for a few days,
and then you realize oh, I forgot goal four. You cannot go back
in history and retractively reprocess data. And this is true for most
analytics tools. There are solutions now where
you can download and do things like that, or some
solutions of data warehouses, things like that. But your core analytics data,
once collected for every vendor, is fairly well set. It’s kind of harder to
go back in history. There are a couple
tools do it. Mostly that you can’t. It’s just the way the data is collected, stored, and processed. Yes. AUDIENCE: What’s the privacy
policy for Google Analytics? AVINASH KAUSHIK:, and
then at the bottom it says privacy– no, I’m
just kidding. There are Terms and Conditions
on every single Google page. You’ll actually see
a privacy policy. It’s very long and expansive, so
please, every single person should read it. No, I’m not kidding. If you’re worried about privacy,
you should read it. Privacy is a question that comes
up often about Google. You should make sure you’re
comfortable with it. Is there a specific aspect
of privacy you wanted me to touch? Because it’s like 12, 15– I don’t know– 900 pages. AUDIENCE: I personally don’t
have a law degree, so I don’t think I’d be able to read that
and truly understand the implications. AVINASH KAUSHIK: What’s
your concern? AUDIENCE: Well, the concern is
of you guys having all that data on a website that might
be [UNINTELLIGIBLE] that who you’d share that with
and something like that. AVINASH KAUSHIK: My recommendation is read the policy. The thing is, if you’re
running a business, it is actually– you’re not being fair to your
business, if you don’t read the policy. Please read it, because
you’re right. It is an important
thing to read. The policy states out very
clearly what Google will and will not do with the data. So, for example, in what you
said, there are things that Google will not do
with your data. They very explicitly
set it out and they touch exactly we said. Are you going to share, are
you going to monetize? Are you going to spy on me? Rather than me saying trust me,
I’m going to say go read the policy. And there is actually a
two paragraph section that touches on that. It’s very short. Find it, read it,
and from your– privacy is in the eye
of the beholder. The best thing you should
do is read the policy. Make sure you’re comfortable
with it, because there are things that Google can do with
your data, and there are things that it will not
do with your data. And those are laid out
fairly clearly. But it is open to
interpretation, which is I find different for
each person. And that’s OK. But it is everywhere. The easiest thing to
find in Google Analytics is not a report. It’s the present policy. AUDIENCE: Are there plenty of
examples of companies that have policies and change them? AVINASH KAUSHIK: Yes. Yes, if Google– AUDIENCE: [UNINTELLIGIBLE] policy and then a year later–
and a new Board of Directors comes in and struck it out? AVINASH KAUSHIK: So most
reasonable companies– I would put Google in the
reasonable bucket– most reasonable companies, by
law, if they change terms and conditions on you will actually
make sure you explicitly are aware of it. AUDIENCE: Yahoo did it. Yahoo changed the policy without
telling anybody. And Hewlett Packard
wonderful thing is if you change your policy without
telling your customers, that customer will show up somewhere
else and tell everyone [UNINTELLIGIBLE]. AUDIENCE: It’s a really
big [UNINTELLIGIBLE]. AVINASH KAUSHIK: No, it is. Most reasonable companies, I
find, that will not change terms and conditions without
informing you about it. It’s the right thing to do. It’s a very Google-y
thing to do. Thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *