The Customer Analytics Blog

Actionable advice on optimizing the customer lifecycle

Woopra: State of the Union for January 2009

Woopra News - January 24th, 2009 by .

0 Flares Twitter 0 Facebook 0 Google+ 0 LinkedIn 0 0 Flares ×

Greetings fellow Wooprites! It’s been too long since my last State of the Union, and for that I apologize. Luckily for me, plenty of you are not shy about prodding for information and updates, and since we aims to please… here come a bunch of them!

Infrastructure

Over the last 90 days Woopra has evolved significantly, though it’s done so in ways not yet visible. You see, all of the heavy lifting is done on the back end to keep everything running smoothly and to add new features. Those of us who actually use the service really only interact with it primarily through the Woopra Client interface, or through the WordPress plugin which embeds stats in our blog.

Let me spend a moment talking about the changes to the back end, which we as users rarely think about:

  • As Woopra expands across the Web onto more and more sites, we keep track of every single page view on every single site. And for every visit there are around 20 pieces of information Woopra collects. Woopra is seeing more than 50 million page views a day across more than 50,000 sites, so that means we’re collecting 1,000,000,000 pieces of raw statistical data each day… and growing.
  • In addition to collecting the data, every time one of us opens our Woopra client we are pulling down streams of that information from the statistics Engines (thats what we call the servers that collect data) in real time. This places additional and constant demand on the infrastructure.
  • Gathering and serving this amount of data alone is quite a technical feat, but planning ahead to handle 100 times this transaction load adds a whole other degree of complexity. And we want to ensure that Woopra runs as well or better when we’re handling data for 500,000 or 5,000,000 sites.
  • Over the last couple of months we’ve scaled up our hosting architecture by 300%. But simply adding more physical machines to handle the load is not good enough. It’s important to ensure that the databases are also optimized, the javascript that sends data to the Engines is speedy, and that the Client can access the data quickly. Efficiency is key when working on a project with this kind of scale.
  • Along with the expanded physical infrastructure, there has been daily ongoing work done to optimize the Engines for performance as well as to add some new features I’m going to let you know about in a moment.

Tackling the issues related to scaling the service was not something we planned on doing while we were actively in development. As some of you may recall, we launched Woopra as a very small closed beta at WordCamp Dallas, but received an unprecedented amount of demand for the service – which we immediately began working to fulfill. Don’t get me wrong, it’s a good problem to have… as problems go. But it has added to the challenges.

Interface

Now, in addition to the challenges of scaling the infrastructure on the back end. We’ve been working hard in a number of key areas to deliver the Woopra statistics to YOU in a dramatically improved manner. But this means a lot more than simply giving you guys an updated Client. It means giving you new and exciting ways to access your stats, communicate with your site visitors, and even add new features to Woopra!

First of all, the new Woopra client, due to be available as a Release Candidate in a couple of weeks, incorporates a number of changes to the user interface designed to improve functionality, fix known bugs, and even add new features. A few things that I really love about it are:

  • A new Webmaster Tools tab will be available from the Home page. I don’t know about you guys, but I’m constantly having to pull up my browser to check different things while working on Web sites. We figured that as long as the Woopra client is running anyway, why not build some tools into it! So if there are specific tools you would like to see added please drop a comment below, and a link to an online version would be very helpful.
  • The Menu area has been compacted and clarified. This is to make room for the addition of plugins. More on that in a minute… It also provides a visual indicator of the number of new Events you have waiting for you to review, in case you missed the live notification or didn’t have one set up.
  • For those of us who are tracking several sites at once, you might not want to be connected to all of them at the same time, and the new client will let you disconnect from the ones you aren’t interested in viewing at that moment. This saves valuable bandwidth on the users end, frees resources on our end, and allows the client to gracefully lose connection to one Woopra Engine without dropping all of the other sites being tracked.
  • The Live visitor view adds new features. The live visitor information boxes will show a mini-map with the location of the user targeted. It also adds a quick link to lookup a visitor’s history, and when known it will even keep track of the original referrer so you can know how they came to the site in the first place.
  • The ability to accommodate user created Plugins. This is huge, and will be one of the biggest differentiators of the Woopra platform. We’re going to open up the Woopra API to allow developers to add the features we’ve been missing. I can’t even imagine what you guys will dream up, but I’m guessing that mashups and integrating Woopra stats with other services will be a big opportunity.

In addition to the client update, we will soon be publishing a guide that allows Webmasters to customize the Chat window on their site when they invite a visitor to chat. Personally, I’ve had very few people respond to the invitations, and I think it’s because the invite isn’t always noticeable. But our internal testing with the changes has proven about 500% more effective! So I can’t wait to get that out to you and hear what kind of difference it makes in real world use.

Business

I also want to briefly touch on one other important aspect that affects Woopra’s growth rate. The economy. As you might imagine, all of the infrastructure and development work that goes into a project of this magnitude is quite expensive. We’ve been working on fundraising for several months, which is extremely difficult in the current climate, and we have had a few offers which we bypassed (luckily we’re not desperate).

Woopra has a small, strong team of people who treat one another as partners, and we don’t want to put our culture at risk of being steered by an external force that views it as merely an “investment”. To be clear, we could absolutely speed up development of the project by taking some of the money we’ve been offered, but we know this would impact the long term goals – and we want to ensure that the only people we answer to are you, our clients.

In addition to raising capital to support Woopra’s growth, one important aspect of Woopra’s long term survival is to be able to offer value added services which we can actually charge for. I’ve mentioned on a couple of other occasions that we’ll roll out high-usage plans and other options as soon as we’ve completed our billing system, but our team feels strongly that we want to ensure the infrastructure and interface issues are ironed out before we start charging. So, Woopra will continue to be free for now, but we sure hope that when the time comes we can count on you guys to make all of this investment worthwhile…

Finally, although I had previously noted that we were planning on closing the Woopra beta, I’ve had way too many people ask me NOT to do that. So we are going to leave it open for signups – though the wait may be long at times. Having said that, there were about 6,000 people who have been waiting patiently for up to two months who we’ve been approving in batches starting yesterday. It takes a couple of days for our batch system to do all of the notifications, but if you’ve been waiting for approval your wait is almost over.

That’s all for today (I know it’s a lot). We look forward to your feedback and suggestions for continued improvement!

0 Flares Twitter 0 Facebook 0 Google+ 0 LinkedIn 0 0 Flares ×