woopra

Greetings fellow Wooprites! It’s been too long since my last State of the Union, and for that I apologize. Luckily for me, plenty of you are not shy about prodding for information and updates, and since we aims to please… here come a bunch of them!

Infrastructure

Over the last 90 days Woopra has evolved significantly, though it’s done so in ways not yet visible. You see, all of the heavy lifting is done on the back end to keep everything running smoothly and to add new features. Those of us who actually use the service really only interact with it primarily through the Woopra Client interface, or through the WordPress plugin which embeds stats in our blog.

Let me spend a moment talking about the changes to the back end, which we as users rarely think about:

• As Woopra expands across the Web onto more and more sites, we keep track of every single page view on every single site. And for every visit there are around 20 pieces of information Woopra collects. Woopra is seeing more than 50 million page views a day across more than 50,000 sites, so that means we’re collecting 1,000,000,000 pieces of raw statistical data each day… and growing.
• In addition to collecting the data, every time one of us opens our Woopra client we are pulling down streams of that information from the statistics Engines (thats what we call the servers that collect data) in real time. This places additional and constant demand on the infrastructure.
• Gathering and serving this amount of data alone is quite a technical feat, but planning ahead to handle 100 times this transaction load adds a whole other degree of complexity. And we want to ensure that Woopra runs as well or better when we’re handling data for 500,000 or 5,000,000 sites.
• Over the last couple of months we’ve scaled up our hosting architecture by 300%. But simply adding more physical machines to handle the load is not good enough. It’s important to ensure that the databases are also optimized, the javascript that sends data to the Engines is speedy, and that the Client can access the data quickly. Efficiency is key when working on a project with this kind of scale.
• Along with the expanded physical infrastructure, there has been daily ongoing work done to optimize the Engines for performance as well as to add some new features I’m going to let you know about in a moment.

Tackling the issues related to scaling the service was not something we planned on doing while we were actively in development. As some of you may recall, we launched Woopra as a very small closed beta at WordCamp Dallas, but received an unprecedented amount of demand for the service – which we immediately began working to fulfill. Don’t get me wrong, it’s a good problem to have… as problems go. But it has added to the challenges.

Interface

Now, in addition to the challenges of scaling the infrastructure on the back end. We’ve been working hard in a number of key areas to deliver the Woopra statistics to YOU in a dramatically improved manner. But this means a lot more than simply giving you guys an updated Client. It means giving you new and exciting ways to access your stats, communicate with your site visitors, and even add new features to Woopra!
(more…)

As a follow up to Lorelle’s previous post, I wanted to report that we believe we’ve stabilized the Web services. Many of you may have noticed some periods of inaccessibility when accessing the Blog, Forums, or occasionally when trying to log into the client.

To provide a little more detail behind the issues (for those of you who are interested) what’s been happening is that as we’ve scaled up the number of users on the system, the number of simultaneous requests for the Blog and Forums have increased – as one would expect. A couple of weeks ago we reached a point at which we decided the Web performance was being degraded, so we needed to shift some attention to the matter.

After doing some brainstorming as to a long term scalable solution to the problem we implemented a new, more complex architecture to handle both Web and login requests. Part of the solution included load balancing the servers. Unfortunately, as with all complex solutions, the Devil is in the details and we had one single little configuration set incorrectly. The result was that the vast majority of requests were working, but every once in a while pages would not load.

Those of you who’ve had to troubleshoot technical gear will probably be able to appreciate the frustration the team had while trying to fix the problem. It’s much easier to find when it’s actually completely broken.

The team did however finally discover the bug, and we’ve squashed it for good! Many thanks to Scott, Greg and the rest of the Layered Tech team for their assistance, as well as to all of you guys for your patience! As of this moment the site is running faster than it ever has in the past, and we hope to keep it that way.

We apologize for the site downtime. We’ve been finishing the migration of all our servers to new, faster servers, including moving our beta testing approval and website. Unfortunately, it hasn’t been a pain-free experience. We anticipated a smoother ride.

We will soon be finished with all the site and server maintenance and migration and Woopra will be better than ever. Thank you for keeping vigilance on the site and reporting on its downtime, and for your patience and support.

Inman News featured Woopra in “Who are they and what do they want?”

There’s more to Web analytics than holding a clicker and counting the number of people who come to your site and watching where they go.

Web analytics is about learning everything you can about your visitors, their intentions and needs. With a deeper understanding of how they use your site, you can tweak it to guide them not only to information they are seeking, but to your “call to action” — whether that’s buying a pair of shoes or contacting a real estate agent about a listing.

…Perhaps the best-known tool for gathering Web site metrics is Google Analytics. But competitors like Visistat and Woopra cater to real estate professionals, and claim to provide more sophisticated capabilities in a more user-friendly format…

Visistat and Woopra even allow users to watch what visitors are doing on their Web sites in real time.

“It’s possible to connect behaviors to who people are by their activities,” Cyberhomes’ Frame said. “You can say, ‘Here is someone who’s been here before — are they doing the same thing, or monkeying around?’”

Using the example of the live conversation feature found in Woopra and other live chat website services, the author, Matt Carter, who previously covered Woopra in Woopra is in demand, explains a key benefit for real estate and service businesses online:

Frame offered an example of the kind of unexpected lessons that can be gleaned from a deeper understanding of users’ behavior. While real estate Web site traffic may decline on the weekend, engagement goes up, he said.

That’s because visitors have entered the WWW zone — wine, women and weekends — “When the kids are in bed and there’s no pressure to get up the next day, so you pour a glass of wine and cruise your favorite real estate site.”

If this engaged visitor is serious, “They will want to talk to a live person” even if it is 10 p.m., Frame said. The upshot is brokers need to make sure their agents can be reached through their Web site at odd hours.

In my recent presentation on Woopra and web analytics at WordCamp Las Vegas, I talked about the statistics that often distract us as they have little value and really little use, yet we become obsessed studying them. The article also makes some of those same points:

• Too much emphasis is made on “above the fold” content and design when the impact of those who make it “below the fold” is greater. They cared enough to read through to the bottom, so reward them when they get there.
• Ask. Stats only give you so much information. If you want information, ask your readers. Have surveys and polls on your site.
• When evaluating how long visitors are staying on a page, do you think about what the page is delivering? What content really keeps them there and why?
• Bounce rates don’t provide enough information. Focus on landing pages. Where are they landing on your site is more important than where they exit. Maybe they got the information they needed and are done, leaving with gratitude. Where they land tells you what caught their interest to make them click through for more.
• Quit giving page views credit. Visitors, especially unique and return visitors, mean more than traffic hits. If a visitor can’t find what they are looking for, they may click all over the place. If they found what they want, they leave happy after visiting one page. Which do you want? Which is really more important?
• System stats are useless information except when you are redesigning your site or writing about system trends, then that information only relates to your site and not the world trend.
• It’s not what sites send you the most traffic but which sites send you the best traffic for conversion rates.

There are a lot of numbers we obsess over without fully understanding their importance to your website, content, and business efforts online. Are you being distracted by distracting statistics?

Demonstrating Woopra recently at WordCamp Las Vegas, I was asked a great question:

What benefit can I really get from live, real-time stats?

While it is cool to see the activity on your website live and in real time, what good is it? After all, isn’t it more important to look at the trends and aggregate numbers to get a real sense of what is happening on your site? Of what is working and what isn’t? Isn’t that more important?

I’m working on an article for the Woopra blog about the benefits of real-time stats and I need your input.

Do you like the live perspective of your traffic because it’s fun? Because it shows you what is happening right now? Or because it gives you insights into your blog’s traffic and activities? Because it gives you the ability to respond to actionable events? What events? What triggers a response?

How do you use the real-time live features of Woopra?

Recently, I presented “By the Numbers” at WordCamp Las Vegas, covering web analytics and blog statistics. I spoke about web analytics, distracting statistics, and ripped apart some myths around the stats we love to watch.

I showcased Woopra, showing the audience how you can see what is happening on your website or blog right now. We watched the traffic and interaction on four WordPress blogs, including tracking the path of participants as they visited the different sites during the presentation.

The Ustream WordCamp Las Vegas Live Stream went out with my presentation, but unfortunately, it wasn’t archived. So this Viddler video is a reproduction of my presentation.

It’s about 53 minutes long and over 240 megs in size. I’ll be releasing shorter versions later, with a captioned version coming soon.

Lorelle VanFossen – By the Numbers and Woopra at WordCamp Las Vegas

Vladimir Prelovac was having an analytically bad day with Google Analytics. According to them, he was getting 10 visitors a day from Google searches for “bioenergoterapia” – a word not found on his blog.

In “Google Analytics might have a problem,” he reported on the detective work that led him to a fascinating discovery. His Google Analytics code was hijacked by a site in Poland.

I made a search on google.pl again found a site called psychoenergoterapia.pl. The site is using my theme, nothing wrong with that, but for some unknown reason they also decided to copy and paste the contents of my site footer, including the Google Analytics code.

And clearly it was working just fine in this alien environment.

…The case basically proves that it is possible to dramatically skew somebodies analytics data and potentially make huge damage to them. Who are them? Companies paying a lot of money for adwords and conversion campaigns for example…

He asked psychoergoterapia.pl to remove his Google Analytics code from their footer, and gave us something to think about.

What if someone is using your Google Analytics or Woopra code on their site because they not only copied your WordPress Theme or website design, but also your analytics JavaScript code? Prelovac admitted he thought this was impossible, that his code was protected. Clearly not.

We here at Woopra considered this security issue from the very beginning. Woopra automatically verifies that the domain name matches the member’s account. If it doesn’t match, it will not track the statistics on that site, nor impact your site’s traffic.

With Woopra, this security protection comes built-in from the start. However, Google Analytics also protects your site’s traffic, if you take the right steps.

Protecting Your Site With Google Analytics

Google Analytics collects data according to Web Property ID and not domain name or website, and you can set your Profile to restrict which sites data will be collected.

In your Google Analytics Profile Settings, create a predefined or custom filter that excludes all traffic from a specific domain or includes only traffic to a specific domain or subdirectory. There are a variety of Include and Exclude filters to choose from that use regular expressions to match the data, tracking only your site in your statistics.

See Google Analytics Domains & Directories, Managing Profiles and Accounts & Profiles, along with “How do I create a filter?” in the Google Analytics Help Documentation for more information on controlling where your stats come from.

In a guest blog post on Problogger, “Statistical Analysis of a Blog’s Traffic” by Eric Rosenberg of The Israel Situation and Narrow Bridge Adventures, he talked about how he is a hobby blogger and MBA student trying to understand how all this works. After taking a statistical analysis class at school, he decided to take his blog for a numbers ride:

As a hobby blogger trying to increase readers to my Israel blog, The Israel Situation, I have always been glued to my site traffic. What does a spike mean? How can I bring more readers? Is there anything I can do to help my blog grow?

…To summarize what I found and apply it to you, the most important factor in growing your blog is persistence. Sticking with it will lead to a real increase in visits. If you write good content, it is inevitable that you will gain more readers. Traffic spikes, like the one I hope to get from guest posting here, are nice. However, most people that visit in a spike don’t really care about your blog, they care about the single post. If you can pick up a few regulars, great, but don’t expect a whole lot.

If you are still starting out, do not give up from a low reader count. I almost did a few times, but decided to stick with it because Israel is something I am passionate about. If you are writing about something you love, you will have a readership increase in time. Other than that, the statistics of blogging are fairly inconclusive.

Eric makes several important points in his article. First, numbers are great, but play with the numbers in different ways and you get different conclusions. If you stop at any of those conclusions, that’s your final answer, but dig deeper, and you might find out even more than you thought about your site and your demographics.

What he discovered is also important. Perseverance does pay off. Blog traffic is not just a numbers game. It’s a patience game. Traffic spikes, then goes away. Over the short term, that’s frustrating. Over the long term, you should hopefully see a rise, but you have to pull back and look across time, not at the moment.

Determining what attracts and keeps people coming back to your site is part of the bigger picture that often goes beyond the numbers.

Have you done some of the statistic testing like he did? Have you found a specific stasticial analysis that works best for you? What’s not working?

Web analytics is still a new industry and we’re learning all the time about which numbers to watch and what they mean. What do the numbers tell you?

Live in Las Vegas? Coming to 2009 International CES (Consumer Electronics Show), the largest technology and electronics show in the world? Or will you be at the more exciting event, WordCamp Las Vegas at the Palace Station Hotel for the first WordCamp of 2009? If so, come see me talk about Woopra and blog analytics.

I’ll be speaking on Saturday at 3PM on web analytics and Woopra, and things you need to consider when staring at all those numbers about your blog’s activities. There are only a few tickets left for WordCamp Las Vegas, so register now.

You’ll be in fine company with the best of blogging and WordPress for this two day event, covering WordPress tips, new SEO strategies, WordPress Theme design and development, WordPress Plugins, integrating multimedia and social into your WordPress blog, monetizing your WordPress blog, and a lot more.

The rest of my impressive fellow speakers include:

• Matt Mullenweg
• Joseph Scott
• Liz Strauss
• Aaron Hockley
• Jim Kukral
• Geoff Kleinman
• Michael Dorausch
• Micah Baldwin
• Dave Taylor
• Micah Baldwin
• Shayne Sanderson
• Niran Amir from Picapp
• Jim Turner of OneByOne Media
• Ethan Gahng from Psolenoid
• Chris Brogan

The event will be live streamed by registration only throug the Ustream WordCamp Las Vegas Live Stream so you can watch and participate no matter where you are this weekend.

I will have only a few of my books, Blogging Tips, What Bloggers Won't Tell You About Blogging, available for sale and autographs, and a few of the last private invitations to use Woopra. Come see me there and bring me a story of how Woopra works for you.

John Pozadzides of One Man’s Blog spotlighted an interview with Woopra co-founder Elie El Khoury on PushingPixels.

The interview highlights Woopra’s development by Elie with Java Swing, the widget toolkit for Java. Swing is an API framework to create a graphical user interface (GUI) for Java programs, the key to making Woopra the “eye candy web analytic program” so many exclaim.

Elie explains in the interview:

It’s true that the browsers today can handle very rich applications, but we cannot forget that the limitations are still endless and the browsers were never meant to handle heavy weight client applications like Woopra.

…I believe the browser is not yet ready to handle apps like Woopra. We all know how much time it takes to create an application that looks the same on IE6 and Firefox, is it worth it?

…Performance is no longer an issue with today’s hardware, but people want to see good-looking things. Adobe Air has done it correctly. You cannot make an ugly application with Adobe Air because they provide you with amazing UI Kits. With Java, it’s very easy to come up with a very ugly application, but what they don’t know is how much Swing is customizable.

After two years of hard work developing Woopra, Elie makes some very good points about the benefits and weakness of writing Java Swing based applications:

Swing’s weak points are not many. But I would say that AWT is better in terms of native dialogs. I prefer the FileChooser instead of the JFileChooser because AWT’s is native. Also, the Anti Aliasing is really weak in Java. It looks different from a platform to another. Adobe Air has done it better!

Not to forget the look and feel, that is the major weakness of Swing.

…The JWebPane of course as mentioned by Sam Berlin of LimeWire. I’m glad they are working on it and have chosen the WebKit renderer. But I don’t believe we will be able to use it before a year of its release unless they provide it as a standalone API. People using PowerPC cannot even run Java 1.6 applications yet, we had to downgrade the application to work with Java 1.5 which is missing a lot of new interesting features like SysTray and more desktop tools.

…One last missing feature in Swing and DesktopTools is the ability to change the screen insets. I would love to create an always on top floating window on the right just like Windows Vista’s Sidebar. I can imagine Woopra on the sidebar and always on top, showing live stats without intruding on other applications (This is doable at the moment but we cannot change the screen insets). That would be awesome.

When I look at Woopra, beyond the powerful analytic features, it’s beautiful. A work of art. Elie says that he spends a lot of his Woopra development time “painting components” and testing colors and gradients. The work that goes into making one of the most exciting live web analytic programs in the world is part skill and a lot of craft. In our hearts and minds, he’s the Claude Monet of Java Swing design.

The full interview by Kirill Grouchnikov of PushingPixels includes some other insights into the development of Woopra and peaks at what Woopra may offer in the future, as Java and Swing and other related applications improve.