Woopra: State of the Union for January 2009
Greetings fellow Wooprites! It’s been too long since my last State of the Union, and for that I apologize. Luckily for me, plenty of you are not shy about prodding for information and updates, and since we aims to please… here come a bunch of them!
Infrastructure
Over the last 90 days Woopra has evolved significantly, though it’s done so in ways not yet visible. You see, all of the heavy lifting is done on the back end to keep everything running smoothly and to add new features. Those of us who actually use the service really only interact with it primarily through the Woopra Client interface, or through the WordPress plugin which embeds stats in our blog.
Let me spend a moment talking about the changes to the back end, which we as users rarely think about:
- As Woopra expands across the Web onto more and more sites, we keep track of every single page view on every single site. And for every visit there are around 20 pieces of information Woopra collects. Woopra is seeing more than 50 million page views a day across more than 50,000 sites, so that means we’re collecting 1,000,000,000 pieces of raw statistical data each day… and growing.
- In addition to collecting the data, every time one of us opens our Woopra client we are pulling down streams of that information from the statistics Engines (thats what we call the servers that collect data) in real time. This places additional and constant demand on the infrastructure.
- Gathering and serving this amount of data alone is quite a technical feat, but planning ahead to handle 100 times this transaction load adds a whole other degree of complexity. And we want to ensure that Woopra runs as well or better when we’re handling data for 500,000 or 5,000,000 sites.
- Over the last couple of months we’ve scaled up our hosting architecture by 300%. But simply adding more physical machines to handle the load is not good enough. It’s important to ensure that the databases are also optimized, the javascript that sends data to the Engines is speedy, and that the Client can access the data quickly. Efficiency is key when working on a project with this kind of scale.
- Along with the expanded physical infrastructure, there has been daily ongoing work done to optimize the Engines for performance as well as to add some new features I’m going to let you know about in a moment.
Tackling the issues related to scaling the service was not something we planned on doing while we were actively in development. As some of you may recall, we launched Woopra as a very small closed beta at WordCamp Dallas, but received an unprecedented amount of demand for the service – which we immediately began working to fulfill. Don’t get me wrong, it’s a good problem to have… as problems go. But it has added to the challenges.
Interface
Now, in addition to the challenges of scaling the infrastructure on the back end. We’ve been working hard in a number of key areas to deliver the Woopra statistics to YOU in a dramatically improved manner. But this means a lot more than simply giving you guys an updated Client. It means giving you new and exciting ways to access your stats, communicate with your site visitors, and even add new features to Woopra!
First of all, the new Woopra client, due to be available as a Release Candidate in a couple of weeks, incorporates a number of changes to the user interface designed to improve functionality, fix known bugs, and even add new features. A few things that I really love about it are:
- A new Webmaster Tools tab will be available from the Home page. I don’t know about you guys, but I’m constantly having to pull up my browser to check different things while working on Web sites. We figured that as long as the Woopra client is running anyway, why not build some tools into it! So if there are specific tools you would like to see added please drop a comment below, and a link to an online version would be very helpful.
- The Menu area has been compacted and clarified. This is to make room for the addition of plugins. More on that in a minute… It also provides a visual indicator of the number of new Events you have waiting for you to review, in case you missed the live notification or didn’t have one set up.
- For those of us who are tracking several sites at once, you might not want to be connected to all of them at the same time, and the new client will let you disconnect from the ones you aren’t interested in viewing at that moment. This saves valuable bandwidth on the users end, frees resources on our end, and allows the client to gracefully lose connection to one Woopra Engine without dropping all of the other sites being tracked.
- The Live visitor view adds new features. The live visitor information boxes will show a mini-map with the location of the user targeted. It also adds a quick link to lookup a visitor’s history, and when known it will even keep track of the original referrer so you can know how they came to the site in the first place.
- The ability to accommodate user created Plugins. This is huge, and will be one of the biggest differentiators of the Woopra platform. We’re going to open up the Woopra API to allow developers to add the features we’ve been missing. I can’t even imagine what you guys will dream up, but I’m guessing that mashups and integrating Woopra stats with other services will be a big opportunity.
In addition to the client update, we will soon be publishing a guide that allows Webmasters to customize the Chat window on their site when they invite a visitor to chat. Personally, I’ve had very few people respond to the invitations, and I think it’s because the invite isn’t always noticeable. But our internal testing with the changes has proven about 500% more effective! So I can’t wait to get that out to you and hear what kind of difference it makes in real world use.
Business
I also want to briefly touch on one other important aspect that affects Woopra’s growth rate. The economy. As you might imagine, all of the infrastructure and development work that goes into a project of this magnitude is quite expensive. We’ve been working on fundraising for several months, which is extremely difficult in the current climate, and we have had a few offers which we bypassed (luckily we’re not desperate).
Woopra has a small, strong team of people who treat one another as partners, and we don’t want to put our culture at risk of being steered by an external force that views it as merely an “investment”. To be clear, we could absolutely speed up development of the project by taking some of the money we’ve been offered, but we know this would impact the long term goals – and we want to ensure that the only people we answer to are you, our clients.
In addition to raising capital to support Woopra’s growth, one important aspect of Woopra’s long term survival is to be able to offer value added services which we can actually charge for. I’ve mentioned on a couple of other occasions that we’ll roll out high-usage plans and other options as soon as we’ve completed our billing system, but our team feels strongly that we want to ensure the infrastructure and interface issues are ironed out before we start charging. So, Woopra will continue to be free for now, but we sure hope that when the time comes we can count on you guys to make all of this investment worthwhile…
Finally, although I had previously noted that we were planning on closing the Woopra beta, I’ve had way too many people ask me NOT to do that. So we are going to leave it open for signups – though the wait may be long at times. Having said that, there were about 6,000 people who have been waiting patiently for up to two months who we’ve been approving in batches starting yesterday. It takes a couple of days for our batch system to do all of the notifications, but if you’ve been waiting for approval your wait is almost over.
That’s all for today (I know it’s a lot). We look forward to your feedback and suggestions for continued improvement!
The new client looks really good!
Wow, this is great. I just want to comment something on viewing the details of visitors. I always get a wrong detail under CITY. Country is correct, but city is wrong. I do not know if this is really important. But from my end, since we live in an archipelago, we need to track the exact city to know if we have a lead to go for a profitable business or not. Knowing the city would give a hint to follow through the visitor or not depending on the business site we have. Thank you.
Awesome I can’t wait for the new client! I still would like to see the issues fixed in: http://bit.ly/fib4 (especially the plugin error).
[...] just posted their State Of The Union They annouced their new client, which personally looks awesome! I guess I can understand why it [...]
Thank you guys for working on this amazing piece of software! I look forward to the new updated client as the features you are describing looks really promising (the API, yay!!).
the same.. looking forward and enjoy`ing the dev of woopra :)
It would really help, you finaly give me an approval on my added website in order I can judge your client myself :-)).
Greetz from the Neths
Some things I´d add to the client are:
1) Know the maximum number of visits my website had at the same time.
2) I have a blog in Blogger and would want to know if the user I see live leaves a message.
3) Statistics on the links clicked all over the website / blog.
Thanks!
Nanopoder
Also, I tried to translate to Russian this and other posts on my blog in category “Woopra”: http://itkaz.ru/category/woopra/ I hope, that I can translate all actual posts of your blog.
Thank you for all the input and recommendations, and support. The Woopra team so appreciates it.
@zhomart: Please do not translate our posts without permission. We’ll be making this site’s information available in other languages in the future and for you to do so violates our copyright policy. Thanks! We appreciate it that you write up in your own words how Woopra works for you.
@Lorelle VanFossen
Okay, I’ll write with my own words :) My second comment came in spam. It was about Russian search engine Yandex. Check it, please.
@ zhomart:
Thanks for letting me know. And thanks again for all your support for Woopra.
Thank you Lorelle and the rest of the Woopra team for this great service
Thank you guys for this nice software. I’m really happy to use ist. ITS GREAT!!!
A good or nice application as probably Woopra is (I didn’t get a change to test it) stands or falls with good support. As I tried to contact Woopra twice, there was no response at all, even on (my) post(s) earlier on this page came no comment or what soever. Only if someone is trying to be usefull by translating the posts for his own country, one can quickly respond and appeal to their copyrights.
See also an post at one of the fora on this issue from “Palamedes”. I understand his comment and join him by uninstalling Woopra an unsubscribe as a member.
Greetz from the Neths…
@hemwebdesign – Your logic is unclear. I have a personal blog which is written in English. If someone were to tell me that they intended to copy my entire blog’s content onto their own site in another language I would ask them not to do that.
On one hand, it could be construed as a complimentary gesture – but on the other hand copying 100% of the content from another site is often described as “Content Scraping”. It is illegal and also infuriating to Web authors. I’d be surprised if anyone would willingly agree to having their site copied.
Lorelle has written extensively on this subject, and I would specifically refer you to a post on “Copyright and Translation: Help Your Community Yourself”
99% of the sites that engage in content scraping do so in order to redirect traffic from the original site so they can monetize it with advertisements. Lorelle has personally experienced this on numerous occasions, and is looking out after the interests of both iFusion Labs, LLC. and our Woopra members when she asks that the content not be copied.
As the CEO, I concur with her request, and I would add that I believe her response was firm, not impolite. Indeed, this is a subject which generates extremely harsh and emotional responses from most Web authors, so Lorelle was actually being quite nice given the subject.
We love it when people talk about Woopra and share the good, neutral and even bad points. It helps us learn and improve on a daily basis and it spreads the word about our work here. All we ask is that no one actually copy the content.
Cheers,
John P.
Can’t wait to get my hands on this tool! Hurry up!
As an update, about 4-5,000 sites were already approved. There are about 1,700 left to go. So if you’ve been waiting, just hang in there and the remainder should be approved shortly!
Cheers,
John P.
As my site’s traffic has grown, I’ve become more and more reliant on Woopra. Thanks so much guys! Keep up the great work!
I haven’t used woopra for a while, but the sites still have the code on them.
I just updated my client and now I’m not able to login, it just says connecting to Woopra…
I’ll try again in the morning, looking forward to the updates!
@brandonks
Your code is most likely the old version with your account # in it. There is now a new code you should use.
We are addicted to Woopra and we are really looking forward to seeing the new design. The only issue we have is that once or twice a week we don’t get stats for 4-6 hours, usually during the night. It’s not a “huge” problem but it does skew the stats. If it’s something we’re doing wrong, it would be good to know what that is but, otherwise, I hope it will corrected in the update. Keep up the great work… we love your product!
HOST IP RESOLUTION – The host IP in the Live Visitor view of your snapshot above is 71.187.21.134 and resolves to the city of Brookside, but when I resolve that address with a couple of different tools, I get the city of Mendham, NJ. I find that to be the case with the IP addresses in my logs. Why am I getting a different city from the one in Woopra?
NOTIFICATION EVENT SOURCES – I have added about 10 notification events, but they report incorrectly. Let’s say once source is named and/or tagged as X, and another, Y. When I connect to my site from site X, I get the correct source notification – X. But when I connect immediately from site Y, the notification still reports X. Regardless of the parameter (IP, tag, city, Referrer URL, or whatnot) that I use, the notification still reports the first site I connected from; in this case, X. This makes it impossible to gather any meaningful data on out new ads. Is this resolved in the upcoming version?
VISITOR #244621 – Does this mean I have had 244,621 visitors to my site? That’s absolutely not the case for my site. How is the visitor number determined?
ANALYTICS – I have never understood how the percentages on the Overview tab are calculated. In fact, none of the data on that tab reconcile with my manual observations. The New Visitors counts are incorrect. Would somebody please enlighten me?
VISIT DURATIONS – We need smaller increments of 15 seconds up to say, 5 minutes, then increments of 1 minute up to say, 10 minutes. That would more accurately reflect visitor durations than the current 0-5 minutes, and so forth.
REFRESH – The most useful page to me thus far is Search, but the Recent Visitors data is always delayed, and the way to refresh that is to close and open the Woopra client. How about a Refresh or Update button to accomplish this? Also, testing can distort some counts, like the number of visits for a tagged user or the number of counts for today. How about a Clear or Reset button to reset various counters?
I am on Beta 1.2, and have many. many more observations. Hopefully, some or all are addressed in the upcoming new release. Love Woopra, but it needs much work. Thanks.
Hi!
http://www.redcarpetflorist.com/glendale-florist/best-sellers-flowers-3584c.asp – Pasadena Flower Delivery
Hi!
http://www.redcarpetflorist.com/glendale-florist/best-sellers-flowers-3584c.asp – Glendale Flowers