November 16, 2012

Making JavaScript MVC applications SEO friendly

If you have played enough with JavaScript MVC libraries then you'll know that they're not really SEO friendly. In fact, if you have made an entire application/webpage using exclusively a JavaScript MVC library chances that your creation will ever appear in search engines results are near 0. Maybe that will change in the future, if search engines start understanding JavaScript, but until then you're pretty much screwed...

...Unless you use a headless browser to make crawlers understand your page!

The problem is simple: Your application/webpage can't be indexed because crawlers don't run JavaScript, which is what generates the real content that browsers display (HTML).

The solution is even simpler: You need something that will run the JavaScript on your application/webpage and output HTML that crawlers do understand.
There are a lot of good headless browsers already, but all of them require some more work to be done on them to start working as you'd expect. Configuration files, libraries, dark magic... Of course, they also give you a lot of options, which you may or may not want. In my case, I wasn't looking for anything special, I just wanted something that will run the JavaScript code and output HTML.

That's why I ended up doing my own headless browser! Just to make things clear, it's not done from 0. In fact, it's less than 50 lines of code. It's based on Qt4 and WebKit and there's even a PHP example! You'll probably want to use something like that. Maybe you'll also want to read Google's advices about all this stuff.

Enough writing, here is the GitHub repo. Enjoy!

Edit (1 year later): I just found Prerender. Haven't tried it yet (lack of time), but I think it gets the job done.