14

There was a blog post from google that they are indexing javascript applications. However, my AngularJS Application www.fore-cite.com seems not to be indexed at all. Is there anything special that I have to do in order to enable indexing and make the content searchable?

paweloque
  • 18,466
  • 26
  • 80
  • 136
  • 2
    Have you used the webamstertools that google provides to see indexing information and test-crawl pages? Are you using HTML5 mode in angular? Is your webserver correctly returning index.html when directly requesting specific routes? – Kevin B Jan 08 '15 at 21:46
  • I've asked google webmaster tools to render the page, but it didn't help. What is the html5 mode in angular? What do you mean by directly requesting specific routes? AngularJS also kicks in when index.html is asked. What shall I do there? – paweloque Jan 08 '15 at 22:57
  • For example, if you ahve an angular route pointing to example.com/about, your webserver should return index.html, and then angular will render the /about route. – Kevin B Jan 08 '15 at 22:59
  • html5 mode is a setting in angular that you can enable to allow for viewing your pages using example.com/about rather than example.com/#/about – Kevin B Jan 08 '15 at 23:03
  • I get the html5 mode. But for the index.html: Do I have to do something special for this to happen? – paweloque Jan 08 '15 at 23:17
  • Yes, it's a setting in your webserver. For example, with apache, you would use .htaccess to redirect all requests to folders that don't exist to index.html, otherwise only the homepage would be indexable. – Kevin B Jan 08 '15 at 23:19

3 Answers3

19

The Google crawler does execute javascript on the pages that it crawls. With AngularJS, there are a few steps you have to take to make sure that your application is getting crawled and indexed properly.

HTML5 Mode

You must use html5 mode.

Webserver Setup

For the html5 mode to work properly, you must configure your webserver so that requests to directories that don't exist get rewritten to index.html.

Sitemap

Google does not properly follow links in angularjs apps yet, therefore you must create a sitemap for all of your routes. This sounds like a pain to do, however, with proper build processes this can be a very automated process. (gulp, grunt, etc.)

Cons

This of course only applies to the google crawler. Other search crawlers such as Bing may not support javascript applications yet, though I wouldn't be surprised if this changes over the next year or two (if it hasn't already.)

Other considerations

One commonly missed problem with indexing angular apps is things like pagination and content that shows up after clicking a button. If these actions do not change the url, google will not crawl it. For example, say you have a page with a table using pagination and it has 3 pages. Google will only crawl the first page unless each page has a different url route such as /table/page/1 /table/page/2 /table/page/3

Community
  • 1
  • 1
Kevin B
  • 94,570
  • 16
  • 163
  • 180
  • With this approach the deep links don't work anymore. If the user enters a deep link as the Url, the server will be asked. We'd need, however, AngularJs to answer the request. Is there a solution for this problem? – paweloque Jan 09 '15 at 10:15
  • Here is a nice blog article describing how to do this rewriting and server configuration directly in django: https://blog.kevinzhang.me/posts/no-nonsense-angularjs-html5mode-url-routing-with-django.html – paweloque Jan 09 '15 at 13:57
  • At your first comment, deep linking should work just fine. Can you elaborate? – Kevin B Jan 09 '15 at 14:39
  • I now got everything to work. Started with the html5mode and then I started to understand why the webserver setup is important. Finally I've found a solution for a setting directly in django and not the webserver. Thanks a lot for you answer. It helped a lot. Btw. the project where this was necessary is: www.fore-cite.com – paweloque Jan 10 '15 at 22:33
  • Could you explain why html5mode is a must? Why doesn't using the /#! and ?_escaped_fragment_= work? – Yaron Feb 23 '15 at 06:48
  • The point was to not have to provide alternative content for crawlers. – Kevin B Feb 23 '15 at 06:49
  • @paweloque Your site is not indexed though. Didn't the solution work? – user1721135 May 24 '15 at 20:20
  • I implemented the solution as suggested, but it doesn't seem to work. Do you have an idea what the problem might be? @user1721135 – paweloque May 25 '15 at 21:19
  • did you add a sitemap and upload it to webmaster tools? – Kevin B May 25 '15 at 21:20
  • @paweloque did you add the site to webmaster tools? When you do that try "render as google bot" it will show you if google understands the JS or not. Im very curious as I have a similar project now. – user1721135 May 27 '15 at 06:15
  • 2
    I did exactly that. Render as google worked fine, i.e. the page was displayed correctly. In spite of that the crawling seems not to work. Currently I've no idea how to analyse and to look for a solution... If you're interested, let's do a skype or exchange mail to brainstorm about this problem. My email is: paweloque@gmail.com – paweloque May 28 '15 at 12:47
  • A SO chatroom would likely be better, so that others can participate. – Kevin B May 28 '15 at 15:57
  • This wouldn't work on a 'Github Pages' setup would it? – JCraine Aug 05 '15 at 10:26
  • 3
    I have the exactly same problem. The Fetch as Google seems working as expected with page rendered properly. But the cache page in google is not javascript rendered, the page's content is not properly indexed. and all the html5 route/url are not appeared in the index. I have all the pages listed in sitemap.xml, no helping. – Wudong Nov 26 '15 at 21:21
  • 1
    How does this apply to angular2? – Ken Feb 05 '16 at 12:01
  • I don't expect it to be any different. – Kevin B Feb 05 '16 at 15:09
  • @zyxue With this setup it worked in one simple case, but I didn't manage to make this work in a more complex application. My conclusion is: use AngularJS 2 and isomophic JavaScript – paweloque Feb 18 '16 at 20:08
  • @paweloque, so you mean that your setup now can make google crawl and index multiple pages in your angularjs app? I thought by a simple case, there is still multiple pages (or urls) in the app, right? – zyxue Feb 18 '16 at 20:25
  • No, with Angular2 and isomorphic JS, the server renders the page and returns to the crawler. In a normal browser, the app then comes to live on the browser, but the first view is already rendered. – paweloque Feb 18 '16 at 20:43
  • @paweloque i think he was referring to *"With this setup it worked in one simple case,"* not the angular 2 part – Kevin B Feb 18 '16 at 20:44
0

You might want to use Angular Universal. It looses the benefit of rendering the angular application on the client however it will allow proper search engine indexing (not only google).

you will have to decide if the trade off is worth it for your requirement

Angular Universal Official Website

RollRoll
  • 8,133
  • 20
  • 76
  • 135
-1

Nowadays you wont get good SEO with AngularJS.

First thing is to get your pages crawled. See this simple JS indexing experiment.

To diagnose your site you can use Chrome 41 used by Googlebot to index JS pages. See this.

TomCZ
  • 354
  • 4
  • 5
  • This contradicts the other, accepted answer, so do you have a source for "not getting good SEO with AngularJS"? – Nico Haase Jan 08 '18 at 21:43
  • There is no contradiction. If Google can index it doesn't mean it will index. One of the sources is in the first link. You can google more. Read it, you will know what to look for e.g this [AngularJS SEO case](https://www.deepcrawl.com/blog/best-practice/angular-js-and-seo/). – TomCZ Jan 12 '18 at 11:46