Magento index – The basics, benefits and tips to avoid errors

What does Magento indexing mean?

Definition 1:

 

Indexing is the process when Magento transforms product, category etc. data in order to achieve the fastest request speed.

 

Definition 2:

Data storage in MySQL in order to have the most appropriate indexes for SQL requests.

 

What would happen if there were not indexing in Magento?

All the following examples below show that Magento indexes have significant benefits in terms of ecommerce store performance. Even in the case of only one product it is worth running the index, though you should keep in mind that with one product, there is only a little increase in speed. So if you did not use Magento indexing, the you would face some serious drawbacks:

  1. Prices would be calculated in terms of download speed, on the basis of shopping cart price rules and customer groups.
  2. Calculation of inventory for configurable and bundle products would take place only after loading the product collection.
  3. „Layered navigation” would be generated in real time on the basis of hundreds of product attributes.
  4. The products within a category’s sub-categories, would be queried in a recursive way.

 

Jargon in Magento indexing

Before describing the process of performing Magento index, it is important to clarify a few basic terms.

  • Indexed data – The data set which defines the user interface display
  • Indexer – Procedure suitable for creating data arrays
  • Index event – The event or moment when the source data is modified
  • Index process – The time when the indexer is running
  • Main controller – It gives tasks to the Index process

 

Magento Index process

Below you can see the flow chart of the Magento Index process:

 

Magento index flowchart

 

Indexing can be started by way of a Magento event and manually as well. During the index process, in the user interface, the data are read from the EAV tables, thus reindexing is not recommended only in special, justified cases.

 

Describing Magento Indexing

Catalog and product indexing is switched off by default with both Magento CE and Magento EE. To switch it on go to System -> Configuration -> CATALOG -> Catalog -> Frontend -> Use Flat Catalog Category and Use Flat Catalog Category lists and select YES.

After saving, you’ll be notified in the Magento notification area that a reindex is necessary.

 

Magento index switch on

The Magento notification area can be found in the admin panel, right below the menu.

 

How to run Magento indexing?

 

1) Magento admin

Go to System -> Index Management. Click th Select All link, select Reindex Data at the Actions option, then click Submit.

 

Magento index run admin

 

2) Shell script

Start a terminal, then go to the Magento source directory and give the following command:

php shell/indexer.php –reindexall

 

Magento index run shell script

 

You also have the possibility of running only one index if you run the indexer.php with the –reindex <indexer> parameter. For example, if you want to reindex catalog URLs, then run the following command:

php shell/indexer.php — reindex catalog_url

 

Magento index run shell script 2

 

If you set the info parameter with the indexer, then you will get a list of all the indexes and the index codes.

 

Magento index run shell script

 

+1 method: from PHP code

Run the following code for the total reindex:

<?php
/* @var $indexCollection Mage_Index_Model_Resource_Process_Collection */
$indexCollection = Mage::getModel('index/process')->getCollection();
foreach ($indexCollection as $index) {
    /* @var $index Mage_Index_Model_Process */
    $index->reindexAll();
}

If you want to update only one index, then the following code  is enough:

<?php
$process = Mage::getModel('index/indexer')->getProcessByCode('catalog_product_price');
$process->reindexAll();
}
instead of catalog_product_price insert the index name that you'd like to update.

 

 

How to speed up reading from Magento index table

It is possible to add MySQL table indexes to index tables. One way to do it is the manual method, e.g. in phpMyAdmin, but after a reindex these indexes will disappear because following every reindex, Magento deletes and creates index tables. So we need a more advanced solution.

In the example below we add indexes to a product index.

Let’s create a module and “subscribe to” the catalog_product_flat_prepare_indexes event in its config.xml in the following way:

<catalog_product_flat_prepare_indexes>
    <observers>
        <aion_catalog_product_flat_prepare_indexes>
            <type>singleton</type>
            <class>aion_catalog/observer</class>
            <method>catalogProductFlatPrepareIndexes</method>
        </aion_catalog_product_flat_prepare_indexes>
    </observers>
</catalog_product_flat_prepare_indexes>

You can see in the xml section above that before the product reindex, the catalogProductFlatPrepareIndexes method of the Aion_Catalog_Model_Observer class will be called.

At this moment the table is created, we only need to add the new indexes to the table:

 

/**
 * Add indexes to product flat table
 *
 * @param Varien_Event_Observer $observer observer
 *
 * @return void
 */
public function catalogProductFlatPrepareIndexes(Varien_Event_Observer $observer)
{
    /** @var Varien_Object $indexesObject */
    $indexesObject = $observer->getIndexes();
    /** @var array $indexes */
    $indexes = $indexesObject->getIndexes();
 
    /**
     * We add indexes to these fields for faster request processes
     */
    $addFields = array(
        'upload_date', 'news_to_date', 'special_price', 'special_from_date', 'special_to_date'
    );
 
    foreach ($addFields as $field) {
        $indexes['IDX_'.strtoupper($field)] = array(
            'type' => Varien_Db_Adapter_Interface::INDEX_TYPE_INDEX,
            'fields' => array($field)
        );
    }
 
    $indexesObject->setIndexes($indexes);
}

 

We can add other attributes to the $addFields array. Please not that it is not all column types to which you can add indexes.

 

Errors that my occur during indexing

It can happen that you see some indexes with a processing status for a long time in the Index Management menu. This means that PHP could not close the index_process_*.lock files in the var/locks folder. The easiest way to solve this problem is to delete the lock files.

Another common error appears when the flat table gets damaged and this makes indexing fail. In such a case, the damaged flat table can be deleted without a problem because when reindexing, Magento checks if the table exists or not and if not, it creates it.

The columns in MySQL has a maximum number of 4096, but in reality it is much smaller. It also depends on how many characters each column can store. Thus it can happen that the table cannot be created. This is a rather hidden error because you can only see in the admin panel that the table cannot be reindexed, but you will have to search for a reason with server settings.

If you encounter such a problem, it’s worth taking a look at the attributes – whether all of them are needed in the flat tables or not. If you manage to reduce the number and/or size of the attributes (e.g. using varchar instead of text), this problem can be easily solved.

Unfortunately, the “trickiest” error is when the index process halts. This can happen when memory or running time limit is low on the web server. You can resolve this issue by increasing memory to be used by PHP and extend runtime.

Another solution is to run indexing with a command which I described at “2. shell script “ in the “How to run Magento indexing?” section.

 

Disadvantages of Magento index

I don’t want to confuse anyone. Magento index is a very useful feature of Magento. But at the same time you need to know about its disadvantages as well. Here are some:

  1. It duplicates data. Since data are written to the EAV data structure in admin, but on frontend the data are read from the index table. Therefore every product data is duplicated in the database. If you do not pay attention to reindex, it can happen that you see different things on the user interface than in the admin area.
  2. It needs abundant resources. Magento indexing uses a lot of server capacity because it reads data from a lot of locations.
  3. It is easy to reach the maximum limit of MySQL column numbers. I’ve mentioned this in the previous section.

 

Summary

I hope I could show properly the possibilities in Magento index and also provided help in solving some common problems that may arise. If you need any help, we’d be happy to assist you. Feel free to ask any questions in the comments field.

 

 

138 replies
  1. Deloris Baiera says:

    I precisely had to thank you so much again. I do not know what I might have made to happen in the absence of the entire advice documented by you on this situation. Entirely was the frightful scenario in my opinion, nevertheless noticing the skilled manner you dealt with it made me to weep with fulfillment. Now i am thankful for the service and even hope that you recognize what a powerful job you’re accomplishing educating the others via a web site. Most likely you have never come across any of us.

  2. Drug Rehab Hospital says:

    Inpatient Substance Abuse Treatment Near Me http://aaa-rehab.com Drug Rehab Centers http://aaa-rehab.com Suboxone Clinic Near Me Free
    http://aaa-rehab.com

  3. Mass hysteria says:

    I’m curious to find out what blog platform you have been using? I’m experiencing some small security problems with my latest site and I’d like to find something more secure. Do you have any suggestions?

  4. get likes in Instagram says:

    I was suggested this blog by way of my cousin. I am now not positive whether or not this post is written by him as no one else recognize such exact approximately my difficulty. You’re wonderful! Thank you!

  5. webmaster says:

    It’s really a cool and useful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.

  6. Jeff Cox says:

    It’s really a great and useful piece of information. I am happy that you shared this helpful info with us. Please keep us up to date like this. Thanks for sharing.

  7. car coating protection says:

    Throughout this awesome design of things you actually get an A+ for effort. Where you misplaced me ended up being in all the particulars. As as the maxim goes, details make or break the argument.. And that could not be much more correct in this article. Having said that, allow me inform you what exactly did do the job. The authoring is certainly highly convincing and this is possibly why I am making an effort to opine. I do not make it a regular habit of doing that. Next, even though I can easily notice a leaps in logic you make, I am not necessarily convinced of how you appear to unite the points which in turn make the conclusion. For now I will, no doubt subscribe to your position but hope in the near future you connect your dots much better.

  8. web hosting services says:

    Hello, i think that i saw you visited my website so i came to “return the favor”.I am trying to find things to enhance my website!I suppose its ok to use a few of your ideas!!

  9. Canada says:

    Someone essentially help to make seriously posts I would state. This is the first time I frequented your website page and thus far? I amazed with the research you made to create this particular publish extraordinary. Fantastic job!

  10. aduslot says:

    Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you’ve on this web site. It reveals how nicely you understand this subject. Bookmarked this website page, will come back for more articles. You, my pal, ROCK! I found just the information I already searched everywhere and simply could not come across. What a great site.

  11. Download Joker123 says:

    Great write-up, I am normal visitor of one’s blog, maintain up the excellent operate, and It’s going to be a regular visitor for a lengthy time.

  12. kevin david bbb says:

    Very well written information. It will be supportive to anyone who utilizes it, including yours truly :). Keep up the good work – i will definitely read more posts.

  13. kevin and david says:

    Thank you for another informative blog. Where else could I get that kind of information written in such an ideal way? I’ve a project that I am just now working on, and I have been on the look out for such information.

  14. FafaSlot Gamming says:

    I’ve been browsing on-line greater than three hours lately, but I never found any interesting article like yours. It is pretty worth enough for me. Personally, if all website owners and bloggers made good content as you did, the web might be much more helpful than ever before.

  15. Slot Joker123 says:

    I’ve recently started a site, the information you offer on this web site has helped me tremendously. Thanks for all of your time & work.

  16. Marcelle Darga says:

    I simply want to say I am just new to blogs and seriously savored you’re blog. Almost certainly I’m going to bookmark your blog . You really come with tremendous articles. Thank you for revealing your web-site.

  17. Daftar VivoSlot says:

    Thanks a lot for sharing this with all of us you really know what you are talking about! Bookmarked. Please also visit my web site =). We could have a link exchange agreement between us!

  18. laudo avcb palhoça says:

    Whats up! I just would like to give a huge thumbs up for the good information you could have right here on this post. I will probably be coming back to your blog for extra soon.

  19. job advertising site says:

    Excellent read, I just passed this onto a friend who was doing a little research on that. And he actually bought me lunch as I found it for him smile So let me rephrase that: Thank you for lunch! “Dreams are real while they last. Can we say more of life” by Henry Havelock Ellis.

  20. Game podcasts says:

    I love your blog.. very nice colors & theme. Did you create this website yourself? Plz reply back as I’m looking to create my own blog and would like to know wheere u got this from. thanks

  21. Webtraffic Webseite und mehr Webbesucher says:

    Oh my goodness! an incredible article dude. Thank you However I am experiencing difficulty with ur rss . Don’t know why Unable to subscribe to it. Is there anybody getting similar rss downside? Anyone who is aware of kindly respond. Thnkx

  22. Janel Rietz says:

    Hey there would you mind letting me know which webhost you’re working with? I’ve loaded your blog in 3 completely different web browsers and I must say this blog loads a lot faster then most. Can you suggest a good web hosting provider at a fair price? Kudos, I appreciate it!

  23. แทงบอล says:

    I loved as much as you’ll receive carried out right here. The sketch is tasteful, your authored subject matter stylish. nonetheless, you command get bought an nervousness over that you wish be delivering the following. unwell unquestionably come more formerly again since exactly the same nearly a lot often inside case you shield this increase.

  24. gclub says:

    My partner and I absolutely love your blog and find the majority of your post’s to be exactly I’m looking for. Would you offer guest writers to write content available for you? I wouldn’t mind creating a post or elaborating on a few of the subjects you write regarding here. Again, awesome web log!

  25. slotxo says:

    Really Appreciate this blog post, how can I make is so that I receive an email sent to me when you write a fresh post?

  26. check says:

    obviously like your web site but you need to check the spelling on quite a few of your posts. Several of them are rife with spelling issues and I find it very troublesome to tell the truth nevertheless I’ll surely come back again.

  27. สล็อต says:

    It?¦s really a cool and helpful piece of information. I am happy that you shared this useful information with us. Please stay us up to date like this. Thank you for sharing.

  28. Battery Delivery JB says:

    This design is spectacular! You definitely know how to keep a reader entertained. Between your wit and your videos, I was almost moved to start my own blog (well, almost…HaHa!) Great job. I really loved what you had to say, and more than that, how you presented it. Too cool!

  29. Login Slot joker123 says:

    Hello, you used to write fantastic, but the last several posts have been kinda boring?K I miss your tremendous writings. Past few posts are just a little bit out of track! come on!

  30. computer screen repair near me says:

    Normally I do not read article on blogs, however I would like to say that this write-up very pressured me to take a look at and do it! Your writing taste has been surprised me. Thanks, very great post.

  31. bebê reborn says:

    I’m not that much of a online reader to be honest but your sites really nice, keep it up! I’ll go ahead and bookmark your website to come back down the road. Many thanks

  32. gclub says:

    I would like to thnkx for the efforts you have put in writing this blog. I am hoping the same high-grade blog post from you in the upcoming as well. In fact your creative writing abilities has inspired me to get my own blog now. Really the blogging is spreading its wings quickly. Your write up is a good example of it.

  33. colchão sem ácaros says:

    Pretty section of content. I just stumbled upon your weblog and in accession capital to assert that I acquire actually enjoyed account your blog posts. Any way I’ll be subscribing to your feeds and even I achievement you access consistently fast.

  34. chá seca barriga says:

    You actually make it seem so easy with your presentation but I in finding this topic to be actually one thing which I think I might by no means understand. It kind of feels too complicated and very extensive for me. I’m looking forward to your subsequent put up, I will attempt to get the grasp of it!

  35. melatonina on como tomar says:

    An fascinating discussion is worth comment. I feel that you must write extra on this subject, it may not be a taboo subject but generally persons are not enough to talk on such topics. To the next. Cheers

  36. click here says:

    Hi, I think your site might be having browser compatibility issues. When I look at your website in Safari, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, fantastic blog!

  37. impressão 3d says:

    Hello! I could have sworn I’ve been to this blog before but after browsing through some of the post I realized it’s new to me. Anyways, I’m definitely happy I found it and I’ll be book-marking and checking back frequently!

  38. learn more says:

    Great post. I was checking constantly this blog and I am impressed! Extremely helpful information specifically the remaining section :) I maintain such information a lot. I was looking for this certain information for a long time. Thank you and good luck.

  39. car battery shop near me says:

    Thank you for any other great article. Where else may just anybody get that type of information in such an ideal method of writing? I have a presentation next week, and I’m on the search for such information.

  40. car battery delivery selangor says:

    You could certainly see your expertise within the paintings you write. The arena hopes for even more passionate writers such as you who aren’t afraid to mention how they believe. At all times follow your heart.

  41. car battery delivery stores says:

    I will right away grab your rss feed as I can not in finding your email subscription link or e-newsletter service. Do you have any? Kindly allow me understand in order that I could subscribe. Thanks.

  42. real time energy data says:

    I’ve been browsing on-line more than three hours these days, yet I by no means found any attention-grabbing article like yours. It’s beautiful worth enough for me. In my opinion, if all website owners and bloggers made good content as you did, the net will probably be a lot more helpful than ever before. “Baseball is 90 percent mental. The other half is physical.” by Lawrence Peter Berra.

  43. car rental singapore malaysia says:

    I’m no longer sure the place you are getting your info, but great topic. I must spend some time finding out much more or understanding more. Thanks for magnificent information I was searching for this info for my mission.

  44. blog de vinhos says:

    Great write-up, I?¦m normal visitor of one?¦s website, maintain up the nice operate, and It’s going to be a regular visitor for a long time.

  45. Banheira Dupla Com Hidromassagem says:

    I keep listening to the news update lecture about receiving boundless online grant applications so I have been looking around for the most excellent site to get one. Could you advise me please, where could i get some?

  46. lojavirtual da muda solucoes says:

    you are really a good webmaster. The site loading speed is amazing. It seems that you are doing any unique trick. Moreover, The contents are masterpiece. you have done a excellent job on this topic!

  47. Handy günstig says:

    When I originally commented I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several e-mails with the same comment. Is there any way you can remove people from that service? Many thanks!

  48. slotxo says:

    Hi, Neat post. There is a problem with your website in internet explorer, would test this… IE still is the market leader and a good portion of people will miss your wonderful writing due to this problem.

  49. Slotxo says:

    Hello there, I found your site via Google while looking for a similar subject, your website got here up, it appears to be like good. I’ve bookmarked it in my google bookmarks.

  50. สล็อต says:

    Magnificent web site. A lot of useful info here. I am sending it to some friends ans also sharing in delicious. And obviously, thank you for your sweat!

  51. imiwin says:

    A lot of of what you mention happens to be astonishingly accurate and it makes me ponder the reason why I had not looked at this in this light previously. Your article truly did turn the light on for me personally as far as this issue goes. Nevertheless there is actually 1 position I am not really too comfortable with and while I try to reconcile that with the actual core idea of the issue, permit me observe exactly what the rest of the visitors have to say.Nicely done.

  52. paisagismo says:

    Hi there! I could have sworn I’ve been to this website before but after browsing through some of the post I realized it’s new to me. Anyways, I’m definitely delighted I found it and I’ll be book-marking and checking back frequently!

  53. my site says:

    Wow! Thank you! I constantly wanted to write on my blog something like that. Can I take a portion of your post to my blog?

  54. Online Shop für Bettwaren und Haushaltartikel says:

    You actually make it appear so easy with your presentation however I in finding this topic to be actually something which I think I would never understand. It sort of feels too complex and very large for me. I am taking a look ahead to your subsequent submit, I’ll try to get the dangle of it!

  55. RudolfAPfuhl says:

    Hmm is anyone else experiencing problems with the pictures on this blog loading?
    I’m trying to find out if its a problem on my end or if it’s the blog.

    Any responses would be greatly appreciated.

    Look at my web page; RudolfAPfuhl

  56. super cha sb says:

    I believe this is one of the such a lot important information for me. And i am satisfied reading your article. But should observation on few common issues, The website taste is great, the articles is actually excellent : D. Good task, cheers

  57. IDNPLAY Poker says:

    I used to be very happy to search out this internet-site.I wished to thanks in your time for this excellent learn!! I undoubtedly enjoying each little bit of it and I have you bookmarked to check out new stuff you weblog post.

  58. quadrinhos para casa says:

    Pretty nice post. I just stumbled upon your weblog and wished to mention that I’ve truly loved surfing around your weblog posts. In any case I’ll be subscribing to your rss feed and I’m hoping you write once more very soon!

  59. Xavier Lobalbo says:

    I simply want to mention I am very new to blogging and site-building and honestly liked your blog. Very likely I’m planning to bookmark your website . You really have excellent writings. Appreciate it for sharing your webpage.

  60. Kanjinuano é bom says:

    Can I just say what a relief to seek out somebody who truly knows what theyre talking about on the internet. You positively know the right way to deliver a problem to light and make it important. More folks have to learn this and understand this aspect of the story. I cant believe youre not more widespread since you definitely have the gift.

  61. website says:

    Muy buen post! Felicidades por la web a menudo cuesta encontrar información sobre como solucionar algunas cosas.Merci por compartir! Yo también tengo una web me puedes leer en el enlace que os pongo. Hasta otra!

  62. exclusive car rental says:

    I will immediately seize your rss as I can’t in finding your email subscription link or newsletter service. Do you’ve any? Kindly let me recognize in order that I may just subscribe. Thanks.

  63. rv battery johor says:

    I take pleasure in, lead to I found just what I was taking a look for. You have ended my 4 day lengthy hunt! God Bless you man. Have a great day. Bye

  64. web hosting providers says:

    Have you ever thought about creating an ebook or guest authoring on other websites?
    I have a blog based upon on the same ideas you discuss and would
    love to have you share some stories/information. I know my readers would enjoy your work.
    If you’re even remotely interested, feel free to send me an e mail.

  65. best hosting says:

    I love what you guys tend to be up too. Such clever
    work and exposure! Keep up the very good works guys I’ve incorporated
    you guys to my blogroll.

  66. music says:

    Do you have a spam issue on this blog; I also am a blogger, and I was curious about your situation; many of us have developed some nice practices and we are lookingto trade solutions with other folks, please shootme an email if interested.

  67. pintura says:

    Hi there, I found your website via Google while searching for a related topic, your site came up, it looks great. I have bookmarked it in my google bookmarks.

  68. web hosting services says:

    You could definitely see your expertise within the article you write.
    The arena hopes for even more passionate writers like you who aren’t afraid to say how they believe.
    All the time go after your heart.

  69. web hosting companies says:

    Fantastic blog! Do you have any tips and hints for aspiring writers?
    I’m hoping to start my own site soon but I’m a little lost on everything.
    Would you advise starting with a free platform like
    Wordpress or go for a paid option? There are so many choices out there that
    I’m completely overwhelmed .. Any suggestions?
    Thank you!

  70. cirurgia plástica em belém says:

    What i don’t understood is in truth how you are no longer actually a lot more neatly-appreciated than you might be now. You are very intelligent. You recognize therefore considerably in terms of this subject, produced me personally imagine it from a lot of various angles. Its like women and men are not involved until it is one thing to do with Girl gaga! Your personal stuffs great. Always take care of it up!

Trackbacks & Pingbacks

  1. rx generic viagra

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  2. lumigan eye drops price

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  3. men over 50 viagra heart condition

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  4. hydroxychloroquine prescription

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  5. what happens if a girl takes viagra

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  6. where to buy tylenol

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  7. cialis online usa

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  8. ciprofloxacina

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

  9. sildenifil for sale

    Magento index – The basics, benefits and tips to avoid errors – aionhill.com

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.