Commonly Asked Question: How many products (SKUs) can Magento handle?

 

One of the first questions business owners ask when trying to determine if the Magento platform is right for them is “How many products can I put on my Magento store?”

One of the biggest projects we have worked on here at Customer Paradigm had 250,000 – 300,000 products. The creation of Magento stores with vast catalogs is possible.

At the base technical level, Magento does not impose any product limits. Theoretically, Magento can handle millions of SKUs. From a practical standpoint, out of the box, Magento is not well optimized to handle large catalogs with massive amounts of SKUs.

Words to Know

SKU – “Stock Keeping Unit” is a number or code assigned to a product to identify the product, options, price, and manufacturer. A SKU is used to help you keep track of your products. Each product must have a unique SKU. SKUs are the same across all of your stores. SKUs can not repeat.

Indexing & Reindexing – Indexing is how Magento transforms data such as products and categories (etc.) to improve your websites performance. As the data changes, the modified data must be updated (reindexed). Without indexing, Magento would have to calculate everything (i.e. every price of every product – taking into account all rules, bundles, etc.in order to) on the fly, causing very long loading times.

Cache & Caching – “Caching” your website stores a full output of your page so that subsequent page loads do not require a lot of work from the server – thus speeding up loading times. Caching helps your store perform better when there is a lot of content in it.

 

Site Performance, Site Complexity, and Site Resources

One of the biggest areas of concern in creating a Magento site with a lot of SKUs is going to be your site performance. The more complex your site becomes, the more resources it will require maintaining an appropriate level of site performance.

In Magento, adding products is not a “one and done” task. For each product you add, you also have to add an image, a category, attributes, give it a SKU and more. Each of these tasks, multiplied by the amount of products your site has, adds to the complexity of your site.

You also have to consider:

• Number of websites and stores
• Number of categories
• Amount of custom code
• Number of extensions
• Theming
• Your server architecture and software
• Caching

The more complex your site is, the more draining it is on your sites performance. The more complex your site is, the slower your site will become. We know that speed matters. Pages with more than a 3 second load time can lose up to 40% of customers.

Key idea: The more products that have been added to your site, the more complicated it becomes, the larger your server resources need to be to maintain a usable speed.

 

On the backend, uploading products, URL rewrites, inventory reindexing and flushing the cache (which you will need to do after each bulk procedure) will take you significant amounts of time and server resources.

Each time you make a change to your site, or a visitor accesses your site, a request is sent to the server. On a larger site, more requests are sent to the server. Large sites will have far greater hosting demands. As such they will need to choose a top tier host that has a specific type of hosting environment.

All of this top shelf hosting comes at a top shelf price and will be a large up front investment into the infrastructure of your website.

 

Management of Large Catalogs

When working with large catalogs of products on Magento, it is important to consider upkeep. If you have 300,000 SKUs and 10% of them need to be updated weekly – that is 30,000 SKUs you have to manage on a weekly basis.

Stores of this size require significant staff time to maintain. All staff must be trained to use the backend Magento interface if they are “non-tech” admin. You or your staff will need to know how to upload and edit products, re-index and flushing the cache after all bulk procedures.

Admin support will be needed to get going and stay going for stores of any size. However, the larger your store, the more administrative upkeep you will have. You need to be able to maintain the level of staff for the amount of products you want to have on your website.

Community vs. Enterprise

If you are planning to have a significant number of products in your Magento store, the Magento platform you are running is also something to take into account. While the Magento Community platform can do a wide variety of things, the out of the box functionality is not as diverse as the Magento Enterprise platform. (Customer Paradigm Founder Jeff Finkelstein talks about the pros and cons of each system at length!)

If you have already invested the money into server and staff infrastructure, it is recommended to use the Magento Enterprise Platform for larger stores for the following reasons:

• Better at handling indexing
• Better search functionality
• Better at handling data flow
• 80% faster in re-indexing a catalog
• Full page caching

Real World Case Study

Currently, Customer Paradigm is working with a client who has 26,000 products. While this is nowhere near our largest project of 300,000 products, it is still an amount that the client is already seeing a pain point at. Customer Paradigm Project Manager Darin has the inside scoop on this project.

When we began working with this client, they had all of their SKUs in one category, which has a negative effect on the front end user experience, conversion rates, and SEO ranking.

With all of the products in one lump category, there was no way for users to easily search the site to find the products they were looking to buy. With no sense of organization for their products, there was no way to easily break down the shopping experience for their customers. If customers cannot navigate through your expansive catalog, then you won’t be able to sell your products.

Darin’s challenge in working with this large number of unorganized SKUs is threefold. First, he needs to collaborate with the client to define and create a series of categories and subcategories for the SKUs, then find a streamlined way to assign all of the SKUs to a subcategory as well as enter all missing Metadata for SKUs and categories. The overarching problem is that all of the options for achieving these goals after the fact are incredibly tedious.

While some aspects of this project are evident – such as:

• The need for an easy to use layered navigation menu
• Utilization of search-friendly URLs (every URL needs to be unique and optimized for the page’s primary keyword)
• The implementation of Meta descriptions on category and product pages

 

The actual assignment of categories to the existing SKUs is a little trickier; there are three main options for doing this:

1. Manually assigning each SKU to a category – essentially going one by one. This option will be the lengthiest depending on how many existing SKUs you already have.

2. Exporting/Importing through a .CSV file – with this option you will export all SKUs to a spreadsheet, and from there update each SKU in the spreadsheet. Each Category will have an ID; you will have to find the corresponding ID and enter it for each SKU. The pitfalls of this method are that you have to ensure 100% accuracy for successful upload, and Magento only lets you upload about 1,500 products at a time.

3. Utilization of an extension – there several extensions on the marketplace that attempt to make bulk processes simpler

Darin is positive that once the new site structure has is in place that managing this large number of products will get easier for the client, as long as they continue to follow the systems put in place. However, he notes, that it is important to think about site structure before the initial creation of your site!

 

Do you need help with your Magento site – big or small? Our expert team can help you set your Magento eCommerce website up for success.  Call 303.473.4400 or visit here to have a real person contact you now >>