Magento ERP Integration – How to Plan it Properly Without Making Critical Mistakes

Integration of ecommerce stores to other systems is one of the most complex challenges with ecommerce businesses, especially when talking about Magento ERP. A fitting trinity of an online store, ERP software and the connection between them can create a solid foundation for the online business and can hugely contribute to the overall success of a company. Generally, an online business is expanding and improving constantly with new approaches and technologies, so there is always something new to incorporate and integrate.



 

First of all, let’s start with defining the most critical points of such an ERP system:

  • A) the pieces of ERP software themselves,
  • B) the ecommerce platform,
  • C) the data flow design between them and
  • D) the implementation of the data flow.

 

Here we will take Magento as an ecommerce platform being, in our opinion, the optimal choice for small and middle sized businesses. We will underline this with a few arguments soon. As far as ERP is concerned, the capability to integrate to ecommerce is only one dimension of the whole picture.

However, an actively developed, modular, expandable system – sized for the company in terms of costs and features –, which supports integration to other systems and ecommerce in particular, seems to be essential. A properly documented, extendable API would be ideal.

 

All this being said the main scope of this post is centred around the remaining two questions: what to sync and how to sync between Magento and an ERP (Enterprise Resource Planning) system.

 

In the following we are going to cover the following topics:

 

Business perspective – the role of the ecommerce store within a business

  • Why to integrate your online store and ERP system?
  • Single point of operation – for simplicity and saving time
  • Implications of company size and the ecommerce side of it
  • Consequences of growth – bias towards the ERP side as the centre of operations

The online store dilemma – why Magento?

  • Feature-rich
  • API (Application Programming Interface)
  • Already integrated

The ERP side of integration

  • ERPs are born out of the natural need for connecting processes
  • The price of flexibility
  • ERP – cloud based or on premise?
  • ERP connectivity – system-wide APIs do the job best

Building the bridge – how to connect Magento with your ERP

  • Theory: Data flow design
  • Practice: approaches to implementation of ERP integration

How to choose? Your Magento ERP integration service provider is key.

  • Some good questions to ask an ERP integration specialist

 

 

Business perspective – the role of the ecommerce store within a business

 

Why to integrate your online store and ERP system?

Whether you are a big company or you have only a hobby estore with a few products, you will love to optimize and automate repetitive tasks. It will save you time and money, reduce the risk of errors, multiply your capabilities and will save you a lot of frustration.

That is why you want to integrate. Different systems are optimized for different tasks – ERP for resourcing management, ecommerce marketing software for online sale, CRMs for managing connections, and you want to have the best of everything.

You have no other choice but to integrate.

 

ERP integration with your ecommerce store can save you a lot of time and money. Click To Tweet

 

Another thing you will realize soon is another huge advantage of integration: you have to perform a certain task only once in one system and let all the other systems catch up automatically – that is another good reason to implement integration. Once you integrate, you will naturally want integration to be seamless and automatic. It is no use integrating if you have more trouble with integration than with the actual work.

 

Single point of operation – for simplicity and saving time

So it is nice to have dedicated systems to get special tasks done at central places and it is also ideal if you only have to handle data in one place. For example, no matter where orders are created, online or offline, it is best to manage them in your ERP.

The same way, if you manage to import, set prices and configure products in your ERP, you do not want to do the same thing again in your online store. Of course it is possible to make compromises, like creating products via batch imports both in the ERP and Magento and sync only the inventory, but to be able to draw conclusions, let us simplify and concentrate on the theoretically, and practically, ideal flows, setting apart the implementation efforts, potential costs and technical difficulties.

 

Implications of company size and the ecommerce side of it

It is easy to realize that the bigger your business beyond ecommerce, the more things you want to do centrally in your ERP and sync it all to Magento. And this holds true as well: the smaller your business beyond ecommerce, the less things are advised to do outside of Magento, since Magento is already equipped with the basic ERP features: stock, invoice and shipment management.

 

magento ERP integration small ecommerce business

 

A startup with only a few virtual products might only want to integrate an invoicing system: only sync orders from the estore and manage everything else directly in Magento.

A merchant with several warehouses and physical shops or a drop shipping company operating globally, on the other hand, has to create, manage products inventory, pricing, invoicing and shipping in an ERP system anyway and, naturally, wants its ecommerce solution integrate seamlessly to the ERP, without even having to log in to the store admin area, except when adjusting some UI or marketing stuff.

 

magento ERP integration big ecommerce business

 

The same applies to all other integrated systems:

CRM, BI, ecommerce marketing, 3PL and so on. The more diverse a merchant’s operations are, the less practical it is to integrate these systems to the ecommerce platform directly and it makes more and more sense to sync them to the Enterprise Resource Management solution where everything is actually managed.

 

Consequences of growth – bias towards the ERP side as the centre of operations

So the rule of thumb for ecommerce integration and integration in general is this:

Do one kind of operation only in one system and use that system as the source point of integration to all other systems.

 

 

The online store dilemma – why Magento?

Feature-rich

Magento has been designed to be the full-featured ecommerce platform since the beginning. Its open source approach, multi-site design, all possible product types, flexible product attributes, customer groups, price rules, layered navigation, payment and shipping integrations, import and export capabilities make it a unique player in the market.

 

API (Application Programming Interface)

 

It is probably the most important benefit of Magento:

Magento has advanced APIs that open the gates for the greatest part of its functionalities. As far as technologies are concerned, Magento has SOAP API v1, v2, XML-RPC, REST, and XMLConnect for mobile. These APIs are quite well documented as well. Magento API has another extremely valuable feature from the point of view of integration: it is extendable in a modular way.

 

Already integrated

Last but not least, Magento is already integrated. There are dozens of out-of-the-box integrations on the market to all major ERPs, such as SAP, Microsoft Dynamics, Sage, NetSuite and so on.

 

The ERP side of integration

ERPs: are born out of the natural need for connecting processes.

 

ERPs are by definition centred around integrating core business processes, such as finance, budgeting, payment management, accounting, invoicing, manufacturing, human resource management, workflow management, quality control, procurement, supply chain management, warehousing, inventory management, customer relationship management and so on.

ERPs, by connecting data and processes, improve quality, efficiency, collaboration, monitoring and facilitate decision making in a company.

 

The price of flexibility

ERPs always have to be tailored and configured to the needs of an actual company, so flexibility is built in the genes of every good ERP. The more flexible and customized an ERP is, however, the more difficult it is to connect it to another, especially another complex system like Magento where potentially different ERP modules handle different connection points in both connected parties.

 

ERP – cloud based or on premise?

Cloud based services are booming today and, as in all segments, in the ERP world too. They have many obvious advantages, some of the most important are the quicker general setup time, reduced overall operational costs, reliability of service, transparent updates and improved API connectivity. All these features make cloud based ERPs an extremely competitive choice for mid-sized companies.

A cloud service by nature has to take special care to a flexible API. That is one of the real points of the cloud, and a good position for anyone who wants to have ecommerce integration.

On premise, self hosted ERPs for middle sized and big merchants of course, on the other hand, give the merchant more freedom, control, data security, and the ability of deeper customizations.

For a small company, a compact ERP with dedicated support on the office computers or in a nearby data centre can be perfectly fine as well. Ecommerce integration, however, in this case, generally requires a deeply customized solution.

 

ERP connectivity – system-wide APIs do the job best

Another historical problem of ERP integration is that ERPs originally focused on integrating different modules internally and interfacing to other systems was at best only an afterthought. This trend is changing rapidly and in today’s word of cloud services, vendors are more and more realizing the value of full connectivity.

 

Full connectivity practically means well documented, extendable, feature-rich, robust and high performance, system-wide APIs.

 

It is a huge advantage that APIs have two major aspects:

  • the business logic that makes the internal processes accessible through controlled data exchange
  • and the standardized and documented interface that sits on top of it.

 

Both are very important. Many of the legacy ERP systems only offer database or file/document based access, even making building the ERP interface the bigger part of any integration effort itself.

Some ERPs have integrated online stores as a module.

If the integration is done via an API on the ERP or the ecommerce side, it is generally possible to replace the estore with reasonable effort to a better one leveraging the same interfaces as the original.

 

Building the bridge – how to connect Magento with your ERP

Once we have an ERP and ecommerce platform that fits our purposes, we have to finally connect them. But before that, when it comes to finding the right ERP platform, in a situation of technology change or a new business, it is the best to apply a global view and not only weigh the ERP and ecommerce systems individually but make sure they can be connected well and by the right partners and with the right solution.

 

magento ERP integration connection

 

Theory: Data flow design

When it comes to ERP–ecommerce integration, and integration in general, there are some major architectural points that might be worth taking into account.  

 

Data flow – source and target

This is the easiest and highest level concept of data flow design. We want data somehow created or updated in one system to be transferred to another system. At this level it does not matter how we do it, whether there are any differences in the data format, we do not care about the channels, we only know that we want it to happen.

For example, if a product attribute is updated in the ERP, we want the modification to be transferred to the ecommerce store.

 

Timing – how much delay is tolerable?

Once we pictured the data flow schema, i.e. what data we want to transfer from point A to point B and vice versa, it is also important to figure out where instant, let us say, minimal delay, updates are needed and where greater delays are acceptable.

In some systems, for example, stock levels and product availability are changing quickly and they have to be as much up-to-date in the ecommerce system as possible. Fastest updates can be achieved if the system, where the change occurs, triggers the update directly but frequent status checks from the target system can be an acceptable compromise.

 

Sync or Async – async processes are more efficient

Another aspect of timing is synchronicity. In a system two operations are synchronous if the system has to wait for the results or termination of a call – local or remote, e.g. data transfer – to be able to perform the next operation.

In the case of async I/O instead of idle waiting the system can do something else useful and come back to the original operation when the call is finished. Sometimes blocking dependency between certain calls is inevitable, but in data flow design it is a very good practice to try to eliminate these kinds of sequential dependencies whenever possible and make the overall design async.

When used in a sensible way, async processes generally scale up much better and give less chance for data “traffic jams” and bottlenecks. Even if we use async processes, however, some feedback on the success of the operation is desirable. This is related to fault tolerance and robustness.   

 

Payment transactional and order creation data – special care is needed!

Sometimes the goal is to make sure a series of operations across two systems happen together or do not happen at all. Payment transactions are among the most trivial examples.

Money has to be removed at one place and added at another at the same time. If anything prevents it to happen, even in the middle of the process, the whole process has to be rolled back to the original state on both ends.

 

Another good example in ecommerce data sync is order creation:

Think of an online store that sells discounted items with very low stock levels from a warehouse that sells offline too.

In this case it is a valid requirement that an order should be created in the estore and the ERP together, or should not be created at all with an error message sent to the customer in the online shop that the product is no longer available in the desired quantity.

 

Practice: approaches to implementation of ERP integration

Now we are beyond the high level design of integration, understanding data sources targets, timing, sync and async processes and transactions and we are ready to think of the actual implementation.

 

Server-Client API

One of the key concepts of data engineering is the server-client design, in which the client party starts the communication, actively pushes or pulls data or sends commands, while the server, as it name implies, performs what it is asked to do, processes, returns data or performs tasks.

Clients call the servers through some channels and the servers process the call, preferably with some prompt response as a feedback. This represents the question/answer or statement/acknowledgement paradigm, so it enables active, synchronous communication between two or more systems.

It is also ideal for handling series of async processes where feedback is required on the success of operation. A server/client setup can easily handle parallel connections with multiple clients so it is well scalable.

 

File transfer and data feeds

This is a practically protocol-less data transfer method where one party simply makes new data available for the other party as a simple document. The easiest version of it is to publish data via a web or FTP server or upload it through FTP to the target machine.

Due to its simplicity, this model works well under plain peer to peer circumstances where the receiving party trusts the data source so no substantial authentication, authorization validation and direct feedback is needed.

Far from being a sophisticated and flexible protocol, it fits perfectly to the one way flow of data provider/data consumer, or more generally broadcaster/receiver paradigm and is extremely common especially for product data transfer even nowadays. As far as transferred data is concerned, by nature it is async, but document delivery and processing itself can be controlled, queued or sequentialized.

This kind of communication also assumes activity both on the sending and receiving party’s side. The sender should actively push or broadcast and the receiver should actively pull and process documents. This generally means custom development on the receiving end.

 

Direct database queries

Using direct database queries to sync data is a powerful approach, but in most cases it is very error-prone and dangerous at the same time, since it circumvents the business logic built around the database.

 

Middlewares

Another pattern is a middleman in the communication: a third system may sit between the systems connected to optimize, monitor, log data flow, translate different formats and cache things. This bridge in between can be a hub of not only two systems, but a junction point of a whole infrastructure of subsystems, connecting everything with everything else.

 

These middlewares are often referred to as Enterprise Service Buses.

A good example of these service hubs is Zapier. It leverages hundreds of different APIs as a client and translates input data format coming from the source system to the export data format of the target system, and offers all this as a service.  

The beauty of this approach is that in a data hub you have to implement the API client to an external system only once. Next time you have to connect to the same system, you only have to map data correctly between them.

These middlewares can be extended with server endpoints, its own API, as well, so that it can be reached actively from outside.

An even more advanced version of the same principle is when input data is translated back and forth to a universal internal format in an additional layer built around the API clients. In this case different server/client endpoints of the data hub speak the same language and no peer to peer translation is necessary between them. Data only has to be routed and spread across the different endpoints. This is a highly idealized model but a very up-to-date trend of data networks.

A number of integrators have realized the advantages of some form of the middleware approach, in fact if someone is in the business of connecting multiple ERPs to multiple ecommerce platforms, to create and maintain several peer to peer integrations requires duplicated efforts, potential issues and costs.

 

Point to point vs. middleware

Point to point integration means that two systems are connected directly. In terms of the server-client point of view, the architecture of P2P connection can be diverse. There are certain cases where the ERP connects as a client to the ecommerce platform, in some cases it is implemented the other way round, and there are situations where the combination of both is used. Sometimes APIs are built especially for the sake of the P2P integration on one or both ends.

In case of P2P integrations, there is generally more space for planning and customization and the actual business needs drive development.

 

How to choose? Your Magento ERP integration service provider is key.

The most important aspect of an integration is that it should fulfil the business needs. The right data flow design and features are some of the key factors, but irrespective of the integrator, technology and implementation, there are some other aspects that can be crucial.

When choosing a specific technology or mix of technologies and ERP integration provider, reliability and fitness for the purpose should be the primary aspects and a right balance should be found between costs, performance, scalability, quality of support and other factors. It is the merchant’s and the general solution provider’s responsibility to have a clear view of priorities and understand the critical points.

 

Some good questions to ask an ERP integration specialist

Finally, we have collected some questions that might help to compare different solutions and to make a choice between them.

 

tips Questions: Reliability: How reliability of the integration is ensured?

If a process is critical for the business, for example stock updates, it should be noted that logging and monitoring, automatic error handling, notification system, and maybe live support become essential.

 

Performance: How performance can be improved? Are there some tests or comparisons available? What are the bottlenecks and limitations of the solution? How can bottlenecks be eliminated? What are the upper limits for individual processes?

The potential edge cases should be defined and matched with the figures of the offered default implementation. We have to formulate our expectations and if it does not match the default figures of the default implementation of a vendor, optimization or an alternative solution is required. Bulk data imports and updates are the best candidates for performance issues, so anywhere a lot of data is synced at the same time, special attention is required.

 

Scalability: Can the performance be improved by upscaling resources? Can the solution be upscaled to handle additional stores?

We have to make sure that the integration is prepared to handle all current or potential future business requirements.

 

magento ERP integration questions

 

Data ownership, security and transparency: Who owns the data? How data security is guaranteed? How much the sync processes are transparent for the merchant?

If we have sensitive data and consider cloud based solutions and SaaS integrations, it is an important point.

 

Custom development: How bugs are fixed and new releases installed? Is the solution extendable by 3rd party providers? Is the code or parts of the code open source? Is there a technical or developers documentation available?

Some kind of customization is necessary in most of the cases and it is common for new demands to arise any time. It gives much more freedom to the client if 3rd party developers can be involved.

 

Code maintenance and active development: Is the code actively maintained? How often do new releases come out? Is there any effort needed by the merchant to install new versions or updates? Does the support package cover these processes?  

 

Costs and delivery deadlines: What are the costs of installation, configuration, support and customization. What services are included in the advertised price? How complex is it to configure the solution? How much customization is needed compared to the default implementation?

Generally, the more of the merchant’s requirements an out-of-box implementation covers, the less risky it is to customize and go live with it. There are many components of the overall costs of implementing and running a solution, and it is wise to be aware all of them.

 

Support: What kind of support is provided to answer general questions or in terms of technical staff who can act immediately in emergency?

Of course, 24/7 technical support is preferred, but in the lack of it, it is good to make sure the reaction time of the vendor is fast enough for you.

 

Conclusion

Integrating Magento with an ERP system is a complex challenge with many aspects to it. Even in simple cases, before plunging into implementation, it is highly recommended to analyze and understand the business needs first, and then create a high level architectural design based on that with a prioritized list of features, performance requirements and other important factors.

It is also very useful to recognize the pros and cons of different implementation methods. Hopefully, the points listed above provide some help in all this.

At AionHill we have successfully integrated Magento with a number of different ERP systems over the years and we are happy to help anyone who needs advice.

 

If you have anything to say in the scope of this topic, please leave a comment below. Should you have any further queries, please contact us, we’ll be happy to assist you.

 


Do you need our support?
  • Magento Site Check
  • Magento Code Audit
  • Magento SEO Audit
  • Magento Project Rescue
Request help

NEED A RELIABLE, PROFESSIONAL MAGENTO DEVELOPMENT PARTNER?

Contact us if you have any question or requirement related to the preparation of a new or renewal of an existing online store.

Next