Thoughts on Content Federation

By Derick Deleo

The concept of content federation has been around for a very long time. Some 20+ years ago at IBM we developed a product focused on content federation and actively tried to sell it; not so successfully though. And analysts had organizations convinced that they needed it; kind of like they did about records management, but that can be another topic for another day. But do organizations REALLY need content federation? And if so, why?

We have all heard for a very long time that many organizations have deployed multiple ECM/CSP (Enterprise Content Management/Content Services Platform) systems over the years. This was often caused by departments or lines of business going off and doing their own thing without anyone trying to stop them. It also happened when a business acquired other businesses who had their own ECM/CSP system. Eventually, the CIO or CTO or often the CFO began to question why they were spending money on multiple systems that basically all did the same thing. And then wheels started turning and ideas began to get tossed around.

Of course, consolidation always came up. But when companies began looking into the cost of migrating not only the content and metadata but the applications built on top of the ECM/CSP they had sticker shock. But at the same time it was in the organization's best interest to provide access to various individuals or groups to all of the ECM/CSP systems. Expecting these people to use multiple interfaces was not a good answer. Then the idea of somehow providing federated access to these systems arose. It was a good idea, but was it practical then and is it practical now?

First, if the cost of running multiple ECMs/CSPs is the major concern, federation not only does not address that problem it adds to the cost, because federation is not free. You need to buy a content federation product, have it implemented, then configure it to somehow normalize metadata across the disparate systems so that the results are understandable to the end user. That means you have the license costs for the product plus the services costs to implement and configure it.

Second, normalizing or making sense out of data across multiple ECMs/CSPs can be a difficult task. I have worked with many customers over the almost three decades I worked in the ECM/CSP space. To cite a simple example, I have yet to meet a customer where these disparate systems use the same field name and format for something like Account Number. Now the field name is easy to map, but trying to map different formats for the data can be tricky; and that is just one example. And different types of content across these systems often don’t even share the same type of data; which can result in a rather ugly result set to be displayed for the end user. Therefore just setting up federation becomes a cumbersome task; which often means it will cost a lot of money to accomplish.

Lastly, adding a layer between these disparate systems means adding latency. The federation system needs to send the query to multiple systems, wait for the results to come back from all queried systems, and then aggregate and normalize these into a single result set. Depending on your infrastructure and the performance from all of these systems this can result in frustrated users who start seeing much slower response. And we all know that end users never like slower responses.

Thought needs to go into why an organization would choose to use content federation. It can certainly provide a short-term solution to the problem described at the beginning of this article. Users can be convinced to live with ugly result sets for a time, and maybe even added latency, but in the end they want a simple, responsive, easy to use, and visually appealing solution. In my opinion, content federation should never be considered as a long-term solution for all the reasons I have already mentioned.

I have always encouraged organizations to seriously consider consolidation as the best solution to the problem of maintaining and paying for multiple ECMs/CSPs. And I strongly suggest thorough planning and design work before even beginning to work toward consolidation. Here are some good first steps to take:

Capture and document ALL of the schemas across these systems. AI can help do this for you. Analyze the schemas to come up with a normalized schema or set of schemas. This will require working with the owners of these various systems to agree on normalization. AI can help with this. The actual process of normalizing can be handled during the migration as an automated transformation process. Capture and document all workflows/processes across these systems. Evaluate these workflows/processes to determine what if any changes are desired or needed and how similar workflows/processes can be combined into one. Often organizations find that some workflows/processes are actually no longer needed or can be streamlined and possibly automated leveraging AI. Capture and document all integration points for all of these systems. Analyze and document how these integrations are developed. Determine what method would best serve your organization to redirect these integrations. Call the new ECM/CSP APIs. This is the fastest way to interact with the new system, but if at a later date a move to a newer ECM/CSP comes along then once again, these will need to be changed. If there are a lot of integrations this is not a practical approach. Develop a common API layer that these integrations call and have that layer then call the underlying ECM/CSP APIs. This makes moving to a newer ECM/CSP at a later time much less cumbersome and costly. And PLEASE, before you even begin to migrate content, determine what content can be purged and perform that purge. Far too many organizations ignore this step.

Migrations are NEVER easy, and there will always be issues with bad content, bad metadata, missing content, or orphaned content (I talked about this in a previous article). I strongly suggest engaging a service provider who has experience in running migrations of ECMs/CSPs to assist you in all of the above steps and the migration itself. And while you are working through all these migrations, federation can help for the short-term. But be honest with your users about the possible implications of using federation.

← All insights