Jump to Content
API Management

Large scale hybrid API management: Common challenges with structuring the right teams and platform

February 8, 2023
Greg Kuelgen

Technical Solutions Consultant

Varun Krovvidi

Product Marketing Manager, Google Cloud

Try Google Cloud

Start building on Google Cloud with $300 in free credits and 20+ always free products.

Free trial


In our discussions with technology leaders from around the world for – The Digital Crunch Time: 2022 State of APIs and Applications – two themes emerged 

#1 Cloud — and hybrid cloud specifically — is becoming a driving factor of success with 59% of respondents saying they were looking to increase their hybrid cloud adoption within the next 12 months.

#2 82% of organizations with a mature API strategy and higher API adoption reported increased efficiency, collaboration, and agility. 

It’s clear that hybrid cloud – usually a combination of on-premise infrastructure (or private cloud) and a public cloud computing environment (like Google Cloud) – and APIs are becoming popular keys to success. The challenge now becomes - how to structure your teams and architect your platform to manage APIs at scale in hybrid environments?

In this two-part series, we’ll look at how successful organizations are managing APIs on a large scale across hybrid environments using Apigee. In this post we’ll look at how to structure the right team and set up the platform to help you thrive. In part 2 of this series, we will explore how to operate the platform with optimum clusters, scaling, and automation. 

Why is it difficult to manage APIs at scale in hybrid environments?

As your business grows, it makes sense that you will need to scale and build more APIs to keep pace. This can mean thousands of APIs with tens of thousands of transactions per second. And these APIs are often operated across completely different business units with many development teams. Like with any large scale operation, building a central governance across these hundreds of APIs is extremely challenging.

Operating APIs in a hybrid environment offers a number of benefits — less latency, compliance with regulations etc., — but to get the most out of them, you need a clinical and pragmatic approach to solving the two challenges outlined below. 

#1 How to structure your API teams?

One of the most successful patterns when structuring your API teams is a Center for Enablement team. This team is made up of many federated API producer teams throughout the enterprise with one central team of deep subject matter experts. This central team builds out the guardrails for the API program, develops reusable content, helps automate things like the CI/CD pipeline, and delivers best practices. 

Note that the level of implementation handled by the centralized team may vary. For example, they may have a full CI/CD pipeline to maintain, or they may simply provide templates for the federated dev teams to build their own pipeline. However implementation is coordinated, having this structure ensures a consistent API model with appropriate governance and security in place across the enterprise.

Another successful strategy is to reduce the size or redefine the role of the central teams over time. For example, one of our customers recently adopted a model where their largest business units are fully responsible for operating their own infrastructure using a GitOps model. In this case, the centralized team created the automation for installing and upgrading Apigee hybrid, but the business units themselves operated their own copy of the platform, allowing them complete autonomy. In this way, gradually increasing the autonomy for the federated teams will help avoid delivery bottlenecks while ensuring consistent governance.

You can visualize this change over time pretty easily using the following diagram.


#2 How to architect your platform? 

Another challenge is designing your Apigee organizations (top-level container that contains all your API proxies and related resources) and environments (software environment for creating and deploying API proxies). To tackle this, you first need to understand the relationship between different entities within an Apigee organization. Apigee organization is the top level entity - nothing shared between Apigee organizations. Using an organization in Apigee, customers can segregate resources and manage access to these resources based on their own requirements. In practice, this means that each Apigee organization has its own apps, API keys, developers, proxies, and so on. The diagram below provides a visual representation of how different entities in an Apigee organization interact with each other.


Another key factor to consider is the technical limits of the platform - designing around these limits leads to higher platform performance and stability.

In the design phase, it is vital to articulate fundamental requirements of your architecture and agree on these baseline principles with all the stakeholders involved. Some of these fundamental questions include:

  • How many regions need to host runtimes?

    • Host runtime instances for organizations that handle production traffic in at least 2 regions to meet aggressive uptime SLAs. During upgrades, ensure only one of these regions upgrades at a time.

  • What are the steps in your Software Development Lifecycle (SDLC) and how does Apigee fit into it?

    • Use Apigee organizations as the SDLC perimeter — such as a dev organization and a test organization — to give yourself the capacity for a large number of proxies. It is also common that the dev organization in this model has relatively permissive access for developers, allowing them to work more efficiently. 

  • What level of access or separation do you need between business units or development teams? 

    • Create separation between different teams by leveraging environments with conditional identity and access management to restrict access as necessary

    • If your team is very large and you plan to have multiple organizations, you may want to divide the operational ownership for these different Apigee organizations among different teams in your enterprise while keeping them operating under the centralized team. 

      • In such a shared responsibility model, the largest consumers of the system stand up and operate their own instances. Teams with many APIs or distinct operational needs can use the automation from the central team to stand up and operate their own Apigee hybrid clusters. This reduces the burden on the centralized team while allowing business units to be self-sufficient and ensuring that the clusters are optimized to meet that team's needs.

      • For very large scale programs where additional separation is necessary, consider creating separate organizations for each business unit, each of which has the full complement of SDLC orgs. Some customers have this level of scale, and often it makes logical sense to divide up into different Apigee orgs because it’s rare that 5000 APIs are all related and used together. 

By working off these baseline recommendations, you will come up with your own logical design of Apigee organizations and environments. Using Apigee organizations to represent your SDLC with business units — or functional areas or some other dividing mechanism — is a good way to design different environments.


For large scale deployments — involving thousands of proxies — we recommend dividing organizations into different logical business units while still following the SDLC model. In many cases, only the largest business units will need their own organizations, and it is possible to have a shared organization that contains multiple business units.



Like with any large scale IT project, there isn’t one“correct” way to operate APIs in a hybrid cloud environment. Finding the right approach for your enterprise starts with knowing the organizational requirements and tailoring Apigee for your use case. In the next installment of this series, we will explore how to operate Apigee with the optimal resources (clusters, automation, and monitoring etc.) for a large scale hybrid API program. 

Check out our documentation to learn more or start using Apigee hybrid.

Posted in