Any orchestration system will operate as intended only when the data it operates on is accurate, unambiguous, and available at a known location in real-time. This implies that the heart of the Smart Infrastructure Orchestrator is a real-time operational data reservoir. This collection of data constitutes a federated data reservoir. The reason it is called a data reservoir and not a data lake is analogous to terrestrial settings, where a lake is a natural body of water, but a reservoir is a man-made collection of water from one or more sources curated for a specific purpose.

There are various formats that allow data to be stored as a single source of truth based on the target application that is consuming the data. The three major methods for storing data to drive business decisions include:

        i.     Data Hub – Data is centralized from multiple locations, harmonized and indexed as per required target application.

       ii.     Data Lake – Moving disparately-formated data from different locations into one system, like Hadoop/HDFS as required by different target applications. The key difference from Data Hub is that data is not harmonized and centrally-indexed as it is not easy to index data of different formats.

      iii.     Federated (Virtual ) Database – Data resides in different formats and different locations loosely tied together using metadata tags to relate data with each other.

For a Smart Orchestration system performing autonomous orchestration of infrastructure, it is imperative that the orchestration system has access to all the required data; historical, configuration details, past and present in real-time of the infrastructure that is being orchestrated. The data injected into such a system is disparate in terms of format, size, and location and needs to be collated as per the requirements of the service that is being rendered.

The optimal data reservoir for a Smart Orchestration system is often a Federated Database. This facilitates data capture at different locations, in different formats, and varying size to be indexed and stored locally for global access by a set of centralized orchestration applications. One of the critical design considerations is ensuring that the data captured is tagged with a well-defined metadata structure. Additionally, each data source should register with a central registry by declaring the type and functional characteristics of the data residing within the source. The meta registry serves as a central repository for the applications and provides the location and format of the data being requested by the application based on a metadata query. The collection of data sources along with the metadata store as shown below is the Single Source Of Truth for a Smart Orchestration System.

[Data Architecture for Smart Orchestrator]

Once the data from the infrastructure is available for the Smart Orchestrator at a single point of entry, it enables the Orchestrator to perform intelligent operations based on real-time and historical data from the infrastructure. With a federated database, the Smart Orchestrator can address multiple use cases.

Here are some of the use cases that can be addressed with the availability of such data:

Intent and Artificial Intelligence (AI) Assisted Provisioning: The provisioning of service on an infrastructure involves multiple different decisions to be taken by the service provisioning system to provide the optimal infrastructure for the intent of the service at that instant of time.

As an example, this service request was to create network connectivity from point A to point B, with a performance service level agreement (SLA) of 1Gbps sustained at all times. An ordinary provisioning system without real-time data available to aid the provisioning would have used the shortest path algorithm to provision the connection, which could result in 4 hops between point A and point B.

The system has no apriori information that the connection will not meet the SLA as the link between hop 2 and 3 is completely saturated and cannot deliver 1Gbps. This would result in a connection being delivered, but intent not met.

With an AI-assisted provisioning system that has access to a single source of truth of real-time data, this would not be the case. The system would first investigate all the hops of the path to ensure that the service can be delivered in compliance with the SLA, discover that the link between hop 2 and 3 is saturated and decide to take a longer path, possibly consisting of six hops instead of four, and deliver the service to be compliant with the SLA, but marked as sub-optimal for later remediation. This is only possible with a provisioning system that has real-time data from the infrastructure and historical data from the provisioning that allows decision making on the fly.

As in the case of any Intent Based Infrastructure Provisioning system, it is imperative for any intent translation, provisioning and assurance engines to have comprehensive real-time data from the infrastructure to provide an optimal infrastructure for the business. The figure below shows that the main and critical input to an Intent Engine is the data from the Network State Data Store or the “Single Source of Truth” to achieve autonomy of the infrastructure.

[Autonomous Enterprise]

Analytics-Assisted Anomaly Detection: As the provisioning of a service is performed based on the current state and historical state of the infrastructure, the Smart Orchestrator is continuously gathering data about each of the services being executed on the infrastructure. This data along with the appropriate analytics algorithm can very easily detect and alert anomalies within the infrastructure in real-time.

AI-Assisted Remediation: Smart provisioning and the appropriate analytics will alert the system in real-time to perform remediation tasks such that the erroneous services can be nudged back towards its SLAs. This combined with AI algorithms and real-time and historical data of the infrastructure can provide an intelligent or smart method to remediate any such events in the infrastructure. The remediation tasks can correct an erroneous service or reprovision the service to utilize the optimal infrastructure needed for the service.

As stated earlier, the AI-Assisted Provisioning system may have provisioned a service to utilize six hops instead of the optimal four hops due to resource constraints. At a later time when the resource constraints no longer exist, the AI-Assisted Remediation can reprogram the service back to its optimal path.

Service Level Rollback: This is an offshoot of Smart Provisioning that is being performed by the system, every service rendered on the infrastructure is well documented in terms of the parameters and the entire set of devices/components on which the services were provisioned. The data stored in the single source of truth will enable rollback of service to its previous state, which implies rolling back many of the infrastructure components in such a way that only one particular service is reverted back to its earlier state.

The single source of truth is the heart of the Smart Orchestration system that collects, collates, and distributes information to the various components of the Smart Orchestrator in a timely and accurate manner. This enables the Smart Orchestrator to automate infrastructure and provide responses in real-time for events occurring across the infrastructure.