Open
Contrail
Nodes
Analytics
nodes collect, store, correlate, and analyze information from network
elements, virtual or physical. This information includes statistics,
logs, events, and errors.
Configuration
Node:
Figure
6 shows the internal structure of a configuration node. The
configuration node communicates with the orchestration system via a
REST interface, with other configuration nodes via a distributed
synchronization mechanism, and with control nodes via IF-MAP.
Configuration
nodes
also provide a discovery service that the clients can use to locate
the service providers (that is, other nodes providing a particular
service). For example, when the vRouter agent in a compute node
connects to a control node1, it uses service discovery to discover
the IP address of the control node. The clients use local
configuration, DHCP or DNS to locate the service discovery server.
Configuration
nodes contain the following components:
A
REST API server provides the northbound interface to an orchestration
system or other application. This interface is used to install
configuration state using the high-level data model.
A
Redis
[redis] message bus facilitates communications among internal
components.
A
Cassandra
[cassandra] database provides persistent storage of configuration.
Cassandra is a fault-tolerant and horizontally scalable database.
A
Schema
transformer
detects changes in the high-level data model over the Redis
message bus and transforms (or compiles) these changes in the
high-level data model into corresponding changes in the low-level
data model.
ZooKeeper
[zookeeper] (not shown in diagram) is used for allocation of unique
object identifiers and to implement transactions.
Analytics
Node:
Figure
7 shows the internal structure of an analytics node. An analytics
node communicates with applications using a northbound REST API
(northbound interface is an interface that
allows a particular component of a network
communicate with a higher-level component), with other analytics
nodes using a distributed synchronization mechanism, and with
components in control and configuration nodes using an XML-based
protocol called Sandesh
designed specifically for handling high volumes of data.
Sandesh
is a southbound interface protocol that is used by the Open Contrail
Analytics engine to gather information from the Open Contrail virtual
routers and other software modules like the control-node, and the
configuration manager, which run as part of the controller.
The
analytics nodes contain the following components:
A
collector
exchanges Sandesh
messages with components in control nodes and configuration nodes to
collect analytics information.
A
NoSQL
database stores this information.
A
rules
engine
automatically collects operational state when specific events occur.
A
REST
API
server
provides a northbound interface for querying the analytics database
and for retrieving operational state.
A
query
engine
executes the queries received over the northbound REST API. This
engine provides the capability for flexible access to potentially
large amounts of analytics data.
Sandesh
carries two kinds of messages—asynchronous messages received by
analytics nodes for the purpose of reporting logs, events, and
traces; and synchronous messages whereby an analytics node can send
requests and receive responses to collect specific operational state.
All
information gathered by the collector is persistently stored in the
NoSQL database. No filtering of messages is done by the information
source. The analytics nodes provide a northbound REST API to allow
client applications to submit queries.
Explanation:
An
analytics node communicates with applications using a northbound REST
API, with other analytics nodes using a distributed synchronization
mechanism, and with components in control and configuration nodes
using an XML-based protocol known as Sandesh designed specifically
for handling high volumes of data.
The
collector collects messages sent by the sandesh to collect the
analytics data and this data is stored in the NoSQL database. To get
all the information from the sandesh these messages have not passed
through filters. When specific events occur rules engine
automatically
collects operational state. A REST API server provides a northbound
interface for querying the analytics database and retrieving the
operational state from the database information. A query engine
execute the queries received over the northbound REST API. This query
engine provides the capability for flexible access to potentially
large amounts of analytics data. The data is analyzed by the
analytics nodes one by one and the process goes on.
REST
API is provided by analytics nodes to allow client applications to
submit queries. Analytics nodes provide scatter-gather logic called
“aggregation.” A single GET request (and a single corresponding
CLI command in the client application) can be mapped to multiple
request messages whose results are combined. The query engine is
implemented as a simple map-reduce engine. The vast majority of
Contrail queries are time series (a series of
data points indexed)
Redis Structure
Redsi is use as a message bus in contrail system. It can use for many
purposes and there are many key-value stores. Contrail populate 4
different key-value stores.
There names are
DB1
This key value store use to store querries coming from API servers.
Then querry engine read querry from DB1 and perform it on Database.
After executing querry Querry engine store data in DB1.
DB3:
This key-value store use to store flow records
DB4:
This key-value store use to store UVES in xml form that we see on
port 8081
DB7:
This key-value store is same as what we see on port 8081
Note: The data store in all the key-value store is distributed using
keys and values concept. So, if you want to retrive data you need key
for that data. If you are a subscriber you received key where the
change occurs and then you perform query on redis to get data.
Cassandra Structure
There are many databases in Cassandra but we focus on Analytics Data
base which is name as ContrailAnalyticsCql
ContrailAnalyticsCql
Database:
Now, these are the tables in contrail analytics database
flowrecordtable
flowtablevrouterver2 statstablebydbltagv3
objectvaluetable
messagetablemessagetype messagetablekeyword
messagetabletimestamp
messagetablecategory statstablebyu64strtagv3
statstablebyu64tagv3
messagetablesource statstablebystrstrtagv3
systemobjecttable
flowtableprotdpver2 statstablebystru64tagv3
objecttable
statstablebyu64u64tagv3 flowtabledvndipver2
messagetable
statstablebystrtagv3 flowtableprotspver2
flowtablesvnsipver2
messagetablemoduleid
Now
we list all tables with their attributes
flowrecordtable
action
bytes
data_sample
destip
destvn
direction_ing
dport
drop_reason
input_interface
key
max_interarrival
mean_interarrival
min_interarrival
mpls_label
nw_ace_uuid
other_vrouter_ip
output_interface
packets
protocol
reverse_uuid
setup_time
sg_rule_uuid
sourceip
sourcevn
sport
stddev_interarrival
tcp_flags
teardown_time
tos
underlay_proto
underlay_source_port
vm
vmi_uuid
vrouter
vrouter_ip
flowtablevrouterver2
column1
column2
column3
key
key2
key3
value
statstablebydbltagv3
column1
column2
column3
key
key2
key3
key4
key5
value
objectvaluetable
Messagetablemessagetype
column1
column2
column3
key
key2
Messagetablekeyword
column1
column2
column3
key2
messagetabletimestamp
messagetablecategory
column1
column2
column3
key
key2
statstablebyu64strtagv3
column1
column2
column3
column4
key
key2
key3
key4
key5
key6
value
statstablebyu64tagv3
column1
column2
column3
key
key2
key3
key4
key5
value
messagetablesource
column1
column2
column3
key
key2
statstablebystrstrtagv3
column1
column2
column3
column4
key
key2
key3
key4
key5
key6
value
systemobjecttable
SystemObjectConfigAuditTtl
SystemObjectFlowDataTtl
SystemObjectFlowStartTime
SystemObjectGlobalDataTtl
SystemObjectMsgStartTime
SystemObjectStartTime
SystemObjectStatStartTime
SystemObjectStatsDataTtl
key
flowtableprotdpver2:
flowtableprotdpver2
column1
column2
column3
column4
key
key2
key3
value
statstablebystru64tagv3
column1
column2
column3
column4
key
key2
key3
key4
key5
key6
value
objecttable
column1
column2
key
key2
key3
value
statstablebyu64u64tagv3
column1
column2
column3
column4
key
key2
key3
key4
key5
key6
value
flowtabledvndipver2
column1
column2
column3
column4
key
key2
key3
value
messagetable
Category
Context
IPAddress
InstanceId
Level
MessageTS
Messagetype
ModuleId
Namespace
NodeType
Pid
SequenceNum
Source
Type
VersionSig
Xmlmessage
key
statstablebystrtagv3
column1
column2
column3
key
key2
key3
key4
key5
value
flowtableprotspver2
column1
column2
column3
column4
key
key2
key3
value
flowtablesvnsipver2
column1
column2
column3
column4
key
key2
key3
value
messagetablemoduleid
column1
column2
column3
key
key2
