Author: Tatsuya Naganawa Translator: TF Chinese community

How to build Tungsten Fabric

Most of the documentation for the repo file here is valid.
https://github.com/Juniper/contrail-dev-env--

yum -y install docker git
git clone https://github.com/Juniper/contrail-dev-env
cd contrail-dev-env
. /startup.sh
docker exec -it contrail-developer-sandbox bash

cd /root/contrail-dev-env
yum -y remove python-devel ##Need to resolve dependency issues
make sync
make fetch_packages
make setup
make dep

To build all modules, you can use this command (it takes 1-2 hours, depending on computer performance).

make rpm
make containers

To build more specific modules, you can also use these commands. One thing to note is that rpm-contrail itself is a big data packet and cannot be broken down into more (controller, vrouter, etc. are included).

make list
make rpm-contrail

make list-containers
make container-general-base
make container-base
make container-kubernetes_kube-manager

- those make targets are included fromthis file:/root/contrail/tools/packages/Makefile
 https://github.com/Juniper/contrail-packages/blob/master/Makefile

This command is more useful if you only build vrouter.ko.

build:
cd /root/contrail
scons --opt=production --kernel-dir=/lib/modules/3.10.0-1062.el7.x86_64/build build-kmodule

clean:
cd /root/contrail/vrouter
make KERNELDIR=/lib/modules/3.10.0-1062.el7.x86_64/build clean

Note: When the kernel-devel package of other distributions (I tried the packages of centos 8 and amazon linux 2) is installed, you can also specify it as kernel-dir.
For example, this command creates vrouter.ko for centos 8.2.

It can be loaded manually through the insmod command.

# rpm -ivh --nodeps kernel-devel-4.18.0-147.8.1.el8_1.x86_64.rpm
# scons --opt=production --kernel-dir=/usr/src/kernels/4.18.0-147.8.1.el8_1.x86_64/ build-kmodule

charm install

Tungsten Fabric can also be installed via juju charm.

Using bionic and Openstack Queens, there are 4 nodes (juju node, openstack controller, openstack compute, tunsten-fabric controller)

# apt update
# snap install --classic juju
# juju add-cloud

Select cloud type: manual
Enter a name for your manual cloud: manual-cloud-1
Enter the controller's hostname or IP address: (juju node's ip)

# ssh-keygen
# cd .ssh
# cat id_rsa.pub >> authorized_keys
# cd
# ssh-copy-id(other nodes' ip)

# juju bootstrap manual-cloud-1

# git clone https://github.com/Juniper/contrail-charms -b R5

# juju add-machine ssh:root@(openstack-controller ip)
# juju add-machine ssh:root@(openstack-compute ip)
# juju add-machine ssh:root@(TungstenFabric-controller ip)

# vi set-juju.sh
juju deploy ntp
juju deploy rabbitmq-server --to lxd:0
juju deploy percona-cluster mysql --config root-password=contrail123 --config max-connections=1500--to lxd:0
juju deploy openstack-dashboard --to lxd:0
juju deploy nova-cloud-controller --config console-access-protocol=novnc --config network-manager=Neutron --to lxd:0
juju deploy neutron-api --config manage-neutron-plugin-legacy-mode=false--config neutron-security-groups=true--to lxd:0
juju deploy glance --to lxd:0
juju deploy keystone --config admin-password=contrail123 --config admin-role=admin --to lxd:0

juju deploy nova-compute --config ./nova-compute-config.yaml --to 1

CHARMS_DIRECTORY=/root
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-keystone-auth --to 2
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-controller --config auth-mode=rbac --config cassandra-minimum-diskgb=4--config cassandra-jvm-extra-opts="-Xms1g -Xmx2g"--to 2
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-analyticsdb --config cassandra-minimum-diskgb=4--config cassandra-jvm-extra-opts="-Xms1g -Xmx2g"--to 2
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-analytics --to 2
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-openstack
juju deploy $CHARMS_DIRECTORY/contrail-charms/contrail-agent

juju expose openstack-dashboard
juju expose nova-cloud-controller
juju expose neutron-api
juju expose glance
juju expose keystone

juju expose contrail-controller
juju expose contrail-analytics

juju add-relation keystone:shared-db mysql:shared-db
juju add-relation glance:shared-db mysql:shared-db
juju add-relation keystone:identity-service glance:identity-service
juju add-relation nova-cloud-controller:image-service glance:image-service
juju add-relation nova-cloud-controller:identity-service keystone:identity-service
juju add-relation nova-cloud-controller:cloud-compute nova-compute:cloud-compute
juju add-relation nova-compute:image-service glance:image-service
juju add-relation nova-compute:amqp rabbitmq-server:amqp
juju add-relation nova-cloud-controller:shared-db mysql:shared-db
juju add-relation nova-cloud-controller:amqp rabbitmq-server:amqp
juju add-relation openstack-dashboard:identity-service keystone

juju add-relation neutron-api:shared-db mysql:shared-db
juju add-relation neutron-api:neutron-api nova-cloud-controller:neutron-api
juju add-relation neutron-api:identity-service keystone:identity-service
juju add-relation neutron-api:amqp rabbitmq-server:amqp

juju add-relation contrail-controller ntp
juju add-relation nova-compute:juju-info ntp:juju-info

juju add-relation contrail-controller contrail-keystone-auth
juju add-relation contrail-keystone-auth keystone
juju add-relation contrail-controller contrail-analytics
juju add-relation contrail-controller contrail-analyticsdb
juju add-relation contrail-analytics contrail-analyticsdb

juju add-relation contrail-openstack neutron-api
juju add-relation contrail-openstack nova-compute
juju add-relation contrail-openstack contrail-controller

juju add-relation contrail-agent:juju-info nova-compute:juju-info
juju add-relation contrail-agent contrail-controller

# vi nova-compute-config.yaml 
nova-compute:
 virt-type: qemu 
 enable-resize: True
 enable-live-migration: True
 migration-auth-type: ssh

# bash set-juju.sh(to check status, it takes 20 minutes for every application to be active)
# juju status
# tail -f /var/log/juju/*log | grep -v -w DEBUG

In order to make it run successfully, there are two points to note.

Since juju uses LXD and its own subnet internally, at least Tungsten Fabric nodes need to have some static routes to this subnet (if via AWS, you can use the VPC routing table, and you need to disable source/destination checks)
Since LXD does not allow docker to run by default, it needs to be set to allow through lxc config.

juju ssh 0
 sudo su -
 lxc list
 lxc config set juju-cb8047-0-lxd-4 security.nesting true
 lxc config show juju-cb8047-0-lxd-4
 lxc restart juju-cb8047-0-lxd-4

Deployment of Tungsten fabric on public cloud

gatewayless and snat
When installed on a public cloud, since there is no available hardware that supports MPLS over IP or VXLAN, vRouter needs to have a floating IP from the underlay IP.

Having said that, since Tungsten Fabric supports the gatewayless function, it will not be too difficult to provide services to floating IP from this virtual network (the way is to connect another IP to ENI and make it the source of floating IP, so that External access to services on vRouter)

Note: When using kubernetes, I personally prefer to set the service network to gatewayless (this setting will not use external IP). If a hypervisor with bare metal instances is used, floating IPs with some gatewayless subnets are preferred.

From vRouter to external network, distributed SNAT function can solve the problem.

SNAT with floating IP can also work well

AZ High Availability WIP
In addition, you can also define two separate [Load Balancing] (https://cloud.tencent.com/product/clb?from=10680) devices on vRouters to access the same application, so that it can be accessed from two different availability zones to ensure higher availability.

To make this setting, you need to configure several things:

vRouter has a gatewayless subnet whose subnet range is not included in the VPC subnet.
Configure an instance route in the routing table, which forwards the gatewayless subnet to one of the vRouter nodes.
ELB will be set as the IP address of gatewayless IP. (When specifying IP, you need to configure "Other IP Address" for "Subnet")
In this case, the security group (security-group) will not automatically allow the address of the ELB, so you need to manually allow the CIDR of the VPC to perform the health check of the ELB to make it work properly.

One limitation of the gatewayless function of vRouter is that only when the destination vRouter and the vRouter that originally received the packet are placed in the same L2 subnet, can the packet be forwarded to other vRouters.

If it is in a different subnet, the vRouter will forward the packet to VROUTER_GATEWAY, which in turn will send it to the vRouter, which will form a loop.

Since the AWS subnet cannot contain the same subnet, to make this setup AZ highly available, two load balancers need to be configured for the same application, each AZ has two different gatewayless subnets.

Since ELB can forward data packets to two vRouter load balancers, it can achieve AZ high availability with the help of ELB.

EKS integration

vRouter CNI AWS EKS is another possible integration solution.

https://www.youtube.com/channel/UCXUny7HKBdyakn3-UOdkhsw
To perform this setting, first configure some IAM users for EKS through the web console.
Since kubectl can be used with the IAM user who created the kubernetes cluster, root access to the web console is not recommended.
Then, the following command will delete the VPC CNI from each worker node.

( laptop)
# kubectl delete ds -n kube-system aws-node(EKS worker node)
# mv -i /etc/cni/net.d/10-aws.conflist /tmp/

Next, you can use the same steps as the URL below to install vRouter CNI.

https://github.com/tnaganawa/tungstenfabric-docs/blob/master/TungstenFabricPrimer.md#kubeadm
Note: Although by default, vrouter.ko in the dockerhub container cannot be loaded into the amazon linux 2 kernel, you can use this step to create vrouter.ko for the kernel.
https://github.com/tnaganawa/tungstenfabric-docs/blob/master/TungstenFabricKnowledgeBase.md#how-to-build-tungsten-fabric

CNI MTU setting

It is worth noting that when vRouter is installed in a public cloud instance, some MTU issues may occur.
Changing the physical interface MTU can solve most of the problems, but when the data packet from the container is fragmented, it may be necessary to change the MTU setting of the CNI.

vi /etc/cni/net.d/10-contrail.conf
{" cniVersion":"0.3.1","contrail":{"meta-plugin":"$KUBERNETES_CNI_META_PLUGIN","vrouter-ip":"127.0.0.1","vrouter-port": $VROUTER_PORT,"config-dir":"/var/lib/contrail/ports/vm","poll-timeout":5,"poll-retries":15,+"mtu":1300,"log-file":"$LOG_DIR/cni/opencontrail.log","log-level":"4"},"name":"contrail-k8s-cni","type":"contrail-k8s-cni"}

https://github.com/Juniper/contrail-controller/blob/master/src/container/cni/contrail/cni.go#L33

Original link:
https://github.com/tnaganawa/tungstenfabric-docs/blob/master/TungstenFabricKnowledgeBase.md

Previous selection

[ Tungsten Fabric knowledge base丨vRouter internal operation exploration](https://tungstenfabric.org.cn/topic/104/tungsten-fabric%E7%9F%A5%E8%AF%86%E5%BA%93%E4%B8 %A8vrouter%E5%86%85%E9%83%A8%E8%BF%90%E8%A1%8C%E6%8E%A2%E7%A7%98) Tungsten Fabric Knowledge Base丨More Exploring Inside Components

Tungsten Fabric Getting Started Collection Series Articles——
1.[ First startup and operation guide](https://tungstenfabric.org.cn/topic/54/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8%E4 %B8%A8%E9%A6%96%E6%AC%A1%E5%90%AF%E5%8A%A8%E5%92%8C%E8%BF%90%E8%A1%8C%E6%8C %87%E5%8D%97)
2.[ Seven "weapons" of TF components](https://tungstenfabric.org.cn/topic/56/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85% B8%E4%B8%A8tf%E7%BB%84%E4%BB%B6%E7%9A%84%E4%B8%83%E7%A7%8D-%E6%AD%A6%E5%99%A8 )
3.[ Orchestrator integration](https://tungstenfabric.org.cn/topic/60/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8%E4%B8 %A8%E7%BC%96%E6%8E%92%E5%99%A8%E9%9B%86%E6%88%90)
4.[ Those things about installation (on)](https://tungstenfabric.org.cn/topic/63/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85% B8%E4%B8%A8%E5%85%B3%E4%BA%8E%E5%AE%89%E8%A3%85%E7%9A%84%E9%82%A3%E4%BA%9B% E4%BA%8B-%E4%B8%8A)
5.[ Those things about installation (below)](https://tungstenfabric.org.cn/topic/65/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85% B8%E4%B8%A8%E5%85%B3%E4%BA%8E%E5%AE%89%E8%A3%85%E7%9A%84%E9%82%A3%E4%BA%9B% E4%BA%8B-%E4%B8%8B)
6.[ Integration of mainstream monitoring system tools](https://tungstenfabric.org.cn/topic/69/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8% E4%B8%A8%E4%B8%BB%E6%B5%81%E7%9B%91%E6%8E%A7%E7%B3%BB%E7%BB%9F%E5%B7%A5%E5% 85%B7%E7%9A%84%E9%9B%86%E6%88%90)
7.[ Start the next day's work](https://tungstenfabric.org.cn/topic/71/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8% E4%B8%A8%E5%BC%80%E5%A7%8B%E7%AC%AC%E4%BA%8C%E5%A4%A9%E7%9A%84%E5%B7%A5%E4% BD%9C)
8.8 Typical faults and troubleshooting Tips
9.[ About cluster update)(https://mp.weixin.qq.com/s?__biz=MzI3ODczMjY5MA==&mid=2247484945&idx=1&sn=590839212fb13b77c193f7491bc472b2&chksm=eb53ccecdc2445fa7cfbf77c1430373zhf1445380c6858cadv15380c6s
10.[ Talk about L3VPN and EVPN integration](https://tungstenfabric.org.cn/topic/77/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8% E4%B8%A8%E8%AF%B4%E8%AF%B4l3vpn%E5%8F%8Aevpn%E9%9B%86%E6%88%90)
11.[ About service chain, BGPaaS and others](https://tungstenfabric.org.cn/topic/80/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8 %E4%B8%A8%E5%85%B3%E4%BA%8E%E6%9C%8D%E5%8A%A1%E9%93%BE-bgpaas%E5%8F%8A%E5%85%B6 %E5%AE%83)
12.[ About multi-cluster and multi-data center](https://tungstenfabric.org.cn/topic/81/tungsten-fabric%E5%85%A5%E9%97%A8%E5%AE%9D%E5%85%B8 %E4%B8%A8%E5%85%B3%E4%BA%8E%E5%A4%9A%E9%9B%86%E7%BE%A4%E5%92%8C%E5%A4%9A%E6 %95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83)
13.[ Multi-orchestrator usage and configuration](https://mp.weixin.qq.com/s?__biz=MzI3ODczMjY5MA==&mid=2247485215&idx=1&sn=32aa9d3ce1e8c3400934f1a9fb6788fc&chksm=eb53cde2dc2444f411b2de46fdd1benescene-directive=eb53cde2dc2444f411b2de46fdd1b97453974