Markdown on WordPress??

WordPress でも(数年前から)Mark Down形式を使えるようになってたので早速試してみる。

が、しかし、Jetpack Pluginを有効にしてもカスタム投稿?になってるせいかそのままでは動作せず。orz
==> Jetpack Plugin の MD を無効にして、代わりに “WP Markdown Editor” plug-in を導入したら表示できました!

ハッシュ1つ

ハッシュ2つ

ハッシュ3つ

ハッシュ4つ

int main() {
  printf "inside code\n";
  return 0;
}
  • List
  • List

Install Lagopus on VirtualBox Lagopus0.1.2 + CentOS6.6 + DPDK2.0.0

Here’s complete list of commands required to install Lagopus on VirtualBox using Lagopus0.1.2 + CentOS6.6 + DPDK2.0.0.

# Ubuntu14.04 version is here.
# Created / Tested on 2015/07/10

VirtualBox setup

Create VM on Oracle VirtualBox.

  • Memory: 4096MB
  • CPU: 2 CPUs
  • DISK: 8GB
  • Network for admin/ssh.
    # You can ssh 127.0.0.1:3022 to the VM with this port forwarding rule.
    Adapter#1 :
    – NAT
    – port forwarding: Protocol=TCP|Host Port=3022|Guest Port=22
    – Promiscas mode = reject (拒否)
  • Network for DPDK.
    # Might be easier to add this AFTER installing OS.
    Adapter#2 & 3 :
    – Host-Only
    – Adapter Type= Intel PRO/100 MT Server (82545EM)
    – Promiscas mode = allow all (すべて許可)

# Actually NIC will also work using “Internal network”, “Intel PPRO/1000 MT Desktop (82540EM)”.

Install Linux (CentOS6.6)

CentOS-6.6-x86_64-bin-DVD1.iso
CentOS-6.6-x86_64-bin-DVD2.iso

Install packages (yum):

Python are mainly for Ryu, so might not be required if you only want Lagopus/DPDK.

$ sudo su
# yum update
# yum groupinstall 'Development Tools'
# yum install kernel-devel kernel-headers
# yum install python-devel
Enable EPEL Repository
wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
# yum repolist # confirm repolist
# yum install python-pip python-eventlet python-routes python-webob python-paramiko \
     python-pip python-pastedeploy python-paste python-twisted python-virtualenv
# pip install -U netaddr six pbr
--
# yum install ncurses-devel 
# yum install libpcap-devel libnet
# yum install expat-devel
# yum install openssl-devel
# yum install gmp gmp-devel

Configure Hugepages:

Configure Hugepages for DPDK.
If you increase your port, you might need more Memory/Pages.

$ sudo vi /boot/grub/grub.conf
     kernel /vmlinuz-2.6.32-504.23.4.el6.x86_64 ... hugepages=256
$ sudo mkdir /mnt/huge
$ sudo vi /etc/fstab
     nodev /mnt/huge hugetlbfs defaults 0 0
$ sudo reboot

Don’t forget to check hugepages are alocated after reboot.

$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
HugePages_Total: 256
HugePages_Free: 256
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

Install DPDK:

Download DPDK source code and make config / make install.

$ wget http://dpdk.org/browse/dpdk/snapshot/dpdk-2.0.0.tar.gz
$ unzip dpdk-2.0.0.zip
$ tar xvf dpdk-2.0.0.tar.gz
Note: export RTE_SDK=
$ export RTE_SDK=/home/ebiken/dpdk-2.0.0
$ export RTE_TARGET="x86_64-native-linuxapp-gcc"
$ make config T=${RTE_TARGET}
$ make install T=${RTE_TARGET}

Some how “make config” on Lagopus 0.1.2 did not work well when build/include/rte_config.h exists so change name.

$ mv build/include/rte_config.h build/include/rte_config.h.back

Note:
# If using VMware, apply patch to lib/librte_eal/linuxapp/igb_uio/igb_uio.c
# http://dpdk.org/dev/patchwork/patch/945/

Install Lagopus

Download zip file with tag 0.1.2 from https://github.com/lagopus/lagopus/tree/v0.1.2
Lagopus0.1.2-download
You can also use git checkout specifying tag 0.1.2 if you are used to it.
Make sure RTE_SDK is exported if you are using differnt session from installing DPDK.

$ unzip lagopus-0.1.2.zip
$ cd lagopus-0.1.2/
$ ./configure --with-dpdk-dir=${RTE_SDK}
$ make
$ sudo make install

Setup DPDK + Lagopus

Load DPDK kernel modules.

$ sudo modprobe uio
$ sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/igb_uio.ko
$ sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/rte_kni.ko

Confirm kernel modules are loaded.

$ lsmod | grep uio
$ lsmod | grep igb
$ lsmod | grep rte

Check NIC asignment and Unbound NICs from ixgbe driver.

$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' if=eth0 drv=e1000 unused= *Active*
0000:00:08.0 '82540EM Gigabit Ethernet Controller' if=eth1 drv=e1000 unused=
0000:00:09.0 '82540EM Gigabit Ethernet Controller' if=eth2 drv=e1000 unused=

Other network devices
=====================

$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py -b igb_uio 0000:00:08.0 0000:00:09.0
$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:09.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' if=eth0 drv=e1000 unused=igb_uio *Active*

Other network devices
=====================

Setup Lagopus configuration file.
You don’t need to modify default configuration file if you are using 2 NICs.

$ sudo cp ~/lagopus-0.1.2/samples/lagopus.conf /usr/local/etc/lagopus/
$ sudo vi /usr/local/etc/lagopus/lagopus.conf
interface {
    ethernet {
        eth0;
        eth1;
    }
}
bridge-domains {
    br0 {
        port {
            eth0;
            eth1;
        }
        controller {
            127.0.0.1;
        }
    }
}

Simple run

$ sudo lagopus -d -- -c3 -n1 -- -p3
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
... skip ...
Logical core 1 (io-worker 0) main loop.
Adding Physical Port 0:08:00:27:d7:e2:45
Adding Physical Port 1:08:00:27:19:b6:e9
Assigning port id 0 to bridge br0
Assigning port id 1 to bridge br0

FYI: Ryu install

Get Ryu: http://osrg.github.io/ryu/
Using pip:

$ sudo pip install ryu

Confirm ryu is installed.

$ ryu-manager --version
ryu-manager 3.23.2
$ find / -name simple_switch.py 2>/dev/null
/usr/local/lib/python2.7/dist-packages/ryu/app/simple_switch.py
$ ryu-manager --verbose /usr/local/lib/python2.7/dist-packages/ryu/app/simple_switch_13.py

Install Lagopus on VirtualBox: Lagopus0.1.2 + Ubuntu14.04 + DPDK2.0.0

Here’s complete list of commands required to install Lagopus on VirtualBox using Lagopus0.1.2 + Ubuntu14.04 + DPDK2.0.0.

# CentOS6.6 version is here.
# Created / Tested on 2015/07/10

VirtualBox setup

Create VM on Oracle VirtualBox.

  • Memory: 4096MB
  • CPU: 2 CPUs
  • DISK: 8GB
  • Network for admin/ssh.
    # You can ssh 127.0.0.1:3022 to the VM with this port forwarding rule.
    Adapter#1 :
    – NAT
    – port forwarding: Protocol=TCP|Host Port=3022|Guest Port=22
    – Promiscas mode = reject (拒否)
  • Network for DPDK.
    # Might be easier to add this AFTER installing OS.
    Adapter#2 & 3 :
    – Host-Only
    – Adapter Type= Intel PRO/100 MT Server (82545EM)
    – Promiscas mode = allow all (すべて許可)

# Actually NIC will also work using “Internal network”, “Intel PPRO/1000 MT Desktop (82540EM)”.

Install Linux (Ubuntu14.04)

ubuntu-14.04-server-amd64.iso
ubuntu-14.04.2-server-amd64.iso (14.04-2 also works)

Install packages (apt-get):

Python are mainly for Ryu, so might not be required if you only want Lagopus/DPDK.

# Some packages might require additional repo depending on how you installed OS.
# Generate apt repo list and replace /etc/apt/source.list if required.
# Maybe this site would help you. http://repogen.simplylinux.ch/

$ sudo apt-get update
$ sudo apt-get install linux-headers-`uname -r`
$ sudo apt-get install git libexpat-dev libgmp-dev libncurses-dev \
build-essential unzip make coreutils gcc binutils \
libssl-dev libpcap-dev byacc flex libreadline-dev
$ sudo apt-get install python-eventlet python-routes python-webob python-paramiko python-pip \
python-pastedeploy python-paste python-twisted python-virtualenv python-dev
$ sudo pip install -U netaddr six pbr

Configure Hugepages:

Configure Hugepages for DPDK.
If you increase your port, you might need more Memory/Pages.

$ sudo vi /etc/default/grub
GRUB_CMDLINE_LINUX="hugepages=256"
# 2M*256 = 512M
$ sudo update-grub
$ sudo mkdir /mnt/huge
$ sudo vi /etc/fstab
nodev /mnt/huge hugetlbfs defaults 0 0
$ sudo reboot

Don’t forget to check hugepages are alocated after reboot.

$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
HugePages_Total: 256
HugePages_Free: 256
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

Install DPDK:

Download DPDK source code and make config / make install.

$ wget http://dpdk.org/browse/dpdk/snapshot/dpdk-2.0.0.tar.gz
$ unzip dpdk-2.0.0.zip
$ tar xvf dpdk-2.0.0.tar.gz
Note: export RTE_SDK=
$ export RTE_SDK=/home/ebiken/dpdk-2.0.0
$ export RTE_TARGET="x86_64-native-linuxapp-gcc"
$ make config T=${RTE_TARGET}
$ make install T=${RTE_TARGET}

Some how “make config” on Lagopus 0.1.2 did not work well when build/include/rte_config.h exists so change name.

$ mv build/include/rte_config.h build/include/rte_config.h.back

Note:
# If using VMware, apply patch to lib/librte_eal/linuxapp/igb_uio/igb_uio.c
# http://dpdk.org/dev/patchwork/patch/945/

Install Lagopus

Download zip file with tag 0.1.2 from https://github.com/lagopus/lagopus/tree/v0.1.2
Lagopus0.1.2-download
You can also use git checkout specifying tag 0.1.2 if you are used to it.
Make sure RTE_SDK is exported if you are using differnt session from installing DPDK.

$ unzip lagopus-0.1.2.zip
$ cd lagopus-0.1.2/
$ ./configure --with-dpdk-dir=${RTE_SDK}
$ make
$ sudo make install

Setup DPDK + Lagopus

Load DPDK kernel modules.

$ sudo modprobe uio
$ sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/igb_uio.ko
$ sudo insmod ${RTE_SDK}/${RTE_TARGET}/kmod/rte_kni.ko

Confirm kernel modules are loaded.

$ lsmod | grep uio
$ lsmod | grep igb
$ lsmod | grep rte

Check NIC asignment and Unbound NICs from ixgbe driver.

$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' if=eth0 drv=e1000 unused= *Active*
0000:00:08.0 '82540EM Gigabit Ethernet Controller' if=eth1 drv=e1000 unused=
0000:00:09.0 '82540EM Gigabit Ethernet Controller' if=eth2 drv=e1000 unused=

Other network devices
=====================

$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py -b igb_uio 0000:00:08.0 0000:00:09.0
$ sudo ${RTE_SDK}/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=
0000:00:09.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:00:03.0 '82540EM Gigabit Ethernet Controller' if=eth0 drv=e1000 unused=igb_uio *Active*

Other network devices
=====================

Setup Lagopus configuration file.
You don’t need to modify default configuration file if you are using 2 NICs.

$ sudo cp ~/lagopus-0.1.2/samples/lagopus.conf /usr/local/etc/lagopus/
$ sudo vi /usr/local/etc/lagopus/lagopus.conf
interface {
    ethernet {
        eth0;
        eth1;
    }
}
bridge-domains {
    br0 {
        port {
            eth0;
            eth1;
        }
        controller {
            127.0.0.1;
        }
    }
}

Simple run

$ sudo lagopus -d -- -c3 -n1 -- -p3
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 2 lcore(s)
... skip ...
Logical core 1 (io-worker 0) main loop.
Adding Physical Port 0:08:00:27:d7:e2:45
Adding Physical Port 1:08:00:27:19:b6:e9
Assigning port id 0 to bridge br0
Assigning port id 1 to bridge br0

FYI: Ryu install

Get Ryu: http://osrg.github.io/ryu/
Using pip:

$ sudo pip install ryu

Confirm ryu is installed.

$ ryu-manager --version
ryu-manager 3.23.2
$ find / -name simple_switch.py 2>/dev/null
/usr/local/lib/python2.7/dist-packages/ryu/app/simple_switch.py
$ ryu-manager --verbose /usr/local/lib/python2.7/dist-packages/ryu/app/simple_switch_13.py

OVS controller connection and fail_mode (secure / stand alone)

OVS controller connection 切れた時の動作を Source Code で確認。
意外とシンプルだったので将来の為にメモ。

Read OVS code related to bridge operation when connection to controller were disconnected.
It’s was small piece of code than expected, so noting them here for future use.

Commands:

Set fail_mode. secure or standalone.
# ovs-vsctl set bridge br0 fail_mode=secure

Confirm currenct configuration:
# ovs-vsctl list bridge

Documents:

File: PORTING on OVS2.3.2

256 The default "fail-mode" for Open vSwitch bridges is "standalone",
257 meaning that, when the OpenFlow controllers cannot be contacted, Open
258 vSwitch acts as a regular MAC-learning switch. This works well in
259 virtualization environments where there is normally just one uplink
260 (either a single physical interface or a bond). In a more general
261 environment, it can create loops. So, if you are porting to a
262 general-purpose switch platform, you should consider changing the
263 default "fail-mode" to "secure", which does not behave this way. See
264 documentation for the "fail-mode" column in the Bridge table in
265 ovs-vswitchd.conf.db(5) for more information.

man ovs-vswitchd.conf.db(5)

fail_mode: optional string, either secure or standalone
...
The standalone mode can create forwarding loops on a bridge that has more than one uplink port unless STP is enabled. To avoid loops on such a bridge, configure secure mode or enable STP (see stp_enable).
When more than one controller is configured, fail_mode is considered only when none of the configured controllers can be contacted.
Changing fail_mode when no primary controllers are configured clears the flow table.

Code:

File: ofproto/ofproto.h

126 /* How the switch should act if the controller cannot be contacted. */
127 enum ofproto_fail_mode {
128 OFPROTO_FAIL_SECURE, /* Preserve flow table. */
129 OFPROTO_FAIL_STANDALONE /* Act as a standalone switch. */
130 };

File: ofproto/connmgr.c

## struct connmgr definition:
196 /* Connection manager for an OpenFlow switch. */
197 struct connmgr {
...
214 /* Fail open. */
215 struct fail_open *fail_open;
216 enum ofproto_fail_mode fail_mode;
## Default setting: OFPROTO_FAIL_SECURE
235 connmgr_create(struct ofproto *ofproto,
...
254 mgr->fail_open = NULL;
255 mgr->fail_mode = OFPROTO_FAIL_SECURE;
## Once controller was disconnected, below function is called and flow entry is added depending on fail_mode.
1979 /* Called by 'ofproto' after all flows have been flushed, to allow fail-open
1980 * and standalone mode to re-create their flows.
1981 *
1982 * In-band control has more sophisticated code that manages flows itself. */
1983 void
1984 connmgr_flushed(struct connmgr *mgr)
1985 OVS_EXCLUDED(ofproto_mutex)
1986 {
1987 if (mgr->fail_open) {
1988 fail_open_flushed(mgr->fail_open);
1989 }
1990
1991 /* If there are no controllers and we're in standalone mode, set up a flow
1992 * that matches every packet and directs them to OFPP_NORMAL (which goes to
1993 * us). Otherwise, the switch is in secure mode and we won't pass any
1994 * traffic until a controller has been defined and it tells us to do so. */
1995 if (!connmgr_has_controllers(mgr)
1996 && mgr->fail_mode == OFPROTO_FAIL_STANDALONE) {
1997 struct ofpbuf ofpacts;
1998 struct match match;
1999
## Add flow with "Match ALL" and "Output: OFPP_NORMAL" (== MAC learning bridge)
2000 ofpbuf_init(&ofpacts, OFPACT_OUTPUT_SIZE);
2001 ofpact_put_OUTPUT(&ofpacts)->port = OFPP_NORMAL;
2002 ofpact_pad(&ofpacts);
2003
2004 match_init_catchall(&match);
2005 ofproto_add_flow(mgr->ofproto, &match, 0, ofpbuf_data(&ofpacts),
2006 ofpbuf_size(&ofpacts));
2007
2008 ofpbuf_uninit(&ofpacts);
2009 }
2010 }

How To Install OVS (Kernel Module) from Source Code

OpenVSwitch (OVS) を Source Code からインストールした際にハマったのでメモ。
ハマったポイントは make modules_install して modprobe しても既存の Kernel Module を掴んでしまい、コンパイルしたのを使ってくれなかったところ。
make modules_install する前に既存の openvswitch.ko を消しておく必要がありました。

Version: OVS2.3.0

Summary:

0. download source code

$ wget http://openvswitch.org/releases/openvswitch-2.3.0.tar.gz
$ tar xvf openvswitch-2.3.0.tar.gz

0. Install required packages mentioned in INSTALL

$ sudo apt-get install autoconf
$ sudo apt-get install libtool
cd ~/openvswitch-2.3.0/

1. ./boot.sh
2. ./configure –with-linux=/lib/modules/`uname -r`/build
3. make
4. sudo make install

# Backup / Remove existing openvswitch.ko Kernel Module.
# /lib/modules//kernel/net/openvswitch/openvswitch.ko

5. sudo make modules_install
6. sudo modprobe openvswitch

# lsmod | grep openvswitch

Why you need to remove existing Kernel Module before “make modules_install” ?
既存の Kernel Module を削除する理由

“make modules_install” すると “Can’t read private key” と表示されるが、これは Sign されてないという warning で実際は /lib/modules/3.13.0-32-generic/extra/ に copy (install) されている。
When you “make modules_install”, message like “Can’t read private key” will be shown. This is warning message that the Module is not signed. It would actually copied (installed) to /lib/modules/3.13.0-32-generic/extra/.

“modprobe openvswitch” した場合:
Now, when you run “modprobe openvswitch” :

1. /lib/modules/3.13.0-32-generic/kernel/net/openvswitch/openvswitch.ko
があればこれを使う。
will be used if it exists.

2. “1.” が無ければ extra/openvswitch.ko を使う。
If not, extra/openvswitch.ko will be used.

自分でコンパイルした extra/openvswitch.ko を使用するためには、パッケージ( pre-installed ) な kernel/net/openvswitch/openvswitch.ko を消す必要がある。
Thus, you need to remove exsiting “kernel/net/openvswitch/openvswitch.ko” to load extra/openvswitch.ko which you compiled from source code.

Installation Log

1. Run boot.sh (autoreconf)

ebiken@ovs01:~/openvswitch-2.3.0$ pwd
/home/ebiken/openvswitch-2.3.0
ebiken@ovs01:~/openvswitch-2.3.0$ ./boot.sh
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-aux'.
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'

2. Configure package by running ./configure –with-linux=/lib/modules/`uname -r`/build
Default directory is /usr/local, while ubuntu general app is /usr.
Check INSTALL if you want to change it inline with ubuntu general App.
# Check for Ubuntu specific info : https://wiki.linaro.org/LNG/Engineering/OVSOnUbuntu

“–with-linux=/lib/modules/`uname -r`/build” is required to build Linux kernel module.

ebiken@ovs01:~/openvswitch-2.3.0$ ./configure --with-linux=/lib/modules/`uname -r`/build

# Note: if you want to use SSL to connect to controler, you need openssl available at this point.

3. Run GNU make in the build directory

ebiken@ovs01:~/openvswitch-2.3.0$ make

4. Run “make install” to install the executables and manpages into the
running system, by default under /usr/local.

# This will also create an empty directory /usr/local/etc/openvswitch/
# Libraries have been installed in: /usr/local/lib

ebiken@ovs01:~/openvswitch-2.3.0$ sudo make install

5. If you built kernel modules, you may install and load them, e.g.:
# first, remove existing kernel module.

$ sudo cp /lib/modules/3.13.0-32-generic/kernel/net/openvswitch/openvswitch.ko
$ sudo rm /lib/modules/3.13.0-32-generic/kernel/net/openvswitch/openvswitch.ko

ebiken@ovs01:~/openvswitch-2.3.0$ sudo make modules_install
ebiken@ovs01:~/openvswitch-2.3.0$ lsmod | grep bridge
ebiken@ovs01:~/openvswitch-2.3.0$ lsmod | grep openvswitch

6. Load Kernel Module

ebiken@ovs01:~/openvswitch-2.3.0$ sudo modprobe openvswitch
ebiken@ovs01:~/openvswitch-2.3.0$ lsmod | grep open
openvswitch 66901 0
gre 13796 1 openvswitch
vxlan 37619 1 openvswitch
libcrc32c 12644 1 openvswitch

ebiken@ovs01:~/openvswitch-2.3.0$ modinfo ./datapath/linux/openvswitch.ko
filename: /home/ebiken/openvswitch-2.3.0/./datapath/linux/openvswitch.ko
version: 2.3.0
license: GPL
description: Open vSwitch switching datapath
srcversion: EC4BFE790C33F314F3B4D09
depends: libcrc32c,vxlan,gre
vermagic: 3.13.0-32-generic SMP mod_unload modversions

ebiken@ovs01:~/openvswitch-2.3.0$ modprobe -D openvswitch
insmod /lib/modules/3.13.0-32-generic/kernel/lib/libcrc32c.ko
insmod /lib/modules/3.13.0-32-generic/kernel/net/ipv4/ip_tunnel.ko
insmod /lib/modules/3.13.0-32-generic/kernel/drivers/net/vxlan.ko
insmod /lib/modules/3.13.0-32-generic/kernel/net/ipv4/gre.ko
insmod /lib/modules/3.13.0-32-generic/extra/openvswitch.ko
ebiken@ovs01:~/openvswitch-2.3.0$ modinfo /lib/modules/3.13.0-32-generic/extra/openvswitch.ko
filename: /lib/modules/3.13.0-32-generic/extra/openvswitch.ko
version: 2.3.0
license: GPL
description: Open vSwitch switching datapath
srcversion: EC4BFE790C33F314F3B4D09
depends: libcrc32c,vxlan,gre
vermagic: 3.13.0-32-generic SMP mod_unload modversions

7. Initialize the configuration database using ovsdb-tool, e.g.:
## only if required: sudo mkdir -p /usr/local/etc/openvswitch

$ sudo ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
ebiken@ovs01:~/openvswitch-2.3.0$ ls /usr/local/etc/openvswitch
conf.db

How to show diff of two Linux releases. (git tags)

2つの Linux リリース(バージョン)間の差分を表示させる方法
# 単に git tags 間の差分を表示させてるだけ。

I wanted to find out what was changed in “linux/net/bridge” between two releases, Linux 3.19 and 4.0-rc2.
Here’s log as an reference.

Notes:

* version / release are used interchangeable
* git command reference: http://stackoverflow.com/questions/3211809/git-diff-between-given-two-tags

Index:

1. Clone Linux git tree to local.

$ git clone https://git.kernel.org/cgit/linux/kernel/git/davem/net.git/ net-davem

2. Move to directory created by git clone.

$ cd net-davem/

3. show stats of changes

$ git diff v3.19 v4.0-rc2 --stat net/bridge/

4. show all commits between two versions (tags)

$ git log v3.19..v4.0-rc2 net/bridge/ > git-log_v.3.19_v4.0-rc2.log

5. Count number of commits.

$ grep "^commit" git-log_v.3.19_v4.0-rc2.log | wc -l

6. Check source code diff.

$ git diff v3.19 v4.0-rc2 --stat net/bridge/ > git-diff-source-code.
$ head -n 20 git-diff-source-code.c

7. Check source code diff of a specific file.

$ git diff v3.19 v4.0-rc2  -- net/bridge/br_if.c

Logs:

1. Clone Linux git tree to local.

$ git clone https://git.kernel.org/cgit/linux/kernel/git/davem/net.git/ net-davem

2. Move to directory created by git clone.

$ cd net-davem/

3. show stats of changes

git diff   --stat 
$ git diff v3.19 v4.0-rc2 --stat net/bridge/
 net/bridge/br.c                 |   52 +++++++-
 net/bridge/br_fdb.c             |   60 +++------
 net/bridge/br_if.c              |   11 ++-
 net/bridge/br_mdb.c             |    5 +-
 net/bridge/br_netfilter.c       |   19 +--
 net/bridge/br_netlink.c         |  300 ++++++++++++++++++++++++++++++---------
 net/bridge/br_private.h         |   12 +-
 net/bridge/br_vlan.c            |    4 +-
 net/bridge/netfilter/ebt_vlan.c |    4 +-
 net/bridge/netfilter/ebtables.c |    2 +-
 10 files changed, 331 insertions(+), 138 deletions(-)

4. show all commits between two versions (tags)

$ git log v3.19..v4.0-rc2 net/bridge/ > git-log_v.3.19_v4.0-rc2.log
[ebiroot@vps diff_3.0_4.0-rc2]$ head -n 20 git-log_v.3.19_v4.0-rc2.log
commit 9672723973f1e70189b8409eb2da189a980481c5
Author: Geert Uytterhoeven 
Date:   Thu Feb 12 15:17:27 2015 +0100

    bridge: netfilter: Move sysctl-specific error code inside #ifdef

    If CONFIG_SYSCTL=n:

        net/bridge/br_netfilter.c: In function ‘br_netfilter_init’:
        net/bridge/br_netfilter.c:996: warning: label ‘err1’ defined but not used

    Move the label and the code after it inside the existing #ifdef to get
    rid of the warning.

    Signed-off-by: Geert Uytterhoeven 
    Signed-off-by: David S. Miller 

commit 2573beec56aa28a0e6d4430fb6796d0c76308bcf
Merge: fd3137c 531c94a
Author: David S. Miller 

5. Count number of commits.

$ grep "^commit" git-log_v.3.19_v4.0-rc2.log | wc -l
23
/* 2 out of 23 are reverting commit and commit which was reverted. */
/* Total 21 commits between 3.19 and 4.0-rc2 */

6. Check source code diff.

$ git diff v3.19 v4.0-rc2 --stat net/bridge/ > git-diff-source-code.c
$ head -n 20 git-diff-source-code.c
diff --git a/net/bridge/br.c b/net/bridge/br.c
index 44425af..fb57ab6 100644
--- a/net/bridge/br.c
+++ b/net/bridge/br.c
@@ -19,6 +19,7 @@
 #include 
 #include 
 #include 
+#include 

 #include "br_private.h"

@@ -120,6 +121,48 @@ static struct notifier_block br_device_notifier = {
        .notifier_call = br_device_event
 };

+static int br_netdev_switch_event(struct notifier_block *unused,
+                                 unsigned long event, void *ptr)
+{
+       struct net_device *dev = netdev_switch_notifier_info_to_dev(ptr);

7. Check source code diff of a specific file.

$ git diff v3.19 v4.0-rc2  -- net/bridge/br_if.c
diff --git a/net/bridge/br_if.c b/net/bridge/br_if.c
index ed307db..b087d27 100644
--- a/net/bridge/br_if.c
+++ b/net/bridge/br_if.c
@@ -424,6 +424,7 @@ netdev_features_t br_features_recompute(struct net_bridge *br,
                features = netdev_increment_features(features,
                                                     p->dev->features, mask);
        }
+       features = netdev_add_tso_features(features, mask);

        return features;
 }
@@ -435,10 +436,16 @@ int br_add_if(struct net_bridge *br, struct net_device *dev)
        int err = 0;
        bool changed_addr;

-       /* Don't allow bridging non-ethernet like devices */
+       /* Don't allow bridging non-ethernet like devices, or DSA-enabled
+        * master network devices since the bridge layer rx_handler prevents
+        * the DSA fake ethertype handler to be invoked, so we do not strip off
+        * the DSA switch tag protocol header and the bridge layer just return
+        * RX_HANDLER_CONSUMED, stopping RX processing for these frames.
+        */
        if ((dev->flags & IFF_LOOPBACK) ||
            dev->type != ARPHRD_ETHER || dev->addr_len != ETH_ALEN ||
-           !is_valid_ether_addr(dev->dev_addr))
+           !is_valid_ether_addr(dev->dev_addr) ||
+           netdev_uses_dsa(dev))
                return -EINVAL;

        /* No bridging of bridges */

How to find PCIe Device information of Ethernet NIC (Linux)

ネットワークインターフェースの PCIe Device 詳細情報インターフェース名からたどる方法のメモ書き。
Memo of how you can find PCIe Device info of a network interface on Linux.
# Tested on Ubuntu 14.04 running on VMware Player.

eth0 の情報を知りたい場合。
Say you want to find info about eth0…

1. Find PCIe bus-info using ethtool.

ebiken@ovs01:~$ ethtool -i eth0
driver: e1000
version: 7.3.21-k8-NAPI
firmware-version:
bus-info: 0000:02:01.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

2. To find VendorID:DeviceID “lspci -nn -s

ebiken@ovs01:~$ sudo lspci -nn -s 0000:02:01
02:01.0 Ethernet controller [0200]: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) [8086:100f] (rev 01)

VendorID: 8086 (Intel)
DeviceID: 100f

3. To show details “lspci -vv -nn -s
# Including Subsystem VendorID:DeviceID

ebiken@ovs01:~$ lspci -vv -nn -s 0000:02:01.0
02:01.0 Ethernet controller [0200]: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) [8086:100f] (rev 01)
Subsystem: VMware PRO/1000 MT Single Port Adapter [15ad:0750]
Physical Slot: 33
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR-
Kernel driver in use: e1000

3. To show Capability info, become root (sudo).

ebiken@ovs01:~$ sudo lspci -vv -nn -s 0000:02:01.0
02:01.0 Ethernet controller [0200]: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) [8086:100f] (rev 01)
Subsystem: VMware PRO/1000 MT Single Port Adapter [15ad:0750]
Physical Slot: 33
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR-

4. To show which driver owns which pcie device.


[ebiroot@vps rgx_driver]$ lspci -knn
00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010]
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Kernel driver in use: ata_piix
Kernel modules: ata_generic, pata_acpi, ata_piix
00:01.2 USB controller [0c03]: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] [8086:7020] (rev 01)
Subsystem: Red Hat, Inc QEMU Virtual Machine [1af4:1100]
Kernel driver in use: uhci_hcd
00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 03)
Subsystem: Red Hat, Inc Qemu virtual machine [1af4:1100]
Kernel driver in use: piix4_smbus
Kernel modules: i2c-piix4
00:02.0 VGA compatible controller [0300]: Device [1234:1111]
Subsystem: Red Hat, Inc Device [1af4:1100]
00:03.0 Ethernet controller [0200]: Red Hat, Inc Virtio network device [1af4:1000]
Subsystem: Red Hat, Inc Device [1af4:0001]
Kernel driver in use: virtio-pci
Kernel modules: virtio_pci
00:04.0 SCSI storage controller [0100]: Red Hat, Inc Virtio block device [1af4:1001]
Subsystem: Red Hat, Inc Device [1af4:0002]
Kernel driver in use: virtio-pci
Kernel modules: virtio_pci
00:05.0 RAM memory [0500]: Red Hat, Inc Virtio memory balloon [1af4:1002]
Subsystem: Red Hat, Inc Device [1af4:0005]
Kernel driver in use: virtio-pci
Kernel modules: virtio_pci

Memo: Facebook “6-pack” Blog まとめ。

追記(2015/02/13):図は省略されてるだけで、実際は各LCから40Gx4で全4つのASICへ接続されているのでは、という意見を Twitter: @takanorisuzuki さんから頂きました。また、OCP Dell Switchの内部構成解説した資料などが掲載されたページもご紹介頂きました。
http://www.opencompute.org/wiki/Networking/Workshop-2014-07#Subi_Krisnhamurthy_.28Dell.29:_switch_contribution

Facebook: Introducing “6-pack” : the first open hardware modular switch

Facebookが発表した 6-pack というシャーシ型スイッチのまとめ。
図を読み解くのに時間かかったのでメモ。

結局本当にどんなトラフィックパターンでも non-blocking な理由を理解できずだが、Open Compute Project にコントリビューションされる予定なので、詳細ドキュメントはそのうちOCPにアップされるはずなのでそれ待ち。

シャーシ/カード構成

シャーシは 6U(Card) + 1U(Power) の6スロット構成。
6U = Line Card (0.5U) x 2 x 4 + Fabric Card (1U) x 2
・Line Card は合計8枚。0.5U なので、1行に並んで2枚入る。
・Fabric Card は 1U が合計2枚。
各 Line Card / Fabric Card には Micro Server が搭載されており、これで管理されている。
例えばLine Card は Wedge という16ポートスイッチとして切り出し可能。
(実際は逆でWedgeをベースに6packが作られている)

ASIC

Line Card / Fabric Card 共通のASICを使用。
ASICは 1.28Tbps = 40Gbps x 32port(Channel)
Line Card はこれを 16port 全面 + 16port 後面(Back Planeに接続) として使う。
結果、シャーシ全体で 40Gbps のポートが 128ポート:16 (ports per Line Card) x 8 (Line Cards)

接続

Fabric Card は ASIC 2個搭載。
各Line Card から 8 Channel(40Gbps) づつ1個のASICに接続。
ASIC 毎に 4 Line Card から 8 Channel の接続がある。
各 ASIC へたすき掛けでLine Cardが接続されて non-blocking 接続を実現。

Facebook BLOGから引用: Introducing “6-pack” : the first open hardware modular switch

図を元にASIC/Line Card (LC)の組み合わせを整理。
解らないのは、ASICまたがるトラフィックをどうやって non-blocking で処理しているのか。(例:LC0-LC5)

Fabric#0/ASIC#0 : LC0, LC1, LC2, LC3
Fabric#0/ASIC#1 : LC4, LC5, LC6, LC7
Fabric#1/ASIC#2 : LC0, LC4, LC2, LC6
Fabric#1/ASIC#3 : LC1, LC5, LC3, LC7

管理ポート

各 Line Card / Fabric Card に搭載された管理ポート(GE)間の接続もFabric Cardで実施。
これは通常のトラフィック処理とは別帯域(out of band)で処理している。

Full Route Subnet Length Distribution (from Soft TCAM)

Visualized Internet Full Route Prefix Length distribution.
Data taken on 2014 Q3 to test “Software TCAM”, project by m-asama to implement Software TCAM.

“SoftwareでTCAMを実装する” という挑戦的なプロジェクト “soft_tcam by m-asama”。
半年ぶりに眺めてたら Full Route のサンプルファイルがあったので、Prefix Length の分布をグラフにしてみた。

# Data source: https://github.com/m-asama/soft_tcam

FullRouteLength-01

Actual Numbers … 実際の数字も。

FullRouteLength-02

The Future of Tipping:電子チップによるサービスの変化

The Future of Tipping (Ribbonfarm: by VENKAT on DECEMBER 2, 2014)

BitCoinを初めとした電子通貨/決済によるチップ(Tip)導入によって、サービスが変わるというお話。

ChangeTip は、チャージ(Top Up)たお金を Twitter 経由で支払い&通知がするサービス。
特徴はチップの単位に「あだ名」(monikers)を付けることができ、「1ありがとう=$1.00 として5ありがとうを支払う」といった使い方ができる。

これは、チップの特徴である「社会性・関係性」を良くとらえた仕組みで、このようなチップ支払方法が広まることでチップの流通量はより大きくなり、現金によるチップよりも、電子通貨や決済により意味を加えられたチップは、より親密な関係性を作る事に貢献する。

以上を

・チップとは何か?
the rich tip to bribe,
the middle class tip out of guilt,
the poor tip out of a sense of solidarity.

・なぜ大企業(Starbucksなど)はチップを避けたがるのか?(品質の均質化)
it isn’t the Starbucks app that breaks tipping culture.
It’s the nature of large corporations.

といった観点から論じている。

※ 英語メモ:
・retaliatory = 報復
prevent retaliatory bad service on future visits
将来に訪問したときの報復的な悪いサービスを防ぐ