-----------------------------------------
Introduction to dpdk-pkt_split_app application
-----------------------------------------

This DPDK application uses two or three ports, two of which are split port and Tap
interface. Third port is PCI/Demo port, which is optional to use.
Split port can be OL (with FMAN micro-code) or ethernet port (without FMAN
micro code).

An OL port is a pair of Frame queues which will be used as Tx and Rx frame
queues. ASK module will send the packets on the Rx FQ using the classification
information provided. The application will send the packets back to ASK module
using Tx FQ.

In case of two ports, packets which are received from split port with the provided
UDP destination port will be sent back on same split port. Source/Destination
IP address and Source/Destination Port will be swapped for these packets. Other
packets will be send to kernel by sending them to TAP interface.
Packet coming from TAP interface will be forwarded back on TAP interface without
any change.


In case of three ports, packets which are received from split port with the
provided UDP destination port will be sent back on PCI port. Source/Destination
IP address and Source/Destination port of the packets will NOT BE swapped. Other
packets will be send to kernel by sending them to TAP interface.
Similarly, Packets coming from PCI port will be forwarded on split port.
Source/Destination IP address and Source/Destination port of the packets will
NOT BE swapped.
Packet coming from TAP interface will be forwarded on PCI interface without any
change.


For two ports and UDP destination port = 2152(GTP packet):

  GTP Packets(Swapped Addresses and ports)         All Packets
            ------                                    ------
           |      |                                  |      |
           |     \|/                                 |     \|/
           ---------         Non GTP Packets         ---------
          |  SPLIT  |------------------------------>|   TAP   |
          |  PORT   |                               |Interface|
           ---------                                 ---------


For three ports and UDP destination port = 2152(GTP packet):

                               All Packets
          ------------------------------------------------------
         |                                                      |
         |                                                      |
        \|/     GTP Packets                                     |
          ------------------------                              |
         |                        |                             |
        \|/                       |                             |
      ---------  All Packets  ---------  Non GTP Packets    ---------
     |PCI/Demo |------------>|  SPLIT  |------------------>|   TAP   |
     |  PORT   |             |   PORT  |                   |Interface|
      ---------               ---------                     ---------



----------------------------
Compilation of dpdk-pkt_split_app
----------------------------

Refer to nxp/README for cross compilation setup.
Section: "Building DPDK: Libraries and Test Applications"

Compilation of dpdk-pkt_split_app application:
  $ meson arm64-build --cross-file config/arm/arm64_dpaa_linux_gcc -Dexamples=pkt_split_app
  $ ninja -C arm64-build install

Location of dpdk-pkt_split_app binary: arm64-build/examples/dpdk-pkt_split_app

-------------------------------------------------------------------
Steps to run dpdk-pkt_split_app on LS1046ARDB when split port is eth port
(Without ASK's FMAN micro code)
-------------------------------------------------------------------

1. fmc -x
2. ifconfig fm1-mac10 <valid IP address>
3. mount -t hugetlbfs hugetlbfs /dev/hugepages
4. echo 512 > /proc/sys/vm/nr_hugepages
5. fmc -c usdpaa_config_ls1046_shared_24g.xml -p usdpaa_policy_24g_classif_frag_gtp_1queue.xml -a
6. Bind PCI device with DPDK driver

   To print the available PCI devices, use:
   $ lspci
   $ dpdk-devbind.py --status

   Run these commands only if you want to use PCI port:
   $ dpdk-devbind.py -u 0001:01:00.0
   $ dpdk-devbind.py --bind=uio_pci_generic 0001:01:00.0
   $ dpdk-devbind.py --status

   Note: If above commands fail because of "python: no such file" error,
         Please try "ln -s /usr/bin/python3 /usr/bin/python"
7. Run the dpdk-pkt_split_app application with three ports(Use PCI port):
   $ ./dpdk-pkt_split_app -c 0xe -n 1 --vdev=net_tap0 -- -p 0xe0 -s 5 -m 8000

   This command will use one eth port, one PCI port and one Tap interface. Tap
   interface's mtu is 8000. Here split port id is 5.
   First port will be PCI port, second port will be eth port and third port will
   be Tap interface.

8. Run the dpdk-pkt_split_app application with two ports(Don't use PCI port):
   $ ./dpdk-pkt_split_app -c 0x6 -n 1 --vdev=net_tap0 -- -p 0x60 -s 5 -m 8000

   This command will use one eth port and one Tap interface. Tap
   interface's mtu is 8000. Here split port id is 5.
   first port will be eth port and second port will be Tap interface.

   Note: number of split port id should not be more than one.

   Note: Please check application logs to identify the ports and their IDs.

   Note: Port sequence: First all DPAA ports, then PCI port(s), if any and tap
	 port at last.

   Note: Please refer to "Steps to increase MTU of kernel interface in VSP mode"
         section to increase the MTU of kernel interface.

-------------------------------------------------------------------
Steps to run dpdk-pkt_split_app on LS1046ARDB when split port is OL port
(With ASK's FMAN micro code)
-------------------------------------------------------------------

Flash the LS1046ARDB board with ASK images and fman microcode.
After booting up the board, run the following commands:

1. Start the ask module:
   $ /etc/init.d/lsdk_ask start

2. Set the eth port UP which will be used as OL port:
   $ ifconfig eth0 1.1.1.1 up

3. Run the DPDK configuration commands:
   $ mkdir -p /dev/hugetlbfs
   $ echo 448 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
   $ mount -t hugetlbfs hugetlbfs /dev/hugetlbfs/
   $ export OLDEV_ENABLED=1

4. Bind PCI device with DPDK driver

   To print the available PCI devices, use:
   $ lspci
   $ dpdk-devbind.py --status

   Run these commands only if you want to use PCI port:
   $ dpdk-devbind.py -u 0001:01:00.0
   $ dpdk-devbind.py --bind=uio_pci_generic 0001:01:00.0
   $ dpdk-devbind.py --status

   Note: If above commands fail because of "python: no such file" error,
         Please try "ln -s /usr/bin/python3 /usr/bin/python"

5. Export environment variable to define backhaul port
   $ export BH_PORT_NAME=<Backhaul port name>

   eg: to use eth0 as backhaul port, run "export BH_PORT_NAME=eth0"

6. Make sure user data comfiguration file exists and have proper
   values. Refer to the "Configure classification information" section for
   details.

NOTE: ARP should be resolved before sending the traffic.

7. Run the dpdk-pkt_split_app application with three ports(Use PCI port):
   $ ./dpdk-pkt_split_app -c 0xe -n 1 --vdev=net_tap0 -- -p 0x7

8. Run the dpdk-pkt_split_app application with two ports(Don't use PCI port):
   $ ./dpdk-pkt_split_app -c 0x6 -n 1 --vdev=net_tap0 -- -p 0x3

   Note: Please check application logs to identify the ports and their IDs.

   Note: Port sequence: First all DPAA ports, then PCI port(s), if any and tap
	 port at last.

------------------------------------
Configure classification information
------------------------------------

Default file named data.input is used to configure the classification and
LGW traffic information for dpdk-pkt_split_app. This file should be present in
the directory from where we are running the application. Optionally, user
can specify the filename by giving "-f" option in application cmdline.
Blank lines and the lines starting with '#' will be ignored.

Classification entries in data.input file:

  1. IPV4_INNER <inner IPv4 address>
     or
     IPV6_INNER <inner IPv6 addresses>

     e.g.:
       "IPV4_INNER 1.1.1.1"
       "IPV6_INNER fe80:0000:0000:0000:0204:9fff:fe06:5112"

  2. IPV4_STATIC <static IPv4 address>
     or
     IPV6_STATIC <static IPv6 addresses>

     e.g.:
       "IPV4_STATIC 172.1.1.1"
       "IPV6_STATIC 1111:0000:0000:0000:0204:9fff:fe06:5112"


  3. DEST_PORT <Space separated list of destination port number>
     e.g.: "DEST_PORT 2152 2123"

	Maximum 2 port numbers can be used. Use 2152 and 2123 for
	GTP-U and GTP-C traffic.

  4. PROTOCOL_ID <IP protocol number>
     e.g.: "PROTOCOL_ID 17"

  5. SEC_DISABLE <Disable IPsec. Value must be 1. By default, IPsec is enabled>
     e.g.: "SEC_DISABLE 1"

  6. LISTEN_MODE <Application traffic listen modes, acceptable values 0, 1, 2>
     0 -  Application is listening on both inner IP and static IP addresses.
     1 -  Application is listening on only static IP address.
     2 -  Application is listening on only inner IP address.

LGW traffic information in file:

  1. LGW_IPV4 <Space separated list of IPv4 subnets>
     and/or
     LGW_IPV6 <Space separated list of IPv6 subnets>

     Minimum one and maximum four IP subnets can be used.
     e.g.:
       "LGW_IPV4 172.16.0.0/16 10.0.0.0/8"
       "LGW_IPV6 2001:0db8:85a3:0000:0000:0000:0000:0000/64"

Note: IPv6 address format must be: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx

-------------------------------------------------------------------
Steps to increase MTU of kernel interface in VSP mode
-------------------------------------------------------------------
There are two ways to increase MTU of kernel interface:

A. Increase the buffer size in kernel:
  1. Compile the kernel after adding/updating following options in .config file:
     CONFIG_FSL_DPAA_ETH_JUMBO_FRAME=y
     CONFIG_FSL_FM_MAX_FRAME_SIZE=9600
  2. Boot up the board with this kernel.
  3. Run the following command to update the MTU:
     $ ifconfig <interface-name> mtu 9578

  This will increase the buffer size in kernel to 9600.

B. Enable Scatter Gather on kernel:
  1. Add the following entry to the bootargs and boot up the board:
     fsl_fm_max_frm=9600
  2. Run the following command to update the MTU:
     $ ifconfig <interface-name> mtu 9578

------------------------------------------------------------------------------------
Steps to run dpdk-pkt_split_app on DPAA2 (when split port is eth port)
------------------------------------------------------------------------------------

1. source /usr/local/dpdk/dpaa2/dynamic_dpl.sh dpmac.1 dpmac.2
2. ./dpdk-pkt_split_app -c 0xf -n 1 --vdev=net_tap0 -- -p 0x7 -s 0

'pkt-split' Application using traffic bifurcation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  pktsplit application has been modified to support traffic bifurcation, where traffic can be
  bifurcated such that some traffic goes to the kernel and other to DPDK using DPDMUX.
  Separate rules can be configured for this bifurcation. Below are some sample commands:

Note:
   We do not require data.input file to configure the classification rules in case of DPAA2.

1. Configure DPDMUX to have two DPNI's, one with kernel and other with user-space

  $ ls-addni --no-link
  	let us assume the dpni and eth. interface created are dpni.z and ethx resp.
  $ source /usr/local/dpdk/dpaa2/dynamic_dpl.sh dpmac.x dpmac.y
	let us assume the interface for dpmac.x is dpni.x and for dpmac.y is dpni.y
	and the container created be dprc.x
  $ echo dprc.x > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/unbind
  $ restool dprc disconnect dprc.x --endpoint=dpni.x
  $ restool dpdmux create --default-if=1 --num-ifs=2 --method DPDMUX_METHOD_CUSTOM --manip=DPDMUX_MANIP_NONE --option=DPDMUX_OPT_CLS_MASK_SUPPORT --container=dprc.1
  $ restool dprc connect dprc.1 --endpoint1=dpdmux.0.0 --endpoint2=dpmac.x
  $ restool dprc connect dprc.1 --endpoint1=dpdmux.0.1 --endpoint2=dpni.z
  $ restool dprc connect dprc.1 --endpoint1=dpdmux.0.2 --endpoint2=dpni.x
  $ restool dprc assign dprc.1 --object=dpdmux.0 --child=dprc.x --plugged=1
  $ echo dprc.x > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
  $ export DPRC=dprc.x
  $ ifconfig ethx 1.1.1.2
  $ ifconfig ethx promisc

2. Run pkt-split application providing 'dpaa2-traffic-split-config' option for configuring DPDMUX rules:

  $ ./dpdk-pkt_split_app -c 0xf -n 1 --vdev=net_tap0 -- -p 0x7 -s 0 --dpaa2-traffic-split-config="(3,2152,2)"

  Here "--dpaa2-traffic-split-config - (type,val,mux_conn_id)" is defined by
	value as 'val' based on which DPDMUX can split the traffic to mux_conn_id

	'type' -
		1: ETHTYPE,
		2: IP_PROTO,
		3: UDP_DST_PORT
		4: IP_FRAG_UDP_AND_GTP
		5: IP_FRAG_PROTO
		6: IP_FRAG_UDP_AND_GTP_AND_ESP
		7. VLAN (val here is the VLAN id)
