Rsync

rsync is a powerful Linux command that allows users to efficiently synchronize files and directories between two locations. It is commonly used for backups, mirroring data, and moving files between servers.

One of the key features of rsync is its ability to only copy files that have been modified, which makes it much more efficient than simply copying all files every time. It also has the ability to compress data and skip files that are already up-to-date, which further improves performance.

Here are three common use cases for the rsync command:

  1. Backing up data: rsync can be used to create backups of important files and directories. For example, the following command will create a backup of the /home directory in the /backup directory:
rsync -avz /home /backup
  1. Mirroring data: rsync can be used to create a mirror of a directory on another server. This is useful for creating a redundant copy of data for high availability. For example, the following command will create a mirror of the /var/www directory on a remote server:
rsync -avz --delete /var/www [email protected]:/var/www
  1. Transferring files: rsync can be used to transfer files between two servers. This is useful for moving large amounts of data quickly and efficiently. For example, the following command will transfer all files in the /data directory to a remote server:
rsync -avz /data [email protected]:/data

I hope this brief overview of the rsync command and its use cases has been helpful. As always, be sure to carefully read the documentation and test the command on a non-critical system before using it in a production environment

Openssl cheat sheet

This is my cheat sheet for establishing Public Key Infrastructure. These are the commands that I usually use in my setups for the PKI

Generate private key:

openssl genpkey -algorithm RSA -out tim.pkey

Get the public key from the private key:

openssl pkey -in tim.pkey -pubout -out tim_public_key

Encrypt file

openssl pkeyutl -in my_private_infor.txt -out encrypted_data.txt -encrypt -pubin -inkey tim_public_key

Decrypt file

openssl pkeyutl -out decrypted.txt -in encrypted_data.txt -decrypt -inkey tim.pkey

Commands to handle CA, generation of CSR and signing the CSR by the CA.

First need to setup the CA

Generate private key for the CA:

openssl genrsa -aes256 -out myCA.key 2048

Generate CA certificate from the private key of the CA

openssl req -x509 -new -key myCA.key -sha256 -days 3650 -out myCA.pem

On the device that I want to request from the CA to sign the CSR:

First generate private key for the device, or some kind of an endpoint

openssl genrsa -out dev.key

Generate CSR from the private key. This CSR needs to be sent to the CA server.

openssl req -new -key dev.key -out dev.csr

Then on CA, sign the CSR, and the recieved certificate to be sent to the device

openssl x509 -req -in device/dev.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out dev.crt -days 365 -sha256

Raspberry pi fan performance

Recently I bought a fan for raspberry pi. The fan is small, and as I could feel the air outtake was not that strong. If I decide to install this fan, I”ll need to drill in the official raspberry pi case. I started to doubt if all the tempering with the official case worth the effort. I decided to perform a test before tempering with the case. I performed three tests with the same command:

sysbench --num-threads=4 --validate=on --test=threads --thread-yields=40000 --thread-locks=6 run

While performing the test I measured the CPU temperature every 5 seconds with the following python code

import datetime
import pause as pause

file="/sys/class/thermal/thermal_zone0/temp"
line=""
output="output.txt"
while (file):
    with open(file,"r") as in_f:
        line=int(in_f.read())
        line=line/1000
    with open(output,"a") as out_f:
        out_f.write("{0},{1}\n".format(datetime.datetime.now().strftime("%H:%M:%S %d/%m/%Y"), line))
    pause.seconds(5)

The first test was with the official raspberry pi case closed, the second test was with the top lid open and the third test was with a lid open and a fan on top of the CPU while I was holding the fan with my hands. Not perfect, but it is what it is.

I arranged the results within a graph with axe Y shows CPU temperature in Celsius, and axe X shows a 5 second tic increments. The results show that with with the lid closed and the lid open the results are fairly close. The difference in graphs (blue and red) exist probably due to a different starting point. The graph shows without any doubt a good boost to temperature handling with a fan (yellow line).

The maximum temperature without a fan in both cases was 81 degrees and with a fan 72 degrees Celsius.

So my conclusion is: if possible, use a fan!

Addition

I tempered with my raspberry pi official case. Added the fan and did a few holes in the case. This is the picture of my crude solution

I added the results with this tempered case to the data from above and the new graph as follows:

Test scenario CPU temperature over time 5 seconds interval

Blue – closed lid
Red – opened lid
Orange – opened lid with fan
Green – lid with installed fan

Temperature goes up to maximum of 71 degrees Celsius with the tempered case

Test scenarioClosed lidOpen lidOpen lid with fanCase with installed fan
Max temperature81817271
Test scenario maximum temperature comparison table

Conclusion stays the same: Use a fan

SSH tunneling

Ssh tunneling allows to route traffic via ssh tunnel. For example, in the following topology I would like to to access 172.217.171.228 on port 80 via 101.1.1.1. But on local port on 192.168.0.10 I would like to go to port 5050.

In that case my command that I”ll issue on 192.168.0.10 will be:

ssh -L 5050:172.217.171.228:80 [email protected]

This command will open ssh to 101.1.1.1 from 192.168.0.10, and will hold local port 5050 on the Client open. Every connection that is made to localhost:5050 on the Client will be forwarded to the SSH session and from the “SSH tunnel intermediate” will open session to 172.217.171.228:80

There also another option for ssh tunnel, where the Server connects to client, and then on the client machine user can open connection to some local port and the session ends up in the server.

The following command will make this happen:

ssh -R 5050:localhost:80 [email protected]

In that case the Server will open ssh session to the Client. Every connection made on the client to port localhost:5050 will be forwarded to 172.217.171.228

With the same topology described above, I want to access any server from any port. In that case, the command will be used:

ssh -D 5050 [email protected]

This will act as a SOCKS server. Meaning, after ssh established to 101.1.1.1 from 192.168.0.10, every connection that is made to localhost:5050 will be forwarded to outside world with random port from 101.1.1.1. This is useful if some proxy is used to firewall your connections to the internet. The initiator of the connection needs to work with SOCKS4 or SOCKS5 protocol. Firefox can be this initiator.

To configure SOCKS in firefox, in address on top write and go to about:preferences. Search for proxy in search field. Click on settings and write localhost in address and port number 5050 under “SOCKs host”

Obviously the ssh server needs to support ssh tunneling. My ubuntu did not support this by default. To enable this in the file /etc/ssh/sshd_config this configuration should be applyed:

AllowTcpForwarding yes
GatewayPorts yes

After this configuration changed, ssh service should be restarted

service ssh restart

TMUX cheat sheet

Tmux is a terminal that I often use. Great benefits are if ssh disconnects the terminal remains and if I run some command, it continues to run. Other benefit is same screen for two or more ssh connections.

These are more common basic command to use in tmux

Start a new session with a specific name

tmux new -s session-5

By default , after entering the command “tmux”, sessions are named with a number

List sessions tmux ls

[email protected]:~$ tmux ls
0: 2 windows (created Sun Oct 11 13:10:12 2020) (attached)
session-5: 1 windows (created Tue Oct 13 11:26:48 2020)
[email protected]:~$

There are two sessions in this computer, named 0 and session-5.

Connect to specific session

tmux attach -t session-5

I”ll use C^ to specify CONTROL and M^ to specify ALT button. The control character for tmux is C^b. After this combination is pressed user enters control mode and perform actions like copy and paste, split windows etc.

Split session vertically 

C^b %

Split session horizontally 

C^b "

The result of splitting first vertically then horizontally looks like the following:

Navigate between open windows

C^b arrows

Open new window

C^b c

Detach – exit from tmux, but leave the session running

C^b d

To copy in tmux, first click C^b [. With arrows go to desired text click C^SPACE. Select desired text with arrows. Click M^w. Text copied.

To paste go to desired location to paste , click C^b ]

Tar cheat sheet

I don’t use tar very often. However every time I do, I don’t remember the switches and options. So I am making this post in order to remember the basics of the tar command.

Store files in a new archive (option -c), in file name arch.tar (option -f indicates file name), while show all output (-v).

tar -c -v -f arch.tar .

This will be the same as :

tar -cvf arch.tar .

From now on will use a short way of giving commands to the tar.

To archive files with compression add switch for compression. For gzip add -z:

tar -cvzf arch.tar .

List files (-t) within the archive (-f) arch.tar, while showing verbose output (-v) :

tar -tvf arch.tar

Extract files (-x) from archive (-f) arch.tar, while showing verbose output (-v):

tar -xvf arch.tar

Extract to specific directory use -C switch. 

tar -xvf arch.tar -C arch

Python libraries for networking

Someone told me about these python libraries for python for networking. I am sure that I will not remember them by heart, so I will write these libraries here, in order to remember

  • Asyncio – Python standard library. Asynchronous I/0, event loops, coroutines and tasks
  • Diesel – coroutine-based networking library for python
  • Pulsar – Event driven concurrent framework for Python
  • NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) is a Python library that implements a set of functions to interact with different network device Operating Systems using a unified API.
  • Twisted – an event-driven networking engine written in Python 

Capturing traffic on an interface and reviewing later in wireshark on Cisco CSR 1000v

TL;DR: On CISCO router, capture traffic locally to a pcap file and send it later to your computer.

I don’t know how did I missed that! I needed to capture a traffic on an interface, however for some reason, i couldn’t use live capture. So I was trying to find a way to capture, using the “?” a lot in CISCO terminal. Accidentally, I found a way to store a capture locally. I pretty sure everyone knows about this but me, but I”ll write this anyway.

After the capture is complete it needed to be sent somewhere. In my case I will send it to tftp server. So make sure you have tftp server running somewhere.

Performing the capture

Configure capture to match some traffic. In my case I want to capture any IPv6 traffic. For me the command would be :

Router# monitor capture cap_name match ipv6 any any interface gigabitEthernet 2 both

The command breakdown with some of the fields explained:

Router#monitor capture cap_name match ipv6 any any interface gigabitEthernet 2 both
                        ^              ^    ^   ^
                        |              |    |   +----------------+
              +---------+  +-----------+    +-------------+      |
              |            |                              |  Destination selection
              +            +                              |
   Capture name    Match traffic type                Source selection
                     any   all packets                 A.B.C.D/nn  IPv4 source Prefix ...
                     ip^4  IP^4 packets only              or
                     ipv6  IPv6 packets only           X:X:X:X::X/<0-128>  IPv6 source...
                     mac   MAC filter configuration    any         Any source prefix
                                                       host        A single source host
                                                       protocol    Protocols

Now I can start the capture:

Router# monitor capture cap_name start

Now the capture runs. It is probably a good idea to have some good match for a specific traffic to make sure to keep the capture file small and memory of the CISCO free.

While the capture runs, I can check it status:

Router#show monitor capture cap_name

Status Information for Capture cap_name
  Target Type:
   Interface: GigabitEthernet2, Direction: both
   Status : Active
  Filter Details:
   IPv6
    Source IP:  any
    Destination IP:  any
   Protocol: any
  Buffer Details:
   Buffer Type: LINEAR (default)
   Buffer Size (in MB): 10
  Limit Details:
   Number of Packets to capture: 0 (no limit)
   Packet Capture duration: 0 (no limit)
   Packet Size to capture: 0 (no limit)
   Maximum number of packets to capture per second: 1000
   Packet sampling rate: 0 (no sampling)

When the capture is done, I can stop it:

Router#monitor capture cap_name stop

And now I need to send the capture to my tftp server:

Router#monitor capture cap_name export tftp://10.0.0.44/my_capture.pcap
!
Exported Successfully

Other destinations where a traffic can be exported to:

Router#monitor capture cap_name export ?
  bootflash:  Location of the file
  flash:      Location of the file
  ftp:        Location of the file
  http:       Location of the file
  https:      Location of the file
  pram:       Location of the file
  rcp:        Location of the file
  scp:        Location of the file
  tftp:       Location of the file

This is it! Just open the file you’ve received in wireshark.

Kernel programming seminar.

I have a kernel programming seminar at my job. The idea is to enrich knowledge and to ‘fortify’ what I already know.

In my experience, in order to truly learn and remember what I learned is to write it down. I will write my notes here in a hope that I don’t forget what I’ve learned. In case I do forget, I always can come here, and remind myself what I’ve learned.

So the next few posts will be my personal notepad available for everyone.

Part 1 – Introduction

One of the method to program in kernel is to use modules. In most cases these are drivers.

To compile a module, it is required to compile the module with the source code of the kernel – so called headers.

When compiling a module, use Makefile:

obj-m += acme.o
KDIR ?= /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)

EXTRA_CFLAGS = -g

default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

The compilation result file ends with .ko. This file needs to be part of kernel. To insert or remove this file from the kernel the following commands can be used:

insmod <file name> – insert module file into the kernel
rmmod <module name> – remove module from kernel
lsmod – list modules loaded into kernel

Module has to have two functions: one activates when loading file to kernel, the other activate when removing module from kernel:
module_init(acme_init);
module_exit(acme_exit);

The function init will be called with the __init preamble :
static int __init acme_init(void)
{
}

The function exit will be called with the exit __preamble:
static void __exit acme_exit(void)
{
}

The module should have these definitions:
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Example character driver");
MODULE_AUTHOR("Free Electrons");

If license differ then GPL, dmesg log will show that the kernel is tainted:
module license 'Proprietary' taints kernel.

kernel module hello world program:

#include <linux/init.h>
#include <linux/module.h>

MODULE_LICENSE("Proprietary");

static int __init hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}

static void __exit hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}

module_init(hello_init);
module_exit(hello_exit);

MODULE_AUTHOR("Tim Goldshmit");
MODULE_DESCRIPTION("Tim's HELLO module");
MODULE_LICENSE("GPL");