Archive for January, 2017

Add an alias on a network interface in Linux

$ sudo ip link set dev enp0s25 alias test
$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s25: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 68:f7:28:84:38:ce brd ff:ff:ff:ff:ff:ff
    alias test
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
    link/ether 60:57:18:cf:0c:6a brd ff:ff:ff:ff:ff:ff

Enable export of all environment variables in bash

When setting up environment variables, for example in .profile you can write the following near the top of the file to enable exporting of all the environment variables without explicitly writing “export” each time:

set -o allexport

with the above, then you can just write the following, for example:

TEST_DIR=/tmp/test

instead of

export TEST_DIR=/tmp/test

To disable the all export:

set +o allexport

You are also able to set environment variables without the export or allexport, however that will NOT make the variable available to sub-processes.

Count bytes and characters with wc

Count bytes and characters with wc

wc -c filename == counts the bytes
wc -m filename == counts the characters

this can be combined with other commands like grep, awk, sed

example with sed:

sed -n '1p' filename | wc -c

the above will print the first line from filename and count the bytes in that line

Fix ^M in Linux/UNIX using vi/vim

Sometimes if you edit a file in windows and then open it in Linux/UNIX it will have the special characters ^M at the end of each sentence. To fix ^M in Linux/UNIX using vi/vim run the following:

%s/[ctrlkey+v and ctrl-key+M]//g

sed tricks to replace multi blank spaces and tabs with single space

Replace all multi-blankspaces (more than one spaces/tabs) in file with mono-blankspace (one space):

sed 's/[<space>][<tab>][<space>][<tab>]*/ /g' filename

can also be:

 sed 's/[<space>][<tab>][<space>][<tab>]*/ /g' filename > newfilename

note: g in the end means GLOBAL

In the above samples, <space> means ‘ ‘ (empty space) and <tab> means ‘      ‘ (tab key press).

See which queries are executing – Oracle

See which queries are currently executing in Oracle (11g).

select s.username, s.sid, s.osuser, sql_text 
from v$sqltext_with_newlines t,v$session s 
where t.address = s.sql_address 
and t.hash_value = s.sql_hash_value 
and s.status = 'ACTIVE' 
and s.username <> 'SYSTEM' 
order by s.sid, t.piece

Finding and cleaning Oracle locks

How to find blocking sessions in Oracle (11g). This is used to make sure that the object which we are trying to update is actually locked by another session:

select
 c.owner,
 c.object_name,
 c.object_type,
 b.sid,
 b.serial#,
 b.status,
 b.osuser,
 b.machine
from
 v$locked_object a ,
 v$session b,
 dba_objects c
where
 b.sid = a.session_id
and
 a.object_id = c.object_id;

And this query tells which are the problem sessions:

select l1.sid, ' IS BLOCKING ', l2.sid
from v$lock l1, v$lock l2
where l1.block =1 and l2.request > 0
and l1.id1=l2.id1
and l1.id2=l2.id2;

So now we have the SID of the problem session and we only need the serial number to kill it.

SELECT serial# FROM v$session WHERE sid=<sid>

And now kill it:

ALTER SYSTEM KILL SESSION '<sid>,<session#>'

Check tablespace free space and total used – Oracle

Use the below SQL query to check the total, used and free space in all the tablespaces in the database

select 
   fs.tablespace_name                          "Tablespace", 
   (df.totalspace - fs.freespace)              "Used MB", 
   fs.freespace                                "Free MB", 
   df.totalspace                               "Total MB", 
   round(100 * (fs.freespace / df.totalspace)) "Pct. Free" 
from 
   (select 
      tablespace_name, 
      round(sum(bytes) / 1048576) TotalSpace 
   from 
      dba_data_files 
   group by 
      tablespace_name 
   ) df, 
   (select 
      tablespace_name, 
      round(sum(bytes) / 1048576) FreeSpace 
   from 
      dba_free_space 
   group by 
      tablespace_name 
   ) fs 
where 
   df.tablespace_name = fs.tablespace_name;

If you are running this in sqlplus you might want to format the columns first

column "Tablespace" format a13
column "Used MB" format 99,999,999
column "Free MB" format 99,999,999
column "Total MB" format 99,999,999

 

Sync servers using multiple streams with LFTP

If you have a remote server that you want to sync a directory you can use for example rsync, however rsync doesn’t offer the multi stream syncing and can be slower if you have many new files. The below solution uses lftp with 5 parallel streams to download and each stream is split into 10 separate segments. This is very helpful if you cannot max your download connection with regular methods and of course if the uploading server has enough bandwidth.

#!/bin/bash
login=your_username 
pass=your_password
host=sftp://download.redoem.com
remote_dir=/media/files/redoem/dir
local_dir=/home/redoer/downloads
 
trap "rm -f /tmp/syncserver.lock" SIGINT SIGTERM
if [ -e /tmp/syncserver.lock ]
then
 echo "Syncserver is running already."
 exit 1
else
 touch /tmp/syncserver.lock
 lftp -u $login,$pass $host << EOF
 set ftp:ssl-allow yes
 set mirror:use-pget-n 10 
 mirror -c -P5 --newer-than=now-15min --log=syncserver.log $remote_dir $local_dir
 quit
EOF
 rm -f /tmp/syncserver.lock
 trap - SIGINT SIGTERM
 exit 0
fi

The above sample should work for most cases – you just need to change the values of the variables (highlighted in red).

This part (–newer-than=now-15min) can be changed based on how old files you want to sync, for example if you synced the files 4 hours and you have new files in the last hour, you want to change this part to ONLY download the new files then change this part to be now-60min or more (but less than 4 hours in this example).

If you want to share from your local machine to the remote server, replace this line as (changed parts highlighted in red):

 mirror -R -c -P5 --newer-than=now-15min --log=syncserver.log $local_dir $remote_dir

Now save the above as a .sh file, example syncserver.sh and you can add it as a cronjob to run every 15 minutes or run it when you have new files.

*/15 * * * * /home/redoer/syncserver.sh >> /home/redoer/sync_cron.log 2>&1

ZNC in a DigitalOcean droplet

It is nice to have an IRC connection that is always available and that doesn’t have your home IP address exposed. With the ZNC setup you can then use your local IRC client (BitchX, irssi, HexChat etc) to connect to the ZNC and you will be able to see all the networks and channels it is connected to. This tutorial will explain how to setup ZNC on a $5/month DigitalOcean droplet (ref link) with Ubuntu 16.04.1. Of course this setup should work on any hosting and setup – DigitalOcean is just given as an example here.

Create a Droplet

1. Press on the Create a Droplet button once you are logged in your DO account

  • Select the OS image (Ubuntu 16.04.X), the Droplet size/price ($5/mo), data center region, add extra features, ssh key if you want that extra security and then press on Create Droplet

2. Login to your Droplet (box) and make sure everything is up to date

sudo apt-get update
sudo apt-get upgrade

Installation and Build of ZNC

1. Grab few essentials that are needed to compile the ZNC from source

sudo apt-get install build-essential libssl-dev libperl-dev pkg-config

2. Grab the latest ZNC source tar ball

cd /usr/local/src; sudo wget http://znc.in/releases/znc-latest.tar.gz

3. Extract the package and enter the directory

sudo tar -xzvf znc-latest.tar.gz; cd znc*

4. Configure the source system wide (you can use ./configure –prefix=$HOME/znc if you don’t want system wide)

./configure

5. Then, compile ZNC (might take few minutes)

sudo make; sudo make install

Configuration of ZNC

1. Create a new user on which we will run ZNC

adduser znc-admin

2. Switch to the new user

su znc-admin; cd ~

3. Start ZNC and its configuration

/usr/local/bin/znc --makeconf

4. Below is a sample configuration (ZNC version 1.6.4) – I highlighted in red all the config that I entered manually, the rest is the default values (what is given in the brackets [ ])

[ .. ] Checking for list of available modules...
[ >> ] ok
[ ** ] 
[ ** ] -- Global settings --
[ ** ] 
[ ?? ] Listen on port (1025 to 65534): 14125
[ ?? ] Listen using SSL (yes/no) [no]: yes
[ ?? ] Listen using both IPv4 and IPv6 (yes/no) [yes]: yes
[ .. ] Verifying the listener...
[ >> ] ok
[ ** ] Unable to locate pem file: [/home/znc-admin/.znc/znc.pem], creating it
[ .. ] Writing Pem file [/home/znc-admin/.znc/znc.pem]...
[ >> ] ok
[ ** ] Enabled global modules [webadmin]
[ ** ] 
[ ** ] -- Admin user settings --
[ ** ] 
[ ?? ] Username (alphanumeric): redoEm
[ ?? ] Enter password: 
[ ?? ] Confirm password: 
[ ?? ] Nick [redoEm]: 
[ ?? ] Alternate nick [redoEm_]: 
[ ?? ] Ident [redoEm]: 
[ ?? ] Real name [Got ZNC?]: Redo'Em
[ ?? ] Bind host (optional): 
[ ** ] Enabled user modules [chansaver, controlpanel]
[ ** ] 
[ ?? ] Set up a network? (yes/no) [yes]: 
[ ** ] 
[ ** ] -- Network settings --
[ ** ] 
[ ?? ] Name [freenode]: 
[ ?? ] Server host [chat.freenode.net]: 
[ ?? ] Server uses SSL? (yes/no) [yes]: 
[ ?? ] Server port (1 to 65535) [6697]: 
[ ?? ] Server password (probably empty): 
[ ?? ] Initial channels: #redoEm
[ ** ] Enabled network modules [simple_away]
[ ** ] 
[ .. ] Writing config [/home/znc-admin/.znc/configs/znc.conf]...
[ >> ] ok
[ ** ] 
[ ** ] To connect to this ZNC you need to connect to it as your IRC server
[ ** ] using the port that you supplied. You have to supply your login info
[ ** ] as the IRC server password like this: user/network:pass.
[ ** ] 
[ ** ] Try something like this in your IRC client...
[ ** ] /server <znc_server_ip> +14125 redoEm:<pass>
[ ** ] 
[ ** ] To manage settings, users and networks, point your web browser to
[ ** ] https://<znc_server_ip>:14125/
[ ** ] 
[ ?? ] Launch ZNC now? (yes/no) [yes]: 
[ .. ] Opening config [/home/znc-admin/.znc/configs/znc.conf]...
[ >> ] ok
[ .. ] Loading global module [webadmin]...
[ >> ] [/usr/local/lib/znc/webadmin.so]
[ .. ] Binding to port [+34125]...
[ >> ] ok
[ ** ] Loading user [redoEm]
[ ** ] Loading network [freenode]
[ .. ] Loading network module [simple_away]...
[ >> ] [/usr/local/lib/znc/simple_away.so]
[ .. ] Adding server [chat.freenode.net +6697 ]...
[ >> ] ok
[ .. ] Loading user module [chansaver]...
[ >> ] ok
[ .. ] Loading user module [controlpanel]...
[ >> ] ok
[ .. ] Forking into the background...
[ >> ] [pid: 19576]
[ ** ] ZNC 1.6.4 - http://znc.in

That is it – ZNC is now up and running!

Connect to ZNC with your local IRC Client

Now find the IP address from your Droplet and setup your IRC client to connect to it.

Example with HexChat:

After connecting to the ZNC all the networks and channels you have added will appear.

Go to Top