sort -u like awk for few items and many dups

| awk '{a[$0]=1} END{for (k in a) print k}'

Performance test:
# create a file with few items and many dups
$ for i in {1..1000000}; do printf "a\nb\nc\n" >> few_items_many_dups.txt; done

# sort uniq using awk
$ time cat few_items_many_dups.txt | awk '{a[$0]=1} END{for (k in a) print k}' > few_items_many_dups.txt.awk_sortuniq
real    0m1.291s

# sort uniq using sort -u
$ time cat few_items_many_dups.txt | sort -u > few_items_many_dups.txt.sortuniq
real    0m36.726s

how-to get IP from hostname using dig

$ dig www.groupon.com +short | tail -n1
184.28.147.170
 

$ dig www.groupon.com +short
www.groupon.com.edgekey.net.
e5352.b.akamaiedge.net.
184.28.147.170

iptables config script with stateful filtering and logs

#!/bin/bash
#
# iptables configuration script

IPTABLES=/sbin/iptables

#
# Flush all current rules
#
$IPTABLES -F

#
# Allow SSH connections on tcp port 22
# This is essential when working on remote servers
# via SSH to prevent locking yourself out of the system
#
$IPTABLES -A INPUT -p tcp --dport 22 -j ACCEPT

#
# Allow HTTP connections on tcp port 80 from 10.20.30.40
#
$IPTABLES -A INPUT -s 10.20.30.40 -p tcp --dport 80 -j ACCEPT

#
# Set default policies for INPUT, FORWARD and OUTPUT chains
#
$IPTABLES -P INPUT DROP
$IPTABLES -P FORWARD DROP
$IPTABLES -P OUTPUT ACCEPT
#
# Set access for localhost
#
$IPTABLES -A INPUT -i lo -j ACCEPT
#
# Accept packets belonging to established and related connections
#
$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT


# Log drops to /var/log/messages
LOGLIMIT="2/s"
$IPTABLES -N LOGDROP
$IPTABLES -A LOGDROP -p tcp -m limit --limit $LOGLIMIT -j LOG --log-prefix "TCP LOGDROP: "
$IPTABLES -A LOGDROP -p udp -m limit --limit $LOGLIMIT -j LOG --log-prefix "UDP LOGDROP: "
$IPTABLES -A LOGDROP -p icmp -m limit --limit $LOGLIMIT -j LOG --log-prefix "ICMP LOGDROP: "
$IPTABLES -A LOGDROP -f -m limit --limit $LOGLIMIT -j LOG --log-prefix "FRAGMENT LOGDROP: "
$IPTABLES -A LOGDROP -j DROP

$IPTABLES -A INPUT -p icmp -i eth0 -j LOGDROP
$IPTABLES -A INPUT -p tcp -i eth0 -j LOGDROP
$IPTABLES -A INPUT -p udp -i eth0 -j LOGDROP




#
# In case you want to save these settings
#
# /sbin/service iptables save
#

#
# List rules
#
$IPTABLES -L -v


ref:
http://www.linuxquestions.org/questions/linux-security-4/iptables-logging-385165/
http://www.centos.org/docs/5/html/5.1/Deployment_Guide/s1-iptables-saving.html

float arithmetic

laptop:~ pablo$ python

CASE 1:

>>> a = 987654321123456.1
>>> a
987654321123456.1
>>> (a*20 - (a-1)*20)/20
1.0
>>> (a*100 - (a-1)*100)/100
0.96
>>> (a*1000 - (a-1)*1000)/1000
1.024
>>> (a*10000 - (a-1)*10000)/10000
1.024

CASE 2:

>>> a = 987654321123456.1
>>> b = 987654321123455.2
>>> a-b
0.875
>>> a
987654321123456.1
>>> b
987654321123455.2

How-To copy column values from a table to another in MySQL

Imagine table A and B like:

SELECT * FROM A WHERE (...) LIMIT 3
id, a, x, c
__________
2, 5, 1, 6
7, 2, 6, 1
9, 8, 2, 2

SELECT * FROM B WHERE (...) LIMIT 3
id, x, b, c
__________
1, NULL, 3, 3
5, NULL, 0, 4
3, NULL, 2, 7

You want to copy x column values from A (i.e. 1,6,2) to B (all NULL).

Solution:

SELECT @i:=0;
SELECT @j:=0;
UPDATE B
JOIN (
SELECT @i:=@i+1 AS tid, B.id FROM B WHERE (...) LIMIT 3
) Bt ON (Bt.id=B.id)
JOIN (
SELECT @j:=@j+1 AS tid, A.id from A WHERE (...) LIMIT 3
) At ON (At.tid=Bt.tid)
SET B.x = At.x;

Linux command line python pretty print

cat somexmlfile.xml | python -c "import sys;import lxml.etree as etree;x = etree.parse(sys.stdin);print etree.tostring(x, pretty_print = True)"

alias pp='python -mjson.tool'

cat somexmlfile.json | pp

How to open port 80 for out connections to a specific user

sudo iptables -A ufw-user-output -p tcp --dport 80 -m owner \
 --uid-owner pepito -j ACCEPT