07 May 2015 
Matteo Mattei 
nfs virtualbox nat 
I am using Ubuntu 14.04 (but it is valid also with other Linux distributions) that runs under Virtualbox. My network interface is set as NAT and I am running nfs version 4 (NFSv4).
Every time I try to mount a remote share I always obtain the following error:
matteo@vm:~$ sudo mount -t nfs xxx.xxx.xxx.xxx:/opt/share /mnt/remote
mount.nfs: access denied by server while mounting xxx.xxx.xxx.xxx:/opt/share
And in the kernel log (dmesg) I see this:
[ 1351.443078] RPC: server xxx.xxx.xxx.xxx requires stronger authentication.
The issue here is that the NFS client is trying to use UDP protocol to access the remote host. You can verify it by capturing the network traffic with wireshark (or tcpdump). In this case is sufficient to force the usage of the TCP protocol when you mount the share:
matteo@vm:~$ sudo mount -t nfs xxx.xxx.xxx.xxx:/opt/share /mnt/remote -o proto=tcp
matteo@vm:~$
Another option is to change your network adapter in Virtualbox from NAT to BRIDGE.
That’s all for the moment.
02 Mar 2015 Matteo Mattei python dymo labels Dymo is a leading company for label printing. Altought several things can be done using their DYMO Label v.8 Software, I found that there was no way to set a specific date in a label layout so that every time I need to print a label I have the current date plus 30 days. I read the manual and I found that to do it we have to use a variable field but this is something that has to be handled using an external program.
So I wrote a little python tool that runs on Windows 7 and that does exactly what I need. Make sure to install pywin32 since we are going to use COM apis.
This is my example label (my.label) and the related Python code:
As you can see I have two fields in the XML with IsVariable set to True and I have assigned to them the name TEXT1 and TEXT2. Now, all the logic happens between line 30 and line 41 of the Python script.
- I initialize the Dispatch for the printer, configure the label I want to use (mylabel) and then I select the printer [lines 30-34].
- Then I apply my rules to the variables TEXT1 and TEXT2 [lines 36-37].
- And finally I launch the print job [lines 39-41].
That’s all! With only few lines of Python code we are able to print a label using our Dymo LabelWriter 450 Turbo printer.
Note: the full code is available on GitHub including the scripts to build a self-contained executable.
13 Feb 2015 Matteo Mattei linux bash backup megatools Mega is a wonderful and secure online cloud service that offers 50GB of free storage. So I thought that it would be great using it as additional backup service for my Linux servers. Fortunately there is a good application library and tools to access Mega storage from Linux command line. This tools are called megatools and it is released under GPLv2 license.
In this post I will show you how to compile, install and configure a full backup system for your server using Mega and MegaTools.
First of all register an account with Mega at http://mega.co.nz then follow these instructions to compile and install megatools.
#!/bin/bash
VERSION="1.9.95"
apt-get install -y build-essentials pkg-config libglib2.0-dev libssl-dev libcurl4-openssl-dev libfuse-dev glib-networking
wget http://megatools.megous.com/builds/megatools-${VERSION}.tar.gz
tar xzf megatools-${VERSION}.tar.gz
cd megatools-${VERSION}
./configure && make && make install && ldconfig
Note: the above instructions are valid for Debian 7 Wheezy and Debian 8 Jessie. In case you have a different Linux distribution please install the required dependencies.
Now that you have MegaTools installed in /usr/local/bin create a configuration file with your credentials in /root/.megarc:
[Login]
Username = Your_Mega_Username
Password = Your_Mega_Password
Since the password is in clear, it is important to protect the file:
Test now your mega installation and login credentials:
root@debian:~# megals
/Contacts
/Inbox
/Root
/Trash
If all goes well you are ready to prepare your backup script. Create a new file called megabackup.sh and place it in /root:
Make it executable and accessible only to root:
chmod 750 /root/megabackup.sh
You only need to set a cron-job now to execute the backup every day:
04 04 * * * root /root/megabackup.sh
Update 2015-02-14: MegaFuse
I recently found a better tool to accomplish my needings, it is called MegaFuse and it works like a Linux mountpoint. First of all download all needed dependencies:
apt-get install libcrypto++-dev libcurl4-openssl-dev libdb5.1++-dev libfreeimage-dev libreadline-dev libfuse-dev make
Then download MegaFuse and compile it:
git clone https://github.com/matteoserva/MegaFuse
cd MegaFuse
make
Now create a configuration file /root/.megafuse.conf with your Mega credentials:
USERNAME = your_mega_email
PASSWORD = your_mega_password
MOUNTPOINT = /mnt
The above configuration assume you will use /mnt as your mountpoint (change it if you want).
Protect it in the same way:
chmod 640 /root/.megafuse.conf
And this is the revisited script to do the backup:
You can now configure the cronjob:
04 04 * * * root /root/megafusebackup.sh
12 Feb 2015 Matteo Mattei linux bash installer In this post I will show you how to develop a self contained Linux command line installer in Bash that will decompress an archive and perform some tasks.
Installer content
Our installer that is basically a self-extracting archive with some logic around, consists in three parts:
- A bash script that performs the extraction of the archive and applies some logic.
- A marker to separate the bash script and the archive.
- An archive containing the actual data to install.
Start now!
Create a new bash script called installer.sh with the following content:
#!/bin/bash
echo ""
echo "My Command Line Installer"
echo ""
# Create destination folder
DESTINATION="/opt/my_application"
mkdir -p ${DESTINATION}
# Find __ARCHIVE__ maker, read archive content and decompress it
ARCHIVE=$(awk '/^__ARCHIVE__/ {print NR + 1; exit 0; }' "${0}")
tail -n+${ARCHIVE} "${0}" | tar xpJv -C ${DESTINATION}
# Put your logic here (if you need)
echo ""
echo "Installation complete."
echo ""
# Exit from the script with success (0)
exit 0
__ARCHIVE__
This script is self-explain but I will try to describe the steps:
- Create a destination folder ${DESTINATION}.
- Find __ARCHIVE__ marker and put the tarball content into ${ARCHIVE} variable.
- Decompress the tarball into the destination folder.
- Eventually apply your installation logic (copy some files, change some others, etc…).
- Exit from the script (this step is mandatory otherwise bash will try to interpret the tarball and will exit with error).
- Add __ARCHIVE__ marker at the bottom of the script. This marker will be used to separate the actual bash script with the tarball content.
Now generate a compressed tarball of your application (I used .tar.xz in the above example):
tar cJf myarchive.tar.xz /folder/to/archive
OK, now append it to the installer bash script and make it executable:
cat myarchive.tar.xz >> installer.sh
chmod +x installer.sh
That’s all! You can now distribute your installer.
Execute your installer
The users will execute your installer simply running:
09 Jan 2015 Matteo Mattei linux server couchdb debian erlang Setup repository and install all dependencies
echo "deb http://packages.erlang-solutions.com/debian wheezy contrib" >> /etc/apt/sources.list
wget -qO - http://packages.erlang-solutions.com/debian/erlang_solutions.asc | apt-key add -
apt-get update
apt-get install -y build-essential curl erlang-nox erlang-dev libmozjs185-1.0 libmozjs185-dev libcurl4-openssl-dev libicu-dev
Create CouchDB account
useradd -d /var/lib/couchdb couchdb
mkdir -p /usr/local/{lib,etc}/couchdb /usr/local/var/{lib,log,run}/couchdb /var/lib/couchdb
chown -R couchdb:couchdb /usr/local/{lib,etc}/couchdb /usr/local/var/{lib,log,run}/couchdb
chmod -R g+rw /usr/local/{lib,etc}/couchdb /usr/local/var/{lib,log,run}/couchdb
Install CouchDB
wget http://apache.panu.it/couchdb/source/1.6.1/apache-couchdb-1.6.1.tar.gz
tar xzf apache-couchdb-1.6.1.tar.gz
cd apache-couchdb-1.6.1
./configure --prefix=/usr/local --with-js-lib=/usr/lib --with-js-include=/usr/include/js --enable-init
make && make install
Create symlinks and start the database
chown couchdb:couchdb /usr/local/etc/couchdb/local.ini
ln -s /usr/local/etc/init.d/couchdb /etc/init.d/couchdb
/etc/init.d/couchdb start
update-rc.d couchdb defaults
Verify that all is working fine
curl http://127.0.0.1:5984/
The output should be like this:
{"couchdb":"Welcome","uuid":"5da242ff50cecec904d6caf36be34194","version":"1.6.1","vendor":{"name":"The Apache Software Foundation","version":"1.6.1"}}
Finalize setup
In order to connect from remote edit /usr/local/etc/couchdb/local.ini and change
to:
Restart the database:
And from a web browser visit the CouchDB Futon:
08 Dec 2014 Matteo Mattei linux server mysql varnish debian php iptables postfix Setup bash and update the system
cp /etc/skel/.bashrc /root/.bashrc
apt-get update
apt-get dist-upgrade
Make sure to have the following two lines (with the same format) at the top of your /etc/hosts file
127.0.0.1 localhost.localdomain localhost
xxx.xxx.xxx.xxx web1.myserver.com web1
Note: xxx.xxx.xxx.xxx is the public IP address assigned to your server.
Install all needed packages
apt-get install php5 mysql-server mysql-client apache2 iptables phpmyadmin varnish shorewall vsftpd php5-cli php5-curl php5-dev php5-gd php5-idn php5-imagick php5-imap php5-memcache php5-ming php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xcache php5-xmlrpc php-apc php-pear php-xml-rpc postfix apg ca-certificates heirloom-mailx
MySQL/PhpMyAdmin:
- mysql root password: xxx
- repeat mysql root password: xxx
- web server to reconfigure automatically: apache2
- configure database for phpmyadmin with dbconfig-common? Yes
- Password of the database’s administrative user: xxx
- Password for phpmyadmin: xxx
- Password confirmation: xxx
Postfix:
- Select Internet Site
- System mail name: (insert here the FQDN, for example web1.myserver.com)
Setup FTP
Stop VSFTP server:
Create backup configuration:
mv /etc/vsftpd.conf /etc/vsftpd.conf.backup
Add new configuration:
listen=YES
listen_port=21
anonymous_enable=NO
local_enable=YES
guest_enable=YES
guest_username=nobody
user_sub_token=$USER
local_root=/var/www/vhosts/$USER
virtual_use_local_privs=YES
user_config_dir=/etc/vsftpd/users
pam_service_name=vsftpd_local_and_virtual
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/chroot_list
ftpd_banner=Welcome to my ftp server
write_enable=YES
download_enable=YES
dirlist_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
xferlog_file=/var/log/xferlog
connect_from_port_20=YES
connect_timeout=60
data_connection_timeout=300
idle_session_timeout=300
local_max_rate=0
max_clients=0
max_per_ip=3
Create an empty chroot_list file:
touch /etc/vsftpd/chroot_list
Start VSFTP server:
Setup Apache
Stop Apache web server:
Backup Apache configuration:
cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.backup
Edit the following lines in /etc/apache2/apache2.conf
- From Timeout 300 to Timeout 45
- From KeepAliveTimeout 5 to KeepAliveTimeout 15
- Change the mpm_prefork_module section like the following:
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 400
MaxClients 400
MaxRequestsPerChild 10000
</IfModule>
Edit /etc/apache2/ports.conf and change the port 80 with 8080 since we are going to use Varnish:
NameVirtualHost *:8080
Listen 8080
Change the port (from 80 to 8080) also in the default virtual host /etc/apache2/sites-enabled/000-default Now restart Apache:
/etc/init.d/apache2 restart
Setup Varnish
Stop Varnish daemon:
Open /etc/varnish/default.vcl and make sure the backend section is like this:
backend default {
.host = "127.0.0.1";
.port = "8080";
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
}
Now edit /etc/default/varnish and set the DAEMON_OPTS variable like this:
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-p thread_pools=4 \
-p thread_pool_max=1500 \
-p listen_depth=2048 \
-p lru_interval=1800 \
-h classic,169313 \
-p max_restarts=6 \
-p connect_timeout=600 \
-p send_timeout=2000 \
-s malloc,2G"
Restart Varnish:
/etc/init.d/varnish restart
Setup MySQL
MySQL is already configured. You only need to log slow queries (that is often usefult during slow load page investigation). Todo it, open /etc/mysql/my.cnf and decomment the following two lines:
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
Copy the default configuration for one interface:
cp /usr/share/doc/shorewall/examples/one-interface/interfaces /etc/shorewall/interfaces
cp /usr/share/doc/shorewall/examples/one-interface/policy /etc/shorewall/policy
cp /usr/share/doc/shorewall/examples/one-interface/rules /etc/shorewall/rules
cp /usr/share/doc/shorewall/examples/one-interface/zones /etc/shorewall/zones
Now open /etc/shorewall/policy file and change the line:
removing info directive given it fills the system logs:
Now open /etc/shorewall/rules and add the following rules at the bottom of the file:
HTTP/ACCEPT net $FW
SSH/ACCEPT net $FW
FTP/ACCEPT net $FW
# real apache since varnish listens on port 80
#ACCEPT net $FW tcp 8080
NOTE: in case you want to allow ICMP (Ping) traffic from a specific remote hosts you need to add a rule similar to the following where xxx.xxx.xxx.xxx is the remote IP address, before the Ping(DROP) rule:
Ping(ACCEPT) net:xxx.xxx.xxx.xxx $FW
Now edit /etc/default/shorewall and change startup=0 to startup=1 You are now ready to start the firewall:
/etc/init.d/shorewall start
Setup Postfix
Stop postfix server:
Edit /etc/mailname and set your server domain name, for example:
Then, in order to monitor mail traffic coming from PHP you need to edit /etc/php5/apache2/php.ini. Go to [mail function] section and set the following two options:
sendmail_path = /usr/local/bin/sendmail-wrapper
auto_prepend_file = /usr/local/bin/env.php
Now create the two files above:
sendmail-wrapper:
#!/bin/sh
logger -p mail.info sendmail-wrapper.sh: site=${HTTP_HOST}, client=${REMOTE_ADDR}, script=${SCRIPT_NAME}, pwd=${PWD}, uid=${UID}, user=$(whoami)
/usr/sbin/sendmail -t -i $*
env.php:
<?php
putenv("HTTP_HOST=".@$_SERVER["HTTP_HOST"]);
putenv("SCRIPT_NAME=".@$_SERVER["SCRIPT_NAME"]);
putenv("SCRIPT_FILENAME=".@$_SERVER["SCRIPT_FILENAME"]);
putenv("DOCUMENT_ROOT=".@$_SERVER["DOCUMENT_ROOT"]);
putenv("REMOTE_ADDR=".@$_SERVER["REMOTE_ADDR"]);
?>
Now make they both have executable flag:
chmod +x /usr/local/bin/sendmail-wrapper
chmod +x /usr/local/bin/env.php
Add also /usr/local/bin/ to the open_basedir php list in /etc/apache2/conf.d/phpmyadmin.conf
php_admin_value open_basedir /usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/local/bin/
Restart Postfix:
/etc/init.d/postfix start
Prepare environment
Create all needed directories and files
mkdir /root/cron_scripts
mkdir -p /var/www/vhosts
mkdir -p /etc/vsftpd/users
touch /etc/vsftpd/passwd
Now download all tools to manage the server locally:
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ALIAS_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DOMAIN_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/MYSQL_CREATE.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/UPDATE_ALL_FTP_PASSWORD.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/UPDATE_FTP_PASSWORD.sh
chmod 770 *.sh
Download also the tools that will be used with cron:
cd /root/cron_scripts
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/cron_scripts/backup_mysql.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/cron_scripts/mysql_optimize.sh
chmod 770 *.sh
- Edit /root/ADD_DOMAIN.sh and change ADMIN_EMAIL variable with your email address.
- Edit /root/MYSQL_CREATE.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.
- Edit /root/cron_scripts/backup_mysql.sh and change the variable DB_PASSWORD with your MySQL root password and MAIL_NOTIFICATION with your email address.
- Edit /root/cron_scripts/mysql_optimize.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.
Edit /etc/crontab and add the following lines at the bottom:
# mysql optimize tables
3 4 * * 7 root /root/mysql_optimize.sh
# mysql backup
32 4 * * * root /root/backup_mysql.sh
