Matteo Mattei

Hello, my name is Matteo Mattei and this is my personal website. I am computer engineer with a long experience in Linux system administration and web software development. My preferred computer languages and technologies are PHP, Python, Bash, HTML, HTML5, CSS3, jQuery, MySQL, Node.js, MongoDB, etc...

© 2006-2014 Matteo Mattei - all rights reserved. About my blog code

linkedin rss twitter google+ github facebook

Full web server setup with Debian 7 (Wheezy)

Setup bash and update the system

cp /etc/skel/.bashrc /root/.bashrc
apt-get update
apt-get dist-upgrade

Configure hostname correctly

Make sure to have the following two lines (with the same format) at the top of your /etc/hosts file

127.0.0.1       localhost.localdomain localhost
xxx.xxx.xxx.xxx web1.myserver.com web1

Note: xxx.xxx.xxx.xxx is the public IP address assigned to your server.

Install all needed packages

apt-get install php5 mysql-server mysql-client apache2 iptables phpmyadmin varnish shorewall vsftpd php5-cli php5-curl php5-dev php5-gd php5-idn php5-imagick php5-imap php5-memcache php5-ming php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xcache php5-xmlrpc php-apc php-pear php-xml-rpc postfix apg ca-certificates heirloom-mailx

MySQL/PhpMyAdmin:

  • mysql root password: xxx
  • repeat mysql root password: xxx
  • web server to reconfigure automatically: apache2
  • configure database for phpmyadmin with dbconfig-common? Yes
  • Password of the database's administrative user: xxx
  • Password for phpmyadmin: xxx
  • Password confirmation: xxx

Postfix:

  • Select Internet Site
  • System mail name: (insert here the FQDN, for example web1.myserver.com)

Setup FTP

Stop VSFTP server:

/etc/init.d/vsftpd stop

Create backup configuration:

mv /etc/vsftpd.conf /etc/vsftpd.conf.backup

Add new configuration:

listen=YES
listen_port=21
anonymous_enable=NO
local_enable=YES
guest_enable=YES
guest_username=nobody
user_sub_token=$USER
local_root=/var/www/vhosts/$USER
virtual_use_local_privs=YES
user_config_dir=/etc/vsftpd/users
pam_service_name=vsftpd_local_and_virtual
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/chroot_list
ftpd_banner=Welcome to my ftp server
write_enable=YES
download_enable=YES
dirlist_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
xferlog_file=/var/log/xferlog
connect_from_port_20=YES
connect_timeout=60
data_connection_timeout=300
idle_session_timeout=300
local_max_rate=0
max_clients=0
max_per_ip=3

Create an empty chroot_list file:

touch /etc/vsftpd/chroot_list

Start VSFTP server:

/etc/init.d/vsftpd start

Setup Apache

Stop Apache web server:

/etc/init.d/apache2 stop

Backup Apache configuration:

cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.backup

Edit the following lines in /etc/apache2/apache2.conf

  • From Timeout 300 to Timeout 45
  • From KeepAliveTimeout 5 to KeepAliveTimeout 15
  • Change the mpm_prefork_module section like the following:
  <IfModule mpm_prefork_module>
      StartServers         5
      MinSpareServers      5
      MaxSpareServers      10
      ServerLimit          400
      MaxClients           400
      MaxRequestsPerChild  10000
  </IfModule>

Edit /etc/apache2/ports.conf and change the port 80 with 8080 since we are going to use Varnish:

NameVirtualHost *:8080
Listen 8080

Change the port (from 80 to 8080) also in the default virtual host /etc/apache2/sites-enabled/000-default Now restart Apache:

/etc/init.d/apache2 restart

Setup Varnish

Stop Varnish daemon:

/etc/init.d/varnish stop

Open /etc/varnish/default.vcl and make sure the backend section is like this:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}

Now edit /etc/default/varnish and set the DAEMON_OPTS variable like this:

DAEMON_OPTS="-a :80 \
                -T localhost:6082 \
                -f /etc/varnish/default.vcl \
                -p thread_pools=4 \
                -p thread_pool_max=1500 \
                -p listen_depth=2048 \
                -p lru_interval=1800 \
                -h classic,169313 \
                -p max_restarts=6 \
                -p connect_timeout=600 \
                -p send_timeout=2000 \
                -s malloc,2G"

Restart Varnish:

/etc/init.d/varnish restart

Setup MySQL

MySQL is already configured. You only need to log slow queries (that is often usefult during slow load page investigation). Todo it, open /etc/mysql/my.cnf and decomment the following two lines:

log_slow_queries       = /var/log/mysql/mysql-slow.log
long_query_time = 2

Configure Shorewall firewall rules

Copy the default configuration for one interface:

cp /usr/share/doc/shorewall/examples/one-interface/interfaces /etc/shorewall/interfaces
cp /usr/share/doc/shorewall/examples/one-interface/policy /etc/shorewall/policy
cp /usr/share/doc/shorewall/examples/one-interface/rules /etc/shorewall/rules
cp /usr/share/doc/shorewall/examples/one-interface/zones /etc/shorewall/zones

Now open /etc/shorewall/policy file and change the line:

net             all             DROP            info

removing info directive given it fills the system logs:

net             all             DROP

Now open /etc/shorewall/rules and add the following rules at the bottom of the file:

HTTP/ACCEPT     net             $FW
SSH/ACCEPT      net             $FW
FTP/ACCEPT      net             $FW

# real apache since varnish listens on port 80
#ACCEPT         net             $FW             tcp             8080

NOTE: in case you want to allow ICMP (Ping) traffic from a specific remote hosts you need to add a rule similar to the following where xxx.xxx.xxx.xxx is the remote IP address, before the Ping(DROP) rule:

Ping(ACCEPT)    net:xxx.xxx.xxx.xxx       $FW

Now edit /etc/default/shorewall and change startup=0 to startup=1 You are now ready to start the firewall:

/etc/init.d/shorewall start

Setup Postfix

Stop postfix server:

/etc/init.d/postfix stop

Edit /etc/mailname and set your server domain name, for example:

server1.mycompany.com

Then, in order to monitor mail traffic coming from PHP you need to edit /etc/php5/apache2/php.ini. Go to [mail function] section and set the following two options:

sendmail_path = /usr/local/bin/sendmail-wrapper
auto_prepend_file = /usr/local/bin/env.php

Now create the two files above:

sendmail-wrapper:

#!/bin/sh
logger -p mail.info sendmail-wrapper.sh: site=${HTTP_HOST}, client=${REMOTE_ADDR}, script=${SCRIPT_NAME}, pwd=${PWD}, uid=${UID}, user=$(whoami)
/usr/sbin/sendmail -t -i $*

env.php:

<?php
putenv("HTTP_HOST=".@$_SERVER["HTTP_HOST"]);
putenv("SCRIPT_NAME=".@$_SERVER["SCRIPT_NAME"]);
putenv("SCRIPT_FILENAME=".@$_SERVER["SCRIPT_FILENAME"]);
putenv("DOCUMENT_ROOT=".@$_SERVER["DOCUMENT_ROOT"]);
putenv("REMOTE_ADDR=".@$_SERVER["REMOTE_ADDR"]);
?>

Now make both executable:

chmod +x /usr/local/bin/sendmail-wrapper
chmod +x /usr/local/bin/env.php

Restart Postfix:

/etc/init.d/postfix start

Prepare environment

Create all needed directories and files

mkdir /root/cron_scripts
mkdir -p /var/www/vhosts
mkdir -p /etc/vsftpd/users
touch /etc/vsftpd/passwd

Now download all tools to manage the server locally:

wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ADD_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/ALIAS_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DEL_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/DOMAIN_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/MYSQL_CREATE.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/UPDATE_ALL_FTP_PASSWORD.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/UPDATE_FTP_PASSWORD.sh
chmod 770 *.sh

Download also the tools that will be used with cron:

cd /root/cron_scripts
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/cron_scripts/backup_mysql.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/LAMP/cron_scripts/mysql_optimize.sh
chmod 770 *.sh
  • Edit /root/ADD_DOMAIN.sh and change ADMIN_EMAIL variable with your email address.
  • Edit /root/MYSQL_CREATE.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.
  • Edit /root/cron_scripts/backup_mysql.sh and change the variable DB_PASSWORD with your MySQL root password and MAIL_NOTIFICATION with your email address.
  • Edit /root/cron_scripts/mysql_optimize.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.

Configure CRON

Edit /etc/crontab and add the following lines at the bottom:

# mysql optimize tables
3  4  *  *  7   root    /root/mysql_optimize.sh

# mysql backup
32 4  *  *  *   root    /root/backup_mysql.sh

How to backup MySQL database using shell and cron

Given I did it dozens of times and everytime I have rewritten the code form scratch, I decided to write a simple script to backup all MySQL databases separately in order to avoid to always reinvent the wheel.

The following script must be configured with:

  • the MySQL root user.
  • the password of MySQL root user.
  • the email address to receive the notifications in case of failures.
  • the destination folder of the backups.
  • the number of copies to keep before overwriting the old backup.

NOTE: given the MySQL root password is in clear is important to limit the access to the script:

chown root.root mysql_backup.sh
chmod 660 mysql_backup.sh

Then, to set a cronjob to do it automatically every day, open /etc/crontab and add the following line at the bottom:

# mysql backup
32 4  *  *  *   root    /root/backup_mysql.sh

How to cross compile CURL library with SSL and ZLIB support

post_image

Every time I have to cross compile a new application or library it is always painful and I often have to spend several minutes (hours) to figure out how to build it. In this case I am going to show you how to cross compile CURL library (and application) with SSL and ZLIB support for an embedded system based on ARM.

First of all I suppose you already have all includes and libraries (static and dynamic) of openSSL and Zlib somewhere in your system (how to cross-compile openssl and zlib is out of the scope of this post).

In my case I have this structure:

.
├── openssl
│   ├── apps
│   │   └── openssl
│   ├── include
│   │   └── openssl
│   │       ├── aes.h
│   │       ├── asn1.h
│   │       ├── asn1_mac.h
│   │       ├── asn1t.h
│   │       ├── bio.h
│   │       ├── blowfish.h
│   │       ├── bn.h
│   │       ├── buffer.h
│   │       ├── camellia.h
│   │       ├── cast.h
│   │       ├── cmac.h
│   │       ├── cms.h
│   │       ├── comp.h
│   │       ├── conf_api.h
│   │       ├── conf.h
│   │       ├── crypto.h
│   │       ├── des.h
│   │       ├── des_old.h
│   │       ├── dh.h
│   │       ├── dsa.h
│   │       ├── dso.h
│   │       ├── dtls1.h
│   │       ├── ebcdic.h
│   │       ├── ecdh.h
│   │       ├── ecdsa.h
│   │       ├── ec.h
│   │       ├── engine.h
│   │       ├── e_os2.h
│   │       ├── err.h
│   │       ├── evp.h
│   │       ├── hmac.h
│   │       ├── idea.h
│   │       ├── krb5_asn.h
│   │       ├── kssl.h
│   │       ├── lhash.h
│   │       ├── md4.h
│   │       ├── md5.h
│   │       ├── mdc2.h
│   │       ├── modes.h
│   │       ├── objects.h
│   │       ├── obj_mac.h
│   │       ├── ocsp.h
│   │       ├── opensslconf.h
│   │       ├── opensslv.h
│   │       ├── ossl_typ.h
│   │       ├── pem2.h
│   │       ├── pem.h
│   │       ├── pkcs12.h
│   │       ├── pkcs7.h
│   │       ├── pqueue.h
│   │       ├── rand.h
│   │       ├── rc2.h
│   │       ├── rc4.h
│   │       ├── ripemd.h
│   │       ├── rsa.h
│   │       ├── safestack.h
│   │       ├── seed.h
│   │       ├── sha.h
│   │       ├── srp.h
│   │       ├── srtp.h
│   │       ├── ssl23.h
│   │       ├── ssl2.h
│   │       ├── ssl3.h
│   │       ├── ssl.h
│   │       ├── stack.h
│   │       ├── symhacks.h
│   │       ├── tls1.h
│   │       ├── ts.h
│   │       ├── txt_db.h
│   │       ├── ui_compat.h
│   │       ├── ui.h
│   │       ├── whrlpool.h
│   │       ├── x509.h
│   │       ├── x509v3.h
│   │       └── x509_vfy.h
│   └── libs
│       ├── lib4758cca.so
│       ├── libaep.so
│       ├── libatalla.so
│       ├── libcapi.so
│       ├── libchil.so
│       ├── libcrypto.a
│       ├── libcrypto.so
│       ├── libcrypto.so.1.0.0
│       ├── libcswift.so
│       ├── libgmp.so
│       ├── libgost.so
│       ├── libnuron.so
│       ├── libpadlock.so
│       ├── libssl.a
│       ├── libssl.so
│       ├── libssl.so.1.0.0
│       ├── libsureware.so
│       └── libubsec.so
└── zlib
    ├── include
    │   ├── zconf.h
    │   └── zlib.h
    └── libs
        ├── libz.a
        ├── libz.so -> libz.so.1.2.8
        ├── libz.so.1 -> libz.so.1.2.8
        └── libz.so.1.2.8

Now download the last version of CURL, decompress and configure it:

$ wget http://curl.haxx.se/download/curl-7.37.1.tar.gz
$ tar xzf curl-7.37.1.tar.gz
$ export ROOTDIR="${PWD}"
$ cd curl-7.37.1/
$ export CROSS_COMPILE="arm-none-linux-gnueabi"
$ export CPPFLAGS="-I${ROOTDIR}/openssl/include -I${ROOTDIR}/zlib/include"
$ export LDFLAGS="-L${ROOTDIR}/openssl/libs -L${ROOTDIR}/zlib/libs"
$ export AR=${CROSS_COMPILE}-ar
$ export AS=${CROSS_COMPILE}-as
$ export LD=${CROSS_COMPILE}-ld
$ export RANLIB=${CROSS_COMPILE}-ranlib
$ export CC=${CROSS_COMPILE}-gcc
$ export NM=${CROSS_COMPILE}-nm
$ export LIBS="-lssl -lcrypto"
$ ./configure --prefix=${ROOTDIR}/build --target=${CROSS_COMPILE} --host=${CROSS_COMPILE} --build=i586-pc-linux-gnu --with-ssl --with-zlib

At the end of the configure you should see a configuration resume similar to the following:

  curl version:     7.37.1
  Host setup:       arm-none-linux-gnueabi
  Install prefix:   /tmp/working_copy/build
  Compiler:         arm-none-linux-gnueabi-gcc
  SSL support:      enabled (OpenSSL)
  SSH support:      no      (--with-libssh2)
  zlib support:     enabled
  GSS-API support:  no      (--with-gssapi)
  SPNEGO support:   no      (--with-spnego)
  TLS-SRP support:  enabled
  resolver:         default (--enable-ares / --enable-threaded-resolver)
  ipv6 support:     no      (--enable-ipv6)
  IDN support:      no      (--with-{libidn,winidn})
  Build libcurl:    Shared=yes, Static=yes
  Built-in manual:  enabled
  --libcurl option: enabled (--disable-libcurl-option)
  Verbose errors:   enabled (--disable-verbose)
  SSPI support:     no      (--enable-sspi)
  ca cert bundle:   no
  ca cert path:     no
  LDAP support:     no      (--enable-ldap / --with-ldap-lib / --with-lber-lib)
  LDAPS support:    no      (--enable-ldaps)
  RTSP support:     enabled
  RTMP support:     no      (--with-librtmp)
  metalink support: no      (--with-libmetalink)
  HTTP2 support:    disabled (--with-nghttp2)
  Protocols:        DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMTP SMTPS TELNET TFTP

Now compile and install it in the previously configured build directory:

$ make
$ make install

Now in the parent directory you should have a new folder called build with the new curl library and application with OpenSSL and Zlib support cross compiled for ARM:

build/
├── bin
│   ├── curl
│   └── curl-config
├── include
│   └── curl
│       ├── curlbuild.h
│       ├── curl.h
│       ├── curlrules.h
│       ├── curlver.h
│       ├── easy.h
│       ├── mprintf.h
│       ├── multi.h
│       ├── stdcheaders.h
│       └── typecheck-gcc.h
└── lib
    ├── libcurl.a
    ├── libcurl.la
    ├── libcurl.so -> libcurl.so.4.3.0
    ├── libcurl.so.4 -> libcurl.so.4.3.0
    ├── libcurl.so.4.3.0
    └── pkgconfig
        └── libcurl.pc

(I voluntary omitted the share folder in the tree exploded).


How to protect wp-content/uploads folder in Wordpress and avoid spam

Usually it is not a problem with Wordpress itself but sometimes we install lot of plugins that come from not well known origins or that are buggy and they can compromise the entire server. So, after dozens of server maintanance sessions, I am going to summarize all the best practice I found.

  • If you have direct control of the admin area of WP you can restrict the filesystem permissions of uploads folder to only user and group:
chmod o-w wp-content/uploads

Remember that in this way you are not able to upload files from the admin area if the web server runs with other privileges.

  • Check the origin of all plugins and make sure to keep your WP installation up to date. In fact attackers often use the last vulnerabilities to attack your server!
  • Use a different user for each domain. This is a general best practice because if an attacker haks your website, he will not be able to access to all other websites in the same server with the same credentials.
  • Deny the usage of scripts in uploads folder using a special .htaccess file placed in wp-content/uploads/

This third rule is very important and saved me bad headaches.


How to use SysBench on Debian, CentOS and Ubuntu

Sysbench tutorial

I need to test the performance of cloudatcost before deciding to purchase a big dog 3 server. Fortunately a guy gave me access to his Big Dog 3 server for 24 hours so that I can test it before buying. So I decided to compare stats between my laptop, the chepest cloudatcost plan that I previously purchased and the Big Dog 3 plan.

My Laptop:

Configuration: Distribution: Ubuntu 14.04 @ 64 bit RAM: 8GB HD: 750 GB hybrid (8GB SSD) CPU: Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz (8 core)

  1. Install sysbench:

    matteo@margot:~$ sudo apt-get install sysbench
    matteo@margot:~$ sysbench --version
    sysbench 0.4.12
    
  2. CPU benchmark:

    matteo@margot:~$ sudo sysbench --test=cpu --cpu-max-prime=20000 run
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          28.1221s
        total number of events:              10000
        total time taken by event execution: 28.1210
        per-request statistics:
             min:                                  2.76ms
             avg:                                  2.81ms
             max:                                 10.89ms
             approx.  95 percentile:               2.88ms
    
    Threads fairness:
        events (avg/stddev):           10000.0000/0.00
        execution time (avg/stddev):   28.1210/0.00
    
  3. I/O benchmark:

    matteo@margot:~$ sysbench --test=fileio --file-total-size=15G prepare
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    128 files, 122880Kb each, 15360Mb total
    Creating files for the test...
    matteo@margot:~$ sysbench --test=fileio --file-total-size=15G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from timer.
    
    
    Extra file open flags: 0
    128 files, 120Mb each
    15Gb total file size
    Block size 16Kb
    Number of random requests for random IO: 0
    Read/Write ratio for combined random IO test: 1.50
    Periodic FSYNC enabled, calling fsync() each 100 requests.
    Calling fsync() at the end of test, Enabled.
    Using synchronous I/O mode
    Doing random r/w test
    Threads started!
    Time limit exceeded, exiting...
    Done.
    
    Operations performed:  12720 Read, 8480 Write, 27014 Other = 48214 Total
    Read 198.75Mb  Written 132.5Mb  Total transferred 331.25Mb  (1.1041Mb/sec)
       70.66 Requests/sec executed
    
    Test execution summary:
        total time:                          300.0118s
        total number of events:              21200
        total time taken by event execution: 158.6743
        per-request statistics:
             min:                                  0.00ms
             avg:                                  7.48ms
             max:                                 68.36ms
             approx.  95 percentile:              26.58ms
    
    Threads fairness:
        events (avg/stddev):           21200.0000/0.00
        execution time (avg/stddev):   158.6743/0.00
    
    matteo@margot:~$ sudo sysbench --test=fileio --file-total-size=15G cleanup
    [sudo] password for matteo: 
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Removing test files...
    

cloudatcost.com - Developer 1 plan:

Configuration: Distribution: Debian 7.6 @ 64 bit RAM: 512MB ECC CPU: 1 Xeon vCPU HD: 10GB SSD

  1. Install sysbench:

    root@debian:~# apt-get install sysbench
    root@debian:~# sysbench --version
    sysbench 0.4.12
    
  2. CPU benchmark:

    root@debian:~# sysbench --test=cpu --cpu-max-prime=20000 run
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    
    Doing CPU performance benchmark
    
    Threads started!
    Done.
    
    Maximum prime number checked in CPU test: 20000
    
    
    Test execution summary:
        total time:                          29.8126s
        total number of events:              10000
        total time taken by event execution: 29.8099
        per-request statistics:
             min:                                  2.93ms
             avg:                                  2.98ms
             max:                                  5.08ms
             approx.  95 percentile:               3.04ms
    
    Threads fairness:
        events (avg/stddev):           10000.0000/0.00
        execution time (avg/stddev):   29.8099/0.00
    
  3. I/O benchmark:

    root@debian:~# sysbench --test=fileio --file-total-size=5G prepare
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    128 files, 40960Kb each, 5120Mb total
    Creating files for the test...
    root@debian:~# sysbench --test=fileio --file-total-size=5G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    Initializing random number generator from timer.
    
    
    Extra file open flags: 0
    128 files, 40Mb each
    5Gb total file size
    Block size 16Kb
    Number of random requests for random IO: 0
    Read/Write ratio for combined random IO test: 1.50
    Periodic FSYNC enabled, calling fsync() each 100 requests.
    Calling fsync() at the end of test, Enabled.
    Using synchronous I/O mode
    Doing random r/w test
    Threads started!
    Time limit exceeded, exiting...
    Done.
    
    Operations performed:  38731 Read, 25820 Write, 82560 Other = 147111 Total
    Read 605.17Mb  Written 403.44Mb  Total transferred 1008.6Mb  (3.3386Mb/sec)
      213.67 Requests/sec executed
    
    Test execution summary:
        total time:                          302.1012s
        total number of events:              64551
        total time taken by event execution: 180.8488
        per-request statistics:
             min:                                  0.00ms
             avg:                                  2.80ms
             max:                               3856.24ms
             approx.  95 percentile:               1.28ms
    
    Threads fairness:
        events (avg/stddev):           64551.0000/0.00
        execution time (avg/stddev):   180.8488/0.00
    
    root@debian:~# sysbench --test=fileio --file-total-size=5G cleanup
    sysbench 0.4.12:  multi-threaded system evaluation benchmark
    
    Removing test files...
    

cloudatcost.com - Big Dog 3 plan:

Configuration: Distribution: CentOS 6.5 @ 64 bit RAM: 8GB ECC CPU: 8 Xeon vCPU HD: 80GB SSD

  1. Install sysbench:

    root@localhost [~]# wget http://www.lefred.be/files/sysbench-0.5-3.el6_.x86_64.rpm
    root@localhost [~]# rpm -ivh sysbench-0.5-3.el6_.x86_64.rpm 
    root@localhost [~]# sysbench --version
    sysbench 0.5
    
  2. CPU benchmark:

    root@localhost [~]# sysbench --test=cpu --cpu-max-prime=20000 run
    sysbench 0.5:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    Random number generator seed is 0 and will be ignored
    
    
    Primer numbers limit: 20000
    
    Threads started!
    
    
    General statistics:
        total time:                          31.0954s
        total number of events:              10000
        total time taken by event execution: 31.0830s
        response time:
             min:                                  3.07ms
             avg:                                  3.11ms
             max:                                  5.86ms
             approx.  95 percentile:               3.17ms
    
    Threads fairness:
        events (avg/stddev):           10000.0000/0.00
        execution time (avg/stddev):   31.0830/0.00
    
  3. I/O benchmark:

    root@localhost [~]# sysbench --test=fileio --file-total-size=15G prepare
    sysbench 0.5:  multi-threaded system evaluation benchmark
    
    128 files, 122880Kb each, 15360Mb total
    Creating files for the test...
    Extra file open flags: 0
    Creating file test_file.0
    Creating file test_file.1
    Creating file test_file.2
    Creating file test_file.3
    Creating file test_file.4
    Creating file test_file.5
    Creating file test_file.6
    Creating file test_file.7
    Creating file test_file.8
    Creating file test_file.9
    Creating file test_file.10
    Creating file test_file.11
    Creating file test_file.12
    Creating file test_file.13
    Creating file test_file.14
    Creating file test_file.15
    Creating file test_file.16
    Creating file test_file.17
    Creating file test_file.18
    Creating file test_file.19
    Creating file test_file.20
    Creating file test_file.21
    Creating file test_file.22
    Creating file test_file.23
    Creating file test_file.24
    Creating file test_file.25
    Creating file test_file.26
    Creating file test_file.27
    Creating file test_file.28
    Creating file test_file.29
    Creating file test_file.30
    Creating file test_file.31
    Creating file test_file.32
    Creating file test_file.33
    Creating file test_file.34
    Creating file test_file.35
    Creating file test_file.36
    Creating file test_file.37
    Creating file test_file.38
    Creating file test_file.39
    Creating file test_file.40
    Creating file test_file.41
    Creating file test_file.42
    Creating file test_file.43
    Creating file test_file.44
    Creating file test_file.45
    Creating file test_file.46
    Creating file test_file.47
    Creating file test_file.48
    Creating file test_file.49
    Creating file test_file.50
    Creating file test_file.51
    Creating file test_file.52
    Creating file test_file.53
    Creating file test_file.54
    Creating file test_file.55
    Creating file test_file.56
    Creating file test_file.57
    Creating file test_file.58
    Creating file test_file.59
    Creating file test_file.60
    Creating file test_file.61
    Creating file test_file.62
    Creating file test_file.63
    Creating file test_file.64
    Creating file test_file.65
    Creating file test_file.66
    Creating file test_file.67
    Creating file test_file.68
    Creating file test_file.69
    Creating file test_file.70
    Creating file test_file.71
    Creating file test_file.72
    Creating file test_file.73
    Creating file test_file.74
    Creating file test_file.75
    Creating file test_file.76
    Creating file test_file.77
    Creating file test_file.78
    Creating file test_file.79
    Creating file test_file.80
    Creating file test_file.81
    Creating file test_file.82
    Creating file test_file.83
    Creating file test_file.84
    Creating file test_file.85
    Creating file test_file.86
    Creating file test_file.87
    Creating file test_file.88
    Creating file test_file.89
    Creating file test_file.90
    Creating file test_file.91
    Creating file test_file.92
    Creating file test_file.93
    Creating file test_file.94
    Creating file test_file.95
    Creating file test_file.96
    Creating file test_file.97
    Creating file test_file.98
    Creating file test_file.99
    Creating file test_file.100
    Creating file test_file.101
    Creating file test_file.102
    Creating file test_file.103
    Creating file test_file.104
    Creating file test_file.105
    Creating file test_file.106
    Creating file test_file.107
    Creating file test_file.108
    Creating file test_file.109
    Creating file test_file.110
    Creating file test_file.111
    Creating file test_file.112
    Creating file test_file.113
    Creating file test_file.114
    Creating file test_file.115
    Creating file test_file.116
    Creating file test_file.117
    Creating file test_file.118
    Creating file test_file.119
    Creating file test_file.120
    Creating file test_file.121
    Creating file test_file.122
    Creating file test_file.123
    Creating file test_file.124
    Creating file test_file.125
    Creating file test_file.126
    Creating file test_file.127
    16106127360 bytes written in 1661.93 seconds (9.24 MB/sec).
    
    root@localhost [~]# sysbench --test=fileio --file-total-size=15G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
    sysbench 0.5:  multi-threaded system evaluation benchmark
    
    Running the test with following options:
    Number of threads: 1
    Random number generator seed is 0 and will be ignored
    
    
    Extra file open flags: 0
    128 files, 120Mb each
    15Gb total file size
    Block size 16Kb
    Number of IO requests: 0
    Read/Write ratio for combined random IO test: 1.50
    Periodic FSYNC enabled, calling fsync() each 100 requests.
    Calling fsync() at the end of test, Enabled.
    Using synchronous I/O mode
    Doing random r/w test
    Threads started!
    
    Operations performed:  45662 reads, 30441 writes, 97408 Other = 173511 Total
    Read 713.47Mb  Written 475.64Mb  Total transferred 1.1612Gb  (3.9637Mb/sec)
      253.68 Requests/sec executed
    
    General statistics:
        total time:                          300.0017s
        total number of events:              76103
        total time taken by event execution: 31.1080s
        response time:
             min:                                  0.00ms
             avg:                                  0.41ms
             max:                                 65.85ms
             approx.  95 percentile:               2.54ms
    
    Threads fairness:
        events (avg/stddev):           76103.0000/0.00
        execution time (avg/stddev):   31.1080/0.00
    
    root@localhost [~]# sysbench --test=fileio --file-total-size=15G cleanup
    sysbench 0.5:  multi-threaded system evaluation benchmark
    
    Removing test files...
    

Network benchmark:

In order to test also the Network performance I use the speedtest-cli tool. But I do this test only in the Big Dog 3 server.

root@localhost [~]# wget -q -O - https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py | python
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from KW Datacenter (167.88.44.169)...
Selecting best server based on latency...
Hosted by Source Cable Ltd (Hamilton, ON) [52.90 km]: 22.833 ms
Testing download speed........................................
Download: 236.30 Mbits/s
Testing upload speed..................................................
Upload: 148.09 Mbits/s

Conclusions:

I finally decided to purchase a Big Dog 2 plan from CloudAtCost and as far as I experimented the performance is not so bad (for the moment)... The only problem I see is that sometimes the network seems missing some packets as long as some sporatic stucks of 2/3 seconds when accessing the filesystem from a SSH connection.


How to directly print from web browser

Fortunately the web browsers have been created to protect the client environment from executing client processes and accessing client resources. However sometimes this limitation is too strong and does not let us to do some simple things like direct print a document without opening an intermediate print pop-up.

I searched in Google for a long time and experimented several tests before summarizing all possibilities I found. There is no a single way to achieve it and every option has some pros and cons depending on the environment and conditions you have.

Direct printing using a system call

Requirements:

  • A web server installed (I only consider a Linux server).
  • A printer directly connected to the web server.

Pros:

  • You have the full control of the printer parameters.
  • This option allow you to use every browser.
  • You can print a file or a document without creating a web layout for it.

Cons:

  • You can use the application only on the same PC that holds the web server (and the printer).
  • At least you can install the web application in the company server with a Network printer.

Details: The idea here is to have a web server running for example Apache and PHP (but every other server-side language would be OK). On the same PC a printer is connected and configured using CUPS (on Linux). When a user clicks a button or a link in the web page, a PHP script executes a system() call to lpr that creates one ore more printer jobs depending on the number of documents passed.

The function above accepts a string of documents (space separated), sends a printer command and then polls the printer queue (the spooler) using lpd command and wait for the printer to return ready. Just for reference, the above function is a snippet of code I used to print barcodes with a Dymo LabelWrite 450 Turbo and 99012 paper labels.

Direct printing using browser options

Requirements:

  • A compatible web browser (Firefox or Chrome).
  • Direct access to the user browser.

Pros:

  • You don't need to have a web server running on the same machine.
  • The web application can be remote.
  • You can print what you see in the browser using just HTML and CSS.

Cons:

  • You need to create a web layout for a document with HTML and CSS.
  • You don't have access to all printer options.
  • Limited number of supported browsers.
  • You must have access to the web browser to change configuration.

Details: In this case we have to modify the preferences of the web browser and/or execute it in a particular mode (like kiosk mode for example). As you know, every browser has a different configuration... From the web side we can use the Javascript window.print() function. Use the following instructions based on the web browser you are using:

  1. Chrome You need to have Chrome version 18.0.1 or higher. From Chrome, open a new tab and type info:config then make sure the Disable Print Preview flag is NOT enabled (Print preview must be enabled for Kiosk Printing to work). Now set your application the default page to open when Chrome launches and then close the browser. Now you have to execute Chrome in kiosk mode. To do it just append --kiosk --kiosk-printing to the chrome executable. Chrome should now start in kiosk mode (full-screen) and from here you should be able to print directly from the browser.

  2. Firefox In Firefox open a new tab and type about:config in the address bar. Then right click on the white space and select NEW > BOOLEAN and in the text edit that appears type print.always_print_silent, hit OK and then select true. This procedure writes a line to prefs.js file (on Windows) or user.js file on Linux and the next time you start the browser, any Javascript print(); function will print directly to the printer using the currently configured print settings.

You can use the following test page to test the direct printing using browser options: