21 Aug 2014 
Matteo Mattei 
linux embedded curl 
Every time I have to cross compile a new application or library it is always painful and I often have to spend several minutes (hours) to figure out how to build it. In this case I am going to show you how to cross compile CURL library (and application) with SSL and ZLIB support for an embedded system based on ARM.
First of all I suppose you already have all includes and libraries (static and dynamic) of openSSL and Zlib somewhere in your system (how to cross-compile openssl and zlib is out of the scope of this post).
In my case I have this structure:
.
├── openssl
│ ├── apps
│ │ └── openssl
│ ├── include
│ │ └── openssl
│ │ ├── aes.h
│ │ ├── asn1.h
│ │ ├── asn1_mac.h
│ │ ├── asn1t.h
│ │ ├── bio.h
│ │ ├── blowfish.h
│ │ ├── bn.h
│ │ ├── buffer.h
│ │ ├── camellia.h
│ │ ├── cast.h
│ │ ├── cmac.h
│ │ ├── cms.h
│ │ ├── comp.h
│ │ ├── conf_api.h
│ │ ├── conf.h
│ │ ├── crypto.h
│ │ ├── des.h
│ │ ├── des_old.h
│ │ ├── dh.h
│ │ ├── dsa.h
│ │ ├── dso.h
│ │ ├── dtls1.h
│ │ ├── ebcdic.h
│ │ ├── ecdh.h
│ │ ├── ecdsa.h
│ │ ├── ec.h
│ │ ├── engine.h
│ │ ├── e_os2.h
│ │ ├── err.h
│ │ ├── evp.h
│ │ ├── hmac.h
│ │ ├── idea.h
│ │ ├── krb5_asn.h
│ │ ├── kssl.h
│ │ ├── lhash.h
│ │ ├── md4.h
│ │ ├── md5.h
│ │ ├── mdc2.h
│ │ ├── modes.h
│ │ ├── objects.h
│ │ ├── obj_mac.h
│ │ ├── ocsp.h
│ │ ├── opensslconf.h
│ │ ├── opensslv.h
│ │ ├── ossl_typ.h
│ │ ├── pem2.h
│ │ ├── pem.h
│ │ ├── pkcs12.h
│ │ ├── pkcs7.h
│ │ ├── pqueue.h
│ │ ├── rand.h
│ │ ├── rc2.h
│ │ ├── rc4.h
│ │ ├── ripemd.h
│ │ ├── rsa.h
│ │ ├── safestack.h
│ │ ├── seed.h
│ │ ├── sha.h
│ │ ├── srp.h
│ │ ├── srtp.h
│ │ ├── ssl23.h
│ │ ├── ssl2.h
│ │ ├── ssl3.h
│ │ ├── ssl.h
│ │ ├── stack.h
│ │ ├── symhacks.h
│ │ ├── tls1.h
│ │ ├── ts.h
│ │ ├── txt_db.h
│ │ ├── ui_compat.h
│ │ ├── ui.h
│ │ ├── whrlpool.h
│ │ ├── x509.h
│ │ ├── x509v3.h
│ │ └── x509_vfy.h
│ └── libs
│ ├── lib4758cca.so
│ ├── libaep.so
│ ├── libatalla.so
│ ├── libcapi.so
│ ├── libchil.so
│ ├── libcrypto.a
│ ├── libcrypto.so
│ ├── libcrypto.so.1.0.0
│ ├── libcswift.so
│ ├── libgmp.so
│ ├── libgost.so
│ ├── libnuron.so
│ ├── libpadlock.so
│ ├── libssl.a
│ ├── libssl.so
│ ├── libssl.so.1.0.0
│ ├── libsureware.so
│ └── libubsec.so
└── zlib
├── include
│ ├── zconf.h
│ └── zlib.h
└── libs
├── libz.a
├── libz.so -> libz.so.1.2.8
├── libz.so.1 -> libz.so.1.2.8
└── libz.so.1.2.8
Now download the last version of CURL, decompress and configure it:
$ wget http://curl.haxx.se/download/curl-7.37.1.tar.gz
$ tar xzf curl-7.37.1.tar.gz
$ export ROOTDIR="${PWD}"
$ cd curl-7.37.1/
$ export CROSS_COMPILE="arm-none-linux-gnueabi"
$ export CPPFLAGS="-I${ROOTDIR}/openssl/include -I${ROOTDIR}/zlib/include"
$ export LDFLAGS="-L${ROOTDIR}/openssl/libs -L${ROOTDIR}/zlib/libs"
$ export AR=${CROSS_COMPILE}-ar
$ export AS=${CROSS_COMPILE}-as
$ export LD=${CROSS_COMPILE}-ld
$ export RANLIB=${CROSS_COMPILE}-ranlib
$ export CC=${CROSS_COMPILE}-gcc
$ export NM=${CROSS_COMPILE}-nm
$ export LIBS="-lssl -lcrypto"
$ ./configure --prefix=${ROOTDIR}/build --target=${CROSS_COMPILE} --host=${CROSS_COMPILE} --build=i586-pc-linux-gnu --with-ssl --with-zlib
At the end of the configure you should see a configuration resume similar to the following:
curl version: 7.37.1
Host setup: arm-none-linux-gnueabi
Install prefix: /tmp/working_copy/build
Compiler: arm-none-linux-gnueabi-gcc
SSL support: enabled (OpenSSL)
SSH support: no (--with-libssh2)
zlib support: enabled
GSS-API support: no (--with-gssapi)
SPNEGO support: no (--with-spnego)
TLS-SRP support: enabled
resolver: default (--enable-ares / --enable-threaded-resolver)
ipv6 support: no (--enable-ipv6)
IDN support: no (--with-{libidn,winidn})
Build libcurl: Shared=yes, Static=yes
Built-in manual: enabled
--libcurl option: enabled (--disable-libcurl-option)
Verbose errors: enabled (--disable-verbose)
SSPI support: no (--enable-sspi)
ca cert bundle: no
ca cert path: no
LDAP support: no (--enable-ldap / --with-ldap-lib / --with-lber-lib)
LDAPS support: no (--enable-ldaps)
RTSP support: enabled
RTMP support: no (--with-librtmp)
metalink support: no (--with-libmetalink)
HTTP2 support: disabled (--with-nghttp2)
Protocols: DICT FILE FTP FTPS GOPHER HTTP HTTPS IMAP IMAPS POP3 POP3S RTSP SMTP SMTPS TELNET TFTP
Now compile and install it in the previously configured build directory:
Now in the parent directory you should have a new folder called build with the new curl library and application with OpenSSL and Zlib support cross compiled for ARM:
build/
├── bin
│ ├── curl
│ └── curl-config
├── include
│ └── curl
│ ├── curlbuild.h
│ ├── curl.h
│ ├── curlrules.h
│ ├── curlver.h
│ ├── easy.h
│ ├── mprintf.h
│ ├── multi.h
│ ├── stdcheaders.h
│ └── typecheck-gcc.h
└── lib
├── libcurl.a
├── libcurl.la
├── libcurl.so -> libcurl.so.4.3.0
├── libcurl.so.4 -> libcurl.so.4.3.0
├── libcurl.so.4.3.0
└── pkgconfig
└── libcurl.pc
(I voluntary omitted the share folder in the tree exploded).
19 Aug 2014 Matteo Mattei linux wordpress email postfix security server spam apache Usually it is not a problem with Wordpress itself but sometimes we install lot of plugins that come from not well known origins or that are buggy and they can compromise the entire server. So, after dozens of server maintanance sessions, I am going to summarize all the best practice I found.
- If you have direct control of the admin area of WP you can restrict the filesystem permissions of uploads folder to only user and group:
chmod o-w wp-content/uploads
Remember that in this way you are not able to upload files from the admin area if the web server runs with other privileges.
- Check the origin of all plugins and make sure to keep your WP installation up to date. In fact attackers often use the last vulnerabilities to attack your server!
- Use a different user for each domain. This is a general best practice because if an attacker haks your website, he will not be able to access to all other websites in the same server with the same credentials.
- Deny the usage of scripts in uploads folder using a special .htaccess file placed in wp-content/uploads/
This third rule is very important and saved me bad headaches.
06 Aug 2014 Matteo Mattei cloudatcost.com sysbench benchmark performance Sysbench tutorial
I need to test the performance of cloudatcost before deciding to purchase a big dog 3 server. Fortunately a guy gave me access to his Big Dog 3 server for 24 hours so that I can test it before buying. So I decided to compare stats between my laptop, the chepest cloudatcost plan that I previously purchased and the Big Dog 3 plan.
My Laptop:
Configuration:
Distribution: Ubuntu 14.04 @ 64 bit
RAM: 8GB
HD: 750 GB hybrid (8GB SSD)
CPU: Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz (8 core)
-
Install sysbench:
matteo@margot:~$ sudo apt-get install sysbench
matteo@margot:~$ sysbench --version
sysbench 0.4.12
-
CPU benchmark:
matteo@margot:~$ sudo sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Doing CPU performance benchmark
Threads started!
Done.
Maximum prime number checked in CPU test: 20000
Test execution summary:
total time: 28.1221s
total number of events: 10000
total time taken by event execution: 28.1210
per-request statistics:
min: 2.76ms
avg: 2.81ms
max: 10.89ms
approx. 95 percentile: 2.88ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 28.1210/0.00
-
I/O benchmark:
matteo@margot:~$ sysbench --test=fileio --file-total-size=15G prepare
sysbench 0.4.12: multi-threaded system evaluation benchmark
128 files, 122880Kb each, 15360Mb total
Creating files for the test...
matteo@margot:~$ sysbench --test=fileio --file-total-size=15G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.
Extra file open flags: 0
128 files, 120Mb each
15Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 12720 Read, 8480 Write, 27014 Other = 48214 Total
Read 198.75Mb Written 132.5Mb Total transferred 331.25Mb (1.1041Mb/sec)
70.66 Requests/sec executed
Test execution summary:
total time: 300.0118s
total number of events: 21200
total time taken by event execution: 158.6743
per-request statistics:
min: 0.00ms
avg: 7.48ms
max: 68.36ms
approx. 95 percentile: 26.58ms
Threads fairness:
events (avg/stddev): 21200.0000/0.00
execution time (avg/stddev): 158.6743/0.00
matteo@margot:~$ sudo sysbench --test=fileio --file-total-size=15G cleanup
[sudo] password for matteo:
sysbench 0.4.12: multi-threaded system evaluation benchmark
Removing test files...
Configuration:
Distribution: Debian 7.6 @ 64 bit
RAM: 512MB ECC
CPU: 1 Xeon vCPU
HD: 10GB SSD
-
Install sysbench:
root@debian:~# apt-get install sysbench
root@debian:~# sysbench --version
sysbench 0.4.12
-
CPU benchmark:
root@debian:~# sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Doing CPU performance benchmark
Threads started!
Done.
Maximum prime number checked in CPU test: 20000
Test execution summary:
total time: 29.8126s
total number of events: 10000
total time taken by event execution: 29.8099
per-request statistics:
min: 2.93ms
avg: 2.98ms
max: 5.08ms
approx. 95 percentile: 3.04ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 29.8099/0.00
-
I/O benchmark:
root@debian:~# sysbench --test=fileio --file-total-size=5G prepare
sysbench 0.4.12: multi-threaded system evaluation benchmark
128 files, 40960Kb each, 5120Mb total
Creating files for the test...
root@debian:~# sysbench --test=fileio --file-total-size=5G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
sysbench 0.4.12: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.
Extra file open flags: 0
128 files, 40Mb each
5Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 38731 Read, 25820 Write, 82560 Other = 147111 Total
Read 605.17Mb Written 403.44Mb Total transferred 1008.6Mb (3.3386Mb/sec)
213.67 Requests/sec executed
Test execution summary:
total time: 302.1012s
total number of events: 64551
total time taken by event execution: 180.8488
per-request statistics:
min: 0.00ms
avg: 2.80ms
max: 3856.24ms
approx. 95 percentile: 1.28ms
Threads fairness:
events (avg/stddev): 64551.0000/0.00
execution time (avg/stddev): 180.8488/0.00
root@debian:~# sysbench --test=fileio --file-total-size=5G cleanup
sysbench 0.4.12: multi-threaded system evaluation benchmark
Removing test files...
Configuration:
Distribution: CentOS 6.5 @ 64 bit
RAM: 8GB ECC
CPU: 8 Xeon vCPU
HD: 80GB SSD
-
Install sysbench:
root@localhost [~]# wget http://www.lefred.be/files/sysbench-0.5-3.el6_.x86_64.rpm
root@localhost [~]# rpm -ivh sysbench-0.5-3.el6_.x86_64.rpm
root@localhost [~]# sysbench --version
sysbench 0.5
-
CPU benchmark:
root@localhost [~]# sysbench --test=cpu --cpu-max-prime=20000 run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Random number generator seed is 0 and will be ignored
Primer numbers limit: 20000
Threads started!
General statistics:
total time: 31.0954s
total number of events: 10000
total time taken by event execution: 31.0830s
response time:
min: 3.07ms
avg: 3.11ms
max: 5.86ms
approx. 95 percentile: 3.17ms
Threads fairness:
events (avg/stddev): 10000.0000/0.00
execution time (avg/stddev): 31.0830/0.00
-
I/O benchmark:
root@localhost [~]# sysbench --test=fileio --file-total-size=15G prepare
sysbench 0.5: multi-threaded system evaluation benchmark
128 files, 122880Kb each, 15360Mb total
Creating files for the test...
Extra file open flags: 0
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
Creating file test_file.5
Creating file test_file.6
Creating file test_file.7
Creating file test_file.8
Creating file test_file.9
Creating file test_file.10
Creating file test_file.11
Creating file test_file.12
Creating file test_file.13
Creating file test_file.14
Creating file test_file.15
Creating file test_file.16
Creating file test_file.17
Creating file test_file.18
Creating file test_file.19
Creating file test_file.20
Creating file test_file.21
Creating file test_file.22
Creating file test_file.23
Creating file test_file.24
Creating file test_file.25
Creating file test_file.26
Creating file test_file.27
Creating file test_file.28
Creating file test_file.29
Creating file test_file.30
Creating file test_file.31
Creating file test_file.32
Creating file test_file.33
Creating file test_file.34
Creating file test_file.35
Creating file test_file.36
Creating file test_file.37
Creating file test_file.38
Creating file test_file.39
Creating file test_file.40
Creating file test_file.41
Creating file test_file.42
Creating file test_file.43
Creating file test_file.44
Creating file test_file.45
Creating file test_file.46
Creating file test_file.47
Creating file test_file.48
Creating file test_file.49
Creating file test_file.50
Creating file test_file.51
Creating file test_file.52
Creating file test_file.53
Creating file test_file.54
Creating file test_file.55
Creating file test_file.56
Creating file test_file.57
Creating file test_file.58
Creating file test_file.59
Creating file test_file.60
Creating file test_file.61
Creating file test_file.62
Creating file test_file.63
Creating file test_file.64
Creating file test_file.65
Creating file test_file.66
Creating file test_file.67
Creating file test_file.68
Creating file test_file.69
Creating file test_file.70
Creating file test_file.71
Creating file test_file.72
Creating file test_file.73
Creating file test_file.74
Creating file test_file.75
Creating file test_file.76
Creating file test_file.77
Creating file test_file.78
Creating file test_file.79
Creating file test_file.80
Creating file test_file.81
Creating file test_file.82
Creating file test_file.83
Creating file test_file.84
Creating file test_file.85
Creating file test_file.86
Creating file test_file.87
Creating file test_file.88
Creating file test_file.89
Creating file test_file.90
Creating file test_file.91
Creating file test_file.92
Creating file test_file.93
Creating file test_file.94
Creating file test_file.95
Creating file test_file.96
Creating file test_file.97
Creating file test_file.98
Creating file test_file.99
Creating file test_file.100
Creating file test_file.101
Creating file test_file.102
Creating file test_file.103
Creating file test_file.104
Creating file test_file.105
Creating file test_file.106
Creating file test_file.107
Creating file test_file.108
Creating file test_file.109
Creating file test_file.110
Creating file test_file.111
Creating file test_file.112
Creating file test_file.113
Creating file test_file.114
Creating file test_file.115
Creating file test_file.116
Creating file test_file.117
Creating file test_file.118
Creating file test_file.119
Creating file test_file.120
Creating file test_file.121
Creating file test_file.122
Creating file test_file.123
Creating file test_file.124
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
16106127360 bytes written in 1661.93 seconds (9.24 MB/sec).
root@localhost [~]# sysbench --test=fileio --file-total-size=15G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
sysbench 0.5: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Random number generator seed is 0 and will be ignored
Extra file open flags: 0
128 files, 120Mb each
15Gb total file size
Block size 16Kb
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Operations performed: 45662 reads, 30441 writes, 97408 Other = 173511 Total
Read 713.47Mb Written 475.64Mb Total transferred 1.1612Gb (3.9637Mb/sec)
253.68 Requests/sec executed
General statistics:
total time: 300.0017s
total number of events: 76103
total time taken by event execution: 31.1080s
response time:
min: 0.00ms
avg: 0.41ms
max: 65.85ms
approx. 95 percentile: 2.54ms
Threads fairness:
events (avg/stddev): 76103.0000/0.00
execution time (avg/stddev): 31.1080/0.00
root@localhost [~]# sysbench --test=fileio --file-total-size=15G cleanup
sysbench 0.5: multi-threaded system evaluation benchmark
Removing test files...
Network benchmark:
In order to test also the Network performance I use the speedtest-cli tool. But I do this test only in the Big Dog 3 server.
root@localhost [~]# wget -q -O - https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py | python
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from KW Datacenter (167.88.44.169)...
Selecting best server based on latency...
Hosted by Source Cable Ltd (Hamilton, ON) [52.90 km]: 22.833 ms
Testing download speed........................................
Download: 236.30 Mbits/s
Testing upload speed..................................................
Upload: 148.09 Mbits/s
Conclusions:
I finally decided to purchase a Big Dog 2 plan from CloudAtCost and as far as I experimented the performance is not so bad (for the moment)… The only problem I see is that sometimes the network seems missing some packets as long as some sporatic stucks of 2/3 seconds when accessing the filesystem from a SSH connection.
21 Jul 2014 Matteo Mattei print web browser Fortunately the web browsers have been created to protect the client environment from executing client processes and accessing client resources. However sometimes this limitation is too strong and does not let us to do some simple things like direct print a document without opening an intermediate print pop-up.
I searched in Google for a long time and experimented several tests before summarizing all possibilities I found. There is no a single way to achieve it and every option has some pros and cons depending on the environment and conditions you have.
Direct printing using a system call
Requirements:
- A web server installed (I only consider a Linux server).
- A printer directly connected to the web server.
Pros:
- You have the full control of the printer parameters.
- This option allow you to use every browser.
- You can print a file or a document without creating a web layout for it.
Cons:
- You can use the application only on the same PC that holds the web server (and the printer).
- At least you can install the web application in the company server with a Network printer.
Details: The idea here is to have a web server running for example Apache and PHP (but every other server-side language would be OK). On the same PC a printer is connected and configured using CUPS (on Linux). When a user clicks a button or a link in the web page, a PHP script executes a system() call to lpr that creates one ore more printer jobs depending on the number of documents passed.
The function above accepts a string of documents (space separated), sends a printer command and then polls the printer queue (the spooler) using lpd command and wait for the printer to return ready. Just for reference, the above function is a snippet of code I used to print barcodes with a Dymo LabelWrite 450 Turbo and 99012 paper labels.
Direct printing using browser options
Requirements:
- A compatible web browser (Firefox or Chrome).
- Direct access to the user browser.
Pros:
- You don’t need to have a web server running on the same machine.
- The web application can be remote.
- You can print what you see in the browser using just HTML and CSS.
Cons:
- You need to create a web layout for a document with HTML and CSS.
- You don’t have access to all printer options.
- Limited number of supported browsers.
- You must have access to the web browser to change configuration.
Details: In this case we have to modify the preferences of the web browser and/or execute it in a particular mode (like kiosk mode for example). As you know, every browser has a different configuration… From the web side we can use the Javascript window.print() function. Use the following instructions based on the web browser you are using:
-
Chrome You need to have Chrome version 18.0.1 or higher. From Chrome, open a new tab and type info:config then make sure the Disable Print Preview flag is NOT enabled (Print preview must be enabled for Kiosk Printing to work). Now set your application the default page to open when Chrome launches and then close the browser. Now you have to execute Chrome in kiosk mode. To do it just append --kiosk --kiosk-printing to the chrome executable. Chrome should now start in kiosk mode (full-screen) and from here you should be able to print directly from the browser.
-
Firefox In Firefox open a new tab and type about:config in the address bar. Then right click on the white space and select NEW > BOOLEAN and in the text edit that appears type print.always_print_silent, hit OK and then select true. This procedure writes a line to prefs.js file (on Windows) or user.js file on Linux and the next time you start the browser, any Javascript print(); function will print directly to the printer using the currently configured print settings.
You can use the following test page to test the direct printing using browser options:
06 Jun 2014 Matteo Mattei backup centos plesk ftp mysql I am working for a client that needs a system to backup a CentOS server hosted by OVH that uses Plesk. I didn’t know that Plesk already implemented a way to backup all domains and MySQL data so I cerated a script to do that:
As you can see this script keeps the last 7 days of backups. However after googling a while I found that the same thing could be done directly using a Plesk utility called pleskbackup, so I developed an alterlate script that uses it:
I hope it can help you.
07 Jul 2013 Matteo Mattei audio skype ubuntu If you have problems with Skype audio on Ubuntu (distortion, croak, noise…) the simple way to get it fixed is editing the file /etc/pulse/default.pa, change one line and restart the system.
From:
load-module module-udev-detect use_ucm=0
To:
load-module module-udev-detect use_ucm=0 tsched=0
For simplicity you can execute the following command that will do all the job:
sudo sed -i "{s/^load-module module-udev-detect use_ucm=0$/load-module module-udev-detect use_ucm=0 tsched=0/g}" /etc/pulse/default.pa
Restart now the system and the Skype audio should work fine!
