HTTP/HTTPS GET and POST requests in Python3 including file upload with builtin modules

26 May 2016

Matteo Mattei

The following script performs GET and POST requests to [httpbin.org] using only builtin python 3 modules. There is also a class to support file encoding for upload.

	#!/usr/bin/env python3
	# This script performs a GET and a POST request through HTTP/HTTPS using
	# builtin python3 moudules. There is also a class to encode files for upload!

	import urllib.request
	import http.client
	import mimetypes
	import codecs
	import uuid
	import binascii
	import io
	import os
	import sys

	REMOTE_PROTOCOL = 'http' # or https
	REMOTE_HOST = 'httpbin.org'
	REMOTE_PORT = '80' # 443 for https
	USERAGENT = 'MySuperUserAgent'

	class MultipartFormdataEncoder(object):
	def __init__(self):
	self.boundary = uuid.uuid4().hex
	self.content_type = 'multipart/form-data; boundary={}'.format(self.boundary)

	@classmethod
	def u(cls, s):
	if sys.hexversion < 0x03000000 and isinstance(s, str):
	s = s.decode('utf-8')
	if sys.hexversion >= 0x03000000 and isinstance(s, bytes):
	s = s.decode('utf-8')
	return s

	def iter(self, fields, files):
	"""
	fields is a sequence of (name, value) elements for regular form fields.
	files is a sequence of (name, filename, file-type) elements for data to be uploaded as files
	Yield body's chunk as bytes
	"""
	encoder = codecs.getencoder('utf-8')
	for (key, value) in fields:
	key = self.u(key)
	yield encoder('--{}\r\n'.format(self.boundary))
	yield encoder(self.u('Content-Disposition: form-data; name="{}"\r\n').format(key))
	yield encoder('\r\n')
	if isinstance(value, int) or isinstance(value, float):
	value = str(value)
	yield encoder(self.u(value))
	yield encoder('\r\n')
	for (key, filename, fpath) in files:
	key = self.u(key)
	filename = self.u(filename)
	yield encoder('--{}\r\n'.format(self.boundary))
	yield encoder(self.u('Content-Disposition: form-data; name="{}"; filename="{}"\r\n').format(key, filename))
	yield encoder('Content-Type: {}\r\n'.format(mimetypes.guess_type(filename)[0] or 'application/octet-stream'))
	yield encoder('\r\n')
	with open(fpath,'rb') as fd:
	buff = fd.read()
	yield (buff, len(buff))
	yield encoder('\r\n')
	yield encoder('--{}--\r\n'.format(self.boundary))

	def encode(self, fields, files):
	body = io.BytesIO()
	for chunk, chunk_len in self.iter(fields, files):
	body.write(chunk)
	return self.content_type, body.getvalue()


	if len(sys.argv) < 2:
	sys.stderr.write('Please pass a file as argument\n')
	sys.exit(1)

	if not os.path.isfile(sys.argv[1]):
	sys.stderr.write('%s is not a valid file\n' % sys.argv[1])
	sys.exit(1)


	# SIMPLE HTTP GET
	req = urllib.request.Request(REMOTE_PROTOCOL+'://'+REMOTE_HOST+':'+REMOTE_PORT+'/get',headers={'User-Agent':USERAGENT})
	with urllib.request.urlopen(req,timeout=10) as f:
	buf = f.read(300).decode('utf-8')
	print("GET RESPONSE")
	print(buf)

	# POST FIELDS AND FILES
	fields = [('param1','paramvalue1'),('param2','paramversion2'),('param3','paramvalue3')]
	files = [('fieldname',os.path.basename(sys.argv[1]),sys.argv[1])]
	content_type, body = MultipartFormdataEncoder().encode(fields, files)
	h = None
	if REMOTE_PROTOCOL == 'http':
	h = http.client.HTTPConnection(REMOTE_HOST,REMOTE_PORT)
	else:
	h = http.client.HTTPSConnection(REMOTE_URL,REMOTE_PORT)
	headers = {
	'User-Agent': USERAGENT,
	'Content-Type': content_type
	}
	h.request('POST', '/post', body, headers)
	res = h.getresponse()
	if not res.status in (200,201):
	sys.stderr.write(res.read().decode('utf-8')+'\n')
	sys.exit(1)
	print("POST RESPONSE")
	print(res.read().decode('utf-8'))

view raw test_http.py hosted with ❤ by GitHub

Cross compile wget statically for ARM

15 Apr 2016

Matteo Mattei

arm wget compile

1 Comment

The following script can be used to statically cross compile wget for ARM.

Requirements:

You need openssl and zlib already present in the current $ROOTPATH directory with related libraries and included respectively inside libs and include folders.
You need a glibc compiled with --enable-static-nss flag so that getaddrinfo and gethostbyname cannot complain at link time.

	#!/bin/bash

	VERSION="1.17"
	if [ ! -f wget-${VERSION}.tar.xz ]; then
	wget http://ftp.gnu.org/gnu/wget/wget-${VERSION}.tar.xz
	fi

	rm -rf wget-${VERSION} build
	tar xJf wget-${VERSION}.tar.xz

	export ROOTDIR="${PWD}"
	cd wget-${VERSION}
	export CROSS_COMPILE="arm-none-linux-gnueabi"
	export CPPFLAGS="-I${ROOTDIR}/openssl/include -I${ROOTDIR}/zlib/include"
	export LDFLAGS="-L${ROOTDIR}/openssl/libs -L${ROOTDIR}/zlib/libs"
	export AR=${CROSS_COMPILE}-ar
	export AS=${CROSS_COMPILE}-as
	export LD=${CROSS_COMPILE}-ld
	export RANLIB=${CROSS_COMPILE}-ranlib
	export CC=${CROSS_COMPILE}-gcc
	export NM=${CROSS_COMPILE}-nm
	export LIBS="-static -lc -lssl -lcrypto -lz -ldl"

	./configure \
	--prefix=${ROOTDIR}/build \
	--target=${CROSS_COMPILE} \
	--host=${CROSS_COMPILE} \
	--build=i586-pc-linux-gnu \
	--with-ssl=openssl --with-zlib \
	--without-included-regex \
	--enable-nls \
	--enable-dependency-tracking \
	--with-metalink \
	--sysconfdir=${ROOTDIR}/build/etc \
	--localedir=${ROOTDIR}/build/usr/share/locale \
	--mandir=${ROOTDIR}/build/usr/share/man \
	--infodir=${ROOTDIR}/build/usr/share/info \
	--bindir=${ROOTDIR}/build/usr/bin

	make
	make install

view raw wget_build.sh hosted with ❤ by GitHub

The resulting binary will be placed into the build folder.

Share internet connection to the the LAN, protect everything with a firewall and setup dhcp server on Linux.

22 Feb 2016

Matteo Mattei

server networking firewall shorewall dnsmasq

0 Comments

I have a router with a public static ip address provided by the ISP and I need to share internet access to all pc in the LAN. To do it I need a server with two ethernet interfaces (eth0 and eth1) that will act as a firewall and dhcp server. That server will also be used as a web server to publish some contents in the LAN and in internet.

My configuration is this:

eth0 with public static IP address provided by the ISP.
eth1 with private static IP address assigned by me and connected to a switch.

Install shorewall:

apt-get install shorewall

Start configuration with two interface shorewall example:

cd /usr/share/doc/shorewall/examples/two-interfaces/
cp interfaces /etc/shorewall/
cp masq /etc/shorewall/
cp policy /etc/shorewall/
cp rules /etc/shorewall/
cp zones /etc/shorewall/
cp stoppedrules /etc/shorewall/

Now configure /etc/network/interfaces (I am using Debian jessie):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address xxx.xxx.xxx.xxx
netmask yyy.yyy.yyy.yyy
gateway zzz.zzz.zzz.zzz

auto eth1
iface eth1 inet static
address 192.168.0.1
netmask 255.255.255.0

where:

xxx.xxx.xxx.xxx is the public IP address provided by the ISP
yyy.yyy.yyy.yyy is the netmask provided by the ISP
zzz.zzz.zzz.zzz is the gateway provided by the ISP

And /etc/resolv.conf:

# Google DNS
nameserver 8.8.8.8
nameserver 8.8.4.4

Now configure shorewall to act as firewall and share internet to all LAN devices.

File: /etc/shorewall/interfaces

#ZONE   INTERFACE       OPTIONS
net     eth0            dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
loc     eth1            dhcp,tcpflags,nosmurfs,routefilter,logmartians

File: /etc/shorewall/zones

#ZONE   TYPE    OPTIONS                 IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall
net     ipv4
loc     ipv4

File: /etc/shorewall/policy

#SOURCE         DEST            POLICY          LOG LEVEL       LIMIT:BURST

$FW             net             ACCEPT
$FW             loc             ACCEPT
loc             net             ACCEPT
net             all             DROP            info
# THE FOLLOWING POLICY MUST BE LAST
all             all             REJECT          info

File: /etc/shorewall/rules

?SECTION ALL
?SECTION ESTABLISHED
?SECTION RELATED
?SECTION INVALID
?SECTION UNTRACKED
?SECTION NEW

#       Don't allow connection pickup from the net
#
Invalid(DROP)   net             all             tcp
#
#       Accept DNS connections from the firewall to the network
#
DNS(ACCEPT)     $FW             net
DNS(ACCEPT)     loc             $FW
#
#       Accept SSH connections from the local network for administration
#
SSH(ACCEPT)     loc             $FW
#
#       Allow Ping from the local network
#
Ping(ACCEPT)    loc             $FW

#
# Drop Ping from the "bad" net zone.. and prevent your log from being flooded..
#

Ping(DROP)      net             $FW

ACCEPT          $FW             loc             icmp
ACCEPT          $FW             net             icmp
#

# custom rules
SSH(ACCEPT)     net             $FW
Web(ACCEPT)     net             $FW
Web(ACCEPT)     loc             $FW

# DNAT rules (useful for natting a service on a device)
# this rule opens port 8080 from internet to port 80 of 192.168.0.2 in TCP
#DNAT            net             loc:192.168.0.2:80     tcp     8080

The above configuration allows SSH connections from local and from remote as well as Web access.

File: /etc/shorewall/masq

#INTERFACE:DEST         SOURCE          ADDRESS         PROTO   PORT(S) IPSEC   MARK    USER/   SWITCH  ORIGINAL
#                                                                                       GROUP           DEST
eth0                    eth1

File: /etc/shorewall/stoppedrules

#ACTION         SOURCE          DEST            PROTO   DEST            SOURCE
#                                                       PORT(S)         PORT(S)
ACCEPT          eth1            -
ACCEPT          -               eth1

Now edit /etc/shorewall/shorewall.conf and set:

STARTUP_ENABLED=Yes
IP_FORWARDING=On

In particular the last line is necessary to share internet to the LAN. Edit now /etc/default/shorewall and set:

startup=1

Now start shorewall:

/etc/init.d/shorewall start

And try to configure a pc in the LAN with a static ip address, for example:

address: 192.168.0.200
netmask: 255.255.255.0
gateway: 192.168.0.1

Yeah, the pc should be able to access internet! But we need to go a little bit ahead because given we don’t want to assign a static ip address to all pc (or devices) in the LAN. So we have to install a DHCP server.

apt-get install dnsmasq

Now backup the default configuration of dnsmasq and create a new /etc/dnsmasq.conf with something like this:

interface=eth1
dhcp-range=192.168.0.50,192.168.0.150,12h
dhcp-host=60:E3:27:8D:88:64,accesspoint,192.168.0.10
dhcp-host=F8:32:E5:84:3A:1F,pc1,192.168.0.11
dhcp-host=F8:32:E5:84:2D:71,pc2,192.168.0.12
dhcp-host=F8:32:E5:84:2D:B6,pc3,192.168.0.13

The first line specifies the interface where the DHCP server is running (eth1 for me1). The second line (dhcp-range) is the range of IP addresses that the DHCP server will provide with a lease of 12 hours. In this case from address 192.168.0.50 to address 192.168.0.150. All other lines are used to define devices with static IPs. The syntax is:

dhcp-host=DEVICE_MAC_ADDRESS,DEVICE_NAME,DEVICE_IP

Dnsmasq beyond dhcp and DNS caching provides also another interesting feature… every entry you set in /etc/hosts of the server is automatically forwarded to all devices in the LAN. This means that if we want to set a simple name for accessing the webserver from the LAN it’s just a matter of editing /etc/hosts and add the server’s name:

127.0.0.1       localhost
192.168.0.1     myserver

This allows all devices in the LAN to access the server using myserver.

Now restart dnsmasq:

/etc/init.d/dnsmasq restart

To list all devices that have received a new IP address the file to look at is /var/lib/misc/dnsmasq.leases

Resources:

Full web server setup with Debian 8 (Jessie)

20 Feb 2016

Matteo Mattei

linux server mysql varnish debian php iptables postfix

9 Comments

Setup bash and update the system

cp /etc/skel/.bashrc /root/.bashrc
apt-get update
apt-get dist-upgrade

Configure hostname correctly

Make sure to have the following two lines (with the same format) at the top of your /etc/hosts file

127.0.0.1       localhost.localdomain localhost
xxx.xxx.xxx.xxx web1.myserver.com web1

Note: xxx.xxx.xxx.xxx is the public IP address assigned to your server.

Install all needed packages

apt-get install php5 mysql-server mysql-client apache2 iptables phpmyadmin varnish shorewall vsftpd php5-cli php5-curl php5-dev php5-gd php5-imagick php5-imap php5-memcache php5-pspell php5-recode php5-sqlite php5-tidy php5-xcache php5-xmlrpc php-pear php-xml-rpc postfix apg ca-certificates heirloom-mailx

MySQL/PhpMyAdmin:

mysql root password: xxx
repeat mysql root password: xxx
web server to reconfigure automatically: apache2
configure database for phpmyadmin with dbconfig-common? Yes
Password of the database’s administrative user: xxx
Password for phpmyadmin: xxx
Password confirmation: xxx

Postfix:

Select Internet Site
System mail name: (insert here the FQDN, for example web1.myserver.com)

Setup FTP

Stop VSFTP server:

/etc/init.d/vsftpd stop

Create backup configuration:

mv /etc/vsftpd.conf /etc/vsftpd.conf.backup

Add new configuration:

listen=YES
listen_port=21
anonymous_enable=NO
local_enable=YES
guest_enable=YES
guest_username=nobody
user_sub_token=$USER
local_root=/var/www/vhosts/$USER
virtual_use_local_privs=YES
user_config_dir=/etc/vsftpd/users
pam_service_name=vsftpd_local_and_virtual
chroot_local_user=YES
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd/chroot_list
ftpd_banner=Welcome to my ftp server
write_enable=YES
download_enable=YES
dirlist_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
xferlog_file=/var/log/xferlog
connect_from_port_20=YES
connect_timeout=60
data_connection_timeout=300
idle_session_timeout=300
local_max_rate=0
max_clients=0
max_per_ip=3

Create an empty chroot_list file:

mkdir /etc/vsftpd
touch /etc/vsftpd/chroot_list

Install PAM module for virtual users:

apt-get install libpam-pwdfile

And configure it creating the file /etc/pam.d/vsftpd_local_and_virtual with this content:

# Standard behaviour for ftpd(8).
auth    required        pam_listfile.so item=user sense=deny file=/etc/ftpusers onerr=succeed

# first try to authenticate local users
auth    sufficient      pam_unix.so

# if that failed, login with virtual user
auth    required        pam_pwdfile.so  pwdfile /etc/vsftpd/passwd

# pam_pwdfile doesn't come with account, so we just permit on success
account required        pam_permit.so

Start VSFTP server:

/etc/init.d/vsftpd start

Setup Apache

Stop Apache web server:

/etc/init.d/apache2 stop

Backup Apache configuration:

cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.backup

Edit the following lines in /etc/apache2/apache2.conf

From Timeout 300 to Timeout 45
From KeepAliveTimeout 5 to KeepAliveTimeout 15

Edit /etc/apache2/mods-enabled/mpm_prefork.conf:

<IfModule mpm_prefork_module>
        StartServers             5
        MinSpareServers          5
        MaxSpareServers          10
        MaxRequestWorkers        150
        MaxConnectionsPerChild   10000
</IfModule>

Edit /etc/apache2/ports.conf and change the port 80 with 8080 since we are going to use Varnish:

Listen 8080

Change the port (from 80 to 8080) also in the default virtual host /etc/apache2/sites-enabled/000-default.conf Now restart Apache:

/etc/init.d/apache2 restart

Setup Varnish

Stop Varnish daemon:

/etc/init.d/varnish stop

Open /etc/varnish/default.vcl and make sure the backend section is like this:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}

Now edit /etc/default/varnish and set the DAEMON_OPTS variable like this:

DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"

Now we have to make some changes also to systemd scripts (this step is mandatory for Debian Jessie!) since systemd does not consider /etc/default/varnish settings:

cp /lib/systemd/system/varnish.service /etc/systemd/system/

Edit /etc/systemd/system/varnish.service and change port 6081 with port 80:

[Unit]
Description=Varnish HTTP accelerator

[Service]
Type=forking
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl
ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/reload-vcl

[Install]
WantedBy=multi-user.target

Restart Varnish:

systemctl daemon-reload
systemctl restart varnish.service

Setup MySQL

Correct the MySQL configuration warning:

sed -i "{s/^key_buffer/key_buffer_size/g}" /etc/mysql/my.cnf

Enable MySQL slow query logging (often useful during slow page load debugging):

sed -i "{s/^#slow_query_log_file /slow_query_log_file /g}" /etc/mysql/my.cnf
sed -i "{s/^#slow_query_log /slow_query_log /g}" /etc/mysql/my.cnf
sed -i "{s/^#long_query_time /long_query_time /g}" /etc/mysql/my.cnf
sed -i "{s/^#log_queries_not_using_indexes/log_queries_not_using_indexes/g}" /etc/mysql/my.cnf

MySQL is now configured, so restart it:

/etc/init.d/mysql restart

Configure Shorewall firewall rules

Copy the default configuration for one interface:

cd /usr/share/doc/shorewall/examples/one-interface
cp interfaces /etc/shorewall/
cp policy /etc/shorewall/
cp rules /etc/shorewall/
cp zones /etc/shorewall/

Now open /etc/shorewall/policy file and change the line:

net             all             DROP            info

removing info directive given it fills the system logs:

net             all             DROP

Now open /etc/shorewall/rules and add the following rules at the bottom of the file:

HTTP/ACCEPT     net             $FW
SSH/ACCEPT      net             $FW
FTP/ACCEPT      net             $FW

# real apache since varnish listens on port 80
#ACCEPT         net             $FW             tcp             8080

NOTE: in case you want to allow ICMP (Ping) traffic from a specific remote hosts you need to add a rule similar to the following where xxx.xxx.xxx.xxx is the remote IP address, before the Ping(DROP) rule:

Ping(ACCEPT)    net:xxx.xxx.xxx.xxx       $FW

Now edit /etc/default/shorewall and change startup=0 to startup=1 You are now ready to start the firewall:

/etc/init.d/shorewall start

Setup Postfix

Stop postfix server:

/etc/init.d/postfix stop

Edit /etc/mailname and set your server domain name, for example:

server1.mycompany.com

Then, in order to monitor mail traffic coming from PHP you need to edit /etc/php5/apache2/php.ini. Go to [mail function] section and set the following two options:

sendmail_path = /usr/local/bin/sendmail-wrapper
auto_prepend_file = /usr/local/bin/env.php

Now create the two files above in /usr/local/bin:

sendmail-wrapper:

#!/bin/sh
logger -p mail.info sendmail-wrapper.sh: site=${HTTP_HOST}, client=${REMOTE_ADDR}, script=${SCRIPT_NAME}, pwd=${PWD}, uid=${UID}, user=$(whoami)
/usr/sbin/sendmail -t -i $*

env.php:

<?php
putenv("HTTP_HOST=".@$_SERVER["HTTP_HOST"]);
putenv("SCRIPT_NAME=".@$_SERVER["SCRIPT_NAME"]);
putenv("SCRIPT_FILENAME=".@$_SERVER["SCRIPT_FILENAME"]);
putenv("DOCUMENT_ROOT=".@$_SERVER["DOCUMENT_ROOT"]);
putenv("REMOTE_ADDR=".@$_SERVER["REMOTE_ADDR"]);
?>

Now make they both have executable flag:

chmod +x /usr/local/bin/sendmail-wrapper
chmod +x /usr/local/bin/env.php

Add also /usr/local/bin/ to the open_basedir php list in /etc/apache2/conf-enabled/phpmyadmin.conf

php_admin_value open_basedir /usr/share/phpmyadmin/:/etc/phpmyadmin/:/var/lib/phpmyadmin/:/usr/local/bin/

Restart Postfix:

/etc/init.d/postfix start

Prepare environment

Create all needed directories and files

mkdir /root/cron_scripts
mkdir -p /var/www/vhosts
mkdir -p /etc/vsftpd/users
touch /etc/vsftpd/passwd

Now download all tools to manage the server locally:

wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ADD_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/ALIAS_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_ALIAS.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_DOMAIN.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DEL_FTP_VIRTUAL_USER.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/DOMAIN_LIST.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/MYSQL_CREATE.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/UPDATE_ALL_FTP_PASSWORD.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/UPDATE_FTP_PASSWORD.sh
chmod 770 *.sh

Download also the tools that will be used with cron:

cd /root/cron_scripts
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/cron_scripts/backup_mysql.sh
wget https://raw.githubusercontent.com/matteomattei/servermaintenance/master/Debian7/LAMP_24/cron_scripts/mysql_optimize.sh
chmod 770 *.sh

Edit /root/ADD_DOMAIN.sh and change ADMIN_EMAIL variable with your email address.
Edit /root/MYSQL_CREATE.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.
Edit /root/cron_scripts/backup_mysql.sh and change the variable DB_PASSWORD with your MySQL root password and MAIL_NOTIFICATION with your email address.
Edit /root/cron_scripts/mysql_optimize.sh and change the variable MYSQL_ROOT_PASSWORD with your MySQL root password.

Configure CRON

Edit /etc/crontab and add the following lines at the bottom:

# mysql optimize tables
3  4  *  *  7   root    /root/mysql_optimize.sh

# mysql backup
32 4  *  *  *   root    /root/backup_mysql.sh

How to build a web kiosk with Raspberry Pi and make the SD read-only.

16 Feb 2016

Matteo Mattei

kiosk raspberry

48 Comments

Download latest Raspbian Lite distribution (Jessie in my case) from the official Raspberry website.

Then unzip the file and flash the image in the SD card:

sudo dd if=2016-02-09-raspbian-jessie-lite.img of=/dev/mmcblk0 bs=1M

Power up the RPi2 (or RPi1) with the SD plugged in and log in:

user: pi password: raspberry

Now configure the network (wired static ip in my case) editing /etc/dhcpcd.conf and adding the following lines at the bottom:

# my custom static settings
interface eth0
static ip_address=192.168.1.130/24
static routers=192.168.1.254
static domain_name_servers=192.168.1.254

Configure Kiosk

Execute raspi-config to cutomize some settings:

sudo raspi-config

In particular,

Set localization (keyboard and system locale)
Enable SSH
Expand root partition
Set autologin on console (B2)

At the end reboot your raspberry pi so that the new filesystem size will take effect.

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install midori matchbox-window-manager xserver-xorg x11-xserver-utils unclutter xinit

Now add tty group to pi user because pi needs to handle /dev/ttyX devices and adjust permissions accordingly at every system startup:

gpasswd -a pi tty
sed -i '/^exit 0/c\chmod g+rw /dev/tty?\nexit 0' /etc/rc.local

Now create a startup script:

vi /home/pi/startkiosk.sh

#!/bin/bash

# disable DPMS (Energy Star) features.
xset -dpms

# disable screen saver
xset s off

# don't blank the video device
xset s noblank

# disable mouse pointer
unclutter &

# run window manager
matchbox-window-manager -use_cursor no -use_titlebar no  &

# run browser
midori -e Fullscreen -a http://www.google.com

and make it executable:

chmod +x /home/pi/startkiosk.sh

Then add the following lines at the end of /home/pi/.bashrc:

if [ -z "${SSH_TTY}" ]; then
  xinit ~/startkiosk.sh
fi

This three lines allow starting X only when we login as pi directly but not from SSH.

Make the SD read-only

Backup your current /etc/fstab and create a new one with the following content:

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    ro                0       2
/dev/mmcblk0p2  /               ext4    ro                0       1
tmpfs           /tmp            tmpfs   defaults,noatime,mode=1777      0       0
tmpfs           /var/log        tmpfs   defaults,noatime,mode=0755      0       0
tmpfs           /var/lib/systemd tmpfs   defaults,noatime,mode=0755      0       0
tmpfs           /run            tmpfs   defaults,noatime,mode=0755      0       0

Then create a little script to help you to change the read-write/read-only mode of the filesystem everytime you need:

vi /home/pi/mountfs.sh

#!/bin/bash

case "${1}" in
        rw)
                sudo mount -o remount,rw /
                echo "Filesystem mounted in READ-WRITE mode"
                ;;
        ro)
                sudo mount -o remount,ro /
                echo "Filesystem mounted in READ-ONLY mode"
                ;;
        *)
                if [ -n "$(mount | grep mmcblk0p2 | grep -o 'rw')" ]
                then
                        echo "Filesystem is mounted in READ-WRITE mode"
                else
                        echo "Filesystem is mounted in READ-ONLY mode"
                fi
                echo "Usage ${0} [rw|ro]"
                ;;
esac

That’s it, reboot your RPi and enjoy your safe web kiosk!

How to create self contained executables in Python

09 Dec 2015

Matteo Mattei

python

0 Comments

This is a very quick guide on how to create self contained Python executables (for all platforms). First of all install PyInstaller (I am using pip3 because I work with Python 3.x):

sudo pip3 install pyinstaller

Now install upx for a better compression:

sudo apt-get install upx

Now you are ready to create your self-contained executable:

pyinstaller \
    --onefile \
    --noconfirm \
    --noconsole \
    --clean \
    --log-level=WARN \
    --key=MySuperSecretPassword \
    --strip \
    myscript.py

Resulting executable will be placed inside dist folder and it will be called myscript. This is what each parameter does:

--onefile allows to create a single self contained binary.
--noconfirm replaces output directory without asking for confirmation.
--noconsole should be used in GUI application with no console. --console should be used otherwise.
--clean cleans PyInstaller cache and remove temporary files.
--log-level=WARN shows only warnings and errors during build.
--key=yourkey uses the given key to encrypt the Python bytecode (yes it’s secure!).
--strip removes debug information to executable and shared libraries.

For windows, if you want to add also the icon to the resulting exe file you can add this additional parameter:

--icon-file=myapplication.ico

The application have only to be recompiled on every platform you want to release your application for. I know, the resulting binary will be a little heavy (~24MB for a PySide GUI application on Linux) but we have to consider that it contains the interpreter itself and all the needed libraries!!!

Older Newer