CakePHP New application database config (never forget the enconding)

Today I will put down the way to begin a new web application project using CakePHP.

The first part is database creation. Database charset must be utf-8 and collation: utf8_unicode_ci. The difference between general and unicode collations is explained in an excellent manner in this post on stackoverlow.com

Next, when you start a new CakePHP project the database config class looks like this

public $default = array(
  'datasource' => 'Database/Mysql',
  'persistent' => false,
  'host' => 'localhost',
  'login' => 'user',
  'password' => 'password',
  'database' => 'test_database_name',
  'prefix' => '',
  // 'encoding' => 'utf8',
);

My lesson today is: Before changing anything else, uncomment the last line!.

... in case you do not then Unicode (Greek in my case) text will still appear correctly in the web application, but phpMyAdmin and sqldump will display garbage. In addition searches with non-latin text will always fail. To make matters worse when I discovered the case of the problem, I realised that I would have to re-enter all my test data.

SchemaSync: An effective and simple way to synchronize MySQL databases

I have spend the last couple of days trying to find out a solution for migrating changes from a development database to the productive. I was looking for a clean, simple and reliable solution that (ideally) runs on Linux.

My first attempt was to to use the official MySQL mysqldiff command. It turned out that in my case, the only suggestion I got was to recreate indexes and drop primary keys every time there was a record number mismatch between tables. I also run into Windows GUI's with a 30 day evaluation period and a cost ranging from $90-$150.

My lack changes when I run into Schema Sync written in Python by Mitch Matuson. The page has everything you need and the setup (if you already have python) takes less that a minute.

Quoting from the utility's examples page, Sync the production db with the changes from the development db

 schemasync mysql://user:pass@dev-host:3306/dev_db mysql://user:pass@prod-host:3306/production_db

This will give you two files in your home directory, one to do the patching and one to revert back the changes if requested. A excellent piece of work, that I am very happy to use. Thumbs up for mr Matuson.

Netbeans 7.x and Debian

Switched my laptop from Fedora 19 to Debian 7.1 for some peace, quiet and stability. It appears that the latest transitions to kernel 3.11 and the issues with the nvidia drivers were too much for it. So here I am, enjoying the Debian way of life without yum and gstreamer.

A goodie available only to Debian and Ubuntu users is the Oracle Java PPA available through this link. Andrew at webupd8,org has set up a PPA that downloads and installs the latest Oracle Java packages keeping your computer up to date along with the rest of normal updates.

So now it's Netbeans turn to get installed. I prefer to download the Java SE version and then manually add the PHP and CakePHP plugins that I require for my everyday use.

The first time I tried to install the normal way I ended up with an Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main". A little bit of digging revealed that that one way to install netbeans is to use the --silent installer option. The will install netbeans to the default /usr/local/netbeans-7.x location which is perfectly all right. After that you can run it from your desktop Development menu. So to start the installer just become root and type:

root@nb-thanassis:/home/thanassis/Downloads/Netbeans# ./netbeans-7.4-javase-linux.sh --silent

in a similar manner, in order to uninstall netbeans from your system, go to the installation directory and type

root@nb-thanassis:/usr/local/netbeans-7.4# ./uninstall.sh  --slinent

There is one small thing that I noticed. in order for all this to work you must cd to the same directory as the installer script before issueing any commands. In my case -- hence the blog post -- all other attempts failed.

.

CakePHP locking tables

Here are my two cents on the issue.

The code below is a function from a behaviour that tries to create an additional unique key on a field named code, by counting the number of records created this year. The important part in the locking procedure is that we must specify the AS clause in the LOCK TABLES statement or otherwise the $model->find() function will not work complaining that the table is locked.

    public function getNextCode(&$model)
    {
        $thisYear = date('Y');
        $dbo = $model->getDataSource();
        $dbo->execute(
            sprintf('LOCK TABLES %s AS %s WRITE;',
                $model->table,
                $model->alias
            )
        );
        $recordsThisYear = $model->find(
            'count',
            array(
                'recursive' => -1,
                'conditions' => array(
                    $model->alias .'.code LIKE' => $thisYear.'%'
                )
            )
        );
        $dbo->execute('UNLOCK TABLES');
        return sprintf('%d-%06d', $thisYear, $recordsThisYear + 1);
    }

The original idea for the post and function cake from a doWeb posting available through here.

Using a raspberrypi as an sftp server

Following a previous post regarding how to use your raspberry-pi device as a file server, we are going to continue amd set up sftp service on the same pi device, so that it may be accessible over WAN.

The complete guide comes from a Mark Van den Borre posting available through this link.In our case however the steps are fewer, since raspberry has already the openssh server set up and running and if you have followed from the previous port we already have a user (bill) and a group (microsoft) to use for sftp service.

To get started let;s give our friend Bill a password:

pi@xena ~ $ sudo passwd bill 
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Next step will be to prevent Bill from interactively logging in. The usual remedy to this problem to use the sftp server as a login shell. After the post is over bill will not be able to access our pi from ssh either

pi@xena ~ $ sudo chsh bill 
Changing the login shell for bill
Enter the new value, or press ENTER for the default
        Login Shell [/usr/lib/tftp-server]: /usr/lib/sftp-server
pi@xena ~ $ 

Now for the sftp configuration itself. (Copying, pasting and adjusting from Mark's post we have something like this:) Open the default OpenSSH server configuration for editing:

pi@xena ~ $ sudo vi /etc/ssh/sshd_config

: and change the default sftp server from:

Subsystem sftp /usr/lib/openssh/sftp-server

to

Subsystem sftp internal-sftp

Some users can only use sftp, but not other OpenSSH features like remote login. Let's create a rule for that group of users. Add the following section to the bottom of /etc/ssh/sshd_config:

Match group microsoft
ChrootDirectory /mnt/SFTP-Data
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

Reboot...

CakePHP and AJAX submitting a form with jQuery

A couple of years back I wrote an article about how to handle the dependent drop down lists problem using CakePHP's Ajax facilities. Today i will put down a trivial example of how to submit a CakePHP created form using jQuery as a small reference that can be easily pasted.

Let me remind you of the CakePHP ajax way. You start by creating a controller method that will "return" the ajax content. For our trivial example, the following controller will be more than enough.

class AjaxController extends AppController {
   var $uses = NULL;

   public function helloAjax()
   {
       $this->layout='ajax';
       // result can be anything coming from $this->data
       $result =  'Hello Dolly!';
       $this->set("result", $result);
   }
}

The corresponding view file view/ajax/hello_ajax.ctp should contain just the following:

<?php echo $result; ?>

Setting up our Ajax call is now as easy as, creating a link or a button that will invoke the asynchronous call and then setting the id of the field that will receive the result. A typical setup would be that the link looks something like this :

<a href="#" id="performAjaxLink">Do Ajax </a>

And then the target field can be created using:

<?php echo $this->Form->input('your_field', array('id' => 'resultField')); ?>

Finally a little script at the end of the file ...

<script>
    jQuery("#performAjaxLink").click(
            function()
            {                
                jQuery.ajax({
                    type:'POST',
                    async: true,
                    cache: false,
                    url: '<?= Router::Url(['controller' => 'ajax','admin' => FALSE, 'action' => 'helloAjax'], TRUE); ?>',
                    success: function(response) {
                        jQuery('#resultField').val(response);
                    },
                    data:jQuery('form').serialize()
                });
                return false;
            }
    );
</script>

The jQuery Ajax API is available here.

Using a raspberry-pi as a UPS server with nut

In this post we will try to install the Network UPS tools on a Raspberry-Pi device, attach a USB connected UPS and use it as a UPS server that will allow all machines sharing the same UPS to shut-down correctly when the UPS runs out of power. Our server will look after two clients; one running EL5 and the other openSUSE 11.4.

At the end of the post we will demonstrate how easy it is to set up your clients once the server is up and running and provide additional instructions for setting up the client software on Fedora 18 and EL6.

Update 2014-02-02: Meanwhile things here at the office have changed. The openSUSE machine is now gone and has been replaced by one running Debian 7. I have now revised the client setup guides for Fedora and EL5, 6 and I also have added one for Debian. The openSUSE "howto" is left as is but I can no longer verify if it works or not :) ..

Server Setup

Before we begin I would like to confess that my first attempt to install a no-name made in China UPS resulted to total failure, so eventually I got an expensive APC BackUPS Pro, that worked without any problems from the beginning, so unless your UPS is one supported by the UPS network tools project drivers, don't even try to follow the tutorial.

A second remark, is that if you are following the tutorial as the standard pi user you will need to prefix almost all commands with sudo. To become root on a standard Raspbian and follow along, you will need to issue sudo su-. (Thanks Derek for pointing it out)

root@raspbx:~# apt-get install nut-client nut-server
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libupsclient1
Suggested packages:
  nut-cgi nut-snmp nut-dev nut-xml
The following NEW packages will be installed:
  libupsclient1 nut-client nut-server
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,583 kB of archives.
After this operation, 3,217 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://archive.raspbian.org/raspbian/ wheezy/main libupsclient1 armhf 2.6.4-2.3 [106 kB]
Get:2 http://archive.raspbian.org/raspbian/ wheezy/main nut-client armhf 2.6.4-2.3 [191 kB]
Get:3 http://archive.raspbian.org/raspbian/ wheezy/main nut-server armhf 2.6.4-2.3 [1,286 kB]
Fetched 1,583 kB in 2s (562 kB/s)     
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libupsclient1.
(Reading database ... 37855 files and directories currently installed.)
Unpacking libupsclient1 (from .../libupsclient1_2.6.4-2.3_armhf.deb) ...
Selecting previously unselected package nut-client.
Unpacking nut-client (from .../nut-client_2.6.4-2.3_armhf.deb) ...
Selecting previously unselected package nut-server.
Unpacking nut-server (from .../nut-server_2.6.4-2.3_armhf.deb) ...
Processing triggers for man-db ...
Setting up libupsclient1 (2.6.4-2.3) ...
Setting up nut-client (2.6.4-2.3) ...
[info] nut-client disabled, please adjust the configuration to your needs.
[info] Then set MODE to a suitable value in /etc/nut/nut.conf to enable it.
Setting up nut-server (2.6.4-2.3) ...
[info] nut-server disabled, please adjust the configuration to your needs.
[info] Then set MODE to a suitable value in /etc/nut/nut.conf to enable it.
root@raspbx:~# 

Don't worry about the nut-server information we shall deal with it later on. Now an optional step that will allow us to to use the lsusb utility will be to install the usbutils package, assuming that it is not already there. So:

root@raspbx:~# apt-get install usbutils

.. and then -- blame me for my Windows habits, I firmly suggest a reboot. When the system is back on, we will make sure that out USB device is nιcely plugged in...

root@raspbx:~# lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. 
Bus 001 Device 004: ID 051d:0002 American Power Conversion Uninterruptible Power Supply
root@raspbx:~# 

Our UPS is there so let's set up the driver for the nut server. Open file /etc/nut/ups.conf and append the following lines at the end.

[apc1200]
        driver = usbhid-ups
        port = auto
        desc = "APC Back UPS Pro 1200VA supporting the two network servers"

You can name your ups anything you like, as far as the driver is concerned my advice is to browse through the official Network UPS Tools drivers list.

Setting up the UPS name and driver is not enough. I have not tested this on any other Debian box, but on raspberry-pi we need an extra step in order to create the /var/run/nut folder and set correct permissions to it.

root@raspbx:~# mkdir /var/run/nut
root@raspbx:~# chown root.nut /var/run/nut/
root@raspbx:~# chmod 770 /var/run/nut/

Now we are ready to test the UPS driver.

root@raspbx:~# upsdrvctl start
Network UPS Tools - UPS driver controller 2.6.4
Network UPS Tools - Generic HID driver 0.37 (2.6.4)
USB communication driver 0.31
Using subdriver: APC HID 0.95
root@raspbx:~# 

Our next step will be to configure upsd and upsmon. The network UPS tools design dictates that upsd communicates with the UPS driver that we just started and upsmon communicates with upsd and actually shuts down the machine in the event of a power failure. By providing this extra level of indirection, nut allows for multiple instances of upsmon to run on different machines. That way they can allow share the same physical UPS and this is what we said that we are going to demonstrate in this posting.

So to enable accessing the upsd via the network, edit the file /etc/nut/upsd.conf and place the following LISTEN directives.

LISTEN 127.0.0.1 3493
LISTEN 192.168.1.137 3493

192.168.1.137, is my pi's IP address -- replace that with your own. Next, we will need to add some kind of security and the next file that we will need to tamper with will be the /etc/nut/upsd.users. Edit it with your text editor and set up the following users

[admin]
        password = myadmpass
        actions = SET
        instcmds = ALL

#
# --- Configuring for a user who can execute tests only
#
[testuser]
        password  = pass  
        instcmds  = test.battery.start
        instcmds  = test.battery.stop

#
# --- Configuring for upsmon
#
# To add a user for your upsmon, use this example:
#
[upsmon_local]
        password  = local_pass
        upsmon master
[upsmon_remote]
        password  = remote_pass
        upsmon slave

Finally the local UPS monitor daemon will need to specify the UPS to monitor and the user credentials from upsd.users file. Open the /etc/nut/upsmon.conf file, locate the monitor section and add the following line:

MONITOR apc1200@localhost 1 upsmon_local local_pass master 

The number 1 after the ups name and host is the power value. The man page for upsd states clearly that:

The "current overall power value" is the sum of all UPSes that are currently able to supply power to the system hosting upsmon. Any UPS that is either on line or just on battery contributes to this number. If a UPS is critical (on battery and low battery) or has been put into "forced shutdown" mode, it no longer contributes.
A "power value" on a MONITOR line in the config file is the number of power supplies that the UPS runs on the current system.

Final steps: Open the /etc/nut/nut.conf file and change the value of Mode to netserver -- making sure that there are no spaces between each side of the = sign. (See NOTE at end of file) and issue the following commands:

root@raspbx:/etc/nut# service nut-server start
[ ok ] Starting NUT - power devices information server and drivers:  driver(s). upsd.
root@raspbx:/etc/nut# service nut-client start
[ ok ] Starting NUT - power device monitor and shutdown controller: nut-client.
root@raspbx:/etc/nut# 

As a last check, verify that both services will start automatically on system (using the update-rc.d command) reboot and yes, our server is ready! ...

root@raspbx:~# ps -ef | grep ups
nut       3275     1  0 Apr09 ?        00:08:24 /lib/nut/usbhid-ups -a apc1200
nut       3278     1  0 Apr09 ?        00:00:19 /sbin/upsd
root      3312     1  0 Apr09 ?        00:00:00 /sbin/upsmon
nut       3314  3312  0 Apr09 ?        00:00:09 /sbin/upsmon
root      4721  4711  0 18:44 pts/1    00:00:00 grep ups
root@raspbx:~#

Clients

Client setup requires more or less three things: One will be to edit the nut.conf file and set the mode variable value to netclient. Next will be to place the correct MONITOR line in the upsmon.conf file and the third will be to start the upsmon daemon.

openSUSE

Our first client is an openSUSE 11.4 machine that I keep saying that I must upgrade. To install nut on openSUSE we need to issue the following command as root.

zypper install nut

openSUSE nut stores the configuration files /etc/ups. By the way the file /usr/share/doc/packages/nut/README.SUSE offer excellent detailed and precise information on how to do things right. So to get things started:

• Add the line MODE=netclient at the end of the /etc/ups/nut.conf file
• Add MONITOR apc1200@asterisk "UPS supporting the main Servers" to /etc/hosts.conf
• Comment out any reference to any UPS at the end of /etc/ups/ups.conf
• Add MONITOR apc1200@asterisk 1 upsom_remote remote_pass slave
• Start the service with etc/init.d/upsd start. (The reload option can be used to reread updated configuration files
• Finally change the system config so that the service starts every time you start your system using the following command: chkconfig upsd on

Reboot and verify :

atlas:~ # ps -ef | grep ups
root      2958     1  0 18:06 ?        00:00:00 /usr/sbin/cupsd -C /etc/cups/cupsd.conf
root      3374     1  0 18:06 ?        00:00:00 /usr/sbin/upsmon
upsd      3376  3374  0 18:06 ?        00:00:00 /usr/sbin/upsmon
root      4776  4732  0 18:13 pts/0    00:00:00 grep ups

You might probably want to test the configuration and whether the upsmon daemon can shut-down your server, so go ahead and ...

atlas:~ # upsmon -c fsd
Network UPS Tools upsmon 2.6.0
                                                                               
Broadcast Message from upsd@atlas                                              
        (somewhere) at 13:57 ...                                               
                                                                               
Executing automatic power-fail shutdown                                        
                                                                               
                                                                              

Broadcast message from root@atlas (Wed Apr 10 13:57:06 2013):

The system is going down for system halt NOW!

Debian/Ubuntu

Perhaps the easiest setup is on a Debian system. You only need four steps:

1. Install just the client: sudo apt-get install nut-client.
2. Edit the file/etc/nut/nut.conf and set the mode to netclient. MODE=netclient (mind that there must be no spaces around the equals sign).
3. Add the monitor MONITOR apc900@xena 1 upsom_remote remote_pass slave command in the /etc/nut/upsmon.conf.
4. Restart the nut-client service
   service nut-client restart
5. Update the system to start the service automatically update-rc.d nut-client defaults

Fedora and CentOS versions 5 & 6

Fedora also stores the nut related data in /etc/ups. Again here we need to perform the three steps we mentioned before, but this time we will need to start the upsmon daemon by hand. So to set up our fedora box as a network client:

• Install the software using yum install nut-client
• Add MONITOR apc1200@asterisk 1 upsom_remote remote_pass slave
• Add /usr/sbin/upsmon start in /etc/rc.d/rc.local to verify that the monitor program will start again after reboot.
  Note: On my Fedora 20 system the file was not present so I had to create it, turn it into a shell script by adding !/bin/sh at the first line and make it executable.

Verify:

[thanassis@skymnos ~]$ ps -ef | grep upsmon
root      1898     1  0 17:37 ?        00:00:00 /usr/sbin/upsmon start
nut       1900  1898  0 17:37 ?        00:00:00 /usr/sbin/upsmon start
500       2142  2118  0 17:38 pts/0    00:00:00 grep upsmon

NUT Monitor

A very good GUI based tool to help test the ups servers. It can be easily installed using the package manager of your distribution -- just search for the nut-monitor package and after you install and run it, it looks like this:

Windows

Winnut is a Windows client, that runs as a 32bit service on Windows 7. The project has not been updated since February 24, 2011. I did install the software on a Windows machine but have not been able to do any serious testing. The program's configuration follows the same rules as the Linux clients. The only thing you have to is click the edit configuration file button

and then add the correct MONITOR Line in the upsmon.conf file that will appear loaded into notepad. On 64bit systems you will also need to change the line

NOTIFYCMD "\"c:\\Program Files\\WinNUT\\alertPopup.exe\""

to

NOTIFYCMD "\"c:\\Program Files (x86)\\WinNUT\\alertPopup.exe\""

NVidia on Centos 6

The post documents the steps I followed in order to install the NVidia drivers using kmod-nvidia and ELPrepo on a fresh CentOS installation.

We begin by importing the ELRepo Project's public key

sudo rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org

The people at ELRepo suggest that this should also be present, so unless you already have it ...

sudo yum install yum-fastestmirror

The next step is to install the ELRepo repo itself

sudo rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm

... and finally the NVidia drivers, which is what we aimed for in the first place.

sudo yum install kmod-nvidia

An optional step would be to install the 32bit compatibility drivers and files

sudo yum install nvidia-x11-drv-32bit

What else is there... ahh yes, reboot!

Fedura 18 update with fedup

After updating two fc17 x86_64 KDE PCs over the network using FedUp, I thought that I would put down my experience and prepare anyone trying to do the same.

To get things started, fedup will probably not exist on your system, so, before starting the actual update process you will probably have to install it using the simple sudo yum install fedup command.

An other thing to mention here is that the current version of google-earth does not install on fedora 18.The actual error message is file /usr/bin from install of google-earth-stable-6.0.3.2197-0.x86_64 conflicts with file from package filesystem-3.1-2.fc18.x86_6. So the only way to do a descent update -- at least for the moment, Jan-16-2013 -- is to remove it: (sudo yum erase google-earth-stable) I am certain that the problem will be fixed, but right now we cannot have them both.

For the sake of completion, after installing fedup and removing google-earth, perform a full system update.

sudo yum -y update

After updating and rebooting, start the actual update process with the command:

sudo fedup-cli --network 18 --debuglog fedupdebug.log

This process (depending on your internet speed) will take quite some time. In the case of my desktop development machine, it downloaded 1755 packages plus an additional 728. When it will eventually be over, we will need to reboot and start the system from a specific "System Update (FedUp)" grub menu entry. FedUp will then do it's magic (press the Escape key if you want to see what it;s doing) and then you 'll have to reboot. The tricky part afterwards, is that KDE does not start complaining about various dependencies being let unsatisfied. The solution is start the system in text mode, make sure that you have an active network connection and then run:

yum distro-sync

This will downgrade approximately 90 packages but things will work out. As the fedora wiki advices, try running package-cleanup --orphans to determine packages left over that will receive no further updates and if possible remove them. (speaking of which a package-cleanup --oldkernels wouldn't hurt either...)