The language of our ancestors

Years ago, I have learned x86 assembly. I have lot of time, and it was a pleasure to dive into the low level programming.   Programming in assembly is quite easy, however tons of code are required to do simple things. So it is very inefficient to solve simple problems in assembly, and it is even more inefficient to solve complicated problems in assembly.

Who uses the assembly language today?

As far as I know, the deepest level o C optimization, is done with inline assembly, many video decoders (H.264, and others) make use of assembly language to optimize most use and time/performance critical parts of the code. Virus programmers, and anti-virus programmers (probably those two groups are the same guys).

Assembly snippets in time sensitive sections be can boost the performance of the application, however it is debugging of the code is quite time consuming.

Recently I have participated in the fourth edition of Olimex Weekend Programming Challenge (http://olimex.wordpress.com/2013/04/15/weekend-programming-challenge-issue-4-solutions/). I played a little with assembly, my solution is quite fast (it is still a little faster then C implementations).

 It was fun, to use this old and maybe forgotten language, to solve the simple problem.

You can see my assembly solution here: https://github.com/OLIMEX/WPC/tree/master/ISSUE-4/SOLUTION-29

After the publication I have optimesed the code little bit, to use registers more often, then the in-memory variables:

Advertisements
Posted in Uncategorized | Tagged , , | Leave a comment

Performance matters.

I have recently played participated in Olimex Weekend Programming Challenge (http://olimex.wordpress.com/2013/04/05/weekend-programming-challenge-issue-3/).
I have coded my solution in Python, and created custom C module to speed up string decomposition.

I like solving puzzles and I have great fun trying to make my code as fast as possible played with sets and dictionaries and found that dictionaries are little bit faster then sets (I was using Python 2.7). To spot the difference I used input file with 221k rows.

excution time [s]
dictionary 14.856
set 15.541

I have performed some tests with various test vectors, to see how the algoritm behaves when input data size changes.

performance-matters

As you can see execution time raises linearly with the number of characters in the input file.

You can see the source code of my soulution at: https://github.com/OLIMEX/WPC/tree/master/ISSUE-3/SOLUTION-10

Posted in Uncategorized | Tagged , , | Leave a comment

kexec – using kexec system call to update live UBIFS root filesystem

I use UBIFS as main general purpose MTD file system in my embedded projects. When customer is updating our embedded device, he updates root filesystem completely. I believe it is more error prone than updating single applications or files. It is not possible to update live attached UBI volume, so if you wish to update flash contents it is mandatory umount root filesystem and detach the UBI volume.

There are few ways to do it:

  1. reboot the device and start it from different (minimal) flash partition as root filesystem (you have to maintain two root filesystems, the real one, and the minimal one used for update),
  2. use pivot_root/chroot pair to switch to different rootfs and then umount main root filesystem (unfortunately, it did not worked for me),
  3. create minimal root filesystem on the fly, and boot into it using kexec kernel system call.

The third way is my favorite, basic steps to perform UBI file system update are as follow:

  1. Create file for temporary file system.
  2. Mount the temporary file as loop device.
  3. Create required directory structure.
  4. Copy all required files (busybox, libraries, kernel modules, modutils, ubiformat, etc) to the mounted root filesystem.
  5. Copy your update script, or update application into the mounted directory
  6. Umount the temporary file.
  7. Use kexec to boot with the kernel, use temporary file as initial ramdisk, execute your update script, or update application as the init process (use “init=/your_update_script.sh” kernel command line parameter).
  8. After “warm reboot” your UBI volume is not attached and you can update it with ubiformat.

I suggest attaching update script to your update package, as it is possible to customize tasks inside this script and it is possible to do whatever you want during this script execution.

Posted in Uncategorized | Tagged , , , , | Leave a comment

MPLAB X – force build process to use more than one thread

I use Microchip MPLAB X IDE under Windows 7 to write, compile and debug code for PIC24 processor. During my work on a small  project I have about  50 source C files. It takes some time to rebuild everything from scratch (one minute and a few seconds) on my iCore 7 PC. MPLAB X rebuilds everything from scratch when new file is added to the project.  It is quite annoying for me I have to wait every time for the MPLAB to build my code. It’s a waste. I going to explain how I managed to eliminate this waste.

MPLAB X is based on Netbeans, and uses Cygwins’ Makefile based build system. When build command is executed Cygwin’s make.exe is called. Unfortunately there is now way to set the number of building threads in the MPLAB X configuration menu. My first idea was change the name of make.exe  (located by default at C:\mplab_msys\bin)  to make-bin.exe to and replace it with simple batch file:

make-bin.exe -j 10 %*

where 10 is count of threads build process should spawn. It didn’t work. MPLAB X calls CreateProcess to execute make.exe, the make.exe have to be binary executable in PE format.

I compiled simple wrapper:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAX_LEN 4096

int main(int argc, char *argv[])
{
       unsigned int len = 0;
       char *cmd = malloc(MAX_LEN);
       memset(cmd, 0, MAX_LEN);

       argv++;
       argc--;

       strcat(cmd, "make-bin.exe -j 20");
       while(argc--)
       {
           len += strlen(*argv);
           len++;
           if (len &gt; MAX_LEN)
               exit(EXIT_FAILURE);
           strcat(cmd, " ");
           strcat(cmd, *argv++);
       }

       printf("%s\n", cmd);
       return system(cmd);
}

and replaced make.exe with it.

I use 20 building threads, builing time is reduced to 17 secodns for my project, so it is more or less 4 times shorter.

I played a little with count of building threads:

  • 1 thread – total time: 1m 3s,
  • 2 threads – total time: 35s,
  • 4 threads – total time: 23s,
  • 8 threads – total time: 18s,
  • 20 threads – total time: 17s,
  • 100 threads – total time 18s.
Posted in Uncategorized | Tagged , , | Leave a comment

Oracle 10g, Xen, and Squeeze

Debian Wiki entry is good starting point for installation: http://wiki.debian.org/DataBase/Oracle.

Another interesting tutorial is here: http://www.linuxinsight.com/oracle10g_on_debian.html.

Important points to take care of:

  • Create .profile file for oracle user
umask 0022
export ORACLE_BASE=/opt/oracle/
export ORACLE_HOME=/opt/oracle/product/10.2.0/db_1
export ORACLE_SID=orcl
export PATH=$PATH:$ORACLE_HOME/bin
export EMHOSTNAME=oracle
export DISABLE_HUGETLBFS=1
export ORACLE_TERM=xterm
if [ -z $LD_LIBRARY_PATH ]
then
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
else
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib
fi
if [ -z $CLASSPATH ]
then
 CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
 CLASSPATH=$CLASSPATH:$ORACLE_HOME/network/jlib
export CLASSPATH
else
 CLASSPATH=$CLASSPATH:$ORACLE_HOME/JRE:$ORACLE_HOME/jlib
 CLASSPATH=$CLASSPATH:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib
export CLASSPATH
fi
  • Don’t create database when installing Oracle.
  • Ignore “collector” error, just click continue.
  • After installation, create simple wrapper to disable HUGE TABLE FS. 
cd $ORACLE_HOME/bin
mv oracle oracle.bin 
cat >oracle <<"EOF"
 #!/bin/bash  
export DISABLE_HUGETLBFS=1 
exec $ORACLE_HOME/bin/oracle.bin $@ 
EOF  
chmod +x oracle
  • Execute dbca and create database.
  • Change ORACLE_HOME_LISTNER in $ORACLE_HOME/bin/dbstart to $ORACLE_HOME
  • Create /etc/init.d/oradb script to control Oracle start and stop
#!/bin/sh 
ORA_HOME=/opt/oracle/product/10.2.0/db_1
ORA_OWNER=oracle
test -f $ORA_HOME/bin/dbstart || exit 0
case "$1" in
	'start')
	   su - oracle -c "$ORA_HOME/bin/dbstart $ORA_HOME" &
   	;;
	'stop')
	   su - oracle -c "$ORA_HOME/bin/dbshut $ORA_HOME" &
   	;;
esac
  • Don’t forget to execute if you create /etc/init.d/oradb
# update-rc.d oradb defaults
  • to start Database Manager Console
oracle@oracle:~$ emctl start dbconsole 
  • last but not least, trailing / in $ORACLE_HOME breaks installation.
Posted in Uncategorized | Tagged , | Leave a comment