Archive for the ‘Uncategorized’ Category

Improved Python Traceback Module

Published January 27th, 2010, updated January 28th, 2010.

Like any modern language, Python comes along with a nice traceback module. This module gives you stack traces from the line of code where an exception is raised up to the next try-except clause. So, you can easily catch exceptions and write stack traces into a debug log. This debugging technique is pretty handy to drill down bugs and I use it a lot in prototyping.

Using the traceback module is straight forward for evident programming mistakes. However, real bugs are context-sensitive and they can hardly be reproduced without the actual data that was processed when an exception was raised. If you can reproduce a specific bug, you can add some logging code in front and inspect the variables the next time the bug is triggered. But if a bug occurs once in a blue moon, you’d be better in logging the data the first time an exception raises.

import traceback

def erroneous_function():
    ham = u"unicode string with umlauts äöü."
    eggs = "binary string with umlauts äöü."
    i = 23
    if i>5:
        raise Exception("it's true!")

try:
    erroneous_function()
except:
    print traceback.format_exc(with_vars=True)

Here’s my solution; an improved Python traceback module the logs variables from the local scope next to the affected code. You can find a working copy in our Mercirual repository (see the below).

Traceback (most recent call last):
 File "test.py", line 16, in <module>
   Local variables:
     erroneous_function = <function erroneous_function at 0x7ff6d82b...
     __builtins__ = <module '__builtin__' (built-in)>
     __file__ = "test.py"
     traceback = <module 'traceback' from '/srv/www/vhosts/dev.teamr...
     __name__ = "__main__"
     __doc__ = None
   erroneous_function()
 File "test.py", line 13, in erroneous_function
   Local variables:
     i = 23
     eggs = "binary string with umlauts \xc3\xa4\xc3\xb6\xc3\xbc."
     ham = u"unicode string with umlauts ???."
   raise Exception("it's true!")
Exception: it's true!

I am not sure if it is the “right” solution as sensitive information might be logged. This might have security implications for some real-world scenarios where webapps report stack traces to the end user (e.g. by using cgitb in production).

Credit: this code was inspired by format_exc_plus by Bryn Keller.

2010-01-28: there’s an active discussion on python-dev.

get latest source code
visit mercurial repository

Adding a Custom LDAP Schema to Open Directory on 10.5+

Published January 15th, 2010.

Open Directory is a key component of Mac OS X Server. It consists of OpenLDAP, MIT Kerberos, Password Server and a tool chain that enables GUI administration. Sadly, adding new ldap schemas to the directory server is not documented in the advanced administration guides and you have to tinker with the command line tools. I could not find any good documentation how you to add a custom LDAP schema, so I’ll show my solution here.

Mac OS X Server 10.5 ships with OpenLDAP 2.3. This release supports run-time configuration, which means that the LDAP schemas are stored within the directory server and you cannot simply put your new schema file in /etc/openldap/schema/; you have to convert it to an LDIF file and load this into the directory itself. This can be done during run-time but it breaks replication if you do so. So, instead you have to create a proper old-style config and run a manual conversion to the new run-time config.

To do so, you need to place the new schema file in /etc/openldap/schema/some-new.schema. This directory is copied  to new replicas when you join them, so you won’t break the Apple tool chain. Then, you need to include the new schema file from /etc/openldap/slapd.conf; this has no direct effect but slaptest(1) uses this to re-create the run-time config. Finally, convert the old-style config to a new run-time config using slaptest(1) like “slaptest -f slapd.conf -F slapd.d” and restart slapd:

cd /etc/openldap
cp some-new.schema schema/
cat >> slapd.conf <<HERE
include /etc/openldap/schema/some-new.schema
HERE
mv slapd.d slapd.d_bak
slaptest -f slapd.conf -F slapd.d
launchctl unload /System/Library/LaunchDaemons/org.openldap.slapd.plist
launchctl load /System/Library/LaunchDaemons/org.openldap.slapd.plist

Beware: we are deleting the old run-time config here and create a new one from the static config. If you have changed the config without adopting the old-style config, you might loose modifications. So, check twice if all required schemas are included from slapd.conf. AFAIK, Kerio Mailserver is troublesome here as it is not adding the include lines to slapd.conf. Though, thise procedure is exactly what the Apple tool chain does on replication and I suggest you do it exactly this way. Good luck!

Eight Questions on Twitter

Published January 7th, 2010.

Why…

  1. is it too slow for real-time communications (that is available on IRC since 1988)?
  2. can’t #hashtags contain unicode?
  3. is there a length limit for my nickname?
  4. can’t I sign on from multiple computers simultanously?
  5. does their web page no auto-refresh?
  6. is it always over capacity?
  7. didn’t they register appropriate country TLDs like .de?
  8. have they shut down their Jabber interface?

    Dragons Everywhere!

    Published December 27th, 2009, updated January 8th, 2010.

    It’s late December and like every year, hackers from accross Europe come together for the Chaos Communication Congress. The congress “is the annual four-day conference organized by the Chaos Computer Club (CCC). It takes place at the bcc Berliner Congress Center in Berlin, Germany. The Congress offers lectures and workshops on a multitude of topics and attracts a diverse audience of thousands of hackers, scientists, artists, and utopians from all around the world.”

    What’s new this year: there are some off-site hackcenters where people join the congress without being there in-real-life. “those unable to attend the Congress in Berlin [are invited] to celebrate their own Hack Center Experience, watch the streams, participate via twitter or chats, drink Tschunk, cook and have a good time.” This is exactly what some of us are going to do: we will meet at the UUGRN hackerspace on Monday and join the congress events.

    Recent Topics of #sickos

    Published October 30th, 2009.

    Here’s a compilation of recent topics from #sickos on SickosNet.

    Oct 27 2009 go go go!
    Oct 17 2009 Everything is OK
    Oct 05 2009 The Truth is out there.
    Sep 28 2009 arrr!
    Sep 18 2009 welcome to the encoding hell
    Sep 07 2009 knights of the infinite loop
    Aug 24 2009 world domination. fast.
    Aug 18 2009 computing sucks.
    Aug 13 2009 hacking at random
    Jul 31 2009 Happy SysAdminDay 2009!!1
    Jul 08 2009 ...
    Jun 29 2009 "The tenth sale is a thousand times easier than the second one (the first one doesn't count... beginner's luck)." --Seth Godin
    Apr 30 2009 SHA-1: Practical collisions are within resources of a well funded organisation.
    Apr 23 2009 "It's a trap!" --Admiral Ackbar
    Apr 10 2009 The road to hell is paved with good intentions.
    Mar 24 2009 "Damn it!" --Jack Bauer, CTU
    Mar 09 2009 "leadership is nature's way of removing morons from the productive flow" --Scott Adams in Dilbert
    Mar 02 2009 Some people, when confronted with a problem, think "I know, I'll quote Jamie Zawinski." Now they have two problems.
    Feb 26 2009 "The key to performance is elegance, not battalions of special cases." --Jon Bentley and Doug McIlroy
    Feb 23 2009 we like monkeys
    Feb 20 2009 cloud computing becomes fog when it goes down.
    Feb 14 2009 happy 1234567890
    Jan 26 2009 time.ctime(1234567890)
    Jan 15 2009 to /b/ or not to /b/...
    Jan 09 2009 beware of 'import skynet'.
    Jan 09 2009 @ack
    Dec 31 2008 happy new 1984
    Dec 27 2008 nothing to hide
    Dec 14 2008 Hail Eris. All hail Discordia.
    Dec 10 2008 rip silcnet. all hail sickosnet. > silc -c sickos.org
    Nov 19 2008 while True: pass
    Nov 06 2008 long live teh king !
    Nov 05 2008 remember remember teh 5th of november
    Sep 26 2008 for(;P("\n"),R=;P("|"))for(e=C;e=P("_"+(*u++/8)%2))P("|"+(*u/4)%2);
    Sep 19 2008 talk like a pirate day
    Sep 19 2008 Ahoy there Landlubbers!
    Sep 17 2008 0^0 := 1
    Jul 11 2008 we're the scene.
    Jul 09 2008 that's like knitting a sweater for a dead squirrel
    Jul 08 2008 computers suck and i hate them
    Jun 23 2008 osascript -e 'tell app "ARDAgent" to do shell script "whoami"'
    Jun 17 2008 Linux was fun
    May 24 2008 don't forget to bring a towel
    May 21 2008 Lost in Hyperspace
    May 03 2008 all your base are belong to us
    Apr 21 2008 look, it's making friends with the roomba!
    Apr 16 2008 The only laws on Internet are assembly and RFCs
    Apr 11 2008 inventors of the infinite loop
    Mar 19 2008 "the project suffers from lack of directions and frequent infighting between its developers" --Distrowatch.com on Gentoo Linux
    Mar 04 2008 "What you see on these screens up here is a fantasy; a computer enhanced hallucination!" --Wargames
    Feb 19 2008 a man collecting shoes... it's just not right.
    Jan 22 2008 | |  |   |     |        |             |                     |                                  |
    Dec 30 2007 pirates are better than ninjas
    Dec 27 2007 virtual congress
    Dec 21 2007 yankee white
    Dec 21 2007 rule 35 of the internet: if it doesn't exist on the internet, it must be created.
    Dec 18 2007 "[PHP] takes the worse-is-better approach to dazzling new depths" --Larry Wall
    Dec 16 2007 "Strong typing is for people with weak memories." --Tom Van Vleck
    Nov 30 2007 beiss mich, kratz mich, gib mir hostnamen!
    Nov 26 2007 tuttle-buttle
    Nov 23 2007 there's a buffer overflow but i won't disclouse
    Nov 22 2007 inventors of the infinite loop
    Sep 13 2007 'The Internet makes it so easy to get solutions to most of the problems that it has taken the fun out of it.' --Miguel de Icaza
    Sep 03 2007 'Sickos can wreak death and destruction from thousands of miles away!' --Arnold Yabenson, Weekly World News
    Jul 23 2007 "History is a set of lies agreed upon" --Napoleon Bonaparte
    Jul 19 2007 "Unix is a glorified video game" --Ed Post
    Jul 16 2007 "Programs should not attempt special solutions to general problems." --Pike & Kernighan
    Jul 09 2007 Hackerbande
    Jun 27 2007 Note: There are two BFGs in Hell.
    May 18 2007 0xdeadbeef
    May 02 2007 09:F9:11:02:9D:74:E3:5B:D8:41:56:C5:63:56:88:C0
    Apr 08 2007 "they seem to think that just because no one has ripped them apart means that no one can" --jf on apple security
    Mar 26 2007 "I think the fundamental mistake was this adoption of a democratic process" --Ian Murdock on Debian
    Mar 14 2007 foo
    Jan 08 2007 DEFCON 3
    Jan 07 2007 alert. blog.fefe.de is down. switching from DEFCON 3 to DEFCON 2.
    Dec 23 2006 Stell dir vor, es ist Congress, und keiner geht hin.
    Dec 14 2006 "Screensavers are sort of a poor man's LSD, without the bad trips." --Larry Wall
    Dec 06 2006 Ministry of Truth
    Dec 06 2006 Into the Box...
    Nov 15 2006 if the price matters, you're not a real gamer
    Nov 06 2006 The Revolution Will Not Be Televised
    Oct 10 2006 tut der router nicht mehr routen musst du booten
    Jul 28 2006 Happy System Administrator Appreciation Day!
    Jun 01 2006 We have done the impossible and that makes us mighty.
    May 31 2006 brilliant but empty
    May 17 2006 i wanna say something meaningful
    Apr 30 2006 wii
    Apr 20 2006 Bridge ahead.  Pay troll.
    Apr 09 2006 Beware! The Blob!
    Feb 13 2006 "Just because you're paranoid doesn't mean they aren't after you" --Kurt Cobain

    Today 16oo: Sickos Hack Nacht

    Published October 24th, 2009.

    We are going to have a Hack Nacht tonight. Sickos and other nerds are invited to exchange ideas, write code and to get to know each other. I’ve some ideas what to do and I’m looking forward to seeing you hackers tonight. I’ll update this article when the event is over, stay tuned (our join us on SickosNet).

    jQuery.postJSON()

    Published October 18th, 2009.
    /*
     * Hello Brandley, tonight I've tried to figure out how to do proper
     * JSON POSTs using "Content-type: application/json" and serialized JSON
     * data in the content portion (body) of my HTTP requests. This is
     * specified in the Twitter API for status updates and Identica JSON
     * webservices (found via Mark Pilgrim).
     * After some tests, I had to recognize that jQuery does not support JSON
     * encoding in the core distribution and aside of $.getJSON(), there
     * is no $.postJSON().
     * Below is an proposed update. As it relies on your json plugin, I'd ask
     * you to add it to your code base so that other jQuery users can benefit
     * of it.
     */
    
    $.postJSON = function(url, data, callback) {
        return jQuery.ajax({
            'type': 'POST',
            'url': url,
            'contentType': 'application/json',
            'data': $.toJSON(data),
            'dataType': 'json',
            'success': callback
        });
    };
    

    visit project

    Mercurial Repositories Available

    Published October 17th, 2009.

    Dear Googlebot, I want to tell you that you can find my latest source code over at hg.sickos.org. That’s where my friend dlat has installed a Mercurial source code management server. I’ve already migrated some projects there and would like you to index these pages. Please note that some projects have individual pages in our Wiki, just follow this category page. See you again!

    Sniffing HTTP Traffic at HAR2009

    Published August 14th, 2009.

    I’m currently visiting har2009, an international IT security conference in the Netherlands. It’s an amazing event with so many nice people, fresh lectures and a wonderful environment. There is a large wired and wireless network and everybody on the campsite is wearing a laptop, a pda or some other device that can connect to the Internet. And because there are so many security people around, I think it would be funny to demonstrate some insecurity here…

    First, there is the Web Proxy Autodiscovery Protocol (WPAD), which is used by your web browser when you use “proxy autoconfiguration” – the default setting on many systems. Second, there is a DHCP server for the campsite that does hostname registration in the DNS server. I asked myself what would happen if I could register the name wpad.visitors.har2009.net?

    Well, I have done so. And I have setup an appropriate proxy that intercepts all traffic that passes this machine. After 24 hours, there were more than 800 different hosts using this malicious proxy server – and many of them signed up to unencrypted web services like Twitter and others. That’s quite impressive as this are about 20 percent of the visitors! Now I’m wondering what happens if I break up SSL…

    Active DNS filtering at DTAG?

    Published July 1st, 2009.

    Today, I’ve found some strange behaviour on my TDSL Business connection. Some DNS queries are being dropped at the routers of Deutsche Telekom and I wonder if this is a malfunction with their Internet filters. When I query random name servers using “dig www.google.com any”, those packets never reach their destination. I’ve asked for confirmation on IRC and it looks like others perceive the same odd behaviour.

    I’ve investigated this a little bit and found that it is only port 53/udp that is being affected; doing dns queries over port 53/tcp is working fine. Thus, it looks like they are using some deep packet inspection at their routers as only special dns queries are being dropped. Can someone else confirm this odd behaviour?

    Python-MagickWand or How to Work With Icons

    Published June 18th, 2009, updated December 8th, 2009.

    I’m currently working on a pet project where I want to convert favicons (in Windows .ico format) to Web standard .png format. I started using the Python Image Library (PIL) for this, which supports plenty of image formats and which is Python’s standard way for doing image manipulations. A basic any_to_png function using PIL looks like this:

    img = Image.open(StringIO(buf))
    img = img.resize((16,16),Image.ANTIALIAS)
    img.save(buf, format='PNG', transparcency=1 )
    return buf.getvalue()

    This is straight forward, but I had to find out that PIL’s .ico support is pretty outdated while Microsoft has updated the specs. Modern .ico files have switched from .bmp to .png format and added alpha masks. There is a patch available that brings you .png support, but alpha masks are still broken.

    So, I’ve searched for another image library that has proper support for icon files and stumpled upon my old friend ImageMagick. I’ve found that there are Python bindings for the MagickWand interface (the C API). Yet, those bindings are incomplete, ugly and not actively maintained. I’ve found alternate Python bindings for the MagickWand interface and those are pretty nice:

    img = Image(StringIO(buf), 'ico')
    if not i.select((16,16)):
        i.alpha(True)
        i.scale((16,16))
    return i.dump('png')

    You see, Ian Stevens‘ CDLL bindings are a straight forward implementation of the MagickWand C API using the CDLL wrapper library. I’ve added some missing functions, documentation and clarified the licensing issues (now available under a BSD license) and I think this is a clean and elegant solution for a long standing problem. You can find a snapshot of Python MagickWand here and the latest source code there. Enjoy.

    download source code
    visit mercurial repository
    visit original project page

    2009-12-08: We’ve cloned python-magickwand and accept patches. You can find our latest work in our mercurial repository. Today, we’ve commited a magickwand6.5 patch, please look at the hg changelog for details.

    Command Line XML Processing

    Published January 12th, 2009.

    XMLStarlet is a command line interface to the Gnome XML- and XSLT libraries (libxml2, libxslt). It supports XPath queries and other usual permutations to XML data on the command line. I’ve stumpled upon this tool some years ago and today, I wanted to use it in practice. However, it looks like development has deceased and the last (yet incomplete) release is from 2005.

    So far, you can easily query XML data, but there is not way to append formatted XML elements (like parts of a tree) somewhere to the tree. Is anyone using XMLStarlet in real-world scenarios like shell scripts? Are there any better tools for DOM manipulations that are handy within shell scripts?

    btw: I’ve fixed the x86_64 compile errors on RHEL5 here.

    Directory Snapshots

    Published September 15th, 2008.

    Creating backups of open files was a challenging endeavor in the past. The main problem here is inconsistency. This is because the original data could get changed while it is read by the backup software. If this happens, it can result in missing references or references pointing at incorrect data. A typical example would be any kind of database. Here we have a lot of indices that are stored aside of the raw data. When the backup software reads the index and an update occurs, the original data cannot be read anymore.

    A simple strategy to deal with this type of problem is the creation of offline backups. They are fairly simple to implement and they ensure exclusive access by the backup software. Though, this approach is inelegant as it requires a downtime for every backup. To overcome this limitation, many applications spawned custom backup mechanisms to enable online backups. These mechanisms include simple dumps, logshipping, single- and multi-master replication and others. Although, they enable online backups, they are proprietary and require application specific know-how.

    A more generic approach is the use of file system snapshots. They create a static copy of the original data within milliseconds. This copy can be backed up while the system is online and the original data gets changed. On Linux, this snapshot functionality is part of the Logical Volume Manager (LVM). It is included in the standard Kernel since version 2.4.x and most modern Linux distributions activate it in their default installation.

    dsnapshot on top of lvm

    To create a file system snapshot, one basically requires the file system to be on a logical volume (LV) and some free space on the underlying volume group (VG). Then, one can tell the logical volume manager to snapshot the particular volume. The LVM holds all IOs and creates a new copy-on-write (COW) volume within milliseconds. This new snapshot volume can be mounted and backed up safely, as it does not change when it is written to the original volume.

    Of course, there are good reasons to use the backup methods that are recommended by some software vendor. But there are also many situations where a generic approach is preferable. Think of small databases, virtual machines, mail servers and other applications that store custom index files. I’ve seen a lot of these situations in the wild and for many times, I desired to have an easy snapshotting functionality. This lead to various snapshotting scripts, consolidation, adoptions and so on. After all, I’ve created dsnapshot which I want to introduce here.

    The dsnapshot script provides a high-level interface to the Linux Logical Volume Manager. It uses its block-level snapshot support to create directory snapshots. In contrast to block-level snapshots, directory snapshots resemble the file system layer. Thus, you can snapshot any directory that is on a logical volume and you don’t have to worry about the actual logical volumes, mount points and paths.

    This is the actual syntax for creating…

    $ dsnapshot --create /srv/mysql/test/
    /var/lib/dsnapshot/srv-fdf2e6dc/mysql/test/

    … and removing a directory snapshot.

    $ dsnapshot --remove /var/lib/dsnapshot/srv-fdf2e6dc/mysql/test/

    I’ve found this script very handy when you need to backup single directories instead of whole volumes.

    download source code

    301

    Published July 9th, 2008, updated September 30th, 2008.

    301 is an uri redirector. It allows you to create short links for complex web addresses. Just submit a longish uri at 301.sickos.org, and you will get a short link that points to the original address. You can pass this on twitter, in irc or whereever you want to avoid complex web addresses.

    301

    This service was inspired by tinyurl.com and monkey.org/sl. In contrast to their services, 301 comes along with full python source code. This give you freedom to run your own 301 service and adopt it to your needs. Get the source code at benjamin-schweizer.de/files/301/.

    hint: use pedit to manage the link database

    use service
    download source code

    Htpasswd Editor

    Published June 13th, 2008, updated September 18th, 2008.

    User authentication on unix systems typically relies upon password files or directory services. Both contain logon names, user ids, passwords, the location of your home directory and other information. The choice of the right authentication backend typically relies upon the amount of users you have to manage and your system environment.
    If you have decided to use simple password files, you can create different files for various services. This gives you the opportunity to separate system users from service users. Further, this enables you to delegate administrative rights to certain people.
    However, user management still requires you to twiddle with command line tools. This is fine if you are a unix lover, but if you want somebody with little command line experience to manager your users, you probably prefer a user interface that guides the unexperienced and reduces the risk of crashing the system.

    Htpasswd Editor

    This is exactly what htpasswd_editor does. It provides a text user interface for htpasswd(1) files and can easily be integrated with popular software like the Apache Web Server, VSFTP Daemon and other PAM-enabled programs (using pam_pwdfile).

    update 2008-09-17: there’s a new bugfix release available

    download source code

    Exploiting Python’s Class Dispatcher

    Published May 16th, 2008.

    In object oriented programming, the class dispatcher is a built-in function that looks up member functions and executes them in the context of a given class. In Python, those lookups are conducted dynamically, enabling one to modify the behaviour of a class without the need of subclassing. Here are some unusual but yet useful examples.

    # class-based programming style
    class Foo:
        pass
    
    class Bar(Foo):
        def bar(self):
            print "bar"
    
    bar = Bar()
    
    bar.bar() # prints "bar"

    So what? If you write all code on your own, you are fine. You can subclass Foo and invoke all methods from the new class Bar. But what, if the instantiation is done in code sections that you cannot modify? Imagine you are writing a plugin and you do not want to touch the code of others. They decided to instantiate Foo and you do not want to change this, nor you want to change Foo.

    # prototype-based programming style
    class Foo:
        pass
    
    foo = Foo()
    
    def bar(self):
        print "bar"
    
    foo.__class__.bar = bar
    
    foo.bar() # prints "bar"

    This second example shows how to “inject” a method into an already instantiated object. In fact, this works because Python uses dynamic delegation. Objects and classes are inspected at runtime and so, the dispatcher finds attributes even if they are added after object instantiation.

    Yahoo! Pipes

    Published May 11th, 2008, updated May 12th, 2008.

    Today, I’ve found some time to play around with Yahoo! Pipes. Basically, it is a visual programming environment, that aims at dealing with XML feeds and other sources from the semantic web. You can use a visual editor (see screenshot) to add various sources and operators. You can filter, rearrange and combine various feeds and finally create a new one.

    Yahoo! Pipes Visual Editor

    Because I use various Web 2.0 services like bookmarks.sickos.org (a del.icio.us clone), Google Reader, Flickr and others, I’ve many blog items at different locations. From there, I could write custom code to integrate these items with my blog or use some standardized technology. I’ve decided to use the latter, and this is where Yahoo! Pipes comes into the game. I’ve created a new pipe there and integrated the combined rss feed into my Wordpress blog.

    An examination of factors leading to abating customer loyalty towards magazine subscriptions

    Published March 31st, 2008, updated April 9th, 2008.

    What makes magazine readers cancel their subscriptions? This is an interesting question that gains in importance for magazine publishers and their service providers. Within a fully developed market, retaining customers is seen as a key to market dominance and long-lasting profits (Rowley and Dawes, 1999). To achieve this, it is important to understand the factors that lead to abating customer loyalty counter these factors.

    From the figures of the advertising market, one finds that the German market spans a volume of 12 billion Euros per year (Rüdell, 2007). It is dominated by traditional media, whereof television, newspapers, magazines and journals are the most important players. About 25 percent of this market is assigned to popular magazines and professional journals. Nearly 6000 different titles have been published in 2004 with a steady upward tendency during the preceding decade (ibid).

    However, this trend might be finished soon. As the magazine market is believed to be fully developed, rivalry among competitors emerges (Gassmann, 2006). This proposition is confirmed by the figures for the German advertising market. Where the number of published magazines increased by 20 percent within the last decade, the net advertising income had a peak in 2000 and then, fell down by 25 percent (Rüdell, 2007).

    Besides this emerging rivalry, changes in European legislation additionally increase the pressure towards the industry. Whereas the national markets were protected by strong market entrance barriers in the past, the ongoing unification of European law might erode these barriers soon.

    For instance, whereas subscriptions in France have to be actively renewed, subscriptions in Germany continue unless a subscriber gives notice. Regulations like this supported separate distribution channels and fragmented the European market into smaller national markets. This leads to a specialisation of many companies within the industry. In Germany, a diverse supply chain emerged with many small companies with a dedicated focus. Today, this supply chain consists of publishers, content holders, news agencies, print offices, distributors, call centres, data processing companies, retailers and others.

    However, with the ongoing unification of European law, these barriers are likely to fade and competitors from other national markets might enter the market. Though, many of the smaller companies are highly specialised and thus, very sensible to changes in this isolated market.

    So, the companies within this market face new challenges and are forced to take action. Due to the unifications in European law and the increasing rivalry with foreign competitors, they have to adopt their current strategies and prepare for stronger competition. From the Boston Matrix (Henderson, 1970), one can find two generic strategies to counter this. First, companies can enter new markets and grow therein. Second, they can focus on their competitors and try to increase their market share within an existing market.

    According to Blank (2006), some US companies decided for the first option: they have placed large investments in online communities and portal sites. This enables them to diverse their investments and allow further growth. However, a similar trend cannot be identified for European companies. Gassmann (2006) states that German publishers are doubtful as most of them have no web strategy at all.

    From there, it is consequential to investigate the second option from the decision matrix: increasing the current market share. To do so, it is important to understand the rules of the market and to perform better than competitors. One important component to achieve this superior performance is seen in customer retention. Therefore, it is essential to understand the motivation that makes customers cancel their subscriptions and to create an authoritative strategy to counter this. It is the aim of this thesis to examine the factors that lead to abating customer loyalty to magazine subscriptions and thus, enable companies within the market to adopt their strategies and increase their customer base.

    preview this book
    buy this book

    Entropy Password Generator

    Published March 27th, 2008.

    Entropy is a password generator. It generates two kinds of passwords: i) low entropy passwords that humans can easily remember and ii) high entropy passwords as commonly used in stored sessions. The low entropy passwords are generated from the Basic English vocabulary by C.K. Ogdeni. The high entropy passwords are random alpha numeric passwords where similar looking characters are stripped.

    Basic English Passwords (low entropy / e=649,527,500)
    note564still             cover624powder           box300person
    discovery371spring       over425such              arm781great
    daughter658advertisement woman600cushion          help695money
    not750sweet              where289brain            present557see
    brain787polish           sticky446change          fly679fear
    body411oven              system475house           frequent497size
    dog303level              cushion435boy            great870language
    porter288doubt           awake847pull             hat783burn              
    
    Mixed Alpha Numeric Passwords (high entropy / e=10^18)
    6rt84tZrvUkLrtE2 AG7HQEjxQDg4Znao v9DUzzJc8X97FQqj cXTQmY3gvvkvwhTx
    VJBEC4RFRtTPNgFA Z4pcMrRPMuE8a4EM EcyJArGdH2D6jZBT wr75cJdmzuF9a9LX
    wce4yXfhdnwjEnU9 hGKfFYuRwQMkAnqg BEmtkbjtLEyKM3YW wVgxoX82TfGmxbuT
    ho3zNKvZCBQ3wgJ6 mvKTTyy6TN9zCCZ8 fKr8eWL34XDNQyKG wCQFtYHQcaxmoAep
    Mp7dMC8gDBMa9qGh TGRKnW58cT8z66a4 dZAt2ghzCbDkdmJA P2XpNxFRDjcfQG83
    gch7TqT2d6RYzpGb xeZWbqDegADXoRnu xmmeJXkFdTXzcWam t9JL3DpKoMPMYrac
    URcVPrCRuQETzVVe aJnw4wghHcj3jCqr 9g9pVYtGtq5RhCaG oJ4y3k8rdjmnUE6w
    aTWyu76uu5TPgkCv aLeffq6MVNfAnxp7 EnqeUkjHPkgwv3AG q5Zmmc3GzJyxneHn
    

    This application is writte in Python and supports both, a CGI interface for your web server and a command line interface. From a security perspective, I strongly recommend the command line version after reading the source code.

    use service
    download source code

    Shift Happens

    Published March 24th, 2008, updated March 25th, 2008.

    The Shift Happens campaign. An interesting video focused on the U.S. education system showing how our world is changing.

    PySqlite2: Unicode Bug while Processing Non-unicode Text

    Published March 5th, 2008, updated September 11th, 2008.

    Pysqlite-2.3.2 accepts binary data on inserts but selects return unicode strings. This results in unicode conversion bugs when non-unicode bytes are stored in the database.

    As sqlite3 accepts binary data in text fields, this seems to be a bug in pysqlite. To fix it, one could i) either restrict inserts to unicode strings or ii) change the result from unicode to binary.

    However, the first would break compatibility with sqlite and that latter would break compatibility with existing code. Thus, this should be discussed with the authors.

    import sqlite3
    
    connection = sqlite3.connect(':memory:')
    cursor = connection.cursor()
    cursor.execute('''CREATE TABLE test (t TEXT)''')
    cursor.execute('''INSERT INTO test (t) VALUES (?)''', (chr(128),))
    cursor.execute('''SELECT t FROM test''')
    # Traceback (most recent call last):
    # File "pysqlite_utf8.py", line 10, in 
    #     cursor.execute('''SELECT t FROM test''')
    #     sqlite3.OperationalError: Could not decode to UTF-8 column 't' with text '?'
    
    print cursor.fetchone()
    connection.close()
    

    pysqlite ticket
    Debian ticket
    download source code

    Web Applications on Mobile Devices

    Published March 4th, 2008.

    Today, Google announced the release of Gears for Mobile Devices. This is good news as it empowers mobile developers to create mighty applications in a wise known from modern web applications. So, what does this mean and what is it good for?

    The mobile world consists of cell phones, smart phones and PDAs/handhelds. Leaving aside the cell phones, one finds useable devices with and without a stylus, capable of getting an Internet connection on their own or via some bluetooth or wifi link. These devices are dominated by two major operating systems: i) Windows Mobile and ii) Symbian OS. There are also some Linux platforms out there (Maemo, OpenMoko, Android), but their market share is not yet relevant. Thus, it is straight forward to focus on the two established platforms in the first instance.

    Developing applications for Windows Mobile is pretty easy; all you need is Visual Studio (not the Express Edition) and you can start with C++, C#/Visual Basic, JScript or ASP.NET. It is quite the same with Symbian OS, wich gives you C++, C#/Visual Basic, Java, Ruby, Python and Perl among others. However, creating cross platform applications gets a bit more complicated – and it is unlikely that you can port them to an emerging platform easily.

    Tough, what both platforms have in common is a capable web browser. From the diversity of microbrowsers, one finds Internet Explorer, Opera Mobile, Web Kit- and Mozilla-based browsers to be the common ones. All those browsers support JavaScript, XMLHttpRequests, CSS and what else you need to create rich client applications.

    Mobile Browser Overview 2008

    What these browsers are lacking is permanent storage: when you loose your internet connection, you cannot save or load any data. This issues was solved for the fully featured desktop browsers recenty (see Adobe Air, Google Gears, or Microsoft Silverlight) and Gears is now available for mobile devices, too. According to the Google Code Blog, they currently support IE Mobile and plan the integration of other mobile browsers in the near future.

    Social Bookmarking

    Published February 29th, 2008, updated August 31st, 2008.

    At sickos.org, Andreas and I run a copy of the social bookmarking software Scuttle. Basically, it is an open source implementation of del.icio.us (or at least, as delicious was back in 2006). Scuttle development has slowed down, but we keep up running this service and patch bugs every now and then. If you want to join us, read the documentation here and feel free to add your bookmarks there.

    Scuttle Social Bookmarking

    use service
    download patches