As shown by previous link the reverse is true. In hindsight it seems to be clear that source of discrepancy is due to shorthands we employed while analyzing the performance.

Why do we need the branch prediction

The simplest way of thinking about processor is as machine executing one instruction at a time. While this is very simple description it doesn’t allow fast computation. Usually there is several stages which can work independently or semi-independently and the processor is similar to assembly line (pipeline) of data. Since each discrete stage needs to fit inside 1 cycle finer splitting allows faster clock rate – and larger throughput (this is simplified view as current processors are superscalar and out-of-order.

There are however few things that prevents reaching the theoretical maximum performance (other then bottlenecks outside processor) called hazard. Consider following program (in pseudo-assembly):

    mov [r0], r0
    jz r0, label
    mov [r1], r0
label:
    add r0, 1, r1

The processor does not know how to proceed after line 2. The simples resolution of a hazard is to stall (do nothing until it is known how to resolve the problem). However in many cases the processor can guess (predict) how to proceed – it requires a bit of bookkeeping in order to maintain consistent state when it fails to guess but it avoids just waiting on solution.

Openness and branch prediction

Traditionally the branches were quite simple – either a branch was taken or not taken. Since the direction of the branch was quite simple (1 bit) it allowed a number of smart methods:

• Counting the number when the branch is taken or not taken – we guess the branch which was taken most frequently in past.
• Considering the history. It is possible to take say 5 past choices of taken/not taken and use it as part of address. It gives performance as single method might be called from 2 different places and have different branching depending on way it is called.

However on modern processors (in practice – any produced during past 30 years or so) we can jump in any place in the code by function pointer:

int main() {
    // f is pointer to some function
    int (*f)() = getF();
    f();
}

The style of calling an unknown function is currently very common – the method call depend on the data that is passed to function instead of place of such call. From processor point of view the virtual method calls, calling closure etc. are calling an arbitrary address. Now instead of potentially two outcomes of branching there can be arbitrary number of them. While usually processors guess that the last taken branch will be taken this time I have not heard about processors doing anything more complicated (the history considered is still from traditional taken/not taken branches from what I understand

Usually the functional and object-oriented languages deal with the problem in various ways. For example they can employ data/control-flow analysis to check what types can occur at give pointers – example from C:

struct A {
    virtual int f() = 0;
};

struct B : A {
    int f() __attribute__((noinline)) {
        return 0;
    }
};

struct C : A {
    int f() {
        return 1;
    }
};

int main() {
    B b;
    A *a = &b;
    return a->f();
}

The gcc can see that nothing but B can be pointed by a. Hence he calls the B::f directly.

	.file	"test.cc"
	.section	.text._ZN1B1fEv,"axG",@progbits,_ZN1B1fEv,comdat
	.align 2
	.p2align 4,,15
	.weak	_ZN1B1fEv
	.type	_ZN1B1fEv, @function
_ZN1B1fEv:
.LFB0:
	.cfi_startproc
	xorl	%eax, %eax
	ret
	.cfi_endproc
.LFE0:
	.size	_ZN1B1fEv, .-_ZN1B1fEv
	.section	.text.startup,"ax",@progbits
	.p2align 4,,15
	.globl	main
	.type	main, @function
main:
.LFB2:
	.cfi_startproc
	leaq	-24(%rsp), %rdi
	jmp	_ZN1B1fEv
	.cfi_endproc
.LFE2:
	.size	main, .-main
	.ident	"GCC: (Gentoo 4.7.2-r1 p1.4, pie-0.5.5) 4.7.2"
	.section	.note.GNU-stack,"",@progbit

However such techniques are (usually) not employed by C compilers as in many cases programmers can go behind its back and change the function pointers – even if programmers know it won’t be changed.

Note about symbols

Potential source of openness is symbol resolution. If the symbol can be outside library the resolution goes through PLT. If f was shared function `int main() {return f ();}` would compile into:

	.file	"test-plt.c"
	.text
	.globl	f
	.type	f, @function
f:
.LFB0:
	.cfi_startproc
	pushq	%rbp
	.cfi_def_cfa_offset 16
	.cfi_offset 6, -16
	movq	%rsp, %rbp
	.cfi_def_cfa_register 6
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret
	.cfi_endproc
.LFE0:
	.size	f, .-f
	.globl	main
	.type	main, @function
main:
.LFB1:
	.cfi_startproc
	pushq	%rbp
	.cfi_def_cfa_offset 16
	.cfi_offset 6, -16
	movq	%rsp, %rbp
	.cfi_def_cfa_register 6
	movl	$0, %eax
	call	f@PLT
	popq	%rbp
	.cfi_def_cfa 7, 8
	ret
	.cfi_endproc
.LFE1:
	.size	main, .-main
	.ident	"GCC: (Gentoo 4.7.2-r1 p1.4, pie-0.5.5) 4.7.2"
	.section	.note.GNU-stack,"",@progbits

The f@PLT is small function which calls appropriate function according to resolution. In general resolve to one additional indirect jump which can be perfectly predictable after first two misses. On x86 (32-bit version) due to technical limitations there is one additional direct jump. The detailed description is in Ulrich Drepper’s guide“How To Write Shared Libraries”.

A note about real performance

Measurement of the problems which can arise can be quite tricky. The modern processors and compilers can be quite smart and microbenchmarking of instruction can be fight against it while at the same time for the result to be meaningful there is need to exploit them.

I tried to use following code in my test:

#include <stdint.h>
#include <stdio.h>
#include <time.h>
#include <inttypes.h>
#include <math.h>

static void f() __attribute__((noinline));
static void g() __attribute__((noinline));
 
static void f() {
    __asm__ __volatile__ ("");
    return;
}
 
static void g() {
    __asm__ __volatile__ ("");
    return;
}
 
 
typedef void (*fptr_t)();
 
#ifndef DIRECT
#ifdef CHANGE
static void swap(fptr_t *f, fptr_t *g) {
  fptr_t tmp = *f;
  *f = *g;
  *g = tmp;
}
#endif
#endif
 
#ifdef VOLATILE
#define LOAD_FPTR(n) \
  volatile fptr_t n ## ptr = n
#else
#define LOAD_FPTR(n) \
  static volatile fptr_t n ## ptr_static = n; \
  fptr_t n ## ptr = n ## ptr_static
#endif
 
int main() {
#ifndef DIRECT
  LOAD_FPTR(f);
#ifdef CHANGE
  LOAD_FPTR(g);
#endif
#else
    fptr_t fptr = f;
#ifdef CHANGE
    fptr_t gptr = g;
#endif
#endif
 
#ifndef TRIES
#define TRIES 10
#endif
    double times[TRIES];
    for (uint64_t j = 0; j < TRIES; j++) {
        struct timespec start, end;
        clock_gettime (CLOCK_THREAD_CPUTIME_ID, &start);
        for (uint64_t i = 0; i < UINT64_C(0xFFFFFFF); i++) {
#ifndef CHANGE
            fptr();
#else
#ifndef DIRECT
            swap (&fptr, &gptr);
            fptr();
#else
            uint64_t f_or_g;
            asm ("and $1, %0" : "=r"(f_or_g) : "0"(i));
            if (f_or_g) {
                fptr();
            } else {
                gptr();
            }
#endif
#endif
        }
        clock_gettime (CLOCK_THREAD_CPUTIME_ID, &end);
        times[j] = (end.tv_sec - start.tv_sec) + 1e-9*(end.tv_nsec - start.tv_nsec);
        printf("Time[%" PRIu64 "] = %f\n", j, times[j]);
    }
    double mean = 0.0;
    double mean2 = 0.0;
    for (int j = 0; j < TRIES; j++) {
        mean += times[j]/TRIES;
        mean2 += (times[j] * times[j])/TRIES;
    }
    double stdev = sqrt(mean2 - mean * mean);
    printf("Mean  = %f\n", mean);
    printf("Stdev = %f\n", stdev);
    return 0;
}

The table is closer to rough guide than any realistic simulation. For example many Vala virtual calls have 2 indirect jumps and the last row have additional stores to the memory.

How does Vala fit into it

Vala uses GObject under the hood and it was to some extend constraint by it. It needs to fit into existing ABI of libraries written in GObject/C. Since the methods were implemented in C by function pointers it is what done by Vala. However by transition the benefits of more complicated optimizations are lost – the C can no longer optimize the code. However using the open style (with methods) is more encouraged in Vala then C.

List<int> list = new ArrayList<int>();
list.add(0);
// In C++ the add could point directly to ArrayList::add
// In Vala we do the indirect call

Even if the analysis was implemented in Vala the optimization could not be easily implemented without tweaks to detect the ABI. It would not be impossible to simply export gee_array_list_add to allow compiler to look up the correct function and use it directly. As it changes the ABI the classes would need to be explicitly annotated.

Other problem present in Vala is the abstract classes deriving from interfaces. If class overrides a method of such interface currently the following happens:

1. Interface method resolution function (my own naming – not a standard one) is called – 1 direct jump
2. The interface vtable is looked up (memory load) and from it is read a location of abstract class resolution function – 1 indirect jump
3. The abstract class vtable is looked up (memory load) and from it the real function is executed – 1 indirect jump
4. The real method is executed

In total it is 2 indirect jumps (depending on memory load) and 1 direct jump. There seems to be at least 2 ways of improving it:

• Instead of overriding just the abstract class the child class might simply reimplement the interface under the hood thus allowing to directly jump to the method. It seems to be possible given existence of g_type_interface_peek_parent.

  As the interface implementation can be per-derived class instead of one per hierarchy the pointer to real implementation can be inserted into the vtable. This allows to skip the detour via abstract class resolution function and one indirect call saving some time.

• While technically not fully compatible with GObject it is possible to directly resolve the method from outside the class. Consider following loop:

  List<int> list = ...
  for (int i = 0; i < list.size; i++) {
      do_something (list.get (i));
  }
  

  It would be compiled into something roughly corresponding to:

  GeeList list = ...
  for (int i = 0; i < gee_list_get_size (list); i++) {
      int tmp = gee_list_get (i);
      do_something (tmp);
  }
  

  Instead it would be possible to simply get the function pointers for the gee_list_get_size and gee_list_get – chances are that register won’t be saved on stack and they would be used right away. In following code it might not do much difference but it more complicated ones it might make an impact.

Conclusion

Control flow may have an impact on the performance of the program. Usually the compiler, JIT and natural flow of language directs to certain usage-patterns which can be exploited. Unfortunately Vala doesn’t have many optimizations implemented (at least yet) and they cannot be done by C compiler.

In many cases people either don’t care about performance (does it matter if calculator displays result in few 17 ms faster?). Once people are interested in performance they can start optimizing even if it is not natural way in a language (for example struct of arrays in C/C++). It is worth to keep in mind for such case that Vala does not optimize the virtual calls as much as C++ or Java.

The problem is that the Make and Shake are both rather quite low-level tools and while improvement in them is beneficial shake, at least currently, is not a replacement for autotools or CMake.

There are number of things user might want to do with build system, which would need to be manually added to make or shake script:

• CC, CXX, VALAC, CFLAGS… environment variables to allow fine-tuning the compilation by user (or developer who wants to for example test with clang instead of gcc).
• Personalizing the prefix to put program in right place (usually /usr/local is used for program compiled by source but it is often not accessible by users)
• Installing in different root hierarchy steered by DESTDIR. This is different from prefix as latter denotes where program is run from while this variable locates a root (/) location during install time. This is used mostly for creating packages.
• distcheck which allows to create tarballs, build from them, run tests etc.
• Cross-compilation
• Compile dynamic libraries on whatever platform I’m on, handling all complexity (correct flags – say -fPIC, tools, …)
• Build in separate build directory (useful for cross-compilation)
• …

Those features are not explicitly supported by Make or Shake as they are more low-level tools – which is fine if we treat them as such and use other libraries or tools automating the compiling and linking (I haven’t seen anyone claiming Shake is anything more, but it is good to keep in mind that build systems can grow quite complex while using it). However for a tool to be usable for wide range of users there should be provided – preferably with consistent interface with existing tools as it makes the integration easier with other tools (say portage) and does not require user to relearn the specifics. Unfortunately most make tutorials (and shake) omit those problems. It can be fine if we made some assumptions – say we have closed environment (so the need is to integrate with internal tools and only them, specific installation etc.) or we know that we need to provide those features. However it can cause problems if they do appear in published repository.

The own build system therefore is not the Make or Shake but rather systems built on top of them. My guess is that when someone knows the full requirements of a build system than writing more highier-level library on top of Shake should be relativly simple (as it’s EDSL) – however it is harder then it looks like.

PS. While looking for links to the post I’ve (re)read similar post.

PPS. The draft version of this post was accidentally published. Sorry for confusion.

As chosen field I tried to use LLRB trees – I have implemented them quite long time ago and I’ve run into problems that violated the pre and post conditions. Even when problem was discovered it took some time to find out the underlaying problems (my inexperience didn’t helped probably either)

I’ve tried to reimplement the code in GHC Haskell with following rules:

• Using only ‘safe’ extensions
• Making all functions total and without use of undefined or error
• Ensuring that all properties of tree are preserved

Unfortunately this was my first time I used some of the concepts (like Zipper) so the implementation and description might be not the best (comments and corrections welcome).

I have used the code closer in structure to traditional implementations then the one described by Sedgewick. I don’t know the performance though as point was more to ensure the properties of the tree. It also includes both 2-3 and 2-4 implementation with latter one commented out.

While it took some time to implement it (for finding out how to convincing the type checker that (n + 1) + m = n + (m + 1)) it ensures the safety (and in total it have taken much less time then de debugging alone last time).

WARNING! The code was written in hurry and while I have learned and I haven’t had time to clean it. It might contain strange variable names etc. It is also not written as efficient implementation but rather a proof of concept.

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE FlexibleContexts #-}

Above code is simply bunch of extentions that are used

data Nat = Z | S Nat

type family Add (x :: Nat) (y :: Nat) :: Nat
type instance Add  Z     Z    = Z
type instance Add  Z     y    = y
type instance Add  x     Z    = x
type instance Add (S x) (S y) = S (S (Add x y))

DataKinds extension allows to promote a type to a kind (type of type). The above code is simply an encoding of natural numbers on type level with addition

data TaggedNat :: Nat -> * where
  TaggedNat :: TaggedNat n

class NatHelper (n :: Nat) where
  nat :: (Z ~ n => a) -> (forall n'. (S n' ~ n, NatHelper n') => TaggedNat n' -> a) -> TaggedNat n -> a

instance NatHelper Z where
  nat z _ _ = z

instance NatHelper n => NatHelper (S n) where
  nat _ s _ = s (TaggedNat :: TaggedNat n)

data ChainNat :: Nat -> * where
  ChainZero :: ChainNat Z
  ChainCons :: NatHelper n => ChainNat n -> ChainNat (S n)

toChain :: NatHelper n => TaggedNat n -> ChainNat n
toChain = nat ChainZero (ChainCons . toChain)

fromChain :: ChainNat n -> TaggedNat n
fromChain ChainZero = TaggedNat
fromChain (ChainCons _) = TaggedNat

prev :: NatHelper (S n) => TaggedNat (S n) -> (NatHelper n => TaggedNat n -> a) -> a
prev t x = case toChain t of
  ChainCons c -> x $ fromChain c

addEq :: forall a n' m'. (NatHelper n', NatHelper m') =>
         (Add n' (S m') ~ Add (S n') m' => a) ->
         TaggedNat n' -> TaggedNat m' ->
         a
addEq x n m = nat (nat x (const x) m) (\n' -> nat x (addEq x n') m) n

In general the type checker in Haskell is not a SMT solver (at least yet). The above code provides prove of 2 properties:

• If we have S n (or n + 1) then n is a natural number.
• Prove by induction (n + 1) + m = n + (m + 1). Unfortunately it currently is not optimized out despite my attempts to convince GHC by rules etc. that it is complex identity. If someone wanted to write the ‘real’ code it might be a good spot to fix as it changes the complexity.

data BlackNode :: Nat -> * -> * where
  Nil :: BlackNode Z a
  BBNode :: BlackNode n a -> a -> BlackNode n a -> BlackNode (S n) a
  RBNode :: RedNode n a -> a -> BlackNode n a -> BlackNode (S n) a
--  RRNode :: RedNode n a -> a -> RedNode n a -> BlackNode (S n) a

data RedNode :: Nat -> * -> * where
  RedNode :: BlackNode n a -> a -> BlackNode n a -> RedNode n a

The above code defines the nodes. The second type parameter denotes number of non-nil black nodes in all paths in subtree. It states that (for uncommented version):

• For black trees the following rules apply:
  • Nil element is a black tree with no non-nil elements
  • If l, r are a black trees with n black non-nil nodes in all paths and v is a element (label) then BBNode l v r is black tree with n + 1 black non-nil nodes in all paths
  • If l(r) is a red tree (respectively black tree) with n black non-nil nodes in all paths and code>v is a element (label) then RBNode l v r is black tree with n + 1 black non-nil nodes in all paths
  • Nothing else is a black tree
• For red trees the following rules apply:
  • If l, r are a black trees with n black non-nil nodes in all paths and v is a element (label) then RedNode l v r is red tree with n black non-nil nodes in all paths
  • Nothing else is a red tree

By construction it ensures the all red-black tree's properties are kept and the tree is left-leaning. The tree, which would violate any property is not possible to construct.

data BlackNode' :: Nat -> Nat -> Nat -> * -> * where
  Top :: BlackNode' n Z n a
  BBNode'L :: (t ~ Add (S n) m) => BlackNode' (S n) m t a -> a -> BlackNode n a -> BlackNode' n (S m) t a
  BBNode'R :: (t ~ Add (S n) m) => BlackNode n a -> a -> BlackNode' (S n) m t a -> BlackNode' n (S m) t a
  RBNode'R :: (t ~ Add (S n) m) => RedNode n a -> a -> BlackNode' (S n) m t a -> BlackNode' n (S m) t a
  RedNode'L :: (t ~ Add n m) => RedNode' n m t a -> a -> BlackNode n a -> BlackNode' n m t a
  RedNode'R :: (t ~ Add n m) => BlackNode n a -> a -> RedNode' n m t a -> BlackNode' n m t a

data RedNode' :: Nat -> Nat -> Nat -> * -> * where
  RBNode'L :: (t ~ Add (S n) m) => BlackNode' (S n) m t a -> a -> BlackNode n a -> RedNode' n (S m) t a
--  RRNode'L :: (t ~ Add (S n) m) => BlackNode' (S n) m t a -> a -> RedNode n a -> RedNode' n (S m) t a
--  RRNode'R :: (t ~ Add (S n) m) => RedNode n a -> a -> BlackNode' (S n) m t a -> RedNode' n (S m) t a

Since the operation of insertion into tree needs to consider sometimes surrounding and we need to reconstruct the tree afterwards the notion of Zipper become handy. The simplest way of thinking about it is that if we view the tree from certain node we can split/cut the tree into subtree and tree with hole. Now we can represent the tree with hole as either empty suptree (the hole is the tree) or as tuple of sibling subtree, value at parent and tree with hole of 1 size bigger.

The parameters are used to keep the correct sizes of three, where first parameter (n) is number of black non-nil nodes in all paths in subtree, second (m) is number of black nodes in path to root and third (t) is number of black non-nil nodes in all paths in whole tree.

find :: (Ord a, NatHelper t) =>
        (forall n m. t ~ Add n m => BlackNode n a -> BlackNode' n m t a -> b) ->
        (forall n m. t ~ Add n m => RedNode n a -> RedNode' n m t a -> b) ->
        BlackNode t a ->
        a ->
        b
find fb fr n = find'B fb fr n Top

find'B :: forall a b n m t. (Ord a, t ~ Add n m, NatHelper n, NatHelper m) =>
         (forall n' m'. t ~ Add n' m' => BlackNode n' a -> BlackNode' n' m' t a -> b) ->
         (forall n' m'. t ~ Add n' m' => RedNode n' a -> RedNode' n' m' t a -> b) ->
         BlackNode n a ->
         BlackNode' n m t a ->
         a ->
         b 
find'B fb _      Nil           ctx _  = fb Nil ctx
find'B fb fr bn@(BBNode l v r) ctx v' = case compare v' v of
  LT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'B fb fr l (BBNode'L ctx v r) v') n' (TaggedNat :: TaggedNat m)
  EQ -> fb bn ctx
  GT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'B fb fr r (BBNode'R l v ctx) v') n' (TaggedNat :: TaggedNat m)
find'B fb fr bn@(RBNode l v r) ctx v' = case compare v' v of
  LT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'R fb fr l (RBNode'L ctx v r) v') n' (TaggedNat :: TaggedNat m)
  EQ -> fb bn ctx
  GT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'B fb fr r (RBNode'R l v ctx) v') n' (TaggedNat :: TaggedNat m)
--find'B fb fr bn@(RRNode l v r) ctx v' = case compare v' v of
--  LT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'R fb fr l (RRNode'L ctx v r) v') n' (TaggedNat :: TaggedNat m)
--  EQ -> fb bn ctx
--  GT -> prev (TaggedNat :: TaggedNat n) $ \n' -> addEq (find'R fb fr r (RRNode'R l v ctx) v') n' (TaggedNat :: TaggedNat m)

find'R :: (Ord a, t ~ Add n m, NatHelper n, NatHelper m) =>
         (forall n' m'. t ~ Add n' m' => BlackNode n' a -> BlackNode' n' m' t a -> b) ->
         (forall n' m'. t ~ Add n' m' => RedNode n' a -> RedNode' n' m' t a -> b) ->
         RedNode n a ->
         RedNode' n m t a ->
         a ->
         b 
find'R fb fr rn@(RedNode l v r) ctx v' = case compare v' v of
  LT -> find'B fb fr l (RedNode'L ctx v r) v'
  EQ -> fr rn ctx
  GT -> find'B fb fr r (RedNode'R l v ctx) v'

Find allows to get the location (node and context) of element labelled by given value. Unfortunately the black case requires additional bookkeeping as type checker is not a SMT solver.

It is generalized function which can be used for implementation of contains, insert, delete etc. The insert function finds location to add - if it is not Nil then value is already present in tree and it aborts. Otherwise it creates a simple red tree and tries to plug it into the hole. There are four cases of plugging in (2 colors of hole x 2 colors of current subtree) starting with putting the red subtree into a black hole created by Nil.

insert' :: forall a t. (Ord a, NatHelper t) => a -> BlackNode t a -> (Bool, Either (BlackNode t a) (BlackNode (S t) a))
insert' a n = find (\n' ctx -> case n' of
                       Nil -> (True, mergeRB (RedNode Nil a Nil) ctx)
                       _ -> (False, Left $! n)) (\_ _ -> (False, Left $! n)) n a

The simplest cases are plugging red subtree into red hole or black subtree into black hole. They are straightforward operations which simply reconstruct tree from location.

mergeRR :: RedNode n a -> RedNode' n m t a -> Either (BlackNode t a) (BlackNode (S t) a)
mergeRR l (RBNode'L ctx v r) = mergeBB (RBNode l v r) ctx
--mergeRR l (RRNode'L ctx v r) = mergeBB (RRNode l v r) ctx
--mergeRR r (RRNode'R l v ctx) = mergeBB (RRNode l v r) ctx

mergeBB :: BlackNode n a -> BlackNode' n m t a -> Either (BlackNode t a) (BlackNode (S t) a)
mergeBB t  Top                = Left $! t
mergeBB l (BBNode'L  ctx v r) = mergeBB (BBNode l v r) ctx
mergeBB r (BBNode'R  l v ctx) = mergeBB (BBNode l v r) ctx
mergeBB r (RBNode'R  l v ctx) = mergeBB (RBNode l v r) ctx
mergeBB l (RedNode'L ctx v r) = mergeRR (RedNode l v r) ctx
mergeBB r (RedNode'R l v ctx) = mergeRR (RedNode l v r) ctx

mergeBR :: BlackNode n a -> RedNode' n m t a -> Either (BlackNode t a) (BlackNode (S t) a)
mergeBR l (RBNode'L ctx v r)                  = mergeBB (BBNode l v r) ctx
--mergeBR l (RRNode'L ctx v (RedNode l' v' r')) = mergeBB (RBNode (RedNode l v l') v' r') ctx
--mergeBR r (RRNode'R l v ctx)                  = mergeBB (RBNode l v r) ctx

blacken :: RedNode n a -> BlackNode (S n) a
blacken (RedNode l v r) = BBNode l v r

mergeRB :: RedNode n a -> BlackNode' n m t a -> Either (BlackNode t a) (BlackNode (S t) a)
mergeRB t                   Top                                = Right $! blacken t
mergeRB l                  (BBNode'L ctx v r)                  = mergeBB (RBNode l v r) ctx
mergeRB (RedNode l' v' r') (BBNode'R l v ctx)                  = mergeBB (RBNode (RedNode l v l') v' r') ctx
mergeRB r                  (RBNode'R l v ctx)                  = mergeRB (RedNode (blacken l) v (blacken r)) ctx
mergeRB l                  (RedNode'L (RBNode'L ctx v' u) v r) = mergeRB (RedNode (blacken l) v (BBNode r v' u)) ctx
--mergeRB r                  (RBNode'R l v ctx)                  = mergeBB (RRNode l v r) ctx
--mergeRB l                  (RedNode'L (RBNode'L ctx v' u) v r) = mergeBB (RRNode l v (RedNode r v' u)) ctx
--mergeRB l                  (RedNode'L (RRNode'L ctx v' u) v r) = mergeRB (RedNode (RBNode l v r) v' (blacken u)) ctx
--mergeRB l                  (RedNode'L (RRNode'R u v' ctx) v r) = mergeRB (RedNode (blacken u) v' (RBNode l v r)) ctx
mergeRB (RedNode l' v' r') (RedNode'R l v ctx)                 = mergeRB (RedNode l v l') (RedNode'L ctx v' r')

Since it would be inconvenient for user to keep the height of tree on type level it is sufficient to provide him or her with opaque wrapper.

data RBTree a where
  RBTree :: NatHelper t => BlackNode t a -> RBTree a

empty :: RBTree a
empty = RBTree $ Nil

insert :: Ord a => a -> RBTree a -> RBTree a
insert v (RBTree t) = let (_, r) = insert' v t
                      in either RBTree RBTree r

contains :: Ord a => a -> RBTree a -> Bool
contains a (RBTree n) = find (\n' _ -> case n' of
                                 Nil -> False
                                 _ -> True) (\_ _ -> True) n a

While I can see that dynamic typing may have advantages in certain areas (say monkey patching plug-ins - like in Gnome Shell or Firefox - although it have some drawbacks too) static typing can be very useful too.

At the same time people learn how to make good presentation. By good presentation it is usually meant a presentation that keeps people engaged and interested. While the advice often vary from person to person some rules usually have similar direction.

• Avoid reading slides, use short sentences (or even not full sentences)
• Avoid too many slides
• Avoid too many data on one slide (in effect speak a lot more then there is on slides)
• …

Later on the presentation may be published on the internet for anyone to view, or even published as main or only documentation. Since bandwidth can be far from unlimited often only the slides will be uploaded… leading to situation when no one who hasn’t attended the original one has no chance of decrypting what the person meant. Yeah, sure it might have nice pictures and supplemented presentation well, keeping audience focused and entertained, but guessing what does a picture of a baby with title “Happy user of program X” actually mean is well beyond of my ability. If the slides are published the game changes – it is no longer the case that slides are supplemental but they become primary source and the presentation primary context is missing from “good presentation”.

Another popular method, which is used if allowed by bandwidth, is to upload video from presentation. If the presentation was “good” then it might have a better chance of being understood then slides alone. However this method is not without it’s own drawbacks as well. During search for something (be it a library, existing research etc.) the skim reading through slides takes 1 minute and gives at which point it can be decided if it is useful or not – task which is much longer during 2h presentation. Similarly listening is much longer tasks then reading, which makes it much more time wasting (it might also depend if someone is oriented on hearing or seeing). Finally the videos are not searchable with current technology. If I know that something was mentioned in the article searching is as easy as pressing Ctrl+F – this is not the case with videos which depend on remembering structure and searching for right moment. Even if the videos was divided into chapters, which is a rare case, this will not help find anything that was say a side remark (say how to set up the environment if it wasn’t mentioned in introduction).

It looks like that the problem is that the rules are meant to be used during good presentation and later on the slides are reused on a web. Given that they become de-facto documents it might be better to have slightly worse presentations if it means better documentation.

Currently various desktop environments provides their own scripting methods (PowerShell, AppleScript, KJSEmbed. It might be good to look if it fits GNOME as well.

Requirements

There are a few requirements I consider basic to operation of scripting in GNOME:

• Language agnostic. Person who writes script might now one language and (s)he might not want to learn another one. (S)he might have other scripts (s)he wants to integrate – if possible the scripting framework should not force the user to change the programming language. Such framework would allow to write even new, user-oriented languages or even ‘GUI’ scripting (programming by blocks dragged’n'dropped instead of text)
• Live ‘outside’ program. It might be useful to script one application but the usecase I have in mind might involve more than one application. For example I might want to open an attachment from Evolution in Evince and print it.
• Hide ‘memory problems’. Text editor might contain thousands of words in a single document – it’s bad idea to have one object per character or even per word even if we want the client to be able to operate on single characters or even ranges of characters. On the other hand we don’t want to leak objects just because someone have forgotten to unref object on client, client crashed or get into infinite loop.
• Support hooks. For example I might want to run script when I receive an email or when battery is low.

Transport protocol

It seems that it would be possible to build the system on top of DBUS. It already have good cross language support, does not involve any other dependencies and with tricks can avoid mentioned before memory problems.

The biggest problem right now is that most binding support only export of heavy objects – i.e. general assumption is that a dbus object corresponds to language object (both in bindings as well as standard interfaces). Fortunately it is not an assumption made neither by dbus routing nor by dbus protocol (or in fact dbus). It is possible to register a wildcard handler in libdbus (but not, for example, in gdbus) to receive all messages that have not been received by anything else.

The last part is sending – such wildcard would require (from what I understand) to send to specific endpoint. It would not only be inconvenient for user but also seems to not be supported by many bindings. Again fortunately the dbus protocol supports filtering messages by beginning of path (for example all to /org/gnome/Evolution/Script).

Solution?

Well – to start with I haven’t written any prototype even – just read the documentation of dbus and libdbus. There are a few things omitted in the text – for example the script might be run in application or by daemon. Each approach have certain benefits/drawbacks.

Running in application might be faster if script just operate on objects in given application. On the other hand spawning a few VM in application might hurt the performance (say we have one script in perl and one in python to run on each message). Also a script might crash/hang whole application

Daemon on the other hand might spawn a pool of VMs (if they are sandboxed) and respawn them if any crashes. The scripts might run asynchronously (at least to some extend) as well. However it makes the architecture a bit more complex and distinguish between one time scripts (which don’t use daemon) and hooks (running in daemon).

Therefore the blog was moved to wordpress.com - please wait until the change is propagated through DNSes.

I’m going to Desktop Summit

I am sorry for lack of the exact quotes or references but I was unable to find them. They were spread over several weeks and I found only paid archives so I am not able to pinpoint exact newspapers.

One of the articles spoke about Wikileaks and tried to compare the current situation to 1968 stating that the current internet is too one-contact-only to provide platform for revolution giving as an example a message from twitter which makes one time contact and that the revolution needs continuity.

I belive that there are 2 problems with such analysis. First of all I’m not quite sure how many contacts are one-time-only. Arguably one of the most popular site on the internet, Facebook, is based rather on the continous contacts rather then one time. Some studies indicates that the our core groups involve around 4-5 people. Setting aside the social networks any other communications involve known people – you relativly rarely send e-mail to stranger.

Secondly the internet is not homogenious. According to wikipedia This was the first generation to grow up with television in their homes. (…) Chain stores and franchised restaurants were bringing shared shopping and dining experiences to people in different parts of the world. These factors all combined to create a generation that was more self-aware and more united as a group than the generations before it. – in other words it was the beginning of the mass market. However the internet seems to be somehow the end of mass market in some sense as it allows the comeback of the niches.

While it may not unify the revolution it provides platforms for revolution as it connects similar-minded people. You can find communities of the conservatives, socialists, libertatian, anarchists of all kinds etc. For some ideologies it is primary platform of propagation. Recent events in Egypt and Algieria had shown that it is useful tool in this role.

I don’t want to imply that revolution will actually happen. The article was more interesting as what people think about the internet from outside then the revolution itself.

PS I have writted main part of article on 14 of January. Therefore it speak about wikileaks as “recent events”. I decided eventually to post it now – especially in context of the events in Egypt and Algerial world. I don’t want to discuss what actually happened there and what are the consequences – the important part from this post POV is that internet proved to be useful as a platform.

I don’t use the recommended by developers method of building for two reasons. First of all I have rather limited amount of disk space, especially on ‘/home’, and the duplication of many libraries and include files (I use Gentoo) seems to not be an option for me. Secondly installing in Jhbuild sandbox prevents from seeing overall impact of changes in Gnome on system (like some packages, including Firefox, failing to build some time ago with new gtk+ 2).

As a result it more often does not work then do. Excluding some ‘obvious’ reasons like crashes it now requires new gnome-settings-daemon which requires libnotify 0.7, which changes abi and api breaking everything that uses libnotify on my system (mostly from Gnome so hopefully most are fixed but not only). Libnotify is not parallel installable so I can either move to JHbuild or resign from using gnome-shell.

For gnome-shell language, JavaScript, was chosen as easily embeddable and widely known. In addition it doesn’t pull in another complicated platform (and it allows to avoid “duplication” with glibraries). The problem pointed by others. I guess that most JavaScript use are through libraries like JQuery. Anyway the problem isn’t the language itself – when I tried to write simple plugin some time ago I found very little resources to get a hello-world example working. While mokey patching may provide certain benefits crashing shell when one tries to find out how to add something to GUI is not preferable. Additionally it seems that there are problems with embedding javascript anyway.

Release late, release never. The gnome-shell seems to have some features present on screenshots and videos (including ones from GUADEC) that I could never found in one I get from git/tarballs – like multiple views in overview mode. On the other hand it constantly relies on git including sometimes releases of gnome-shell depending on gjs from git. Finding working configuration seems to be hard – while possibly it is made easier by JHbuild I don’t want to use it for reasons above.

Last point is ongoing discussion about the gnome-shell hardware requirements. Contrary to gnome-panel and metacity gnome-shell requires 3D acceleration and while I’m “lucky” (I have new integrated intel card – not great for gaming but works on Linux) some reported issue on NVidia. Even on intel card sometimes it seems that the system slows down during heavy loads (although it improved in 2.6.36 kernel).

Finally I’d like to remaind that the post is written from my own perspective and I may have biased view (including different method of installing and not being involved into design process). In some parts I exagareted (“Release late, release never”) for only reason of joking – and I hope noone will feel offended because of it. However I have my concerns about Gnome-Shell and Gnome 3.0 which, I hope, are invalid.