Daniel's Computer Blog

Better, type-safe dependency properties in C#, WPF

2011-10-23T10:14:00.000-07:00

WPF is a great toolkit for graphical user interface programming where you can truly (and easily) separate program logic from user interface.
The dynamic and flexible binding mechanism is powerful, yet easy to use (once you grasp it :D). Because of it's dynamic nature, type safety is often an issue. Even the most fundamental constructs, like declaring a property that you can use as a binding target (called DependencyProperty) requires you to use casts. Here as a simple example that declares a DependencyProperty called Number of type int:

using System.Windows;

class ProgramLogic : DependencyObject {

    public static readonly DependencyProperty NumberProperty = DependencyProperty.Register("Number", typeof(int), typeof(ProgramLogic));

    public int Number {
        get { return (int) GetValue(NumberProperty); }
        set { SetValue(NumberProperty, value); }
    }
}

As you can see, there are some constructs that you would usually avoid, such as using typeof and - the single worst construct in every programming language - a cast to int in the property's getter.
While describing this to a friend lately, we said to ourselfs: "Why did they choose this construct? C# does have Generics, so why didn't they use it? Even if it would be just a helper function or class, it'd be much better than that.". So we came up with a wrapper to DependencyProperty, which is a bit shorter, and, most importantly, does not require a cast.

class DepProp <PropertyType, OwnerType> where OwnerType : DependencyObject {

    public DepProp(string name) {
        property = DependencyProperty.Register(name, typeof(PropertyType), typeof(OwnerType));
    }

    private DependencyProperty property;

    public DependencyProperty Property {
        get { return property; }
    }

    public PropertyType Get(OwnerType owner)
    {
        return (PropertyType) owner.GetValue(property);
    }

    public void Set(OwnerType owner, PropertyType value)
    {
        owner.SetValue(property, value);
    }
}

The class DepProp (pick a better name when you decide to use it, please) uses two generic parameters. The first one is the type of the property, the second one is the owner's type, which must be derived from DependencyObject. IMHO, using generic parameters is much better than using typeof. These generic parameters are used to guarantee that later calls to Get() and Set() are called with - and return - the correct types. This way, we made the DepdencyObject.GetValue() call type-safe. Sure, the implementation still uses a cast, but what I find most important is that there are no casts spread throughout the whole project. A neat side-effect is that using this class requires a tiny little less typing than the original DependencyProperty. Another nice feature is that you don't need to call a static member function to register your dependency property. Instead, you use new, just like you would do with any other object. Here is the implementation of ProgramLogic using the new DepProp:

class ProgramLogic : DependencyObject {

    public static readonly DepProp<int, ProgramLogic> NumberProperty = new DepProp <int, ProgramLogic>("Number");

    public int Number {
        get { return NumberProperty.Get(this); }
        set { NumberProperty.Set(this, value); }
    }
}

As we can see, the required code is a little less. But the real nice thing is that the compiler can help you pointing out refactoring errors. Let's imagine that you would want to change your NumberProperty to be of type string instead of int. When you change the generic parameter of your NumberProperty declaration and initialization, the compiler would spill out an error in the "normal" property's getter and setter, because the property is still of type int.
I hope that you find this (really) small helper class useful and that it inspires you to write more code like this that makes your overall codebase more maintainable and readable.

GridLayout for WPF: Escape the Margin hell

2011-07-13T10:48:00.000-07:00

Working with WPF, you surely have come across the System.Windows.Controls.Grid class. It is a derivation of System.Windows.Controls.Panel, which in turn is a base class for layouters. You can use a Grid to create windows that have their child elements arranged in a uniform manner.

When you have worked with the Grid class, you probably know about this issue: Adding elements to the Grid will make them be strung together without any spacing between them. When you want to add spacing, you have to set each element's Margin property accordingly. This wouldn't be half as bad - it's just that, depending on where in the Grid your element is, you have to set the left, top, right and bottom margins differently. This can become quite a tedious task and makes your XAML look far more complicated than it needed to be. And that's not all - imagine you putting together a prototype of a very complex UI. Just think about one of those equalizer/sound processor programs that have dozens of controls arranged in a grid. You finally finished your prototype - you think it looks awesome - and show it to your boss. His face turns to a disgusted visage and he says "Why is it all so clung together? You can barely tell the buttons apart since they are too near to each other. Can't you add more space between them?". With the standard Grid class, you would gasp, look a bit depressed and tell your boss to come again in an hour or two. And when he comes around and wants to look at your updated prototype, you are still not finished changing all the margins correctly.

Forget that scenario. Use the GridLayout class from the download below and change all margins at once. It automatically sets a uniform margin to all child elements of this specialized Grid class. And even better - it is smart enough to set the Margin.Left of elements in the first column to 0. The same goes with the Margin.Right in the rightmost column, Margin.Top in the first row and Margin.Bottom in the last row. With this class, you simply change the newly introduced ChildMargin property and be done - you will see the effect of the changes immediately, live in the WPF designer and at runtime.

Here is a screenshot of a simple form done with the default System.Windows.Controls.Grid class - without setting any special margins. You can see how narrow the space between the elements is and that it does not look very appealing.

And here is the same form done with the GridLayout. Note that only the class of the Grid has been changed - the child elements are exactly the same.

The best of all is that this has been done in very few lines of code. It just took two steps: The first was to introduce a new property in a class that derives from System.Windows.Controls.Grid that would define the margin that the cild elements of the new Grid would use. The second set was overriding

protected void MeasureOverride(Size)

that has been introduced in FrameworkElement. The overwritten version of MeasureOverride enumerates all children and changes their Margin accordingly. After that, the base version of MeasureOverride is called. This makes sure that, before each arrangement, the margins are updated correctly.

Here is the complete source-code of the GridLayout:

using System.Windows;
using System.Windows.Controls;

namespace GridLayoutTest
{
    // The GridLayout is a special Panel that can be used exactly like the Grid Panel, except that it
    // defines a new property ChildMargin. ChildMargin's left, top, right and bottom margins will be applied
    // to all children in a way that the children will have a vertical space of ChildMargin.Top+ChildMargin.Bottom
    // and a horizontal space of ChildMargin.Left+ChildMargin.Right between them.
    // However, there is no margin for the borders of the internal widget, so that the GridLayout itself can be
    // aligned to another element without a margin.
    // It's best to have a look at TestWindow, which effectively tests all possible alignments of children.

    public class GridLayout : Grid
    {
        public static readonly DependencyProperty ChildMarginProperty = DependencyProperty.Register(
            "ChildMargin",
            typeof(Thickness),
            typeof(GridLayout),
            new FrameworkPropertyMetadata(new Thickness (5))
            {
                AffectsArrange = true,
                AffectsMeasure = true
            });
        // The child margin defines a margin that will be automatically applied to all children of this Grid.
        // However, the children at the edges will have the respective margins remove. E.g. the leftmost children will have
        // a Margin.Left of 0 and the children in the first row will have a Margin.Top of 0.
        // The margins that are not set to 0 are set to half the ChildMargin's value, since it's neighbour will also apply it,
        // effectively doubling it.

        public Thickness ChildMargin
        {
            get { return (Thickness)GetValue(ChildMarginProperty); }
            set 
            {
                SetValue(ChildMarginProperty, value);
                UpdateChildMargins();
            }
        }

        // UpdateChildMargin first finds out what's the rightmost column and bottom row and then applies
        // the correct margins to all children.

        public void UpdateChildMargins()
        {
            int maxColumn = 0;
            int maxRow = 0;
            foreach (UIElement element in InternalChildren)
            {
                int row = GetRow(element);
                int column = GetColumn(element);
                if (row > maxRow)
                    maxRow = row;
                if (column > maxColumn)
                    maxColumn = column;
            }
            foreach (UIElement element in InternalChildren)
            {
                FrameworkElement fe = element as FrameworkElement;
                if (null != fe)
                {
                    int row = GetRow(fe);
                    int column = GetColumn(fe);
                    double factorLeft   = 0.5;
                    double factorTop    = 0.5;
                    double factorRight  = 0.5;
                    double factorBottom = 0.5;
                    // Top row - no top margin
                    if (row == 0)
                        factorTop = 0;
                    // Bottom row - no bottom margin
                    if (row == maxRow)
                        factorBottom = 0;
                    // Leftmost column = no left margin
                    if (column == 0)
                        factorLeft = 0;
                    // Rightmost column - no right margin
                    if (column == maxColumn)
                        factorRight = 0;
                    fe.Margin = new Thickness (ChildMargin.Left   * factorLeft,
                                               ChildMargin.Top    * factorTop,
                                               ChildMargin.Right  * factorRight,
                                               ChildMargin.Bottom * factorBottom);
                }
            }
        }

        // We change all children's margins in MeasureOverride, since this is called right before
        // the layouting takes place. I was first skeptical to do this here, because I thought changing
        // the margin will trigger a LayoutUpdate, which in turn would lead to an endless recursion,
        // but apparantly WPF takes care of this.

        protected override Size MeasureOverride(Size availableSize)
        {
            UpdateChildMargins();
            return base.MeasureOverride(availableSize);
        }


    }
}

You can download a complete project with an exhaustive test scenario here:

Edit: I forgot to take the ColumnSpan and RowSpan into account in the first version. This lead to margins being too large when an element with a Column- or RowSpan was spanning to the rightmost or bottommost row or column. This has been fixed in version 1.1. Download link has been updated.

Using .NET and WPF in Win32 legacy applications, Part 1: The basics

2011-05-18T11:21:00.000-07:00

Introduction

There are applications that are just impossible to port to any new technology, because of sheer size. Unfortunately, most of these applications are very important to their creators, such as commercial applications that have been developed for decades. Those applications often matured over the time and are satisfying a broad range of users. Rewriting these in a new programming language or a new framework, while being economical, would be impossible.

That's why interoperation and downwards-compatibility is something every new platform should provide. .NET, with it's new Windows Presentation Foundation (WPF) is one of these platforms. You can use .NET in the (from Microsoft's point of view) "legacy" C++ programming language through C++/CLR. And you can use WPF controls and windows in the now deprecated WinForms classes using the "interop" class System.Windows.Interop.HwndSource. Fortunately, since WinForms is built on raw Win32 API under the hood, this HwndSource class is suitable to be used to integrate WPF with raw Win32 applications as well. And it's not that hard at all.

For the impatient among us, you can directly skip to the download and look at the source code. It's only a few dozen lines of code.

Getting started

To get started, we will create a new, empty C++/CLR project. We need C++/CLR, since, obviously, we need to access .NET from either C or C++ source code. I will assume that the "legacy" application you want to use WPF in is a C++ project, since C projects can easily be ported or integrated with C++ code.

Step 1: Creating a new C++/CLR project

In Microsoft Visual C++ 2010 Express, you do that by choosing File -> New -> Project from the menu. Then you select Visual C++ > CLR > CLR Empty Project as the template. I picked win32wpfinterop as the project name, but you can use any name you want. When you hit OK, you will have a new, empty project.

Step 2: Building the Win32 window skeleton

Since C++/CLR is a superset of the "normal" C++ that you write in Visual C++, you can use the Win32 API just as you please. The window example skeleton does not differ from normal C++ at all. Add a new C++ File to the Source Files folder (filter) and add your Win32 window code. Here's how my code looks:

#include <Windows.h>

// Constants
namespace {
   TCHAR * windowClassName = TEXT("win32host");
   TCHAR * windowTitle     = TEXT("Win32 Host (Win32 WPF Interop)");
   int          windowWidth     = 200;
   int          windowHeight    = 100;
}

// Window message procedure
LRESULT CALLBACK WindowProc(
  HWND hwnd,
  UINT uMsg,
  WPARAM wParam,
  LPARAM lParam)
{
   switch (uMsg) {
   case WM_DESTROY:
      ::PostQuitMessage (0);
      return 0;
      break;
   default:
      return ::DefWindowProc (hwnd, uMsg, wParam, lParam);
   }
}

// Main program entry  point
[System::STAThread] // This is IMPORTANT, but it's for in C++/CLR  only
int CALLBACK WinMain(
   HINSTANCE hInstance,
   HINSTANCE hPrevInstance,
   LPSTR lpCmdLine,
   int nCmdShow)
{
   // Register our Window class
   ::WNDCLASS wndclass;
   wndclass.style = CS_VREDRAW | CS_HREDRAW;
   wndclass.lpfnWndProc = &WindowProc;
   wndclass.cbClsExtra = 0;
   wndclass.cbWndExtra = 0;
   wndclass.hInstance = hInstance;
   wndclass.hIcon = NULL;
   wndclass.hCursor = NULL;
   wndclass.hbrBackground = reinterpret_cast <HBRUSHgt; (COLOR_BTNFACE + 1);
   wndclass.lpszMenuName = NULL;
   wndclass.lpszClassName = windowClassName;
   ::RegisterClass(&wndclass);

   // Create our main, raw win32 API window
   // We create the window invisible (meaning that we do not provide WS_VISIBLE as the window style parameter), because making it visible and then
   // adding a HwndSource will make it flicker.
   HWND mainWindow = ::CreateWindow(
      windowClassName,
      windowTitle,
      0,
      CW_USEDEFAULT,
      CW_USEDEFAULT,
      windowWidth,
      windowHeight,
      NULL,
      NULL,
      hInstance,
      0);

   // Now that setting up the HwndSource is finished, we can finally make our window visible
   ::ShowWindow (mainWindow, SW_SHOW);

   // Start message processing
   ::MSG message;
   while (::GetMessageA(&message, 0, 0, 0)) {
      switch (message.message) {
      case WM_QUIT:
         break;
      default:
         ::TranslateMessage(& message);
         ::DispatchMessage(& message);
         break;
      }
   }
   return 0;
}

Download here

Yeah I know, it's a bit length ;-) but that's just how Win32 code is. The good news is that adding WPF components is much less code than this skeleton. Here is how your window should look like:

You need to add the User32.lib to your references, since the functions we used are defined in the Windows' User32 library. You need to right-click on your project in the Solution Explorer and choose Properties. Then navigate to Configuration Properties > Linker > Input and add User32.lib as an Additional Dependency.

Step 3: Create a HwndSource

Before we will be going to use WPF classes, we need to add the required WPF Assemblies to our project references. Right-click your project in the Solution Explorer again, but this time choose References. Now press Add New Reference and get a cup of coffee, while Visual Studio loads every registered .NET assembly. (This has become much faster in Visual Studio 2010, but it is still unbearably slow.) We will need the following assemblies to our project: System (for String), WindowBase and PresentationCore (for HwndSource) and PresentationFramework (for Label).

As the simplest example, we will be adding Label (type System.Windows.Controls.Label) to the window in this tutorial. But we can not just add a Label object to the Window -- the Label class has no members that would take a HWND as the parent class or anything like that. Instead, we need to create an intermediate object that builds the very bridge between Win32's HWND and WPF's System.Windows.UIElement. That intermediate object is of the aforementioned type System.Windows.Interop.HwndSource. Creating a HwndSource-object is very similar to creating a new win32 control - The HwndSource constructor basically takes the same parameters as WinAPI's RegisterClass () and CreateWindow() do. We create a HwndSource using this call:

System::Windows::Interop::HwndSource ^ hwndSource = gcnew System::Windows::Interop::HwndSource (
      CS_VREDRAW | CS_HREDRAW, // window class styles
      WS_CHILD,                // window flags
      0,                       // extended windows styles
      0,                       // x position (will be overridden later)
      0,                       // y position (will be overridden later)
      "WPF Interop",           // window title (not visible)
      static_cast <System::IntPtr> (mainWindow)); // parent window

HwndSource is, at the same time, a pure HWND-style window and a WPF UIElement. This means it can be hosted by another HWND (because we passed the WS_CHILD flag), but can host a WPF UIElement itself. And since being able to host one element in WPF means that you can host any number of elements (e.g. by using a Grid), you have the ultimate freedom to embed complex controls written in .NET and WPF in your Win32 application.

Step 4: Finally, add the Label

It's a bit tricky to get hold of the HWND window handle of HwndSource, because of .NET and C++ type differences. The HWND is available as a System.IntPtr. You need to call ToInt32 and reinterpret_cast it to a HWND like so:

HWND hwndSourceHandle = reinterpret_cast <HWND> (hwndSource->Handle.ToInt32 ());

Now we can call any Win32 API functions on that HWND, such as SetWindowPos:

::SetWindowPos (
      hwndSourceHandle,
      NULL, // ignored
      10,   // x position
      10,   // y position
      180,  // width
      80,   // height
      SWP_NOZORDER | SWP_SHOWWINDOW);

The HwndSource now exists, but that alone is not much of a gain. It's not even visible. You need to create some visible WPF control and add that to the HwndSource. Adding to the HwndSource means setting the control as the HwndSource's RootVisual. Here's how you can do this:

System::Windows::Controls::Label ^ label = gcnew System::Windows::Controls::Label ();
label->Content = gcnew System::String ("WPF Label -- it works!");
label->Background = System::Windows::Media::Brushes::White;

hwndSource->RootVisual = label;

When you insert this code between the call to ::CreateWindow() and ::ShowWindow(), you will experience your first moment of success and a window like this should be visible:

Controls that don't required keyboard focus work very good. Controls with keyboard focus work good, too, as long as there are no other controls in the host window that may steal the focus from WPF. The next part will be about making keyboard handling and focusing work correctly, so stay tuned.
(Don't expect it before June or July 2011, though)

There's a complete example project for Visual C++ 2010 (build with VC++ 2010 Express) with commented source code for download.

And besides: You can build a window's complete content in a C# class library e.g. as a UserControl, add that class library's assembly as a Reference in your C++/CLR's project and go ahead and use it in this interop framework we just built. This basically means that you can embed C# windows and controls in your legacy C++ application.

Avoiding Anti-Aliasing when drawing Rects, Lines in DrawingVisual's DrawingContext

2011-04-01T11:20:00.000-07:00

When building my own high-speed grid control recently, I came across the problem that, when drawing the grid lines, they were anti-aliased and looked very ugly. I searched the web and came up with nothing for phrases like "anti aliasing drawingvisual" or "blurry lines drawingvisual" or even "crisp lines drawingvisual". None of them returned any result, until I came across an answer on StackOverflow (which I currently can't find again).

The solution is very easy, while not even remotely logical to anyone not knowing the deep internals of DrawingVisual. First, here is what you usually would try to draw lines from position (10, 10) to position (200, 10):

drawingContext.DrawLine(new Pen (Brushes.Black, 1), new Point(10, 10), new Point(200, 10));

The result would like something like this:

As you can see, the line is not really black. And it is not one pixel wide. That's because it is rendered "blurry".

The obvious, totally intuitive way (oh, did you notice the irony?) to avoid this, you have to do this:

drawingContext.DrawLine(new Pen (Brushes.Black, 1), new Point(10.5, 10.5), new Point(200.5, 10.5));

And here is our desired result:

Yes, you have to add 0.5 to your fraction-less position. And you will get pixel-perfect lines. This works for DrawRect, too.

Here's a full example with some grid lines:

class DrawElement : FrameworkElement
    {
        protected override void OnRender(DrawingContext drawingContext)
        {
            double x = 10.1;
            Pen pen = new Pen(Brushes.Black, 1);
            while (x < ActualWidth)
            {
                double actualX = Math.Ceiling(x) + 0.5;
                drawingContext.DrawLine(
                    pen,
                    new Point(actualX, 0),
                    new Point(actualX, ActualHeight));
                x += 10.1;
            }

            double y = 10.1;
            while (y < ActualHeight)
            {
                double actualY = Math.Ceiling(y) + 0.5;
                drawingContext.DrawLine(
                    pen,
                    new Point(0, actualY),
                    new Point(ActualWidth, actualY));
                y += 10.1;
            }
            base.OnRender(drawingContext);
        }
    }

And here's the result:

Listen to what Microsoft has to say about using colors in user interfaces

2010-12-07T00:29:00.000-08:00

I just stumbled upon a very interesting and important article on MSDN about colors and their use in user interfaces.

After the fundamentals of different colors spaces are explained, the article continues with design concepts and guidelines that are important to every software developer not exclusively programming on monochrome displays. It highlights the fact that a large portion of the male population has difficulties distinguishing colors and links to the publication Can Color-Blind Users See Your Site?, which describes this topic in more in-depth. But not only color blind people will get annoyed by software that uses colors that are hard to distinguish.

Another highlighted aspect are themes and the possibility for users to change their software visuals to their liking. Those wishes should be respected by every software developer and designer.

The primary goal of a user interface is to ease the user's perception of what is going on and/or how he can accomplish what he currently wants to do with your software. There are many different ways to make this as easy as possible, and icons and colors are a very important part of it. But there are many things you can do wrong when styling your application. You have to take factors like different display hardware and lighting conditions, people's unique color perception and individual themes into account.

What include guards in C++ are, and what they are not

2010-08-18T10:41:00.000-07:00

And why they do not solve "multiple definition" errors.

Include guards are heavily used in C++ and you see them in virtually any code base (that has more than two files). Sometimes there is a bit of confusion what they do, and what they can not do. I decided to explain everything in detail, so I ended up explaining the whole C++ compilation process of preprocessing, compiling and linking your application or library.

Introduction to include guards

First, it's important to know that include guards are not a language feature. They are a technique that use standard preprocessor features to solve a common issue. Namely, they avoid that one header is included multiple times. Let's assume a simple example of two files: test.cpp and the corresponding test.hpp:

test.hpp:
class Test
{
   public:
      void foo();
};

test.cpp:

#include <test.hpp>
/* Some more includes */
#include <test.hpp> // Error! class Test is already defined

void Test::foo()
{
   // Do something here
}

int main()
{
   Test test;
   test.foo();
}

As you can see, test.hpp is included two times in this example. This might not be common in the same implementation file, but when the list of include files is very long, this might very well happen. Remember that an #include is basically the same as when you copy & pasted the content of the included file at this very position. Only that this is done for you by the preprocessor. The resulting code when you run test.cpp through a preprocessor will look like this:

test.cpp - preprocessed

class Test
{
   public:
      void foo();
};

class Test
{
   public:
      void foo();
};

void Test::foo()
{
   // Do something here
}

int main()
{
   Test test;
   test.foo();
}

It should be obvious now why the error occurs. You can't define two classes with the same name in the same translation unit. And that's where the include guards come to the rescue. The basic idea behind include guards is: Do something in the header that will make the preprocessor not "copy & paste" the content into the including implementation file a second time. Besides getting rid of duplicate declarations, this even speeds up compiling, since otherwise the compiler would have to process the same content twice. This goal is achieved by checking whether a certain preprocessor macro is defined. If not, the macro will be defined so that a second check will notice that it is already there, which means that the header is already included. Here's how it is be done:

#ifndef TEST_H
#define TEST_H

class Test
{
   public:
      void foo();
};

#endif

This is what happens, from the viewpoint of the preprocessor:

1st Include of test.hpp
- Is TEST_H defined? - no
- Define TEST_H
- Include content of test.hpp
2nd Include of test.hpp
- Is TEST_H defined? - yes
- Skip to the #endif direction
- Include everything in test.hpp after the #endif
  
  (In 99.9%, this should be empty)

This leads to the desired result for the preprocessed test.cpp, even when we include test.hpp twice:

test.cpp - preprocessed with include guards
class Test
{
   public:
      void foo();
};

// Here should be the second include, which has been avoided by the include guard

void Test::foo()
{
   // Do something here
}

int main()
{
   Test test;
   test.foo();
}

Compiler-specific directions, #pragma once

Since include guards are so common, some compiler vendors (read: Microsoft) decided to create a special directive for them. You might have come across the statement #pragma once while reading through code. This #pragma once is all three preprocessor-directives in one. It replaces the #ifndef, the #define and the #endif (where the #endif is always at the very end of the file). As this is a vendor-specific directive, not all compilers support it. One reason for this may be because it does not solve a problem that couldn't be solved otherwise. This is why you should obide the following, simple rule:

Don't use #pragma once

Because:

It's compiler-specific and is not portable
Because of this, not everyone reading your code might know it
It does not offer big advantages over the "classic" include guards.

(I said "big" advantages. As noted on the Wikipedia article for #pragma once, Visual C++ includes optimization code that makes headers using #pragma once be skipped faster than classic include guards. While this might be desirable, I can't imagine that in practice it will be of great benefit. And, after all, GCC includes optimization for classic include guards, too, which might render the speed improvements of #pragma once on Visual Studio minimal.)

The compile process: Preprocess, compile, link

To fully understand the issue at hand, you have to know how the whole compilation procedure for C++ works. It is divided into three steps.

The first step is the preprocessor. Everything that starts with a # sign is a preprocessor directive. The most common ones are #define, #ifdef or #ifndef, #endif and so on. The preprocessor is essential to C++ because it's the only pragmatic way to split up declarations and definitions and make the same function or class usable from multiple implementation files. The preprocessor is called upon each implementation files (to make this clear again: These are usually called .cpp). The implementation files include header files (and should never include other implementation files). All preprocessor directives in the header files will be processed, too, which means that you can #include other files in the header, #define macros and create include guards. The result of a preprocessed implementation file is a large file that includes every included header, and of course the header included of those headers, and so on. For headers that use libraries such as the standard library, boost, Qt or something like this, the result will often be huge.

The second step is the compilation. This huge mess - also called the translation unit - will now be run through the compiler. Yes, the compiler only compiles one file. And this is not the .cpp file you might have passed on the command line or that you added to the project. It will be a file generated from the preprocessor that does not even remotely resemble your original .cpp file (except for the very bottom part). Most noteworthy, you will not find a single preprocessor directive (like #define) in this resulting file, since they all have been already processed by the preprocessor. The result of the compilation is the object file. The object file is not human readable and does not include source code anymore. Instead, it includes so-called symbols and their content, which might be variables, constants, functions and virtual function tables. This is why you usually have one object-file in your compiler's working directory (this is the Release and Debug directories for Visual Studio. GCC puts the object-files right where your .cpp file is, if you do not specify a file explicitly) for each implementation file in your project. You don't have object files for headers, because they are not individually compiled. They are only "glued" on top of your implementation files. At this point it is important to know that when the compiler creates a function call, it does not include the absolute address of the function in the generated code. Instead, it includes the symbol of that function. In C, this simply is the function's name. In C++, it's the function's name mangled with some meta-information that depends on the compiler. It's very similar with global variables and virtual function tables. They are not referenced by an absolute address, either, but via their symbol name. This means that the object files created by the compiler do not include machine code that is ready to be executed. The symbols have to be replaced with their absolute addresses. And this is where the linker jumps in.

The third and last step is the linking. Just like the compiler, the linker does not know about header files. And it does not know about implementation files, either. It only knows the object files and the linker's job is to link all object files together to one executable or library, that can finally be interpreted by the hardware. It does this by pasting all object files into one, and whenever a symbol occurs (this might be in a virtual or static function call or when referencing a global variable, for example), it replaces this symbol with it's absolute address. That is why you sometimes get errors like "Symbol xyz is already defined in foo.obj". This means that you defined a function or variable in two different translation units, which will generate the same symbol. This is not allowed per One Definition Rule because the linker would not know which of them it should use.

Multiple definitions

There are two ways you can cause multiple definitions of the same symbol: You accidentally define the same global variable or function at two distinct places (e.g. one time in the header and a second time in the implementation file) or you include a header that defines a global variable or function in two separate translation units. The first issue is handled by the compiler. As this double definition is in but one translation unit, the compiler notices that the identifier in question is already declared (and probably defined) and bails out. Here is an example of a double definition of a function foo():

test.hpp
void foo() { } // Note the empty function body!

test.cpp

void foo()
{
   /* Do something */
}

This sometimes can happen when changing a function to be inline and forgetting to remove the original definition.

The second issue is a bit more tricky.
Remember that a translation unit is the implementation file prepended by all included headers. So we take the same header as before, defining an inline function void foo() { }. But this time, it is included from two files in the project, one named test.cpp and the other named test2.cpp:

test.hpp
void foo() { }

test.cpp
#include "test.hpp"

// The actual content of the program is irrelevant
int main ()
{
}

test2.cpp
#include "test.hpp"

// Here are some functions defined in test2.cpp

When you compile this project, you will get an error message along the lines of

multiple definition of `foo()'

. This is because both translation units define a symbol foo, which is prohibited by the holy One Definition Rule.

Include guards to the... rescue?

So, you might think .o( When the function may not be defined twice, let's wrap test.hpp in an include guard! ). Try it and see for yourself. It will not make a difference. This is because the preprocessor will be run for each implementation file separately, leading to two distinct translation units. When test.cpp is preprocessed, TEST_H will not be defined and the header's content will be included in the resulting translation unit. Then the resulting code is compiled, leading to a test.o or test.obj, without any error. There is only one declaration and definition of the function in this translation unit, after all. When the second implementation file is being preprocessed, the preprocessor starts over, which means that #defines made in the first translation unit will not be available in the second or any other preprocessing. This means that TEST_H is still not defined for test2.cpp, so foo()'s definition will be included a second time. It will now be compiled into test2.o or test2.obj. foo() will be defined in both, test.obj and test2.obj.

The final step is the linking, which should lead to an executable or machine-readable library. But the linker notices that foo() is defined two times, prints the aforementioned error message and aborts it's task. Because the linker does not even look at the source code that the object files were compiled from, it does not know where the double definitions occured. In fact, the linker even works without the source code being available.

This means that include guards do not solve multiple definition errors.

Samsung N150 Eliah Netbook review

2010-04-23T10:20:00.000-07:00

I recently bought a white Samsung N150 netbook because I need it for my upcoming vacation in Japan. There are two target audiences for netbook users: Customers with a limited budget and customers that appreciate the small size and high portability of such devices. In my case, both was true. Since I've spent most of my money on my trip to Japan, I was not able to buy a laptop for like 600-700€ or something. Additionally, I wanted something small that would easily fit in my backpack, including a protectional case, and that does not weigh too much. With it's 10" display and a total size of 180x264mm (10.4"x7.4") and a weight (including battery) of 1240g (2.73 lbs), it fully meets my mobility expectations. You don't even notice the weight when carrying it in a backpack or bag.

Specs

Here are some additional specs: Like almost every netbook, the N150 features an Intel Atom at 1,6 GHz. It already uses the newer generation that has been introduced at the beginning of this year. The screen's resolution is 1024x600 pixels, displayed by an Intel GMA 3150 chipset. The resolution can be scaled to 1024x768 for applications that need this resolution, but it then obviously looks ugly. There is an external VGA plug that can output up to 2048x1536 pixels. Neat! I already tried to connect my HDTV and it perfectly works a Full HD 1920x1080 resolution. The N150 ships with Windows 7 Starter and a few drivers, utility software and games pre-installed. On the first boot, the installation is automatically being finalized, which took something like 2 hours, d'oh. In the setup process, you can choose the sizes of the two partitions that will be created. You have a total of 250GB harddisk available for your needs. I chose to have a 80GB system partition and left the rest for data storage. Because Windows 7 Starter only supports 1 GB of memory - you have to upgrade to Windows 7 Home Premium for something around 80€ when you want more - the netbook has only 1 GB of DDR2-800 installed. The memory can easily be upgraded, because there is a special removable cover to access the module. Another limitation in Windows 7 Starter is that you can't use the external monitor as an extended desktop. Only cloning of the image is allowed. The problem with not being able to change your desktop wallpaper on Windows 7 Starter can be circumvented using this trick.

Connectors

It comes with a total of 3 USB ports, two on the right and one on the left. The one on the left is the only one that can be used with devices that draw power from the USB port and is labeled with a small "power" symbol. You can enable and disable the power output on this port with a special utility that comes pre-installed. Microphone and external speakers or headphones can be connected via two separate plugs. The aforementioned VGA connector is on the right side, while the ethernet cable can be plugged in on the left side, next to the round, relatively small power connector. Only 10 and 100mbit ethernet are supported - so no gigabit ethernet on the netbook :-( The AC adapter works with 100 to 240 volts, making it perfectly suitable for my stay in Japan. Quite hidden is an SD-card slot beneath the touchpad with the typical, somewhat fiddly cover. It supports the SD, SDHC and MMC formats.

Internal peripheral

That's it for the connectors for possible external peripheral. There already are some nice hardware pieces built in. One of it is the mandatory 802.11 b/g/n WLAN, but Bluetooth is also included, which can be quite handy to transfer pictures from your cell phone when you don't have a suitable (and usually expensive) cable at hand. While you can plug in an external microphone, there is already one built in. It's located next to the touchpad, which was kind of a stupid decision because you easily cover it with your hand when using the touchpad. I used it to Skype with friends and they said that the quality was very good - which surprised me, because it is so tiny. The webcam has a resolution of 320x200 pixels and is very slow. It's not that much fun doing video-chat with it, because you hardly see your movements. A software from Cyberlink is included that can be used to toy with the recorded picture. But it's actually not that great, either. What really bothers me is that there is no hardware switch or other means to disable or cover the webcam. When you got malware installed that can remote-control your webcam, you will never notice. There is no indicator whether the webcam is active or not, either.

Keyboard and touchpad

Being very small, the keyboard obviously does not come with a standard layout. The keys Insert, Delete, Enter, Page up, Page down and the arrow keys are directly accessible. Home and End, on the other hand, are only accessible by pressing the Fn key that's located between the left Ctrl and Windows keys. This is actually the only thing that requires some time getting used to. The rest of the layout is nice and I could immediately touchtype on the keyboard. Too much typing leads to numb fingers (at least for me, of course), because the keys are pretty hard. The numblock keys are located on the regular character keys and can be activated by pressing Numlock (only availble via Fn key). When pressing Fn and the designated numblock keys, they have the same function as the deactivated numblock keys. The F-keys have special functions that can be used by pressing the Fn key:

Escape: Sleep
F2: Battery status
F3: Euro sign (€)
F4: Switch monitor modes (when an external monitor is attached)
F5: Switch backlight on/off
F6: Mute sound
F7: Samsung Support Center
F8:
F9: Disable/Enable WLAN
F10: Disable/Enable Touchpad
F11: Numlock on/off
F12: Scroll lock on/off
Insert: Pause
Directional up/down: Brightness
Directional left/right: Sound volumne

All in all, the keyboard is very usable, despite the Home and End keys and the mislocation of the <>| key.

The multi-touch touchpad has two buttons - no middle mouse button! Scrolling is done with a two-finger move, which sometimes just doesn't want to work, especially when not sitting directly in front of the keyboard. But most of the time scrolling works very neat. There's also a "three-finger-flick" movement available that's supposed to switch between tabs in browsers, but it doesn't work for me (using Google Chrome as my brother on the netbook). The touchpad's sensitivity can be configured and after configuring for my needs, it works flawlessly (well, most of the time, at least). I bought an external mouse that matched the colors of the netbook, but rarely use it - only when gaming. My overall impression of the touchpad is, keeping it's size constraints in mind, very good. And that's although this my first laptop and I'm not used to touchpads at all.

Software

I removed all pre-installed games before trying them, so I can't tell you whether they are fun or not. A 60-day trial of Microsoft Office 2007 and a full version of Microsoft Works is shipped with the N150, too. The Samsung Recovery Solution is pretty neat. You can easily back up your system partition to your data partition and restore it with only a few mouse-clicks. I think that Samsung Recovery Solution is a re-branded Acronis software. Since I own Acronis True Image Home, I've created a bootable USB-stick with Acronis that I use instead of the Samsung Recovery Solution, though. There are other Samsung tools for: Extending battery life, enable/disable the chargable USB port, manage the display, resolution and network settings and a software to update all Samsung tools. As with all other Windows 7 versions, Windows Live comes for free, which is not at all that bad. For example, you can use the Windows Live Movie Maker to convert movies to a lower resolution when playback is sloppy (which is the case with 720p material).

Working with the N150

When I bought this netbook, I thought that I'd have to live with a number of limitations due to the low-end hardware. I was wrong. The 1.6 GHz Atom with 1 GB of RAM actually performs much better than I had expected. I never ran into memory shortage, even when browsing with a number of open tabs, some of them containing a flash stream. Skype works fast and neat, too, although it warns me that my hardware was too slow. Video playback works great for standard definition videos. 720p videos are a little bit sloppy. It feels like only a few Hertz are missing to play them without stuttering. :-( Using VLC I get much better results than with Windows Media Player or Media Player Classic. I noticed that Flash video streams are very sloppy sometimes, though. I haven't figured out why, yet.

Even when under load, the netbook does not get very hot. It's more like a convenient hand warmer than something that would bother you. The fan and the harddisk are quiet, too, although the constant sound of the harddisk seeks can get on your nerves. You don't hear anything at all when there is some background noise (like a TV or something).

The display's low resolution is less of a problem than I initially thought, too. While there is quite a bit of vertical scrolling, it's not that bad because of the two-finger scrolling move (that works vertically and horizontally, btw). Some applications, however, expect a minimum of 1024x768. That's why you can switch to this resolution and scale it to 1024x600 (which looks ugly, of course, but may be necessary for some buttons to be reached).

But the CPU's missing performance shows in heavy computational tasks such as video transcoding. As I said, 720p videos are not playing without disturbance, so I tried to recode it to 480p using Windows Live Movie Maker (since that's what I had at hand). On my desktop system (a Core i7 860), it took something like 7 minutes, while the netbook was working on it for 1:15 hours. Admittedly, it's not very fair to compare the N150 to a system in which the CPU alone costs as much as the complete netbook.

Gaming

I would never have thought that a netbook could be such a neat gaming device. Of course, it's only older games or games with 2D graphics that run on the N150. But since there are awesome classic games, you can have quite a lot of fun with gaming on this netbook. For instance, you can run Quake III in 1024x600 with something between 30 and 80 fps. Yay! You can play Diablo II on it, too, but it becomes very sloppy, sometimes only one frame every few seconds, when there are many enemies on screen. Go check abandonia.com for old games for free, such as Nightmare Creatures or Blood. There's also gog.com that sell classic games for relatively low prices. Modern "mini-games" like Plants vs. Zombies (you can get it on Steam for half the price) or the extremely famous FarmVille (on facebook) are perfectly playable, too. I even play Beat Hazard (on Steam) using a wireless XBox 360 Gamepad. But of course the device has some limits. I tried Torchlight, a modern Diablo-clone from the original Diablo programmers, but the game was running very sloppy and not playable, although it has a "Netbook mode". You need an Ion-netbook to run it smoothly, I guess. Basically every game released before 2000 should be ok. For later games, I'd try the demo before buying it. I ran the oldest available 3DMark, 3DMark03, and scored 670 points. Doesn't sound amazing, but seems to be enough :-)

Conclusion

I am very happy with this device. The price-tag is friendly to your wallet, the size is portable and the material quality is great. There are a few drawbacks (such as not being able to turn off the camera and a missing middle mouse button), but when you can live with them, there's nothing you can complain about. I have had much fun with it so far, and am sure it'll be a good companion on my trip to Japan. Great buy!

C++ Trivia: Another reason to avoid function-style casts

2010-04-15T05:33:00.000-07:00

I was asked to solve a problem that two of my collegues were puzzled by. The source code was short and minimalistic, but the error was not very obvious. Here's the code (modified, because the original included dependencies to Qt):

#include <iostream>
#include <string>

class Test
{
public:
   enum Enum
   {
      First,
      Second,
      Last
   };

   Test(Enum e): m_value(e) { }

   std::string name() const
   {
      switch (m_value)
      {
      case First:
         return "First";
      case Second:
         return "Second";
      case Last:
         return "Last";
      }
      return std::string();
   }

private:
   Enum m_value;
};

int main()
{
   for (int i = 0; i < Test::Last; ++i)
   {
      Test test(Test::Enum(i));
      std::cout << test.name() << std::endl;
   }
}

Can you spot the error without compiling? Don't worry. Even compilers do not agree on this one. Visual C++ 2005 and gcc 4.4.1 both spit out a similar error message in line

"std::cout << test.name() << std::endl;"

. gcc, by the way, has the more helpful error message here:

test.cpp:39: error: request for member ‘name’ in ‘test’, which is of non-class type ‘Test(Test::Enum)’

Does this help you? Have a close look at the code in the for-loop. Only look at the solution when you are out of ideas!

Klick here to see the solution.

The statement

Test test(Test::Enum(i));

is a function declaration. There is a rule in C++ that everything that can be read as a function declaration will be a function declaration. In this particular case, it's a function returning an object of type Test, taking an enum value of type Test::Enum with the parameter name i. It's important to know that you always can put parameter names in parantheses. This means that

void test(int i);

is the same as

void test(int (i));

The failed attempt to cast i to a value of type Test::Enum lead to a construct that looks like a function parameter. So test is a function name, which obviously can't be used in the manner of test.name(). Hint: This wouldn't have happened if static_cast had been used.

Change wallpaper background image on Windows 7 Starter for netbooks

2010-04-10T09:33:00.000-07:00

This came as quite a surprise. I knew that Windows 7 Start might have some limitations. Luckily, the limitation of started applications has been dropped. But the only really annoying thing that it is missing is the ability to change your wallpaper. This is really a bummer and I can imagine that quite a few people will pay the $80 bucks for an Anytime upgrade to Windows 7 Home Premium just for this reason. But I don't want to throw my money at Microsoft for such a silly reason, so I investigated whether it is possible to change the wallpaper through some registry-hacks or something. I found out that many just replaced the original image and it worked for them. They were using older (pre-release) versions of Windows 7 Starter and this has been "fixed" in the final release. So the only way is using tools like WindowBlinds by Stardock. However, this tool will replace the complete look and feel of your Windows installation. Since I like the original look and feel and don't want to replace a large part of my system with 3rd party software, so I looked into more possible solutions. The best tool that I could find is available at a french site and called Starter Background Changer. This tool replaces the "Personalize" screen of Windows with a home-made interface (that has a few funny translations, at least in German) that lets you change your desktop's background image and some other settings. Here's a screenshot of the interface in German:

Hardware price/performance guides for processors and graphics cards

2010-02-04T07:25:00.000-08:00

What started with graphic card manufacturer's mangled naming schemes has long been continued by the CPU manufacturers. The days where you knew that a "GeForce 3" was faster than a "GeForce 2" and a Pentium 500MHz is faster than a Pentium 400MHz are gone now. CPU's clocks and number of cores can not be measured in a linear fashion, hence price comparison became very hard in the last few years. Luckily, a few tools are available to us consumers that make this a bit easier.

CPUs

First of all, there's a very neat CPU price/performance comparison list at pulsiageek's site. It's especially useful because you can sort it by price, performance or price/performance-ratio.

Graphic cards

Then there's a list of graphics cards containing cards from the old 2MB 3dfx Voodoo graphics card up to the latest GeForce GTX 295. The list over at gpureview.com does not contain a benchmark result, unfortunately, but it contains the MSRP.

Mobile graphics

Mobile GPUs are a whole different story, so it's good to know that there is a separate list at notebookcheck.net that contains the 3DMark01, 3DMark03, 3DMark05 and 3DMark06 score, which is probably the easiest indicator for performance. While there are numerous restrictions you can apply to the list to filter out specific graphics cards, there is no MSRP or reseller price mentioned. There's also a list available that includes actual FPS rates for popular games for each mobile GPU.

Scroll to the last highlight in irssi

2010-02-02T00:14:00.000-08:00

Perhaps you know this situation: You have irssi running in screen and have been away for a few days. You come back and see that you have been highlighted a few hours or maybe even days before. Irssi usually shows you the highlighted line in the server window and it may be something like "daniel: What? Nooo!". Now you're wondering "What the heck was he refering to?", since you don't remember the conversation at that time. Now you would start scrolling back, probably hundreds of lines. When you are connected to your irssi via ssh over a slow line, this may take a second or more per page. It often happens to me that it takes 3 minutes or more to scroll back, just to find out that he mistyped another nick.

I now finally learned about a way to scroll to the highlighted position directly. With the command

/scrollback goto [dd-mm] HH:MM

you can scroll directly to a point in time. And since the timestamp is usually displayed next to the highlight in the server window, it's possible to jump straight to the highlighted position. You can then use

/scrollback end

to go back again to the most recent message in the window.

I was annoyed by that tedious scrolling task for years now, and asked in #irssi every now and then about it, but until today nobody could give me a solution.

Thanks to jink from #irssi on freenode for telling me about /scrollback goto!

Caviar Green, WD10EADS: Green is not my color

2010-01-06T12:19:00.000-08:00

I recently bought a completely new PC, because my water-cooling leaked and I was fed up with it and wanted to start from scratch. The new system has top-notch components that deliver extremely fast performance in games, video-encoding, etc. I "accidentally" bought two 1TB Western Digital "Caviar Green" (product-code WD10EADS).

These disks are more than mysterious. Nobody seems to know how many rpm they spin at. Some resellers list them with 7.200rpm, others with 5.400rpm. There are even specifications stating "5.400-7.200rpm", which could be an indicator that they are "Green", because they spin down when not heavily used. Seems like a nice technology - get all the performance when needed, keep noise and power consumption low when not. It's the first time I've heard about this and it's still only an assumption. Western Digital states disk-speeds for their blue- and black-series with 7.200rpm, but the green-series is just missing the specs.

My actual experience with these disks is... frankly put, horrible. I created a RAID1 of two of these disks and made one 100GB partition for my Windows 7 installation and the rest for game-installations, movies and all the other stuff. Windows-installation was quite smooth (if that's even possible), but system startup and program startup times were disastrous. When the system booted with nothing but Firefox installed, while it was still doing it's usual post-login stuff, starting Firefox took something around 3 minutes. This, of course, does not apply when the system startup was completed and the disk was idle. My impression is that overall data throughput is quite ok with these disks, but seeking and concurrent accesses totally kills performance. I'm sorry I did no benchmarks with actual values I could present, but my subjective impression was more than enough for me to decide that I'll return these disks. I think I'll choose the Western Digital Black series instead (which has double the power consumption according to the manufacturer).

Conclusion

Don't get me wrong - I don't think these disks are bad overall. They are just put to use in the wrong place. I can see good usage for them in external usb-housings or as a second data-drive in an office computer. Or an internal or external backup medium. But not as a system drive or a drive that is accessed frequently and concurrently. What I am most angry about is Western Digital's information policy. Why can't they clearly state how the disks work and how much rpm they spin at at what conditions? Holding back information about your products is nothing I can accept.

Bye bye data-loss, thanks to root on USB-flash

2009-12-26T04:23:00.000-08:00

I still use my server, that is basically a self-made NAS additionally used for irc that's always online (via irssi and screen) large over-night downloads (so I don't have to keep my energy-hungry main pc turned on). I occasionally use it to test out some Linux-stuff (like testing C++-code on different versions of GCC), too.

The server houses 6 SATA drives that are all members of a software-RAID5 and one IDE hard-disk I keep the OS on (previously Debian). I did not want to install the OS on the RAID, because I wanted it to store data only. Since I had bad experience with HDDs in general and IDE-disks especially, this one IDE drive was always something I wanted to replace. Unfortunately, all SATA ports of the motherboard and the additional Adaptec controller are already in use, so I had to stick with the IDE port.

Recently I could not access this box anymore and when I hooked up a monitor to check out what's wrong with it, the console was filled with i/o errors on hda. "Now is the time to replace that last piece of possible failure", I said to myself. A reboot brought no failures, though, and till today I still think the disk is okay and it was a software-failure, but I still wanted to replace the drive with an USB-stick. I had read about booting from USB before and already had an USB-stick with an Ubuntu Live-CD on it. So I ordered another one to use as the OS's root drive. Well actually I used the stick I already had and replaced it's day-to-day usage with the new one, since it was smaller and looked nicer. I did not want to boot a Live-CD, but actually install Ubuntu on the stick, though. When I googled for "Ubuntu USB install" or similar keywords, I always ended up with a Live-CD or a read-only root, and even that took quite a tedious process. But then I stumbled upon this tutorial that made clear the obvious: You can simply put in your Ubuntu CD and your USB stick and install to it like you would to any other drive. Damn! It was so easy :-) The only caveat, as mentioned in the tutorial, is that on the last step of the installation you have to explicitly choose the USB-stick as the boot-device. By the way, it's freaking neat to be able to manually select the boot-device! And on another side-note, the new Ubuntu installer itself is pretty neat, too. It's easy, even not so tech-savvy people could use it without problems. So I burned Ubuntu 9.04 to a CD, booted from it on my main PC (since the server does not have a CD-ROM), plugged in the USB-stick and installed Ubuntu on it. Then I plugged it into the server, did some last fiddling with the settings, copied over the content of the IDE-disk to a backup-directory on the RAID, turned the system off again and unplugged the IDE-disk.

It all worked pretty well, except that the device's name of the USB-stick changed after unplugging the hard-drive. I only had to manually edit GRUB's boot-command (that is another feature that has saved me hours of fiddling already) and later edit it's default configuration. I just guessed that sdh1 has become sdg1, which wasn't too difficult. :-)

Finally! A completely IDE-less system! And additionally, I have minimized the risk of data-loss, because I'd suspect the chances of a head-crash on an USB-drive to be pretty low. Sure, flash-memory wears off after time, but I hope that time-span is much larger than the durability of a hard-disk.

Qt-widget to visually select the corner of a screen (for popup-windows)

2009-12-24T01:01:00.000-08:00

I recently had to create a popup-window that displays active tasks for a user. Because I am frequently annoyed by popups showing at exactly the wrong monitor or position, I included a feature to change the monitor and that monitor's corner that will be used as the popup-position. I'm using QDesktopWidget to find out the available monitors and calculate the correct position for the popup. In the first version of this popup-window, the user had to select the position and monitor using a standard menu. Because that's not very intuitive, I decided to create a widget that displays the available screens in a miniature-preview. The corners of the screens are highlighted and can be selected with the mouse. When the mouse hovers a corner of the preview, a small popup-like window is displayed at that corner's position of the real monitor.

Here are two screenshots:

You can view and download the source in my github-repository and you can download a sample-application for Windows here.

Firebird database-corruption and what you can do about it

2009-12-04T01:26:00.000-08:00

There are usually only two reasons why a database can be corrupted in Firebird: 1) You copied the fdb-file while the Server was accessing it and 2) You don't have forced writes enabled and the server shut down unexpectedly (e.g. power-outage).

In these two cases you often get error-messages like "Internal gds software-consistency check", "Database file appears corrupt", "Wrong record length" or similar when trying to access certain tables or datasets. This is quite bad and means that your database is corrupted and that you have to try to recover it. Because recovery may fail and other steps like using IB Surgeon may be neccessary, it's a good advice to first stop the Firebird server, copy the fdb-file and start the server again. This copy can be used to start over using a different approach or send to a database-repair-service.

Database recovery is done in four steps. Here's how you do it:

Validating the database

The first step for recovery is letting gfix validate the database structure. This is done with the parameters -v -f. A call to gfix could look like this:

gfix -v -f -user sysdba -pass masterkey server:/path/to/database.fdb

gfix may then report errors in the database-structure. When it does not, your database is fine and recovery is not necessary.

Preparing for backup

The second step is preparing the database for a backup, using gfix again. The parameter is -mend in this case:

gfix -mend -user sysdba -pass masterkey server:/path/to/database.fdb

Creating a backup

The third step is a critical one. If this step fails, recovery using tools shipped with Firebird may not be possible. In this step we will create a backup of the fdb-file using gbak. gbak may fail to create the backup because unrecoverable errors exist in the database. Creating the backup is the same as always:

gbak -b -v -user sysdba -pass masterkey server:/path/to/database.fdb /path/to/backup.fbk

Note that paths of the database are remote (that means the path has to exist on the server), while the path to the backup-file is local (and has to exist on the machine you are executing gbak at). When you are lucky, gbak finishes this task without errors. You can then be certain that your data has been saved!

Restoring the backup

The fourth and last step is to restore the backup as usual, again with gbak:

gbak -c -r -v -user sysdba -pass masterkey /path/to/backup.fbk server:/path/to/database.fdb

Finished!

Now everything should be fine. But how can we prevent this from happening again? I initially mentioned two reasons for database-corruption to happen, which both can easily be avoided.

Never copy a database-file when the Firebird-server is accessing it.

Or better yet, never touch the file when the Firebird-server is running! That's the safest way to ensure that your database-file is copied uncorrupted and your data stays safe.

Enable forced writes unless you have very good reasons not to do so

Forced writes are a mechanism by the Firebird-server to tell the operating system to disable any disk-caching mechanisms for this file. When forced writes are not enabled, Firebird does not know what parts of the file are physically stored to the medium. That's one reason why having databases on a network-share is not possible and/or highly discouraged. Because disk-caches are not invented without a reason, this comes with a drawback. Writing will be a tad slower with forced writes disabled, so there might be a reason you don't want to enable it. When you disable forced writes, you have to ensure that the server is connected to an UPS and will always properly shut down. This means hitting the reset-button is a big no-no with forced writes disabled!

When following these rules, it should never be neccessary to recover a database again.

Start application without elevated privileges on Windows Vista and Windows 7

2009-10-08T10:44:00.001-07:00

I finally found out how you do this. You sometimes have applications that have something like "install" or "setup" in their names. Windows Vista and Windows 7 are stupid enough to enforce administrator privileges when starting these applications. But often this is absurd and breaks the applications. Let's say it's a command that's called "InstalScript" that installs a new script to your application's settings directory, which is in the user's APPDATA path. Someone uses your application in a company that has a Windows Domain in-place and his user-account does not have administrator privileges, which is usually the case. This user would now be unable to use your script, because he can't run the InstallScript application.

Since the release of Vista I was always wondering how you could prevent this, but never took the time to google for a solution. At some point, I even assumed it was impossible. Well, it nearly is ;-) Since you are reading this article the chances are high you are developing for Windows. And the chances that you stumbled upon manifest-files are high in this case, too. If you ask me, .manifest is short for "Manifestation Of The Royal Pain In The Ass". But heck, this article is not about ranting against manifest-files. In this case, they are here to help you. You can specify which kind of privileges your application needs in your manifest-file, and when you set "requestedExecutionPrivileges" to "asInvoker", you are free to go! That's the information I found on many websites. Unfortunately, little of them mentioned where this actually should go, and because I'm no meanie, here's an example I took from MSDN:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <assemblyIdentity version="1.0.0.0"
     processorArchitecture="X86"
     name="ShrinkyInstaler"
     type="win32"/>
  <description>Installs Shrinky on your computer</description>
  
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel
          level="asInvoker"
          uiAccess="false"/>
        </requestedPrivileges>
       </security>
  </trustInfo>
</assembly>

Now I can finally call Shrinky's Installer "ShrinkyInstaller.exe" and still debug it! (I put the manifest-file into the debug directory and named it "ShrinkyInstaller.exe.manifest). Yay!

Shrinky Pre-Release

2009-10-04T15:05:00.000-07:00

I had this idea in mind for a long time now. People are often asking me how they can resize images to a smaller file-size, most often because they want to send them via e-mail and don't want to wait 30 minutes for the e-mail to be sent. You can use Microsoft Paint which ships with every Windows for this and it's actually relatively easy to explain how. But most people can't remember the procedure and are lost when they have to do this the next time. So I came up with the idea to write an extremely easy application that can do this and is optimized for high usability and an easy interface. And here's my shot at it: Shrinky (doesn't that sound cute?).

The only thing you have to do when loading Shrinky is clicking on the logo above, select the image-file you want to shrink and then hit "Shrink it!". This will create a file that's about 150-200kb in size, which is quite handy for uploading and sending via e-mail. A new file with the same file-name as the original, with a "-small" appended, will be created. You can tell Shrinky how to resize the image by clicking on "I want to decide myself".

You can shrink images in three different ways:

First, you can enter a file-size you want to have. This can be anything from "100kb" to "1,5 mb" (this is currently the only thing that does not work reliably, because floating-point values are not handled correctly). Shrinky will then try to create a file that matches this file-size as good as possible.

The second way to shrink a file is to enter the width and height directly. This can become handy when resizing an image so it fits in a certain space, such as on a webpage or something.

And the third way is by entering a percentage of the original resolution. Good for situations where you want to, for example, make an image half as big as it currently is.

When you are done setting up Shrinky the way you want, just hit "Shrink it".

Shrinky is completely written in C++ with Qt and is platform-independent. But obviously, since it's aimed at pc-beginners, the most important platform is Windows. That's why I've created binaries for Windows only (I don't have access to a Mac, anyways). See below for a download-link. You can access the source-code via the public git repository on github. Shrinky's currently translated to English and German only. Japanese will surely follow and some nice folks in #qt on freenode already offered translations, too. Man, open source is really nice. ;-)

There's still much to do to make Shrinky a polished product and publicly release it. Here's my todo-list for this project:

Shell-integration for Windows. Easily shrink images via right-click. That's especially important in the file-selection dialogs. With a tight integration, you can easily shrink images while selecting them in your e-mail application.

I need to write an installer for Windows, because extracting a ZIP-archive is not that easy for the target audience I am aiming at.

Processing of multiple files would be neat, too.

A web-page including a donation-facilty via paypal. Albeit being open-source, I'd like to make a few bucks with this application.

You can download the pre-release of Shrinky here. Just extract the zip-archive anywhere on your hard-drive (or usb-stick, for that matter) and run Shrinky.exe. You can post anything regarding your experience with Shrinky as a comment on this blog-post!

Setting the mouse-cursor for Qt's item-views via model

2009-09-30T07:08:00.001-07:00

I like Qt's Model/View architecture very much. Writing models is, after a few short headaches in the beginning, quite intuitive and straight-forward. And it becomes even easier when you don't use QAbstractItemView directly, but a more specialized class like QAbstractListModel or QAbstractTableModel. The basic idea is that the model provides information to the views that is needed to present the individual items to the user. This does not only include the text of the item, but also a decoration (icon), a tool-tip, font, background-styling and more.

One thing I found missing, however, is the mouse-cursor that should be used for the item. For me, it's a common situation that you display a table and one of the columns is clickable and executes an action (such as deleting the row maybe). The user obviously will not understand that he can click cells in this column, unless you change the cursor to something like Qt::PointingHandCursor, for example.

I guess most people that faced this problem derived from the view they are using and override mouseMoveEvent() and do things there. This solution is not in the mind of Model/View, though. The mouse-cursor for an item is something that belongs into the model. So here's what we are going to do:

Create a new Role for the MyMouseCursorRole

Create new subclasses from the item-views we want to use (Qt's only item-views are QListView, QTableView and QTreeView)

In those sub-classes, make sure that the model's data() is called with the MyMouseCursorRole when the mouse moves to a new item (or moves to a position that is not an item)

After this, we can start writing models and handle the MyMouseCursorRole there. In the example, I derive from QStandardItemModel and only override data() and return different shapes for certain items.

In this blog-post, I'm including a small example for QListView. You can get the full source-code for the other views at the bottom of this post.

Here's the header:


#include <QListView>

// Define our new role that can be used in the model.
const int MyMouseCursorRole = Qt::UserRole + 1;

// Define a new constant for user-defined roles in this application.
const int MyUserRole = MyMouseCursorRole + 1;

// MyListView requests the MyMouseCursorRole from the model when the mouse 
// moves over a new row. It does not need to use any tricks like MyTableView
// or MyTreeView, because it does not contain headers.

class MyListView: public QListView
{
   Q_OBJECT

public:
   MyListView(QWidget *parent = 0);

protected:
   virtual void mouseMoveEvent(QMouseEvent *event);

private:
   // m_LastRow stores the row the mouse was over the last time
   // mouseMoveEvent() was called. This is used to minimize calls to the
   // model's data() function. m_lastRow is -1 when no valid row was hovered by
   // the mouse.
   int m_lastRow;
};

And here's the implementation:

MyListView::MyListView(QWidget *parent):
   QListView(parent),
   m_lastRow(-1)
{
   // We need to enable mouse-tracking because we need to know
   // about every mouse-movement.
   setMouseTracking(true);
}

void MyListView::mouseMoveEvent(QMouseEvent *event)
{
   QAbstractItemModel *m(model());
   // Only do something when a model is set.
   if (m)
   {
      QModelIndex index = indexAt(event->pos());
      if (index.isValid())
      {
         // When the index is valid, compare it to the last row.
         // Only do something when the the mouse has moved to a new row.
         if (index.row() != m_lastRow)
         {
            m_lastRow = index.row();
            // Request the data for the MyMouseCursorRole.
            QVariant data = m->data(index, MyMouseCursorRole);

            Qt::CursorShape shape = Qt::ArrowCursor;
            if (!data.isNull())
               shape = static_cast(data.toInt());
            setCursor(shape);
         }
      }
      else
      {
         if (m_lastRow != -1)
            // Set he mouse-cursor to the default when it isn't already.
            setCursor(Qt::ArrowCursor);
         m_lastRow = -1;
      }
   }
   QListView::mouseMoveEvent(event);
}

For QListView, that's all! It gets more complex for QTableView and QTreeView, because they contain headers and mouseMoveEvent() is not called when the mouse moves to these headers. But the cursor set with setCursor() is used for the headers, too. This means that, when the mouse moves from an item with a special cursor-shape directly to a header-control, the mouse-shape does not change back to the default. To circumvent this, we have to implement quite a hack, so I'm not covering it in this blog-post. You can see my solution in the example source-code.

To test our new functionality, we have to create a model that handles MyMouseCursorRole and returns a valid Qt::CursorShape. As noted earlier, I derive from QStandardItemModel, because QStandardItemModel already implements needed standard functionality. QStandardItemModel uses QStandardItems as it's soure of information. Because QStandardItem obviously does not have a property for the cursor's shape, I use a trick to store the cursor-information in the item's text. By convention, when the item's text starts with {hand}, Qt::PointingHandCursor is returned in MyMouseCursorRole and {hand} is snipped off of the text for Qt::DisplayRole.

Here's the header's code, it's quite simple:

class MyTestModel: public QStandardItemModel
{
   Q_OBJECT

public:
   QVariant data(const QModelIndex &index, int role) const;
};

And here's the implementation's code:

QVariant MyTestModel::data(const QModelIndex &index, int role) const
{
   if (role == MyMouseCursorRole)
   {
      // We use a little hack here so we don't need to add an extra element to
      // QStandardItem. When the item's text starts with {hand}, we return
      // the Qt::PointingHandCursor cursor-shape.

      QVariant text = QStandardItemModel::data(index, Qt::DisplayRole);
      if (text.toString().startsWith("{hand}"))
         return Qt::PointingHandCursor;
      else
         return QVariant();
   }
   else if (role == Qt::DisplayRole)
   {
      // Cut out the {hand} from actually displayed content.
      QVariant result = QStandardItemModel::data(index, role);
      QString text = result.toString();
      if (text.startsWith("{hand}"))
         return text.mid(6);
      else
         return result;
   }
   return QStandardItemModel::data(index, role);
}

As you can see, I'm using a trick to call QStandardItemModel::data() because I want to access the "raw" data without re-implementing QStandardItemModel::data()'s functionality. What's quite handy is that you can pass a different Qt::ItemDataRole to data(), so I can get the value for Qt::DisplayRole when preparing the values for MyMouseCursorRole.

Now the only component that's missing to see the result is a demo-application that uses the test-model and the new MyListView-class. That's only a few lines:


#include <QApplication>

int main(int argc, char *argv[])
{
   QApplication app(argc, argv);

   MyTestModel model;

   // Demo data
   QList row;
   row.push_back(new QStandardItem("A 1"));
   row.push_back(new QStandardItem("{hand}B 1"));
   row.push_back(new QStandardItem("{hand}C 1"));
   model.appendRow(row);

   row.clear();
   row.push_back(new QStandardItem("{hand}A 1.1"));
   row.push_back(new QStandardItem("B 1.2"));
   row.push_back(new QStandardItem("{hand}C 1.3"));
   model.item(0, 0)->appendRow(row);

   row.clear();
   row.push_back(new QStandardItem("{hand}A 2"));
   row.push_back(new QStandardItem("B 2"));
   row.push_back(new QStandardItem("C 2"));
   model.appendRow(row);

   MyListView listView;
   listView.setModel(&model);
   listView.show();
   app.exec();
}

That's it. I have compiled a complete project that includes derivations from QListView, QTableView and QTreeView, the test-model and the test-widget. However, this is not a complete solution and by far not suitable to, say, be included in Qt's main releases. There are a few things that need to be solved. For example, using setCursor() to set the cursor for the whole widget is completely broken with these changes.

You can download the example source-code here.

Exceptions: Do's and Don'ts

2009-09-29T04:08:00.000-07:00

Exceptions in C++ are a very complex topic. In addition to the standard-facilities of throwing and catching them, there are things like exception-specifiers and throwing arbitrary objects. Here are some guidelines on what you should do with exceptions and what you definitely should not do.

Do's

Use exceptions. But only for exceptional situations. Don't use exceptions for error-conditions that are likely to occur, like misconfigurations by the end-user, etc. However, when the configuration is already supplied by the software and not influenced by the user, it's indeed an exceptional situation where you should throw a C++-exception.

Derive all your exception-classes from a standard exception-class such as std::runtime_error or at least std::exception.

Create your own base-classes for exceptions in your application or library so you can easily distinguish your exceptions from others'. This is especially important for libraries.

Don't create exceptions on the heap. Throw them by value and catch them by value or const-reference.

Make your exception's error-information accessible as atomic as possible. This helps the user of your exception to create meaningful error messages for different kinds of end-users. For example, when there's a database-error, make the database-function that was executed, the line and part of the SQL-statement and the error-message accessible separately.

Also provide a convenience-function in your exception to create a human-readable error message.

The more different exception-classes you provide, the easier it is to handle certain types of exceptions. On the plus side, more exception-classes does not mean more work when you don't care what kind of exception is thrown when following the guideline to have a standard base-class for your application's exceptions.

Don'ts

Don't throw anything that does not derive from std::exception.

Try not to depend on catch(...) because you can't identify what kind of error has occured. std::exception should be the "widest" scope you catch.

Don't use exception-specifiers. The C++ committee has done a half-assed job when inventing those and didn't correct their mistakes in C++0x. There are several reasons why exception-specifiers are broken by design:
- If any of the called functions throws an exception that's not listed in your exception-specifier, std::terminate will be called. This may happen when modifying a function that's used by any function that uses exception-specifiers, without verifying the whole code after the change.
- When you use exception-specifiers in functions that call third-party code, it's possible that they break with an update of that library. This update could even silently happen by exchanging a dynamic library that's installed system-wide, without you doing anything.
- Execption-specifiers always cost speed. Because they are not checked at compile-time, the compiler needs to add code to check for validity of the thrown exceptions, which slows down execution time even when not caring for exceptions at this point.

Don't throw exceptions from extern "C" functions. This causes undefined behaviour, and in this case it means a crash on most popular platforms.

Don't throw exceptions in destructors. Since destructors usually are called implicitly when the object runs out of scope, they're easy to miss. Additionally, it's not a good idea to prevent your application from freeing up resources.

Create your exception-classes exception-safe. For example, using std::string can potentially lead to a std::bad_alloc exception being thrown.

Baffled by templated operator=

2009-08-14T06:14:00.001-07:00

Now this really took me by surprise and I just believed it after extensive debugging. Here's the punchline: A templated operator= is not used for assignments from the same data-type. That means when test has a templated operator=, this does not call it:

test a, b;
a = b;

On a second thought, this is actually comprehensible behaviour. The compiler generates a default operator= for assignments of it's own type that calls operator= for each member. This default operator is even generated when a templated operator exists. And because overloaded functions have stronger "binding" than template-functions, the implicitly created and therefore overloading operator= is called. Here's an example:

#include <iostream>

struct test
{
   template<typename T>
   test &operator=(const T &value)
   {
      std::cout << "template operator=()" << std::endl;
      return *this;
   }
};

int main()
{
   test a, b;
   b = 10;
   a = b;
}

You might think that the output is

operator=()
operator=()

but it isn't. The output contains only one line, because the generated operator= does not create any output.

It is extremely important to know this rule and don't fall into it's trap like I did. In my case I had a pointer-member and simply assigning it resulted in a double-free in the dtor of the last destroyed copy of that object. Here's the correct way to circumvent this:

#include <iostream>

struct test
{
   template<typename T>
   test &operator=(const T &value)
   {
      std::cout << "template operator=()" << std::endl;
      return *this;
   }
   test &operator=(const test &value)
   {
      return operator=<test>(value);
   }
};

int main()
{
   test a, b;
   b = 10;
   a = b;
}

Shrinking Qt widgets to minimum needed size

2009-08-13T00:21:00.000-07:00

Shrinking a Qt widgets to the minimum size they can become because of size constraints or child widgets is not a difficult task. In fact, it's simply one call to resize(0, 0) that does just that. Because even resizes by the programmer are bound to the usual constraints, this resizes to the smallest possible size.

But when trying this in practical situations, it might become tricky. Shrinking your widgets means it becomes smaller. A widget (or dialog, or window) usually becomes smaller after removing or changing stuff. You will notice, however, that after removing a widget, a call to resize(0, 0) will not resize your widget to the size it would occupy without the removed widget, but instead it behaves like the widget was still there. This is because the widget is not completely removed until the next processing of the message loop. There are two ways to make certain that the message loop is being processed. One is calling QApplication::processEvents(). The other is using a QTimer::singleShot() and doing the task you want to do in the provided slot. I've tried both in regards to shrinking the widget and noticed that using a singleShot is more flicker-free than calling processEvents(), at least on the Windows platform.

I've created a demo-application to visualize the problem and check which method works better on a certain platform. Here are some screenshots:

This is the window after opening the application. You can choose between directly shrinking the widget after removing the last button, using a singleShot() and using processEvents().

Two buttons have been added, the dialog has automatically grown.

The last button has been removed, resize(0, 0) has been called directly after removing the widget. You can see that the size is not appropriate, the space the button took is still there. Notice the wider gaps between the widgets.

The last button has been removed, resize(0, 0) has been called in a slot that was activated by a QTimer using a singleShot(). The size of the window is correct, no flicker was visible.

The last button has been removed, QApplication::processEvents() has been called before calling resize(0, 0). The size of the window is correct, but there was flicker visible.

You can download the source-code of the demo-appliction here and try it out yourself.

Background animation for Qt-Widgets

2009-07-16T05:16:00.000-07:00

Sometimes you want to load data from a specific datasource (e.g. a database or the internet), but let the user work on with your application while it is loading. When doing this, it is essential that the user can see that data is being loaded and immediately notice when loading has finished.

In my situation it is a table view that gets populated with entries and then validates every entry, which can take quite some time. So I wanted to show a "loading" animation in the background of the table, since this is unintrusive, but still easily notable. I created an animation using ajaxload and saved it as Load.gif in my application's directory.

Enabling GIF-support in Qt

The first hurdle you have to take is getting gif-support in your Qt installation. In order to do so, you need to pass -qt-gif to configure and have the qgif library (libqtgif.so or qtgif.dll) in the subdirectory imageformats in the Qt plug-in directory. I'm not getting into more detail here, since it's not the main topic of this post.

Drawing directly on your widget

Next you want to be able to draw directly on the background of your widget. You do this using the paintEvent() and a QPainter. Here's a very minimal example how to do it:

Header:

#include <QWidget>

class TsTestWidget: public QWidget
{
   Q_OBJECT
public:
   virtual void paintEvent(QPaintEvent *event);
};

Implementation:

#include "Test.h"

#include <QApplication>
#include <QPainter>

void TsTestWidget::paintEvent(QPaintEvent *)
{
   QPainter painter(this);
   painter.drawRect(10, 10, 50, 50);
}


int main(int argc, char *argv[])
{
   QApplication app(argc, argv);

   TsTestWidget test;
   test.resize(100, 100);
   test.show();

   app.exec();
}

Result:

Animating the background

The second step is animating this gif. Qt has the QMovie class for this. QMovie is not a widget, it's just managing the animation's frames and timers and provides you the current frame's pixmap. It's frameChanged(int) signal is emitted whenever the next frame should be displayed, usually using a QLabel. In our case we don't use a QLabel to display it, but directly paint it on the widget as we did in the previous step. The tricky part is that paintEvent() is only called when an external event happend that requires repainting of the widget. So we use the frameChanged(int) signal and repaint the widget every time the current frame has changed. Here's a simple example:

Header:

#include <QWidget>
#include <QMovie>

class TsTestWidget: public QWidget
{
   Q_OBJECT
public:
   TsTestWidget();
   virtual void paintEvent(QPaintEvent *event);
private:
   QMovie m_movie;
private slots:
   void paintNewFrame(int);
};

Implementation:

#include "Test.h"

#include <QApplication>
#include <QPainter>
#include <QPaintEvent>

TsTestWidget::TsTestWidget():
   m_movie(qApp->applicationDirPath() + "/Load.gif")
{
   connect(
      &m_movie,
      SIGNAL(frameChanged(int)),
      this,
      SLOT(paintNewFrame(int)));
   m_movie.start();
}

void TsTestWidget::paintEvent(QPaintEvent *event)
{
   // First we extract the current frame
   QPixmap currentFrame = m_movie.currentPixmap();

   QRect frameRect = currentFrame.rect();

   // Only redraw when the frame is in the invalidated area
   frameRect.moveCenter(rect().center());
   if (frameRect.intersects(event->rect()))
   {
      QPainter painter(this);
      painter.drawPixmap(
         frameRect.left(), 
         frameRect.top(), 
         currentFrame);
   }
}

void TsTestWidget::paintNewFrame(int)
{
   repaint();
}

int main(int argc, char *argv[])
{
   QApplication app(argc, argv);

   TsTestWidget test;
   test.resize(100, 100);
   test.show();

   app.exec();
}

Result:

Item views are special

I first tried doing this on a QTableView and failed, frustrated. There is one tricky thing about QAbstractItemViews: You don't paint on the widgets, but on the viewport(), because they're derived from QAbstractScrollArea. Because I didn't immediately realize this, I always got a confusing error message when creating a QPainter in the paintEvent() using the this-pointer. That's because I need to create the painter for the viewport() widget. After finding this out, it became straightforward implementing the animation in the QTableView, just replacing this with viewport().

More re-usable approach

So here we are, having our very own background animated widget, even supporting item views. I could stop this post right here and leave you alone with implementing it into your favorite widgets. But there' still one thing that bugs me: You have to do this for every widget you want an animated background in. When you're using QListView, QTableView and QTreeView in your application, you have to extend all three classes with this functionality, which is error-prone and, bluntly put, lame.

So here's my solution how you can minimize the code you have to write for each new widget-class you want to integrate it into. We create a helper class called "TsBackgroundAnimation". This class does all the work and thanks to multiple inheritance, you just have to derive a class from your Widget and TsBackgroundAnimation, override paintEvent() and call TsBackgroundAnimation's paintAnimation()-function there. There's a a "workaround" we have to make here. Qt does not support deriving from multiple QObject-decendants, hence TsBackgroundAnimation may not derive from QObject. This means that TsBackgroundAnimation can not have slots and it's impossible for it to connect the frameChanged(int)-event to a slot that repaints the widget. So instead of QMovie a TsMovie class is used that provides a frameChanged()-event without the int parameter. That way TsBackgroundAnimation can connect that signal to the slot repaint() of the widget. This made the final implementation a bit larger, so I did not directly paste the sourcecode here. You can download the complete project from here:

BackgroundAnimation.zip

BackgroundAnimation.tar.bz2

And here's a screenshot of the final result:

Coolest postcard ever

2009-06-30T05:03:00.001-07:00

I received the maybe coolest postcard ever today. It's from a pal I know from IRC (#c++.de on quakenet) who has been living in Madagascar for a few years now. He's planning on moving back to Germany soon, so I asked him to send me a postcard before he leaves Africa.

Here's what he has written (click to enlarge):

Have fun decoding it. ;-) (The resulting message is in German, by the way)

Compile-time-array in C++ using templates

2009-06-08T12:37:00.000-07:00

We all know that it's possible to use templates for stuff that they were not quite invented for. In this fine example, an arbitrary number is tested for being prime at compile-time: here, see the first screenshot. A friend of mine now came up with the idea to solve this programming contest problem by pre-calculating the needed values using templates and filling a compile-time-array with them. Well, it ended that this was not a feasible solution for that specific task (because of the resulting sourcecode size), but I still wanted to give the compile-time-array a try. And that's what I came up with:

#include <iostream>

struct Null
{
};

template<int V, typename T = Null>
struct Element
{
   typedef T Base;
   static const int value = V;
};


template<typename Array, int Index>
struct At
{
   static const int result =
      At<
         typename Array::Base,
         Index - 1>::result;
};

template<typename Array>
struct At<Array, 0>
{
   static const int result = Array::value;
};

int main()
{
   using namespace std;
   typedef Element<18, Element<23, Element<42> > > Array;
   cout << At<Array, 0>::result << endl;
   cout << At<Array, 1>::result << endl;
   cout << At<Array, 2>::result << endl;
}

Pretty nice I'd say, huh? ;-)

Thanks to mandrill for pointing out that storing the index in Element, as I did in my first approach, is not needed at all.

New interesting "chimaera" programming language: Scala

2009-04-14T07:52:00.000-07:00

Object oriented programming is currently (or rather: still) state of the art and most commonly found. Functional programming always has been a topic, too, but it never really got the break-through that many people think it deserves. Lisp is one pragmatic example of how object oriented and functional programming can go hand-in-hand, although it's not really clean or even pretty (except for it's S-Expression-based syntax).

There's a relatively new programming language on the field that recently even got attention on a spin-off of Germany's biggest IT news site heise Developer. You can read the german article here or learn more about Scala on it's official website or find news related to the language at scala-blogs.org.

Scala is mixing up object oriented and functional programming. The most important aspects of functional programming (at least to me) are closures and being (optionally) free of side-effects. Scala obviously meets both criteria. It is much more practical than most other functional programming languages because it's based on Java and/or .NET and therefore can use a huge amount of existing third-party libraries out of the box. The most annoying thing about Lisp and, let's say, Haskell, is the absence of existing, maintained libraries to actually do something useful without re-inventing the wheel. Well, there's L Sharp .NET, but well... it's not the real deal and it's not as mature as Scala seems to be. It's even rumored (on Slashdot) that Twitter is moving to Scala.

German community effort to create a good (tm) C++ Tutorial

2009-03-06T02:09:00.000-08:00

I have started to gather people to help me write a C++ tutorial that will just blow away everything that's been around. It will be standards compliant, teaching clean, good coding style, fun to read and practical. We put emphasize on making C++ tasty to programming beginners by providing them the necessary tools to easily create graphical output. Our opinion is that tutorials that teach console programming are boring the read too fast, so we've created a "playground" that the reader can use to write graphical programs without first having to learn how to use complex GUI or graphics libraries. We believe that this approach will attract many readers and will make C++ more attractive to beginners (or maybe even advanced programmers that are tired of wading through a console tutorial).

The team currently consists of roughly 6 people from #c++.de on irc.quakenet.org, but I constantly try to recruit new writers. The tutorial's first version and initial draft are all done in German, because the team consists of Germans only and we'd like to keep the barrier to start helping and writing articles as small as possible. We hope that the final product will be completely available in English some time.

Featured interview at FirebirdNews.org

2009-03-05T01:43:00.001-08:00

Marius Popa from FirebirdNews.org interviewed me as a part of the "People around Firebird" series. I had a chance to talk about myself, my projects and my views on topics like Firebird and Qt.

It's an interesting read, don't miss it!

fbsql: Clean client-libraries for Firebird and Interbase databases

2009-02-26T08:40:00.001-08:00

I've written this quite some time ago (and it didn't receive much love since then, but it's functional and should be bug-free), but I just noticed I haven't mentioned it on my blog: fbsql is a clean C-API that can be used to create bindings for other languages (such as Python or Ruby). It's much simpler to use than the raw Interbase/Firebird API and it uses IBPP internally. So it's basically just forwarding the function calls to IBPP and abstracting the IBPP classes to a C interface that is straight forward and intuitive.

I have included a binding to Lisp, since this was my personal main goal with this project. I noticed that there is no way at all to access Interbase and/or Firebird from Lisp, and since I like toying around with Lisp, I thought that writing small database helper applications with it is a good way to get comfortable with it. The Lisp binding has two layers: First is the CFFI that is used to access the native C functions from the fbsql C-library and the second, more lispish layer is a bit more advanced and better to use. It includes thinks like


(with-statement (st database transaction) 
   (statement-prepare st "select * from table")
   (statement-execute st)
   (while-statement-fetch st
      (format t "~a~%" (statement-get-short st 1))))

This little script creates a statement to the given database and transaction, executes "select * from test" and prints everything value in column 1 (which should be a short).
Lisp is so wonderfully compact. ;-)

ZeroBUGS: Commercial C++ debugger for Linux

2009-01-25T23:59:00.000-08:00

Whenever working with GDB on Linux you get the feeling that it does the basics pretty well (and sometimes even crashes on those basic tasks), but a few things are just missing or too complicated. While GDB is actually a very usable debugger and freely available on virtually any Linux box, it's not 100% stable and sometimes not as high quality as you'd like it to be.

I've just found out about ZeroBUGS, a commercial debugger for Linux (and Linux only!) that's targeted to be better where GDB is lacking. I learned about it from Andrei Alexandrescu's homepage (see the "Zero" link to the left). Although it is a commercial product, there's also a free version available, but it's limited in some ways. ZeroBUGs comes with a GUI and console interface out of the box.

Here's the feature-list, taken from the official site:

Command Line

Graphical Interface

Expression Evaluation

Conditional breakpoints

Memory watchpoints

Compatible with Intel Compiler

Works with GCC 2.95 through 4.3

Supports pthread applications

Debug events can be disabled by thread

Support for debugging forked processes

Scriptable with Python

Support for wide strings and Qt strings

Custom visualization of data structures (via Python script)

It does not support remote-debugging, though (but you can of course use SSH or X-Forwarding for this), but I especially like that it's scriptable in Python. It even works together with valgrind, see the FAQ.

The main difference between the free and the commercial version is that the free version is BETA with untested features. Additionally, some functionality is trimmed down (only the Python console is mentioned on the website) and the free version is not optimized for speed. There's no source code of the free version either, so it's only "free as in beer".

The price is as low as $39.95, so it's even affordable to hobbyists when the free version just is not enough. Unfortunately I've never used it (well, I've just found out about it a few minutes ago), but I'll surely try it out next time I'll be working on a bigger Linux-project. I hope I can provide some actual field-experiences with ZeroBUGs anytime soon.

Bjarne's new book: Programming: Principles and Practice Using C++

2009-01-15T09:03:00.000-08:00

I've reported about this book earlier last year. The original release-date was slated for august last year, but it seems it was delayed till last month.

It's in stock at amazon.com now. I think I'm gonna order a copy of it for our trainees at work. There's never enough of good C++ teaching books.

German translation of Firebird security article

2009-01-15T08:54:00.000-08:00

There's an article on the Firebird Documentation web-page about Firebird File and Metadata Security. Since this is an interesting topic to our customers, I took the time to translate it to German.

The article describes the fundamental problems in securing network connections. One of the biggest problem is key management. Since the Firebird server is usually controlled by and installed at the client's site, you don't have any control over it. So, theoretically, someone could always build his own Firebird server to spy on the password. But that's only one of the problems discussed.

Many thanks to Geoff Worboys, the author of the original article, and Paul Vinkenoog for publishing it on the official web-site.

Embedding Qt Widgets into QtWebKit

2008-12-03T07:23:00.000-08:00

Qt has it's awesome built-in WebKit support which makes it extremely easy to have full-featured browser/html-viewer capabilities in your application (including JavaScript!).

It is possible to embedd any Qt Widget into your QWebPage. The necessary steps for that are quite simple. You have to derive from QWebPage and overload the createPlugin() function, make sure that PluginsEnabled is set for the QWebPage's settings and assign that WebPage to any QWebView.

It is now possible to embed widgets that are known to Qt's runtime MetaType-system into a WebView. You can make a widget accessible using both, the Q_DECLARE_METATYPE macro and the qRegisterMetaType function.

To show the widget, you have to add an HTML object-Tag to your page, like this:


<object type="application/x-qt-plugin"; classid="YourClass" name="myObject" />

It's now visible and can even be manipulated through JavaScript. You can access it's properties and it's public slots.

I've create a small demo that shows you how to do it and what is possible. It consists of a QMake-project (.pro-file), two pairs of header and implementation files for the MyWebKit/MyWebPage and MyWidget classes and a demo HTML page. It should compile and run on any supported platform (Windows, Linux, Mac). Of course only when QtWebKit is enabled in the Qt installation.

Step 1

First, we should derive from the necessary QtWebKit-classes to create our own MyWebView class that always has Qt plug-ins enabled.

MyWebKit.h


#ifndef MY_WEBKIT_H
#define MY_WEBKIT_H
#include 
#include 

// Derive from QWebPage, because a WebPage handles
// plugin creation
class MyWebPage: public QWebPage
{
   Q_OBJECT
   protected:
      QObject *createPlugin(
         const QString &classid, 
         const QUrl &url, 
         const QStringList ¶mNames, 
         const QStringList & paramValues);
   public:
      MyWebPage(QObject *parent = 0);
};

// Derive a new class from QWebView for convenience.
// Otherwise you'd always have to create a QWebView
// and a MyWebPage and assign the MyWebPage object
// to the QWebView. This class does that for you
// automatically.
class MyWebView: public QWebView
{
   Q_OBJECT
   private:
      MyWebPage m_page;
   public:
      MyWebView(QWidget *parent = 0);
};

#endif

MyWebKit.cpp


#include "MyWebKit.h"

#include 

MyWebPage::MyWebPage(QObject *parent):
   QWebPage(parent)
{
   // Enable plugin support
   settings()->setAttribute(QWebSettings::PluginsEnabled, true);
}

QObject *MyWebPage::createPlugin(
   const QString &classid, 
   const QUrl &url, 
   const QStringList ¶mNames, 
   const QStringList & paramValues)
{
   // Create the widget using QUiLoader.
   // This means that the widgets don't need to be registered
   // with the meta object system.
   // On the other hand, non-gui objects can't be created this
   // way. When we'd like to create non-visual objects in
   // Html to use them via JavaScript, we'd use a different
   // mechanism than this.
   QUiLoader loader;
   return loader.createWidget(classid, view());
}

MyWebView::MyWebView(QWidget *parent):
   QWebView(parent),
   m_page(this)
{
   // Set the page of our own PageView class, MyPageView,
   // because only objects of this class will handle
   // object-tags correctly.
   setPage(&m_page);
}

It's now possible to use Qt classes using the above-mentioned object tags.

Step 2

The second step is to create a class that's known by the Qt runtime meta type system. We can't directly use Qt widgets in this way, because runtime meta types need copy-
constructors. So we derive from a Qt widget and add a kinda dull copy-constructor to it.

MyWidget.h


#ifndef MY_WIDGET_H
#define MY_WIDGET_H

#include 
#include 

class MyCalendarWidget: public QCalendarWidget
{
   Q_OBJECT
   public:
      MyCalendarWidget(QWidget *parent = 0);
      // Q_DECLARE_METATYPE requires a copy-constructor
      MyCalendarWidget(const MyCalendarWidget ©);
};
Q_DECLARE_METATYPE(MyCalendarWidget)


#endif

We use a calendar widget because it's something that doesn't already exist in HTML and could be quite useful. Additionally, it has some few properties that we'd want to access from JavaScript.

Step 3

The final step is to build the HTML page that embeds the widget and executes some JavaScript on it. Here's my example:

Test.html


<html>
   <head>
      <title>QtWebKit Plug-in Test</title>
   </head>
   <body>
      <object type="application/x-qt-plugin" classid="MyCalendarWidget" name="calendar" height=300 width=500></object>
      <script>
         calendar.setGridVisible(true);
         calendar.setCurrentPage(1985, 5);
      </script>
   </body>
</html>

The example set the gridVisible property to true and shows the month that I am born in. Of course, the possibilities seem endless! :-)

Conclusion

The only thing that I am still missing is connecting signals to JavaScript functions, similar to what you do with AJAX. It's possible to export non-visual objects and use them from within JavaScript, too (think of a database connection, for example).

You can download the complete project, which should work out-of-the-box when you have Qt with WebKit support installed, here

Nekthuth - Making Vim Love Lisp

2008-09-12T06:55:00.000-07:00

My friend DieMumiee pointed me to a project call Nekthuth that is supposed to be a mini-version of slime, just for my beloved Vim. The web-site looks promising, there are even screenshots and short, but good documentation. Using it it's quite easy to get Nekthuth installed and getting started.

However, my current SBCL is kinda broken. Dunno what I did to it, but after an apt-get update it segfaulted while compiling some scripts and just seems defective. I tried Nekthuth anyways, just to experience a broken pipe. Ouch. :-(

So you're invited to test it out and send reports to me and the original author. I'm quite interested in whether this project could become at least a partial slime-replacement and drive more Lispers to use Vim, and make Lisp available to Vimmers.

Google's own browser: Chrome

2008-09-01T12:51:00.001-07:00

I usualy don't like blogging about a blog-post, but I'll make an exception for this one:

Philipp Lenssen writes in his blog that he received a comic-book from Google that describes a new, upcoming browser called Chrome. You can read about everything announced in his blog-post. IMHO there's nothing special mentioned in the list, nothing that hasn't been done before. Probably the most "exciting" thing about Chrome is that the tabs are above the address-bar, which basically makes no difference, but looks different to all the other browsers.

My guess about some technical details about Chrome is that it is built using WebKit as it's back-end. More specifically, using QtWebKit from the new Qt 4.4 release. As Google has been a customer of Trolltech for Google Earth and maybe other tools before, QtWebKit is just the best tool to build a browser from-scratch.

Because usually Google-tools are high-quality and sometimes even revolutionary, we probably can expect more than has been mentioned in the comic from the browser. Nobody will give away his secrets before launch, anyways.

Update:There is coverage of this on heise, too: Google Chrome: Google greift Microsoft mit eigenem Browser an, but it's a bit too focused on Internet Explorer, imho. Some more interesting things about the relation of Google to the Mozilla Foundation, how it evolved and how Google Chrome might change it, can be found here. In this article I've been proven right that they use WebKit as their rendering engine. No word on Qt, though. The probably most interesting thing about Chrome is separating browser tabs in individual processes.

BeagleBoard - mini-PC at mini-Price

2008-08-28T01:34:00.000-07:00

The guys over at BeagleBoard created an integrated chipset including CPU and graphics processing for as little as $149. With this small thingie you could do gorgeous stuff while using very little room. It'd be perfect for a Car-PC, for example. And for only $23 you even get a transparent housing for that little fella. And there's effort to port and maintain ffmpeg for this platform, making it quite multi-media-enabled.

This is probably as cool as Bug Labs wants to be. At least hardware-wise... a raw chipset is not quite as cool as a BUG, is it?

New watercooling setup

2008-07-23T09:59:00.000-07:00

I've expanded my watercooling-cycle by one graphics card, namely my new (and second) GeForce 8800 GTS 640MB. My modified Thermaltake Orchestra now cools my CPU, the RAM (OCZ-RAM with factory watercooling support) and two GPUs. The only piece that is missing is the motherboard's chipset. I think I will not add watercooling to my current motherboard, but wait till I buy a new one.
Here are a few pics that show how beautiful it looks now.

Evangelion 1.01: You are (not) alone

2008-06-19T14:25:00.000-07:00

Since anime and computer-enthusiasm are strangely connected together, here's some anime-news.

You are (not) alone is a new movie that re-tells the first few episodes of the original Neon Genesis Evangelion anime (and manga) and is the first of a series of four movies. We all know that Neon Genesis Evangelion is one of the best animes ever created and trying to remake it is a very delicate task.

I think, however, that the studios have done a great job with this first movie! The first fifteen to twenty minutes are nearly identical to the original, with a few scenes left out and short cg-scenes added (which I think look a bit misplaced and don't fit in that well). But the later the movie progresses the more scenes are replaced, and as the story develops (which has changed a little bit, too), the movie features quite much new, high-quality content. I don't know whether it was a sound problem on my system, but there's a high pitch sound in the background every time someone talks. I imagined that it could be caused by filtering out background sounds from the original material, but I hope this is not the case. I'll try the DVD on a normal DVD-player when I get the chance for it.

Overall, I am very satisfied with this movie. As I don't have subtitles for it, yet, I didn't understand a few scenes so I couldn't follow the story 100% (my japanese is still bad). I'll watch it with subtitles again as soon as I get the hands on some and find the time, and will post an update.

New introductory book on C++ by Bjarne himself

2008-06-16T02:57:00.001-07:00

Bjarne Stroustrup, the "inventor" and first implementer of C++, is about to publish a new book. It's titled Programming - Principles and Practices Using C++ and is going to be published in August. It's already listed at amazon.com. In contrast to The C++ Programming Language, this book is on an introductory level to programming, not only C++. It's something that the C++ community was lacking for years now, with the only alternative of Accelerated C++.

xstartonce: Boost your productivity

2008-06-04T15:45:00.000-07:00

I've grown accustomed to a way of working on my Desktop which greatly enhances the speed and therefore productivity of working by accelerating switching to certain applications. There are relatively few applications which I use on a daily basis. I'll mention just a few examples here: I use Firefox (well, now it's Arora) for browsing, Vim to edit files, and a shell for doing various stuff. I have to switch between those frequently. Often when coding stuff in Vim, I have to look up information on the net, so I have to switch to Firefox. When I'm done looking up stuff I'll try it out on the console and then switch back to Vim to integrate it into the program. Switching back and forth between applications can be quite a hassle because of the "most recently visited"-paradigm usual implementations of Alt+Tab (or similar) use. The most recently visited window is the one that'll appear first when switching windows. This works great if you have only two windows. It'll get irritating and slow when you have more than two windows and switch back and forth in an unstructured order.

So I've invented this methodology of work: Every regularly used application is assigned a shortcut. It's Ctrl+Alt+F for Firefox, Ctrl+Alt+V for Vim and Ctrl+Alt+C for the console. The program I use for shortcut is selfmade, because it needs one special feature: When the shortcut is activated, the program looks whether there's already a running instance of the application I want to switch to. If there is none, it'll be started (and automatically get the focus). If it is already running, the existing instance will be focused. This works great. When I need my browser, I don't have to think about whether I've already started it, and if so, where have I started it or anything. I just press Ctrl+Alt+F and have a Firefox or Arora window activated.
I have shortcuts for about 8 applications at work, including one or multiple local and several remote consoles, the aforementioned Arora and Vim, Delphi, Microsoft Visual Studio, InstallShield, the production-copy of the application I'm writing and a few more.
Without my shortcut-application I'm really at a loss and struggling with cycling through windows with Alt+Tab.

Until now, there was only one unreleased, Windows-based program which would enable this work-methodoligy: It's called KeyboardGuy, and it's written by me and exclusively used by me.

When I switched to Linux recently, I was really missing that comfort. So I looked for a way to accomplish this with as little work as possible.

The very first try was a shell-script that uses wmctrl to find out the Process- and Window-IDs that I needed. It looked up the process-name via ps and would activate the window if the basename of the specified application and the process-name in ps matched. With my relatively limited knowledge of shell-scripting, I came up with a script like this:

#!/bin/sh

PROCESSES=`wmctrl -l -p | awk '{print $3}'`
DOEXIT=
ps h -p $PROCESSES | while read i;
do
   ID=`echo $i | awk '{print $1}'`;
   NAME=`echo $i | awk '{print $5}'`;
   NAME="`basename "$NAME"`";
   if [ "$NAME" = "$1" ]; then
      wmctrl -l -p | while read WID;
      do
         WPID=`echo "$WID"|awk '{print $3}'`;
         if [ "$WPID" = "$ID" ]; then
            wmctrl -i -a `echo $WID|awk '{print $1}'`
            exit
         fi
      done
   fi;
done

echo $DOEXIT
"$1"

There exists one problem, however: The script needs to completely terminate after wmctrl -a is executed. The call of exit that should accomplish this, however, did not work as expected and the line "$1" (which starts the application given as the first parameter) is always executed, even when the window was activated first. Because I was rather unsatisfied with the whole thing and couldn't get it to work, I decided to rewrite it in C++, using Qt (because Qt makes many things much easier for me here, and I wanted to get this done quickly).
The final result is the application which I call xstartonce.

xstartonce

The current version is the second iteration of the C++/Qt-version. The first version would match the basenames of the given application and the path found in ps's output. This leaves one big limitation, though: You can't assign shortcuts to one and the same program. I need this, for example, because I want shortcuts for a local and several remote consoles. Those would look like "urxvt" for the local console and "urxvt -e ssh melchior" for the remote console. Because the basenames match (both are urxvt), the shortcut-application would not be able to switch between those two distinct consoles.

Configuration file

That's why I invented the "named shortcuts". This needs a configuration file (~/.xstartonce), which has a very simple structure: It's <shortcut-name>=<command --param>. My ~/.xstartonce looks like this:

urxvt-local    = urxvt
urxvt-melchior = urxvt -e ssh melchior
firefox        = firefox
assistant      = assistant
gvim           = gvim
vim            = urxvt -e vim

Currently, only named shortcuts are possible. This means that for each application you want to launch, you need a configuration-file entry. I've totally abandoned the formerly discussed algorithm to match the basenames of the processes in favor of this approach. I will, however, make both possible in the future. This has an advantage that was bugging me with the Windows-version already: When opening firefox from anything else but my shortcut-app, it'll not get "recognized" by it and a second instance will be opened when using the shortcut next time. With the basename-matching algorithm, I can use apps that I don't want to open twice with distinct shortcuts better.

Temporary Process-ID list

xstartonce creates an entry in /tmp/xstartonce-db.<user-name> for each started named shortcut. The structure is like the configuration-file: <name>=<process-id>. When executing a shortcut, first the name is searched in the configuration. If it is found, the xstartonce-db is searched for the name, and if it is found there, the window-id matching the process is looked up via wmctrl -l -p. If a window is found, it is activated. If either no window or no process is found, the command is executed.

Getting xstartonce

xstartonce has tree dependencies: A C++-compiler, Qt and a UNIX-like operating system with the X window system running. If you meet those criteria, you can download the source-code in my github-repository.

Notice:I'll clean up this blog-post and correct spelling and grammar errors tomorrow. It's already past bedtime, so I'm off to get some sleep :-)

Using keyboard-shortcuts

What I haven't mentioned, yet, is how this all works together with keyboard-shortcuts. Well, the answer is easy: Use whatever tool you know for executing keyboard-shortcuts and run xstartonce instead of the application you want to launch. I'm currently using KDE's shortcut-enabled menus together with xstartonce. You can edit KDE's menu by right-clicking it and choosing "Edit menu". Then you can add new menu-items and assign a shortcut to them.
Here's a screenshot how I do this:

Arora Configuration System Proposal

2008-05-28T14:39:00.001-07:00

Introduction

First, I'd like to show you two mock-ups that I've created so you get
a feel at where I am going.

Here's the first screenshot where I used colors for each individual
section:

And here's another with the same colour for each section:

Every webbrowser has a multitude of configuration settings. Some of
them should be easily accessible by the user and some of them are for
fine-tuning and usually only the defaults should be used. There are
settings of different complexity that require different degrees of
prior (technical) knowledge. For example, an easy setting that every
user should understand is "Restore window position and size when
starting". This is just a simple checkbox in the settings dialog that
a user can either check or not check . A far more complex setting
would be a default stylesheet that should be applied to all web-pages.
First, creating such a stylesheet requires technical knowledge and
second the implications of providing such a stylesheet is not very
easy to understand.

Suggestions

My opinion is that there should be a settings-dialog that is
convenient and includes just the settings that Arora chose to be
accessible to the normal user. For advanced users, there is a
about:config page which includes all settings. But the about:config
should be nothing like Firefox's about:config. Firefox's config-page
is only for hackers and lacks usability. Here's how I'd imagine
Arora's about:config:

First of all, Opera has a nice config-page which we can base on.
You can see a screenshot here:

Navigate to opera:config and you can see it. It consists of many
sections which can be opened and closed. Every section includes
multiple settings, which sometimes have custom editors. The editors,
however, are not as convenient as they could be. For example, to enter
a color, you have to type in the hexadecimal color-value instead of
using a color-picker dialog. Every section includes a "Save" and
"Abort" button, which is very reasonable. If there was no explicit way
to save, you'd have to save after every change of the setting. And
saving often means rebuilding or redrawing something, which can be
quite annoying when just changing one simple value.

Opera has meaningful captions of each setting, too. Firefox, on the
other side, only displays the internal setting-name (which often is
quite verbose, but not as good as Opera's captions). What even Opera
is missing, though, is a description of a setting. I find that very
important, although it is, of course, very much work.

A search-functionality is also present. It even behaves awesome. If
an item in section matches, the section is displayed. Inside the
section only matching items are displayed. This way you get to the
settings you're looking for quite quickly. I think, additionally, the
config-page could include some more filters. Something like filters to
show only users or experts settings. With these filters, the
config-page could even fully replace the settings-dialog. IMHO, this
would be really cool. But again, I think the settings-dialog can
provide a much more optimized display and user interface than an
automatically created config-page ever could, so we should keep a
separate dialog that's hand-coded.

There are settings that are added to the system on-the-fly. For
example, extensions should be able to add settings and have them
displayed in the about-page like any other setting.

Implementation

To sum up which information is needed on each configuration option
that is available, it comes down to this:

Category

Internal name

Display name

Type or Class

Editor (optional)

Value-constraints (optional)

Short, meaningful description

Default value

Flags

The Flags can be used to mark a setting as "For Experts", for example.
In the next chapter I'll discuss another usage of a flag that a
setting could have.

A Class is a combination of Type, Editor and Constraints. The
Editor and Constraints can still be overwritten when using a Class,
though.

A category, in turn, needs some more information than it's bare name,
too.

Internal name

Display name

Icon (optional)

Color

Meaningful description, may be a bit longer

To make the config-page visually appealing and to easily find
regularly visited options, a category should have a special color. You
can see that this looks good in the mockup I provided.

With these informations, it should be relatively straight-forward
to implement some settings-classes to read and write settings and to
create the config-page.

Reading settings should be as easy as

Settings::Download::Style style = readSetting<Settings::Download::Style>("Download.DownladStyle");

Writing, however, should always occur in a group. Hence there should
be a class SettingsWriter that can be used to write settings. When the
object runs out of scope, it flushes all the changes that have been
made.

SettingsWriter writer;
writer.set("Download.DefaultDirectory","/home/user/downloads");
writer.set("Download.DownloadStyle",Settings::Download::AlwaysUseDefaultDir);

If this proposal is accepted, I will work out some class-prototypes
for further discussion.

Dreams come true

To have an absolutely awesome and really magnificent
settings-system, you could think about automatically storing chosen
settings online. For example, the tab-behaviour usually should not
change no matter from where you are browsing. And now imagine that
certain settings that are marked as shared are stored on your
online-profile and automatically loaded when using Arora from a
different box with the same profile. This online-profile should, of
course, include bookmarks and history, too. Because those are more
sensitive than settings are, they are of course optional.

Arora: The first usable QtWebKit-based Browser

2008-05-16T07:12:00.000-07:00

Arora is the name of the first QtWebKit-based browser-project that aims to become usable as a day-to-day browser for the masses. It's a spin-off of the QtWebKit demo-browser which is included in Qt and currently doesn't feature much more stuff than the demo does. It's code is managed in a git-repository at github.com and it's therefore very easy to hack on it. I have already created a git-branch and committed a small bug-fix and other changes myself, too.

This project is just a few days old and can already be used without bigger flaws. It will definitely catch on to Firefox (for basic features) soon. This is very exciting, imho, and I can't wait to see what cool things will happen to Arora in the future.

Here's a screenshot of the version that I compiled myself using Microsoft Visual Studio 2005 Express Edition:

New ELF Linker for GNU Binutils: gold

2008-05-16T07:04:00.000-07:00

This totally slipped my radar until now. Ian Lance Taylor, an employee from Google, wrote a new ELF linker that is meant to replace ld as Linux's default linker from scratch. The new linker is called "gold" and is approximately "five times faster linking large C++ applications", according to Ian's blog-post.
It has been released to the public open-source crowd as of March 21st with this announcement.
gold is written in C++ and consists of as little as 50,000 lines of code.
Nice!

Passwords in Firebird

2008-04-16T02:31:00.000-07:00

Firebird has some funny ways of handling passwords. The maximum length of passwords that is evaluated is 8 characters. Every character after the 8th is silently ignored. That's especially funny because the 'default' password for a Firebird-installation is 'masterkey', which has 9 characters. You can, however, successfully log in to freshly installed Firebird-servers providing the password 'masterke'.
I'm working with Interbase and Firebird for more than four years and just now realized that when a co-worker at our company found it out while learning SQL.
The only program that I know that makes note of that is gsec, which prints a warning when setting the password to something longer than 8 characters.

Why Crisis Core - Final Fantasy VII disappoints me

2008-03-28T04:10:00.000-07:00

Man was I glad when my Crisis Core package arrived. I've been already playing this game since Wednesday and have played 7:30 hours until now.

Crisis Core, in contrast to Final Fantasy VII, does not have tactical battles. In fact, they're the direct opposite: They're random. Really random. Well, of course entering a battle was always random for Final Fantasy VII, but in Crisis Core even your special attacks, materia upgrade and Level-Ups are random! Who on earth had that stupid idea? There's a slot-machine-like thing called DMW (Digital Mind Wave - wtf?) in the upper left corner which spins while you beat your enemies. It spins pictures of characters and numbers. When the pictures match on the first and last slot, the "Limit Verge" appears and depending on the outcome of the middle slot you'll execute a Limit Attack.

Everything here is absolutely not influenced by you in any way. You can't tell the slots when to stop, you can't do nothing. Depending on the outcome of the numbers of the Limit Verge you or your materia might level up if the numbers come out right. Theoretically this means that you can gain two levels in twenty seconds or gain no level at all in 5 hours. More fighting doesn't necessarily increase your level either. Well, obvisouly Squeenix was smart enough not to make it as random as a true random number generator, but it's still unpredictable.

The Limit Attacks are somewhat strange, too. Sometimes there are some FMV sequences that tell a part of the story -- right while you're battling totally unrelated enemies or you are on a mission or something, before the attack begins. I don't understand this at all, it makes no sense to me.

You don't have weapons with materia slots, either. In fact, you don't even have different weapons! Instead you have a fixed number of materia slots where you can put your materia, which sometimes increases when finding special items or so. At least you have two Accessorie-slots in the beginning... but again, no weapons. This actually pisses me off most. There are no connections between materia-slot, no slots with doubled materia-exp-increase, no new weapons that do more damage, nothing.

And then there's Material Fusion. You can fuse any two materia together to gain a new one. The outcome, however, is always sooo teeny-weeny better than the originals that it almost never brings you a significant advantage. Plus, you can't predict the outcome at all and sometimes you basically just lose one of the materia, while the other stays the same. Absolutely useless.

Then there's the difficulty of the game. Being a hardcore-gamer I picked the Hard Mode over the Normal Mode. It lasted 30 minutes and I nearly threw my PSP through the room, out of desperation. I died multiple times on the first hard enemy, and the bad thing about that was that there's 5 minute sequence that YOU CAN'T SKIP before that battle. Oh jesus how I hate this! That was the reason why I didn't finish God of War, too, by the way. Un-skippable sequences are the most sinful thing a game-developer can ever do. (By the way: In Crisis Core you can ALWAYS interrupt the game by pressing Start. This somewhat makes up for not being able to skip sequences... but only somewhat.

So I decided to start from the beginning in the Normal Mode. This was a good decision, because the game basically becomes child's play after that. It's actually so easy that it's making it boring at times. Most of the time, when hitting the enemy, it's pushed back and gives you time for the next slash. This makes matches where you don't get a scratch on more than common.

The mission system is used for leveling and resembles Monster Hunter in many ways. Because you're more often in areas without enemies than you were in Final Fantasy VII, it's harder to run around and level up as I often did in the original. The missions kick in here, which you can enter on every savepoint. Pick a mission by category and difficulty, enter it, beat it, get a reward, go on to the next mission. Those are always in the same environments, like the various levels in Monster Hunter. Missions are actually a bit less boring than running around on the field to fight the same monsters over and over again. That's what I did in Final Fantasy VII for hours... or well, actually even for days and weeks. :-)

Oh and here's the moooooost disappointing thing ever: You don't have a party! You're always running around alone as Zack and you can't switch party members! That's so unbelievably boring!

There are good things about the game anyways and it's quiet fun to play, but I must say that I would have more fun if I put FFVII on my PSP again and played that for the nth time. It's just so much more tactical and muuuuch deeper than Crisis Core. Crisis Core seems to be meant for the occasional gamer and not the typical Final Fantasy RPG fan. It's more an action game with a good story than an RPG if you ask me. I hope that there are at least 60 hours play-time in that game, but I'm not too sure about that yet... I'm looking forward to find out more as the story continues, maybe I'll write some positive aspects of the game after I've finished it.

That being said, the game is still much fun. But again, it's not as much fun as Final Fantasy VII was.

New hardware for my lil server

2008-02-17T10:02:00.001-08:00

I recently ordered new hardware for my little file-serving, downloading and IRCing box. As the nature of this 'server' is to be always on and not demanding powerful hardware, I went for the following components:

Intel Celeron 420 (1.6GHz, Single-Core, 35W power-consumption)
ASUS P5B-VM Motherboard (5 SATA-ports, GBit ethernet, on-board VGA)
Crucial 1GB DDR2-667

The hardware that was being replaced was:

Tyan Tiger motherboard
2x Pentium III 1GHz
2x 512MB SD-RAM
2x Adaptec 1210SA
1x Intel 1GBit Ethernet (eepro1000)
ATI Radeon 8500

(Note that I currently have the Intel Ethernet adapter still plugged in since network didn't work out of the box and I haven't found the time to fiddle with it, yet).

As you can see this box is optimized on low cost, low power consumption and replacing as many of the old parts as possible. At a total of 136 EUR (the CPU was only 35 EUR!) including shipping, I'm really satisfied. One additional Adaptec controller would've cost 50 EUR and would've added only 2 SATA ports, so this was probably the best solution. I still have 2 SATA ports unused on the mainboard and have 2 SATA-controllers lying around ready to be plugged in if even those will be used up. Finallly I have potential to grow my RAID even bigger. :-)

Fixing raid-5 failures, the adventurous approach

2008-02-17T09:19:00.000-08:00

You might remember the trouble I had with my raid5 before. Well, it's still not 100% sorted out, but I know the cause now. It really was a faulty drive! I came to notice that after I replaced the motherboard, CPU and RAM with new components. After I've added them and booted into the system (which worked flawlessly on the first try, by the way, although the hardware is absolutely unrelated to the previous one) I noticed a click-sound from one of the harddisks. I immediately realized that I bought new hardware for nuthin. But at least I was sure which component was causing the failure now, plus I got 5 free SATA ports to upgrade the RAID. Previously I had non unused ports, leaving no potential for a possible upgrade. But somehow the raid got messed up in the process. I wasn't able to assemble it with the remaining 3 discs because one disc was always added as spare. So I had 2 functional devices and one spare added, which is obviously not enough to run the raid. This is due to some corrupted superblock, but luckily the superblock is just metadata which can be recreated. If I knew the correct devices and slots they corresponded to before all this happened, I could've created the array with mdadm --create and the correct params. Unfortunately, I did not know the exact params so I had take a more... adventurous approach. There's a perl-script on the linux-raid wiki which permutates over each possible combination of devices (including one missing device) and tries to mount the created array. It does everything in read-only mode so no actual data is being touched, only metadata. If it could mount the raid it prints the mdadm --create command used to build it, stops the array and goes on. You can then execute the creation-commands yourself and see if everything's right. In my case, luckily it was and I got all my data back. Note that I had to connect the failed drive for this to work because it always replaces one given device with 'missing' ('missing' tells mdadm that this device is, well, missing) instead of adding 'missing' to the devices-list. This is because it's not supposed to recreate a partial, but only a complete array. So you need to provide ALL raid-members to the command-line, otherwise it won't work. It should be fairly easy to hack the script to work for partial arrays, too, but it was easier for me to add the drive again than to hack perl-code.

After this the raid was up and I needed to mark the drive as faulty and remove it so it can't cause problems anymore. It's always a bit problematic to map the device-names (/dev/sdx) to the real harddrives and you might pull out the wrong one, possibly leading to more problems. I found out a reliable way to identify the drives:

hdparm -I /dev/sdx | grep 'Serial Number'

This will print the serial number, which usually is visible on the actual discs, too. Somehow the -I option to hdparm never occured to me before. The serial-number matched one of my disks and so I was able to locate and remove the faulty drive.

Yay!

Next step is to contact the reseller for a replacement. I hope the next bad drive will be less problematic.

Qt debugging with Visual Studio 2005

2008-02-14T01:29:00.001-08:00

zbenjamin, one of the fine folks from the Qxt-project just gave me his additions to the AutoExp.dat-file that lets you debug native Qt-types (e.g. QString) far more easily. Here's the before/after-comparison:

Before

After

How it's done

And here's what you have to do to use it yourself:
First, open up the file

C:\Program\ Files\Microsoft\ Visual\ Studio\ 8\Common7\Packages\Debugger\autoexp.dat

Important: Under Windows Vista, you need to open the file as Administrator, because it is not writeable by the user and the program-files-virtualisation will get in your way.

Then, add the following lines under the [AutoExpand]-mark:

QObject =classname=<staticMetaObject.d.stringdata,s> superclassname=<staticMetaObject.d.superdata->d.stringdata,s>
QList<*>=size=<d->end,i>
QLinkedList<*>=size=<d->end,i>
QString=<d->data,su> size=<d->size,u>
QByteArray=<d->data,s> size=<d->size,u>
QUrl =<d->encodedOriginal.d->data,s>
QUrlInfo =<d->name.d->data,su>
QPoint =x=<xp> y=<yp>
QPointF =x=<xp> y=<yp>
QRect =x1=<x1> y1=<y1> x2=<x2> y2=<y2>
QRectF =x=<xp> y=<yp> w=<w> h=<h>
QSize =width=<wd> height=<ht>
QSizeF =width=<wd> height=<ht>
QMap<*> =size=<d->size>
QVector<*> =size=<d->size>
QHash<*> =size=<d->size>
QVarLengthArray<*> =size=<s> data=<ptr>
QFont =family=<d->request.family.d->data,su> size=<d->request.pointSize, f>
QDomNode =name=<impl->name.d->data,su> value=<impl->value.d->data,su>

Now restart Visual Studio and you should be good to go.

More slime goodness

2008-02-11T07:15:00.000-08:00

Remember the Slimy Lisp Video I've posted earlier this year? Well, some other blogger, Peter Christensen has put more effort into it and has written a reference/annotation for the video, including a timeline, a transcript of important parts and introductory explanations on how to set up SLIME to use that video. This will give SLIME-beginners an even better kickstart.
Thanks very much for your effort, Peter!

Fixing my software raid-5 with mdadm

2008-02-07T13:51:00.000-08:00

On my server-box I have a software raid-5 /dev/md0 consisting of 4 500GB SATA-harddisks, namely sda1, sdb1, sdc1, sdd1. When I was working on my other box where I was pulling off some dd-stunts on a lvm-volume my raid suddenly died on the server. There was some output in dmesg that both sda and sdb are somewhat corrupt and that they've been removed from the raid, leaving it unfunctional (you need to have at least n-1 disks in a raid-5 to keep it operational). I was very shocked by this. I restarted the PC and tried to re-assemble the raid with

mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1

to no avail. mdadm said 'bad superblock on device /dev/sda1' (or similar) and leaving out sda1 worked and I had the mdadm assembled with 3 out of 4 disks. That's of course not satisfactory. I stopped the raid and ran S.M.A.R.T.-checks on each of the 4 disks with

smartctl -t long /dev/sdx1

. This took over an hour so I went to sleep and checked the results the next day -- 100% error-free, according to smart! That's really strange. Assembling the array still does not work because of sda1. I opened up sda in cfdisk and saw the exact same partition-size as on sdb and the others, but I knew that something was corrupt. So I wrote the partition-table to a file to back it up, removed the partition and re-added it. Then I used

mdadm --add /dev/md0 /dev/sda1

to re-add the partition that was formerly part of the array, anyways... mdadm did it's job and recovered the raid. You can watch the progress by doing

cat /proc/mdstat

. It took around 7 hours or so to complete, and now the raid5 is fully functional again.
What a horror-trip! I'm still wondering what was going on and why sda1 has been kicked out of the array.
A small addition: After I've fixed the raid it was ok for a day or two, but then one day when I came home I noticed it broke again. I remembered that I stepped onto the USB-keyboard that was attached to the server right after I came home and found an unhandled IRQ-oops in the kernel-log at what happens exactly that timespan. So my suggestion is that the USB-handler somehow messed up something, which in turn has killed the RAID again. But I'm still investigating the issue, for now rebooting and forcing the assembly worked fine. I hope I'll not have any more problems with it...

Converting lvm to a normal partition

2008-02-07T13:42:00.000-08:00

I've recently set up a new gentoo-box and first decided to use lvm2 on my root. Well, I ran into some issues with the kernel and initrd which I could figure out and fix. But then I noticed that, because of the lvm, I won't be able to access the disk from Windows with the free ext3-drivers that are available. Linux will even boot faster because I'll have no need for the initrd anymore. That's when I decided to get rid of the lvm.And that's actually easier than you'd think: If you have a spare-partition oder -harddisk around that is at least the size of the logical volume that you'd like to convert to a partition, you can easily do this with dd. Imagine that /dev/vg/volume is a logical volume that consists of only one partition, /dev/sday:

sh# dd if=/dev/vg/volume of=/dev/sdbx bs=8M
sh# dd if=/dev/sdbx of=/dev/sday bs=8M

That's it. This will back up the logical, continuous data that's hosted on the lvm to a partition. After the first dd you'll be able to mount /dev/sdbx and see how the content of /dev/vg/volume has been copied. The mounted partition's usable size will be exactly the same as the volume's size, even if the partition itself is much bigger. That's because the filesystem on it will still be the same size it was before. You could (but it wouldn't make much sense because we want to move the data to the other partition anyways) fix this with resize2fs (if you use ext2 or ext3, that is).
The second dd copies the data back to the partition that it was formerly stored on, but without the additional lvm-abstraction. The lvm will be overwritten by the 'flat' filesystem-data. If sdbx happens to be bigger than sday, an error will be printed that dd reached the end of the partition. This is nothing to worry about since the data left on sdbx is not interesting to us anyways.
You can fix the filesystem-size to the actual partition size with resize2fs. Since the lvm itself needs some space, too, it will be slightly (a few bytes) larger now.

Accessing MS SQL UID-fields with Qt

2008-02-01T00:51:00.002-08:00

When working with a database that relies heavily on uniqueidentifiers, I experienced problems with handling those fields with Qt's built-in SQL-classes.
First, I connect to the database via the QODBC-driver. Then I fetch the results of table 'a' and tried to fetch the corresponding results in table 'b', which are referenced by foreign keys. Here's a code-snippet:

QSqlQuery a(db);
a.exec("select id from a");
a.next();
QSqlQuery b(db);
b.prepare("select id from b where b.id_a=:id");
while(a.isValid())
{
   b.bindValue(":id",a.value(0));
   b.exec();
   // ERROR: Operand type clash: image is incompatible with uniqueidentifier
   a.next();
}

So Qt converts the binary data it received from the uniqueidentifier to a binary blob of type image, it seems.
There's a simple way to convert the GUID that is stored in a.value(0) to a formatted UID-string, which in turn can be used to bind the value of the second query.

   QString uuidToString(const QVariant &v)
   {
      // get pointer to raw data
      QByteArray arr(v.toByteArray());
      std::string result(arr.constData(),arr.size());
      assert(result.size() == 16);
      const char *ptr = result.data();
      // extract the GUID-parts from the data
      uint   data1 = *reinterpret_cast<const uint*>(ptr);
      ushort data2 = *reinterpret_cast<const ushort*>(ptr+=sizeof(uint));
      ushort data3 = *reinterpret_cast<const ushort*>(ptr+=sizeof(ushort));
      uchar  data4[8] =
      {
         *reinterpret_cast<const uchar*>(ptr+=sizeof(ushort)),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr),
         *reinterpret_cast<const uchar*>(++ptr)
      };
      // create a uuid from the extracted parts 
      QUuid uuid(
         data1,
         data2,
         data3,
         data4[0],
         data4[1],
         data4[2],
         data4[3],
         data4[4],
         data4[5],
         data4[6],
         data4[7]);
      // finally return the uuid as a QString
      return uuid.toString();
   }

Using this function, you can easily bind the values to the second query:

b.bindValue(uuidToString(a.value(0)));

Edit:Starting from Qt 4.4.0 (I used the latest snapshot) QVariant supports GUIDs and hence this function fails AND is unneccessary.

SQL proposal: Select deleted and updated records

2008-01-31T10:40:00.001-08:00

Here's another proposal for an addition/extension to SQL. In the earlier post I said that it should be possible to get feedback from delete- and update-statements. Here's an even better proposal that adds some syntax to SQL and allows you to fetch the records that have been updated or deleted.
Here's how it should look like:

delete from customers where marked_as_deleted="true" select id, name

Executing this will behave like a normal select-statement, while the results will be the id and the name of the customers that you have just deleted. With this it is possible to create a message like

The customers "Tom", "Bill" and "Lara" have been successfully deleted.

This is a good idea because it gives the user more feedback on the operation that just has been completed. With update-statements it's very similar:

update customers set marked_as_deleted="true" where last_action > "01.01.1998" select id, name

A possible user-feedback would be

The customers "Tom", "Bill" and "Lara" have just been marked for deletion, because they are too old.

You can simulate this, of course, by first selecting the customers and then running an update/delete with a) the same where-clause (which is unsafe because the data might've been changed in the meantime) or b) for each fetched id, which is slower because it doesn't let the server handle the whole operation on it's own.

Clean SQL-Server Client-API

2008-01-31T03:32:00.000-08:00

Introduction

I often stumble upon some limitations or an awkwardness when dealing with the Firebird SQL-server. These are the big points which you can't fix in your client-application, because the API and some design choices do not allow it:

There's no progress feedbeck except when fetching datasets. Updating and deleting, however, is just one (potentially huge) blocking call.

There's no clean way to keep my datasets current. When they've been changed from a different (commited) transaction, I don't get any kind of notification.

Buffering datasets is a tremedously hard task to program, if you want to do it efficiently (both speed- and memory-wise).

Multithreading is a no-go optimization, because you need to establish a new connection for each thread. If you want to do intelligent multithreading with Firebird, the task becomes exponentially more difficult and error-prone.

Requirements

So here are the requirements I would expect from a well-designed client-API for Firebird, which would allow me to solve the aforementioned problems.

(Cumulated) progress feedback

This is applicable to update and delete (and maybe other) statements only. For select-statements, it's not needed. The client-app should be notified about the progress of the operation it just executed. Otherwise, (unexpectedly) very long executions may lead to the wrong conclusion that the server 'hangs' or there is something else going wrong. You should be able to create visual feedback on the client-side for better end-user experience. I often see myself writing client-side loops instead of simple updates with where-clauses for the sole reason to provide a progress-bar to the end user, since it is a strong usability-requirement of good UI design. Of course you can't know how many datasets are to come, but you at least should know how many datasets have been processed.
There are two ways in which I can imagine the Firebird-API to support this:
a) provide a call-back function to the execute-function:

      int execute_progress_info(int datasets, void *data, some_other_args) {
        if( time_elapsed > 10000 )
           return 0;
        else
           return 1;
     }

If the call-back function returns 0, execute aborts it's operation (if possible).
b) make the client call the execute-function repeatedly and have it return the number of processed datasets. If the execute-function returns 0, the client stops to call it:

     int datasets = 0, temp;
    while(temp = sql_server_execute(statement)) {
       datasets += temp;
       printf("%d datasets processed.",datasets);
       /* possibly process some window- or system-messages here */
    }

Integrated SQL-parser and editor

I often see the requirement for the client-application (especially for client-library wrappers) to parse and understand the SQL-query. For example, the IBObjects Delphi library does this to modify the SQL-string. Currently, there is much of the SQL-parsing logic re-implemented by IBObjects, while this should be accessible through the SQL-server's API to remove code-duplication and incompatibilities.
It should be possible to create a program-readable SQL-'tree' from SQL-code and vice-versa. With SQL-'tree' I mean some kind of data-structure that represents the SQL-code in-memory, much like how it should look like when it is compiled the server. This in-memory structure can then be modified (e.g. add or remove parts of the selected fields or add an expression to the 'order by'-clause) by the program in a error-proof and logical fashion and converted back to SQL-code or directly executed at API-level. This could be used for SQL-editors that have rich completion-features like today's IDEs like Visual Studio, NetBeans, etc. have. There are SQL-editors that provide those, but they duplicate the SQL-parsing-code, too, and are likely to break with unknown or new syntax. I will provide another good example for this later.

Partial exposure of SQL-subsystem functionality

What I mean by that is that I want to be able to check whether an existing in-memory dataset (that may or may not be in the database) matches a specific where-clause or order datasets that exist in the client's memory only with a given 'order by"-clause.

Event-notification systems with additional information

The event-mechanism as it is right now is kinda useless for many purposes I'd like to use it for. It's design is very simple, but this limits it to a very narrow range of applicable use-cases. For example, when monitoring a dataset for server-side changes, I can currently do the following: I have a base-name for the event, which ideally corresponds to the table's name. Let's say I want to monitor the datasets in the table 'customers' that I've partially fetched in a list. To monitor changes to each existing dataset, I would need two different events: customers_del_$ID and customers_upd_$ID, where $ID is replaced with the dataset's ID. I have two triggers for the table which fire off the corresponding events like this:

  create trigger customers_del_event for customers active after delete as
 begin
    post_event 'customers_del_'||old.id;
 end^

and

  create trigger customers_upd_event for customers active after update as
 begin
    post_event 'customers_upd_'||old.id;
 end^

I then register to two events for each dataset to monitor changes or deletions. When the upd-event triggers, I can re-fetch the dataset by ID and if the del-event is triggered, I can simply remove it from the list. But there's one caveat: When the ID of the dataset changes, I have no chance at all to notice that. There's no way the client can be notified about the ID-change by the server. Of course there are workarounds for this limitation, but none of the ones I can think of come with zero overhead.

Now what I would like to see is the ability to pass (arbitrary) parameters to the events. If it's possible to pass a 'new_id' parameter to the upd-event, all would be fine. I could simply re-fetch the dataset with it's new ID. Or even better, pass all the required fields to the event and eliminate the need to re-fetch the dataset alltogether, because the client can update it's data from the received event's parameters. Here's how I would imagine the syntax:

  create event customer_upd(
       id integer,
       name varchar(100),
       street varchar(100),
       city varchar(100))^
 /* When parameters for events are possible, the event needs to be declared first */

 create trigger customers_upd_event for customers active after update as
 begin
    post_event customer_upd(new.id,new.name,new.street,new.city);
 end^

Thread-safe Client-API

Thread-safeness for the client-API would be a real gain. The whole programming world becomes more and more parallelized to increase speed and responsiveness of programs. More so with Dual-Cores and Quad-Cores on desktop-PCs.
One possible approach that does not provide more speed, but would make it possible to access the same connection from multiple threads would be to have one 'db-access-thread' automatically started in the client-API that simply communicates in a thread-safe way with every client-API-function (e.g. pipes or something similar). That way, each thread would still have to wait for the other thread's operation to finish, but it would make it much easier to program more responsive applications by putting database-operations that will otherwise block the GUI-thread into a separate thread, without the need to create a new connection (which would decrease performance and, additionally, I reallly like to have only one connection per application). This can of course be implemented in the client-application (or 3rd-party client-library) itself, but I currently know no implementation which bothers to do so. Having such neat features in the client-API makes Firebird a better product 'out of the box', IMHO. Another idea is a completely asynchronous API, which would be the best solution. This would be done by making every API-call non-blocking and have an event delivered to the application when it finishes. Asynchronous handling should always be prefered to threaded handling when it comes to networking stuff, since it usually scales better and puts less stress on the client. On the other hand, it's usually far more complicated to program.

Dataset-buffer in Client-API

When programming lists that display datasets, you usually fetch the datasets from the database and then put them in some kind of buffer of your own. Programming that buffer can become a major task if the list is very large. There are techniques to speed up fetching this list, for example what I call 'lazy' data-reading:
You first read in all the dataset's IDs and store them in a structure that looks like this:

  struct dataset {
    db_key id;
    struct dataset_content *content;
 };

content is initially NULL and when the program tries to read the content for the first time, it's fetched from the database with a separate cursor than the original that fetched the IDs. The two SQL-queries would resemble something like this:

  select id from customers where city='New York' order by name; /* Fetch all the IDs */
 select name, street, city from customers where id=? /* fetch a specific dataset */

This 'lazy fetching' speeds up the creation of lists that *require* to know the total size of the result-set, because fetching the IDs is a few times faster than fetching the data, and fetching the data is delayed until it's displayed. Most lists require the total size of the result-set, because implementing the scrollbar-logic for those lists without that information is not possible in any usable fashion. Nearly all SQL-based programs have very bad handling when it comes to lists and scroll-bars. Additionally optimization of small-object-allocation is nearly a MUST in this situation. Otherwise performance is likely to degrade to unusable levels. This is true for any list-buffer, though, and I think it should be part of the client-API so the application does not need to worry about it. As much optimization as possible should be included in the client-API, because it'll make Firebird look like a more stable and faster database than others in subjective comparisons. Again, application-developers are unlikely to write these kinds of optimizations themselves in today's age, because bigger hardware is cheaper than software-optimization (especially at the client-side). Combining this with a multi-threaded or asynchronous approach is the ideal solution. I have created a proof-of-concept implementation with the techniques mentioned above and I think I'll write a detailed describtion of the process and the results, soon. IMHO the usablity and responsiveness of what I've created is as far as it can possibly get.

Creating a 'live' list

Without those features it's still possible to write a client-application that shows a 'live' list of datasets of a specific range in a specific order. With 'live' I mean that new, deleted and updated entries are visible immediately in real- time, without any polling going on. This of course decreases network-traffic and at the same time increases speed. First, I'd like to describe the approach that I see optimal for implementing this without the aforementioned desired features. There are a few database-prerequisites that each table you want to watch 'live' has to meet: It needs a 'updated'-field which is set in 'before insert'- and 'before update'- triggers to the current datetime. Then it needs to post the events 'table_inserted', 'table_updated' and 'table_deleted' in the corresponding 'after'-triggers. The client-app, in turn, first selects the desired data and listens to the table_xxx triggers. Each time a _inserted or _updated trigger is fired, it adds a 'and updated>=:last_update' to it's original select-statement to fetches only the updated datasets. Existing datasets (identified by the ID) are updated accordingly, and new datasets can be added to the list. Since you don't know how the list might be sorted (after all, it's hidden in the select-statement which may be user-provided), the best bet is to add those datasets to either the top or the bottom of the list. The 'if the ID changes, I'm out of luck'-problem applies here, too. In that case, the updated dataset with the new ID would be interpreted as a new dataset and I would see the same dataset twice in the list.
For best performance, you should include the 'lazy fetching'-optimization I described earlier. You first fetch the IDs only, then fetch each dataset as it is required. You need to separate selects for this. The first select is like the original, but with the fields replaced with the 'id'-field only. The second adds an 'and id=:id' to the where-clause, with all previous where-clauses put into parantheses so that no operator-precedence messes up the result. Those are both difficult tasks to program, because it requires some effort to parse and modify the SQL structure. The lazy fetching is difficult, because, when done multi-threaded or asynchronous, it'll quickly become complex and confusing. Parsing the sql-statement is very complex, too, since writing a bullet-proof, Firebird-compatible sql-parser isn't an easy task. That's why there should be a 'cursor'-like object in the Firebird-API that provides optimal fetching and buffering means, and some functions to parse and modify an sql-statement.

Got BUGs?!

2008-01-16T04:42:00.000-08:00

You just have to check this out, it's extremely cool:
http://www.buglabs.net/products
Bug Labs manufactures and sells the BUG device, which is a small micro-controller in a very stylish housing. It got an ARM CPU at it's core (no clock-speed mentioned) and 128MB of main memory. It features hardware graphics acceleration and MPEG4 decoding and encoding, WiFi, USB, Ethernet and a hell of a lot more interfaces.
If it wasn't for the 300 bucks this BUG costs and world-dominating gadget thingie... or whatever.

Slimy Lisp video

2008-01-07T08:36:00.001-08:00

There's a great video that gives you a kick-start in learning SLIME, the extremely advanced Lisp-mode for Emacs.
You can find it on common-lisp.net here: http://common-lisp.net/movies/slime.mov
It's very interesting even for non-Lispers, since it shows off features the average or professional C++, C# or Java IDE can only dream of!

C and C++ interpreter

2007-12-28T21:32:00.001-08:00

I've heard about C interpreter before, but a mostly functional and compliant C++ interpreter was new to me. I just found out about one which you can find on the CERN homepage: http://root.cern.ch/twiki/bin/view/ROOT/CINT. It supports quite a large portion of ISO C++. The differences are explained in this document: http://root.cern.ch/viewcvs/trunk/doc/limitati.txt?root=cint.
I haven't tried it out myself, yet.

C++ glue

2007-12-26T07:44:00.000-08:00

My currently biggest concern with programming languages is missing interoperability. When you write libraries in C++ you can't use them in *any* other language without quite some effort. Good examples are the Qt and KDE-libs. Other languages are completely incompatible with C++ and it's impossible to write really reusable code in C++. Since C++ is one of my favourite programming languages, I'd really like to create a 'bridge' which converts C++-interfaces to pure C-interfaces, which in turn can be used from any programming language. It is possible to create C-interfaces from within C++ with the extern "C" keyword. This way you can have the implementation in C++, but the header-file will be completely valid C-code. C-interfaces are the smallest common interface which usually every sane language supports. If it has dynamic object loading, it usually has support for objects using C-interfaces. This is a good thing, because it is possible to wrap C++-interfaces in C. There are, however, a few things you need to take care of.

Objects	C does not know about classes and objects with methods. You can simply simulate it with naming- and parameter-conventions with C-functions. This is somewhat a no-brainer.
Operators	Operators are nothing but simple functions with special syntax in C++. You can't use that syntax in C, so I'll convert operators to methods with special names like "assign_foo()" for the assignment-operator "=".
Templates	In the first step I'll ignore templates. In the second step, you'll have to provide a list of types you want wrappers for template-classes and functions to be generated for. You'll need to say "I want the template-function 'foo' to be available for the types 'bar' and 'baz' in the C-interface".
RAII and auto-objects	Most (nearly all) languages don't support stack-based user-defined objects, so I'll only work with pointers in the C-interface. There'll be no RAII, sorry.
Overloading	C does not support function-overloading, so with overloaded functions I need to append a number to the function-name. This means that "void foo(int); void foo(float);" will convert to "void foo(int); void foo_2(float);".
Constructors/destructors and new/delete	Since there are no stack-based objects in the glue-code, there'll only be 'new'ed objects. This means that in the wrapper-interface creating an object and calling the constructor is stricly bound together. For each class "foo" I'll create constructor-functions "create_foo" and "create_foo_2" and so on, for each ctor defined. The copy-constructor however will get a special name of "copy_foo()". dtors are expressed by a prepending ~ in C++. This character is no valid character in a function-name otherwise (neither in C nor C++). The dtor will be called "destroy_foo()". Because normal methods start with the classes name, special functions like ctors, dtors or operators start with the action's name instead to make sure there'll be no name-clashes.
Exceptions	Exceptions are a complicated topic. There's a whole topic dedicated to it later.

For a very simple class "foo" with a ctor, cctor, dtor and a function, there'll be the following wrapper-functions without name-mangling and with C-linkage:

struct foo *create_foo();
struct foo *copy_foo(const struct foo *obj);
void foo_set_name(struct foo *obj);
void destroy_foo(struct foo *obj);

Exceptions

In C++ you can throw any classes you want as exceptions and those classes are not explicitly marked as being used as those. This means that the glue-generator has NO idea what could've been thrown by the C++ functions (nor does the C++ libraries user, by the way). That's why there needs to be some configuration where the user can declare which exception base-classes will be caught. std::exception will always be caught of course. The C-interface will provide an error-message and an error-code to the C-user. The error-code will always be 0 for std::exception derived classes by default and only the error-message is filled with content. There'll be a source-code file called exception_translators.cpp which includes functions that are used to extract error-message and error-code from exception objects. Because the C-interface will rewrite all by-value return-types to by-reference (pointers) except for built-ins, it's possible to simply 'return 0' in case of an exception. But this does not mean that you should check for 0 in code that uses the C-interface. Always use the function last_exception()! last_exception returns an exception-info-object with the methods "const char *exception_get_message(const struct glue_exception *obj)" and "int exception_get_code(const struct glue_exception *obj)".
There are the following functions for exception-handling:

struct glue_exception *last_exception();
const char *exception_get_message(const struct glue_exception *obj);
int exception_get_code(const struct glue_exception *obj);
void destroy_exception(struct glue_exception *obj);

And they should be used after *each* call of a wrapped function which can potentially throw. This is a major headache of course, but that's just how error-handling in C HAS to work. There's absolutely no other way. Here's an example on how to "catch" an exception:

name = foo_get_name(t);
if( e = last_exception() ) {
   printf("Exception: %s\n",exception_get_message(e));
   destroy_exception(e);
} else
   printf("Name: %s\n",name);

This should be everything needed to finish this project. I'm sure I'll post more on this topic later and will probably even get a working implementation soon, so stay tuned.
I already have created wrapper-code that examplifies how the generated code should look like. In http://viming.de/cppglue.tar.bz2 there's the "C++ library" consisting of test.h and test.cpp and the wrapper-code consisting of "result.cpp" and "result.h", while result.h is compilable with both C++ and C compilers. The main.c is a C-application which uses the C++ classes. Code speaks more than a thousand words. :-)

German introduction to the C++ standard library

2007-12-17T09:51:00.001-08:00

Earlier this year I've written an introduction into the Standard C++ Library. The text fills four pages and includes containers, strings and streams. It should fit the needs to German-speaking C++ newcomers quite well and can be used as some kind of a kick-start.
Here's the link:
http://viming.de/cpp_stdlib.html

Showing framerate and temperatures in-game with RivaTuner

2007-12-07T07:42:00.000-08:00

RivaTuner is a very tricky program. It's user-interface is like a maze and it's virtually impossible to find some features it provides for the average user. The most interesting feature I know of is displaying the CPU and GPU-temperature in an on-screen-display in-game. With this you can easily check your temps while playing.
To do this, you need to first start the hardware-monitoring server. You need to create a Launcher-item to configure this. Switch to the register "Launcher" and press the Plus-button below the list.

Then you need to select "RivaTuner module activation item" in the following dialog.

In the module picker use the following settings: Module type is "Low-level module" and Module name is "Hardware monitoring". You can leave the name empty and it'll create a default-name for you. After you've pressed "Ok" it'll show the new item in your Launcher-list.

Launch the Hardware monitoring sub-system by double-clicking it in the launcher list or picking it from the context-menu of the tray-icon (if enabled). You'll be presented a new window with lots of bar-charts and a few buttons.

It doesn't seem to work on the system I'm creating this tutorial on, but this shouldn't be a problem.
Now right-click one of the bar-charts and bring up the Setup dialog.

This beast has only one important setting right now. You need to enable "Show xxx in on-screen display" and run the monitoring server. After you've done that, there'll be a new tray icon in your system for the monitoring server where you can enable and disable the on-screen display. It's enabled by default, so we shouldn't worry about that right now.
You should already have an OSD in your games now. You can try it by running any DirectX or OpenGL application. I recommend the Quake 3 Arena demo for this. :-)
Back in the window with all the bar-charts you can click the "Setup" button in the lower right corner. After this the hardware monitoring setup will pop up and you'll be able to select all the plugins you want to have displayed in the OSD. Just enable the desired plugin, press Setup and enable "Show xxx in on-screen display" like we did before.
There are some RivaTuner-plugins around which are not shipped with the default-installation, so you should try google when you don't have access to all temperatures you want to display.
This is what it looks like with my current settings:

RAII as an alternative to Garbage Collectors

2007-11-27T11:57:00.001-08:00

This text I've written is actually already pretty old, but I never really 'published' it on the web, because I was kinda too lazy. :-)
The topic is comparing RAII (Resource Acquisition Is Initialisation) to common Garbage Collector practices and why, in my opinion, Garbage Collectors are not needed in this world.
So here's a link, enjoy:
Garbage Collectors vs. RAII

boost::asio, Visual Studio 2005 and Windows 2000

2007-11-22T05:24:00.001-08:00

As we upgraded our compiler to 2005 earlier this year, I converted one project from Visual Studio 6 to the 2005 version now. It is a service-application and I struggled with it because it did not start. Then I ran the .exe from Explorer instead of the Services-management console to see that it required the (POSIX) function "freeaddrinfo", which is not present in Windows 2000's ws2_32.dll. This is a serious problem, because we want to support Windows 2000 Server edition for that software, especially since we run that version ourself! So I digged into it and found out about this:

Lost backwards-compatibility

According to this Microsoft-Blog the old versions of Visual Studio (read: 6 and earlier) defined getaddrinfo, freeaddrinfo and others as macros that called in-line functions that were declared in the now-legacy wspiapi.h header. When including Winsock2.h nowadays with VS 2003 (.NET) and higher, you don't include the wspiapi.h header anymore and use the usual function-declarations from ws2tcpip.h instead. Those functions now are resolved at run-time to be loaded from ws2_32.dll. Windows 2000's version of this dll does not provide these functions, because at that time they did not yet exist and were expanded to in-line functions instead.

The problem with asio

And here comes the trouble with boost::asio: boost::asio does not let you include the old Winsock-headers when compiling for a Windows-target greater than NT-version 5.1 (this means XP). It will #error out with the message "WinSock.h has already been included".

Resolution

To circumvent this, you need to define _WIN32_WINNT to something lower than 0x0501. 0x0500 is quite resonable for this.

Making .NET-assemblies available to native languages

2007-11-16T01:42:00.001-08:00

As jb from #mono pointed out, there's already a tool that converts Assemblies to corresponding glue-code in C, called cilc. It's included in mono and uses mono runtime libraries, so you need to have mono installed to use it.
I could only find a man-page here, but no documentation so far, but it should be fairly easy to use and understand the C-code, I guess.

Making C++ libraries available to .NET and native languages

2007-11-16T01:08:00.001-08:00

I just got the following idea on how I could expose my pure, unmanaged C++ interface to .NET-languages. The idea is taking two steps:

Step 1

I create a C-interface from my C++ classes, which should be rather easy. Take the class "class Foo { void bla(); };" for example, which would be wrapped into C-code like "struct foo *foo_create();" and "void foo_bla(struct foo*);".
With this, I have a language-neutral interface that I can put into a library. With language-neutral I mean that nearly every language can interact with C-interfaces. This can be done manually or you can write a 'converter' by parsing the C++-headers and auto-generating C glue-code.
I've done something similar while creating the fbsql Interbase/Firebird-API for Lisp, but that was all manual work.

Step 2

Then I'll need to find a way to create ILC-code that resembles this object oriented C-interface and creates a real class from it, fills the method implementations with stubs that load the library and finds the symbols to P/Invoke the functions, respectively.
An alternative to directly writing the ILC-code would be writing C#-code instead. But I'd like to eliminate the dependency on a C#-compiler by directly writing ILC-code, if this is not incredibly hard. I don't have a clue about how ILC looks like and how complex it is.

Direct solution

There is, however, an even more advanced approach. Theoretically I can parse C++-headers (with gcc-xml) and extract all classes and methods. I create ILC code from that which loads the library and the (mangled) symbols and do a bit voodoo to fake C++'s object mechanism. It's actually not that hard, I only need to allocate memory and call the ctor on that memory with the correct calling-convention to create an object.
I like having the C-interface, though, because it gives me more flexibility and I can use it from non-.NET-languages, too. This way I have a truly language-independent interface that should be easy to use from anywhere.

New AMD Phenom X4 fails to impress

2007-11-07T12:28:00.001-08:00

The Phenom X4 is about to hit the shelves. There's been
word about pricing already, but none about performance so far. Well, until now. There's a Crysis benchmark over at expreview.com. I think many expected a miracle happening on the AMD side, because their CPUs, humbly spoken, suck at this time. The results are disillusioning, though:
While the E6850, QX6850 and QX9650 (all 3GHz CPUs) run at the same average of 49 FPS, the Phenom X4 only does 46 FPS at the same clock-rate of 3GHz.
All intel CPUs use a 333MHz (quad-pumped) Front-Side-Bus and a low multiplicator of 9, while the Phenom X4 has a low FSB of 200MHz and a high multiplicator of 15. That means that the communication of the CPU to the periphery is 40% slower, which might, in my humble no hardware-geekish opinion, be the reason for the slowdown.
And now here's the even more demolishing opinion of mine: As you might know I own a Q6600 which is clocked at 2.4GHz. I overclocked it to 3GHz, the same clock-speed the benchmark used, and noticed virtually *no* difference. This could mean that the Phenom X4 is even slower than the Q6600 with 600Mhz less clock-speed!
It's still very early and I don't want to jump to conclusions as of yet, but I don't expect much from AMD's upcoming CPU generation.

Learning a foreign language? Try Anki!

2007-11-05T12:53:00.000-08:00

There are many approaches to learn vocabulary of foreign languages, and one of those are flashcards. There's an awesome and at the same time lightweight software out there that helps you learning with flashcards. It's localized to English, Spanish, French, German, Japanese and Dutch and comes with example decks for Japanese and Russian words.
If you're interested, try Anki now at http://ichi2.net/anki/.
It's totally free and even open-source.

Efficient and reliable file management

2007-11-05T08:40:00.000-08:00

We all know Microsoft explorer. We all know the millions of products that copy it. We also know, from earlier times, the Commander software and we should at least have seen a few of the millions of products that copy it. None of them suits my needs, unfortunately.

Status Quo

Efficiency is everything in my daily life with computers. You should be able to accomplish every task with the minimal count of keystrokes, and have every task freely pausable, stoppable and resumable, if technically possible. And, of course, you need to parallelize tasks and have them reliable work next to each other.
For the Microsoft Explorer, for example, reliability is something unknown. In the default configuration, if one explorer window, or some seemingly unrelated system-task crashes, it takes all your copy operations with you. And it's impossible to resume copy operations with the Microsoft Explorer. I have to admit that the new version shipped with Windows Vista is a complete overhaul and does address some of the main issues I had with XP and earlier Windowses. First, I like the Task-dialogs. Task-dialogs give you big, recognizable buttons with labels that actually tell you what they do after you click them, including a small, detailed explanation beneath them. This is a good example for intuitive, yet non-intrusive user-interface design. But the new Explorer still fails at reliably resuming copy operations or predicting the results of those. For example, you can't specify whether you want the timestamps of the files to be copied stay the same or are updated to 'now'. When moving files, the timestamps don't change. When copying files, they're updated. But, especially when copying files when transferring some large amounts of data that's in daily use, I want the timestamps to stay the same! And it may actually be a big problem if they change, for backup-reasons. No chance with Explorer.

Requirements

Additionally, I want to see both progress in bytes and percent. And I want to see speed in a unit like MByte/s or MBit/s and a reliable prediction of the ETA. Here, too, the new Vista Explorer does a better job than it's XP counterpart. I guess everybody already became victim of a 500kb file that, according to Explorer, would take 32038 days to copy from the CD to your hard disk.
Then, to increase throughput, you might want to fine-tune some parameters like the buffer-size. The buffer-size can be very important in copy operations depending on the source and destination. For example, for a network source from a SMB-share to your local hard-drive or from one hard-drive to the other, you should use very different buffer-sizes. The hard-drives should cope with much bigger buffers than the network-protocol-bound SMB transfer (this is still speculation at this time, though).
Finally resume-operations should be fully customizable. I want options like 'Always overwrite newer', 'Ask if size differs', 'Always overwrite smaller', 'Check file-contents if smaller' and much more. I need those for various situations. You use file-copy operations in many different scenarios that all require different behavior.
A great feature would be to postpone files that the software is unsure about to the end. It happened *so* often that I started a large (like a few hours) file-operation and came back just to see that it asked me for the fifth file whether I want to hug, seize or overwrite it. If it skipped that file and already copied all the others, it would've saved me hours of either my free-time or my work-time.

User-interface

Let's take the Explorer-example, again. In Explorer, you can do everything with the keyboard. But maybe not as easily as you wanted. Jumping to a new address can become a hassle. I noticed that the hotkey to jump to the address-pane changed in every small update the explorer ever got. I think it was already set to each character that you can find in 'address', namely a, d, r, e and s. In the Vista-explorer, you don't even know how to jump to the breadcrumb-style address-pane. But I'll tell you a secret: It's Alt+D. For now, it might change next week.
But still, everything you can do in Explorer you can relatively "easy" do with they keyboard. But there are just things you can't do with Explorer. Selecting files by pattern? Nope. You can use the search-function as a workaround, which isn't optimal and only suits some limited needs. Imagine a dir filled with hundreds of seemingly random files and you want to delete all that start with an A and end in .xxx. Maybe only if they have the archive-bit set or have some special UNIX-permissions. Or only directories. You simply can't, and have to pick them by hand. I want a mixed Regular Expression and file-attribute filter to select my files to delete, copy or move them.

Roundup

So here's a feature round-up that I demand from a file-management-software:
- Large number of options for file-transfers
- Resumable operations
- Independent, parallel operations
- Postponing files that it is not sure about
- User-interface completely keyboard-compatible
- Selection by regex and file-attribute filtering, maybe recursive
- Command-line as an alternative to GUI would be awesome!

High-performance, high usability SQL-List

2007-11-05T07:30:00.000-08:00

This is about list-controls in graphical user interfaces that display query-contents from a sql-query. I think most of us know the hassles of scroll-bars in lists that are dynamically filled with result-sets from sql-queries. The problem with it is that after you began to fetch datasets, you don't know how many to come. It may be 20, or it may be 20.000.000.

Status Quo

So most application-writers are lazy and just fetch some more than fit into the list, and fetch more as the user tries to scroll down. That's actually not as lazy as it sounds, since even this simple solution is problematic to implement depending on GUI-toolkit or API. Hooking up with the correct scroll-bar-events and making it at least a bit bearable to the user can become quite a hassle in itself. And when you've managed to make it work, it's still far from usable: The user doesn't know how many datasets are displayed in the list, either, and the scroll-bar and other visual implications might let the user think that there are only a few datasets. After he scrolled down for 5 minutes, he might finally understand that there's much more data than he might initially have thought. There's the Ctrl+End shortcut that jumps to the end of lists. Most implementations display an hourglass-cursor or a progress-bar in this situation. It might be slow, because they fetch all the content in the list when you want to reach the end.

There are better ways to do this!

Incrementally filling the list in the background (a second thread) might or might not be the best solution to this problem. First, you definitely need an activity-implicator, like a small text saying "Fetching..." somewhere. And you need a way to cancel the fetching in case the user doesn't even remotely care about any but the first 30, 10 or even 2 datasets, or simply cancels the operation. The implementation should be smooth enough so the user doesn't actually care whether it's still fetching, or not, anyways. But being able to cancel it is not a bad idea, either.
Now to more detailed implementation suggestions: Fetching in the background is, of course, nice to have. The user can interact with the software as early, or even earlier, than with the naive 'fetch-first-20'-approach, and still, after some time, knows how many datasets there are. He can't immediately know how many there are, because it's technically impossible. You could do a select count(*) from table beforehand, but it's not even guaranteed the second select will yield the same number of datasets and it may cost a LOT of initial wait-time and up to double the server-load than the following approach. Fetching all available datasets takes some time, too, but we have no other choice but to let the user wait until we gathered the needed information. But it's good to let him interact with the parts of the list we already fetched.

Speeding it up

So there are two things you're eager to know: You need to know the contents of the first X datasets, where X is the number of datasets displayed in the list. And you need to know how many datasets are available in total. Fetching the first X datasets is easy, speeding up fetching all datasets is a bit tricky, though.
The trick is to fetch only the primary key of the relevant datasets. This will be magnitudes faster than fetching the whole dataset, which might include large blobs of text or pictures. Let's imagine a table "list" that consists of the fields ID (integer, primary key), Name (varchar) and Description (Text). The list is generated from the query select * from list. What the implementation does is convert this sql into two sql-statements: select id from list and select * from list where id=?. You execute the first query, and check after each fetch whether a dataset's content needs to be fetched. In the display-routine of the list, you mark each dataset that needs to be displayed as 'needs to be fetched' (if it isn't already available, of course). If there are datasets that need to be displayed, the fetching-thread stops, fetches the relevant datasets and sends a signal to the list that those datasets are ready to be displayed. This works remarkably well. The only pitfall is the thread-synchronization-issue, which might or might not be easily possible in your programming language of choice. The fetcher-thread signals new datasets to the list-view, so the list-view can increment it's item-count to update the scrollbar's appearance. It's wise not to do this after each fetch. If you did so, the signal-queue might fill up and slow down the application. You should only send an update-signal to the list every 500ms or so. This is easier on the eyes for the user and it'll yield better performance and usability. Depending on the list-implementation, the slider of the scrollbar should be usable even while filling the list. With Qt's QListView, for example, this works like a charm.

Conclusion

So we've managed to overcome every shortcoming that SQL-based lists usually yield:
- The user sees the first datasets immediately
- There's virtually no wait-time
- The total number of datasets will be available as fast as technically possible
- Only the content of datasets that need to be displayed will be fetched
- Even skipping to the end will not fetch every dataset
- The network-traffic is therefore kept MINIMAL for the desired behaviour

Overclocking the Q6600 on a P5N32-E SLI

2007-10-29T11:43:00.001-07:00

So at friday my Q6600 (Core 2 Quad 2.4GHz with G0 stepping) and the ASUS P5N32-E SLI arrived and I was, of course, eager to try it out. Well, my system is water-cooled, so it's quite a hassle to change the mobo, of all parts.

Initial problems

First thing I noticed after installing the new stuff was that the PS/2 keyboard did not work. HOLY SHIT, it's a freaking PS/2 keyboard and it does not work with that motherboard! I was (and still am) very shocked about that. It's like building a car with the steering wheel not working. Just not as dangerous.

So I hooked up my USB-keyboard (which is not very comfortable, since it's a compact keyboard) and used that for the day. I installed XP and afterwards windows and, except some small networking quirk in XP, everything worked pretty darn well so far. Nice Vista drivers, even 64bit. It wasn't all that easy the first time I installed Vista on my older P5WD2-E Premium, since it hadn't that great Vista support (how could it if it was build before Vista being around, anyways).

Here we go

After a little bit of initial gaming on stock-clockspeeds I decided to test it's overclocking capabilities. Now here's some more information on my system, first:
It's watercooled and it's pretty well cool I'd say, since both CPU and GPU-temps are okay. The only temperature I'm worried about is the motherboard's: The BIOS says it's 46°C hot, which I think is just too much. I installed a big fan later and it dropped to under 40°C. I have watercooled 2GB (2x1GB) OCZ-RAM with 5-5-5-15 timings, DDR2-800. Then there's the XFX GeForce 8800 GTS 640MB which is clocked to 513MHz core stock, I run it at 700MHz for games and it doesn't even break a sweat with under 52°C. So I went for 3.0GHz for a start, and it worked nicely.

Overclocking

I simply changed the FSB to 333 (1333MHz quad-pumped). Windows boots, benchmarks run, everything's fine. Whenever I try to go just one MHz above 333, the system won't boot up. It won't even finish it's POST. I think that's somehow ridiculous, I should definitely be able to do more with that CPU, it's a G0-stepping, after all. Updating the BIOS just did not do good. After I've updated it to Revision 1205, I couldn't even get it to boot nicely with the 3,0GHz. The speeker beeped like crazy every second time I booted the PC and I reverted my BIOS back to to the second-oldest version, which is 1103. I might add that with the 1205, the PS/2 keyboard worked. It won't with the 1103 I'm currently running. Additionally, the newest BIOS gave crazy voltages to my CPU. It went to 1.38V, although the CPU is spec'ed to a max of 1.37V, iirc. With the older BIOS-revisions, the vcore was much lower.

My current computer-setup

2007-10-29T11:32:00.000-07:00

After upgrading my box this weekend, it currently comes down to this:

CPU	Core 2 Quad Q6600	watercooled, clocked at 3.0 GHz
Graphics	GeForce 8800 GTS 640MB	watercooled, XFX, clocked at 700/2200MHz
Mainboard	nForce 680i SLI	P5N32-E SLI
RAM	2GB DDR2-800	watercooled, OCZ