Those of you that are familiar with eCraft and the types of applications we create, know that we mostly work with business applications and shuffle data back and forth between different back office systems or create applications that lets users interact with back office systems through more lightweight user interfaces tailored for their daily tasks. Therefore it was very interesting to be able to do yet another Microsoft Surface project, this time where the application was not data-centric but instead the focus was shifted towards the presentation end of the spectrum. We were given the opportunity to develop a Surface application to be used at the Nobel festivities with a very tight deadline for the development work. You can read a blog post about our involvement in the project here (in Finnish).

The design called for the Surface background to be sensitive to touch so that the users would be able to paint with their fingers on the screen and so that the paint would gradually fade out over time. The Surface SDK provides a SurfaceInkCanvas control that would easily handle the painting part, but achieving the kind of fade out that we wanted did not seem possible that way. Instead I started looking into the possibility of programmatically generating and updating a bitmap and discovered that it is possible through the use of the WriteableBitmap to do just that. In case you want to do this, make sure that you are using a recent version of the .Net Framework. At least .Net 3.5 and newer have implemented WriteableBitmap using double buffering and other techniques that guarantee it is performant enough for updating the bitmap often. We ended up using a bitmap image of size 1027x768 (Surface resolution) as the background of the application and any finger touch on this would start painting with a small 60x60 brush on the bitmap. We randomly used one of several slightly different brushes to achieve a more “alive” stroke as we painted. The brushes we used were 24-bit PNGs with about 50% alpha in the middle increasing to completely transparent towards the perimeter.

The following functions shows how we used WriteableBitmap to achieve the intended effect of blending together the brush bitmap with whatever was already on the background. Rows 6-11 make sure that in case the contact point is close to the edge of the background bitmap,only the correct part of the brush will be used to update the background. And in lines 26-47 we extract the Red, Green, Blue and Alpha values from both the background and the brush image and blend them together and construct the new image pixel.

private void UpdateBitmap(WriteableBitmap bitmap, BitmapImage brush, Point centerPoint)
{
    if ((int)centerPoint.X >= 0 && (int)centerPoint.X < bitmap.PixelWidth
         && (int)centerPoint.Y >= 0 && (int)centerPoint.Y < bitmap.PixelHeight)
    {
        int bitmapX = Math.Max((int)centerPoint.X - brush.PixelWidth / 2, 0);
        int bitmapY = Math.Max((int)centerPoint.Y - brush.PixelHeight / 2, 0);
        int brushX = Math.Max(brush.PixelWidth / 2 - (int)centerPoint.X, 0);
        int brushY = Math.Max(brush.PixelHeight / 2 - (int)centerPoint.Y, 0);
        int brushWidth = Math.Min(brush.PixelWidth - brushX, Math.Min(bitmap.PixelWidth - bitmapX, brush.PixelWidth));
        int brushHeight = Math.Min(brush.PixelHeight - brushY, Math.Min(bitmap.PixelHeight - bitmapY, brush.PixelHeight));

        Int32Rect brushRect = new Int32Rect(brushX, brushY, brushWidth, brushHeight);
        Int32Rect bitmapRect = new Int32Rect(bitmapX, bitmapY, brushWidth, brushHeight);

        int stride = (brushRect.Width * brush.Format.BitsPerPixel + 7) / 8;
        var brushPixels = new UInt32[brushRect.Width * brushRect.Height];
        var imagePixels = new UInt32[brushRect.Width * brushRect.Height];

        brush.CopyPixels(brushRect, brushPixels, stride, 0);
        bitmap.CopyPixels(bitmapRect, imagePixels, stride, 0);
        bitmap.Lock();

        for (int i = 0; i < brushPixels.Length; i++)
        {
            UInt32 brushA = (brushPixels[i] & 0xFF000000) >> 24;
            UInt32 brushR = (brushPixels[i] & 0x00FF0000) >> 16;
            UInt32 brushG = (brushPixels[i] & 0x0000FF00) >> 8;
            UInt32 brushB = (brushPixels[i] & 0x000000FF);

            UInt32 imageA = (imagePixels[i] & 0xFF000000) >> 24;
            UInt32 imageR = (imagePixels[i] & 0x00FF0000) >> 16;
            UInt32 imageG = (imagePixels[i] & 0x0000FF00) >> 8;
            UInt32 imageB = (imagePixels[i] & 0x000000FF);

            //(Rs As + Rd (1 - As), Gs As + Gd (1 - As), Bs As + Bd (1 - As), As As + Ad (1 - As))
            imageR = (brushR * brushA + 255 * imageR - imageR * brushA) / 255;
            imageG = (brushG * brushA + 255 * imageG - imageG * brushA) / 255;
            imageB = (brushB * brushA + 255 * imageB - imageB * brushA) / 255;
            imageA = Math.Max((brushA * brushA + 255 * imageA - imageA * brushA) / 255, imageA);

            imageA = imageA << 24;
            imageR = imageR << 16;
            imageG = imageG << 8;
            //imageB = imageB;

            imagePixels[i] = imageA | imageR | imageG | imageB;
        }

        // copy in the changed pixels
        bitmap.WritePixels(bitmapRect, imagePixels, stride, 0);

        // Specify the area of the bitmap that changed.
        bitmap.AddDirtyRect(bitmapRect);

        // Release the back buffer and make it available for display.
        bitmap.Unlock();
    }
}

In order to achieve the wanted fade out effect we had another function that we called from a timer that would adjust the alpha channel value of every pixel in the background bitmap.

private void FadeBitmap(WriteableBitmap bitmap, Int32Rect rect)
{
    var imagePixels = new UInt32[bitmap.PixelWidth * bitmap.PixelHeight];
    bitmap.CopyPixels(rect, imagePixels, bitmap.BackBufferStride, 0);
    bitmap.Lock();

    for (int i = 0; i < imagePixels.Length; i++)
    {
        UInt32 imageA = (imagePixels[i] & 0xFF000000) >> 24;
        imageA = imageA < 10 ? 0 : imageA - 10;
        imageA = imageA << 24;
        imagePixels[i] = imageA | imagePixels[i] & 0x00FFFFFF;
    }

    // copy in the changed pixels
    bitmap.WritePixels(rect, imagePixels, bitmap.BackBufferStride, 0);

    // Specify the area of the bitmap that changed.
    bitmap.AddDirtyRect(rect);

    // Release the back buffer and make it available for display.
    bitmap.Unlock();
}

In our implementation we faded the entire background image during every cycle. In a slightly more fancy version we could partition the screen into smaller spaces and keep track of which partitions actually contain any pixels that needs fading, increasing the complexity a bit but drastically cutting down on the amount of bitmap updates needed. That improvement is left as an exercise for the reader :)

In closing, I found that this was a very interesting project to work on and it was really invigorating to be forced to think about bitwise operations and bit shifting. In the normal projects we do, there are seldom any need for going into that level of low-level funkiness.