Correction: The result was of course closer to composition than to continuations, as was pointed out to me by a colleague.

#include <stdio.h>
#include <string.h>

/* Copyright (c) 2014 Eugene Crosser.                                      */
/* License: CC0 (http://creativecommons.org/choose/zero/)                  */

/* Toungue in cheek "continuation passing" implementation of FizzBuzz      */
/* Inspired by this:                                                       */
/* http://themonadreader.files.wordpress.com/2014/04/fizzbuzz.pdf          */

/* With Strings and garbage collection we would have used strings objects. */
/* As we don't want to muddle the code with memory managenemt, we just use */
/* static buffers here. Make them long enogh for the longest output.       */
typedef struct {
	char value[16];
	char dflt[16];
} fbstate_t;

fbstate_t test(int what, int against, char *v, fbstate_t ost) {
	fbstate_t nst;

	if (what % against == 0) {
		strncpy(nst.value, v, sizeof(nst.value));
		strncat(nst.value, ost.value, sizeof(nst.value));
		nst.dflt[0] = '\0';
	} else {
		nst = ost;
	}
	return nst;
}

fbstate_t dflt(int what) {
	fbstate_t nst;

	nst.value[0] = '\0';
	snprintf(nst.dflt, sizeof(nst.dflt), "%d", what);
	return nst;
}

fbstate_t final(fbstate_t ost) {
	fbstate_t nst;

	strncat(nst.value, nst.dflt, sizeof(nst.value));
	nst.dflt[0] = '\0';
	return nst;
}

fbstate_t run(int what) {
	return final(
		test(what, 3, "Fizz",
		  test(what, 5, "Buzz",
		    dflt(what))));
}

int main(int argc, char *argv[]) {
	int i;

	for (i = 1; i <= 100; i++) {
		fbstate_t st = run(i);
		printf("%s\n", st.value);
	}
	return 0;
}

After some fiddling with exitwp, I noticed this fork which addressed hackyll’s specific idiosyncrasy to the tag names containing underscore, and allowed to preserve the document tree structure that I had for my permalinks (it was my goal to preserve permalinks).

This is what I came up with by now:

--------------------------------------------------------------------------------
{-# LANGUAGE OverloadedStrings #-}
import           Data.Monoid (mappend)
import           Hakyll


--------------------------------------------------------------------------------
main :: IO ()
main = hakyll $ do
    match "images/*" $ do
        route   idRoute
        compile copyFileCompiler

    match "css/*" $ do
        route   idRoute
        compile compressCssCompiler

    match "about/index.markdown" $ do
        route   $ constRoute "about.html"
        compile $ pandocCompiler
            >>= loadAndApplyTemplate "templates/default.html" defaultContext
            >>= relativizeUrls

    match "posts/*/*/*/*" $ do
        route $ gsubRoute "posts/" (const "") `composeRoutes`
                gsubRoute ".markdown" (const "/index.html")
        compile $ pandocCompiler
            >>= saveSnapshot "content"
            >>= loadAndApplyTemplate "templates/post.html"    postCtx
            >>= loadAndApplyTemplate "templates/default.html" postCtx
            >>= relativizeUrls

    create ["archive.html"] $ do
        route idRoute
        compile $ do
            let archiveCtx =
                    field "posts" (\_ -> postList recentFirst) `mappend`
                    constField "title" "Archives"              `mappend`
                    defaultContext

            makeItem ""
                >>= loadAndApplyTemplate "templates/archive.html" archiveCtx
                >>= loadAndApplyTemplate "templates/default.html" archiveCtx
                >>= relativizeUrls
                >>= removeIndexHtml


    match "index.html" $ do
        route idRoute
        compile $ do
            let indexCtx = field "posts" $ \_ ->
                                postListCont $ fmap (take 3) . recentFirst

            getResourceBody
                >>= applyAsTemplate indexCtx
                >>= loadAndApplyTemplate "templates/default.html" postCtx
                >>= relativizeUrls
                >>= removeIndexHtml

    match "templates/*" $ compile templateCompiler

    create ["atom.xml"] $ do
    route idRoute
    compile $ do
        let feedCtx = postCtx `mappend` bodyField "description"
        posts <- feedList
        renderAtom myFeedConfiguration feedCtx posts

    create ["rss.xml"] $ do
    route idRoute
    compile $ do
        let feedCtx = postCtx `mappend` bodyField "description"
        posts <- feedList
        renderRss myFeedConfiguration feedCtx posts


--------------------------------------------------------------------------------
postCtx :: Context String
postCtx =
    dateField "date" "%B %e, %Y" `mappend`
    defaultContext


--------------------------------------------------------------------------------
postList :: ([Item String] -> Compiler [Item String]) -> Compiler String
postList sortFilter = do
    posts   <- sortFilter =<< loadAll "posts/*/*/*/*"
    itemTpl <- loadBody "templates/archive-item.html"
    list    <- applyTemplateList itemTpl postCtx posts
    return list

--------------------------------------------------------------------------------
postListCont :: ([Item String] -> Compiler [Item String]) -> Compiler String
postListCont sortFilter = do
    posts   <- sortFilter =<< loadAllSnapshots "posts/*/*/*/*" "content"
    itemTpl <- loadBody "templates/post-item.html"
    list    <- applyTemplateList itemTpl postCtx posts
    return list

--------------------------------------------------------------------------------
--feedList :: ([Item String] -> Compiler [Item String]) -> Compiler String
feedList = fmap (take 10) . recentFirst
    =<< loadAllSnapshots "posts/*/*/*/*" "content"

--------------------------------------------------------------------------------
removeIndexHtml :: Item String -> Compiler (Item String)
removeIndexHtml item = return $ fmap cuttail item
  where
    cuttail = withUrls $ replaceAll "/index.html" (const "/")


--------------------------------------------------------------------------------
myFeedConfiguration :: FeedConfiguration
myFeedConfiguration = FeedConfiguration
    { feedTitle       = "Average Blog"
    , feedDescription = "Random Ramblings"
    , feedAuthorName  = "Eugene Crosser"
    , feedAuthorEmail = "crosser@average.org"
    , feedRoot        = "http://www.average.org/blog/"
    }

There is a thing that I’ll need to address at some point: I’d prefer to keep the markdown source in a flat directory, and the posts be put in the tree based on the posting date. In the existing code, the route to the article is derived directly from the path to the markdown source.

Now I know.

What is event correlation? Imagine that you are monitoring the responsiveness of the web server, disk space, load average and “pingability” on a machine. If this machine is disconnected from the network or crashes, without event correlation, you will get four alarms, one for each of the monitored attributes. You don’t want that, you want to know just that the machine is down (“unpingable”), the rest is unhelpful noise. To avoid superfluous notifications, you need to arrange the monitored attributes into a dependency tree, and if some attribute becomes “failed”, suppress notifications about the failures of its dependent attributes. Quite simple, and, yes, the term “event correlation” is misleading, but never mind that.

My monitoring tool reports status change immediately when the probing completes, and different attributes are probed independently and in parallel. I could check if any of the upstream dependencies of an attribute are in “failed” state before reporting, but it is quite probable that after a failure, a dependent attribute will be probed earlier than its dependency, and be reported nevertheless.

And here, at last, is the solution to this problem:

When we notice status change of an attribute that has dependencies, queue the report without sending it. When we have “success” of a probe of an attribute that has dependants, and the previous status was “success” as well, send the queued reports of the dependants; if the combination of the current and previous statuses is different, discard queued reports. State changes of an attribute that has no dependencies are reported right away without queueing.

When there is more than one level of dependencies the scheme becomes only a little bit more complicated: when you need to release the queued reports, you don’t send them but rather re-queue them up to your upstream dependency.

And that’s it.

Recently I got a new router and installed fresh version of openwrt. It has unbound in the default app repo so I soon switched off the DNS part of dnsmasq and enabled unbound instead.

This has a drawback of not having the local network served by dynamic dns of dnsmasq, so I could no longer access mobile devices by their names. It is not such a big deal, but it irritated me. So I decided to chain the two dns servers together, to have both a validating resolver and a properly handled domain for the home network.

Unfortunately, the build of unbound on openwrt is quite rough. It is not integrated with Luci, and does not even work out of the box until you manually disable the dns functionality of dnsmasq by setting the listen port to 0. There is a howto on the web on chaining unbound “behind” dnsmasq. I wanted it the other way around, something that the guy could not achieve.

First, we want to put dnsmasq on the side. The easiest way is to change the listening port to something non-standard. This can be done from the GUI:

Network -> DHCP and DNS -> Advanced Settings

set DNS Server Port to 5553, and hit Save&Apply;. (At this point DNS will stop working on your network until you configure and launch unbound.)

Now, to the magic part. We will be configuring unbound to serve all the world as a normal caching resolver, except the local domain and the reverse zone for the local network, for which it will act as a forwarder. In the examples we assume that the local domain is ‘lan’ and the local network is 192.168.<something>. We will need to add something to the configuration file “/etc/unbound/unbound.conf”.

First, the basic forwarding directives:

forward-zone:
        name: "lan."
        forward-addr: 127.0.0.1@5553
	
forward-zone:
        name: "168.192.in-addr.arpa."
        forward-addr: 127.0.0.1@5553

These are self-explanatory. But this is not enough. We have to add a number of directives to the server section. First, unbound refuses to send DNS requests to localhost by default. To persuade it, we need this directive:

        do-not-query-localhost: no

Then, by default, unbound will try to establish trust chain. Serving a query in the “lan.” domain, it will try to get the DS record form the parent domain, in this case “.”. Of course the Internet root servers have no idea about our local “lan.” domain and will respond with NXDOMAIN. Which will break the trust chain. To prevent unbound from attempting to build the trust chain, we need to write these directives (second is for the reverse zone):

        domain-insecure: "lan."
        domain-insecure: "168.192.in-addr.arpa."

Next, by default, unbound refuses to return As and AAAAs that point to private (RFC1918) addresses. We have such addresses in the “lan.” domain, so tell it to unbound:

        private-domain: "lan."

On the other side, unbound refuses to deal with PTRs that come from private addresses, so we must override that too:

        local-zone: "168.192.in-addr.arpa" nodefault

That’s all. Now save the file, “Enable” and “Start” the “unbound” service in System -> Startup and it should work.

Should you need to troubleshoot, run "unbound -vvv" and "logread -f". With some luck, the messages will hint to the problem.

> module Main where

Generate digits of Pi as an infinite list.
Use Unbounded Spigot Algorithm from this article:
http://www.cs.ox.ac.uk/jeremy.gibbons/publications/spigot.pdf

> piDigits :: [Integer]
> piDigits = g(1,0,1,1,3,3) where
> g(q,r,t,k,n,l) = if 4*q+r-t<n*t
>   then n : g(10*q,10*(r-n*t),t,k,div(10*(3*q+r))t-10*n,l)
>   else g(q*k,(2*q+r)*l,t*l,k+1,div(q*(7*k+2)+r*l)(t*l),l+2)

Find palindromes of specified length in an infinite list of digits.
Result is returned as an infinite list of Integers, each element
a palindrome when represented as digital characters. E.g.:
"findPali 3 [1,2,3,2,1,7,9,7]" returns "[232,797]".

> findPali :: Int -> [Integer] -> [Integer]
> findPali n x
>   | length pref < n      = []  -- Not necessary for infinite lists
>   | pref == reverse pref = toint pref:findPali n (tail x)
>   | otherwise            = findPali n (tail x)
>     where
>       pref = take n x
>       toint = foldl (\acc x -> acc*10+x) 0

Check if an Integer is a Prime number. Function copied from here:
http://stackoverflow.com/questions/4541415/haskell-prime-test
It is a "naive" implementaion but fast enough for our case, and
readily understandable.

> isPrime :: Integer -> Bool
> isPrime x = not $ any divisible $ takeWhile notTooBig [2..]
>   where
>     divisible y = x `mod` y == 0
>     notTooBig y = y*y <= x

Find and return the first element in a list based on predicate.
Our list is infinite so we don't need the result to be of "Maybe"
type like the "canonical" implementation in Data.List.

> find' :: (a -> Bool) -> [a] -> a
> find' p (x:xs)
>   | p x       = x
>   | otherwise = find' p xs

Print the first 7-digit palindrome of consecutive digits of Pi
that is a Prime number.

> main = putStrLn $ show $ find' isPrime $ findPali 7 piDigits

This is not the fastest thing you can invent (prime check is very simplistic), but it gives the result in under a minute on my system.

UPDATE 2012-04-16: Found the link. Not writing it here ;)

So I got this idea: I have the call log in my phone. If I could make a wall display at my mom’s place that shows when the last call happened that might help. For good measure, I could analyse my location and show if I am at home, or at work, or elsewhere.

The first question that arised was what to use as the display. A computer monitor or an electronic photo frame would irritate mom with the glow. Ideally I would want e-ink display like in an ebook reader. Making custom electronics to drive the display is out of my league (or at least would require much more effort than I was ready to invest), and most existing readers would be difficult to hack to show the data that I need. But not all. I learned that Sony PRS-T1 runs Android, has WiFi, and can be easily rooted. So it was settled.

With Android systems on both ends, using Google services for intermediary was an obvious choice. I already had Latitude enabled on my phone, and the call log synched to a dedicated Google calendar by SMS Backup+. (The latter I later replaced with CallTrack because it creates the calendar events in near-real-time when the phone call completes.)

The next question was where to process the data. For a while, I considered the idea of running the processing “in the cloud” (on my own collocated server), making API calls to Google services, and cooking the text for displaying on the reader device. Or even the image. The latter would be the only choice if I was using a photo frame. In the end, I decided to cut the number of intermediaries and do the job on the reader device itself.

And here is the git repository.

Edit Feb 10: this is the problem that I mentioned above. So far, I just introduced a couple of retries in case of error, but I hate it this way.

(This was posted as a comment in a discussion about the redesigned Google Reader on G+)

I’ve been mulling this idea for a long time. In one line:

One man’s +1 is another man’s -1

When reddit became popular, the biggest frustration for me was the abundance of kittens. (Don’t get me wrong, I love kittens, I just come to reddit for other things.) Now, if I consistently downvote kittens, then the system should be able to determine that my taste is opposite to the users who consistently upvote them, and count their upvotes as if they where downvotes, when creating the display for me. Conversely, my downvote will be counted as upvote in relation to those people.

This way, people with opposite tastes will help produce better experience for each other, instead of disrupting it like they do in all current rating-based systems.

Next step will be automatic grouping of people with similar tastes. Friend suggestions based on actual personal qualities instead of incidental features of the social graph, isn’t it fascinating?

Of course, for this idea to work, there needs to be a -1 button like on reddit. Google ±, anyone?

Update Aug 3 2013: nice illustration!

lucky_list =
    [[a,b,c,x,y,z] | a<-[0..9], b<-[0..9], c<-[0..9],
                     x<-[0..9], y<-[0..9], z<-[0..9],
                     a+b+c == x+y+z]
main = putStrLn $ show $ length lucky_list

this runs in 650+ ms on my system. Can we improve on that? Let’s iterate over possible sums (which are from 0 to 27 since 9+9+9=27). For each sum, combine the distinct triples that sum to that. We can iterate over possible pairs of digits (x,y), getting the third digit as (z = sum-x-y). Here is what we get:

lucky_list =
    [x++y | s <- [0..27], x <- triples_with_sum s, y <- triples_with_sum s ]
  where
    triples_with_sum s =
      [[x,y,s-x-y] | x<-[lx .. hx], y<-[ly x .. hy x]]
        where
          lx   | s < 18     = 0
               | otherwise  = s-18
          hx   | s > 9      = 9
               | otherwise  = s
          ly x | s-x-9 < lx = lx
               | otherwise  = s-x-9
          hy x | s-x > 9    = 9
               | otherwise  = s-x
    -- This expression is less optimized but is much simpler:
    -- a <- [0..9], b <- [0..9], let c = s-a-b, c >= 0, c <= 9]
    -- Speed difference is not noticable
main = putStrLn $ show $ length lucky_list

That’s better: about 80 ms. Now we try to do it in C:

#include <stdio.h>

typedef void (*iterator_t)(int a, int b, int s, void (*func)(), int x, int y);

static int count = 0;
#define lsize 65535
static char lucky_list [lsize][6];

static void
give_result(int a, int b, int s, iterator_t iterator, int x, int y)
{
	//printf("[%d,%d,%d,%d,%d,%d]\n", a, b, s-a-b, x, y, s-x-y);
	if (count < lsize) {
		lucky_list[count][0] = a;
		lucky_list[count][1] = b;
		lucky_list[count][2] = s-a-b;
		lucky_list[count][3] = x;
		lucky_list[count][4] = y;
		lucky_list[count][5] = s-x-y;
	}
	count++;
}

static void
iterate_triplets(int a, int b, int s, iterator_t iterator, int p, int q)
{
	int lx = s-18;
	int hx = s;
	if (lx < 0) lx = 0;
	if (hx > 9) hx = 9;

	//if (a >= 0)
	//	printf("%d: %d, %d, %d [%d..%d]\n",
	//		s, a, b, s-a-b, lx, hx);

	for (int x = lx; x <= hx; x++) {
		int ly = s-x-9;
		int hy = s-x;
		if (ly < lx) ly = lx;
		if (hy > x) hy = x;

		for (int y = ly; y <= hy; y++) {
			(*iterator)(x, y, s, give_result, a, b);
			if (x != y) {
				(*iterator)(y, x, s, give_result, a, b);
			}
		}
	}
}

int
main(int argc, char *argv[])
{
	for (int s = 0; s<=27; s++) {
		iterate_triplets(-1, -1, s, iterate_triplets, -1, -1);
	}
	printf("%d\n", count);
	return 0;
}

This was so much longer! And harder to understand. But the speed is so high we could not measure it!.

Not that everyhting is rosy - centralized propriatory “social networks” are still far ahead in terms of usability and feature-completeness, but at least there is some progress.

Disclaimer: I am now employed by a company for which data storage and backup are parts of its business, but the ideas in this post are from earlier time, when I worked for a (relatively big) ISP, and are not connected with my current job.

We have been backing up the data from our disks to tapes ever since the disks came to existence (we already had tapes by that time). But with the development of technology during the past decade or two, this approach is becoming increasingly problematic. The main trouble is that the density of data on disk storage devices grows faster than the speed at which this data can be copied to/from tape backup devices. It may be tolerable when backing up some database takes several hours (if you can do that without taking it offline), but if recovering it from the backup, in case of loss of “online” data, takes several hours, it means that the disaster recovery procedure becomes a disaster of its own.

On the other hand, the price of disk storage is falling at a higher rate than that of tape storage, and proposition of buying twice or thrice more disk arrays instead of a tape library looks more and more viable. Disks are still more expensive than tape, but not that much. There are even commercially available “solutions” that do backup on disks instead of on tapes. The problem is, just replacing the medium does not help that much with speeding up the process.

Now, let’s try to decompose the problem that we are solving when we establish backup on our system. There are two kinds of errors that a proper backup can mitigate if they happen: equipment imposed and application imposed. I am not calling the former “hardware errors” because I also include firmware/software errors (RAID implementation, block device drivers, filesystem code) into this class. And I am not calling the latter “human errors” because, well, errors in the filesystem code are human errors too, and I am clumping together a sloppy sysadmin who accidentally typed “rm * .bak” and a sloppy web application programmer who failed to validate input and allowed an sql injection.

For the first class of errors, “equipment errors”, there is much better remedy than tape backup, it’s RAID technology. In fact, most shops run RAIDs anyway, to reduce the impact of long restoration times, and from experience, there are very little cases when you have to fall back to tape backup after equipment failures. All you need to do is choose a “right” RAID solution (not all RAID controllers are born equal), and manage it in a “right way” (i.e. actually notice when there is a failed element, and replace it quickly). I’d hazard to say, if you have a decent RAID storage system, you can stop bothering about equipment failures at all. And it won’t cost you a fortune.

The second class of errors, the “human”/“application” errors, is much worse, because although getting rid of them “almost completely” is possible (I, for one, certainly hope that the procedure for launching a nuclear attack is made relatively error-resistant!), it is not practical for most “normal” businesses to hire twice the staff and establish approval procedures for every keypress. So, there must be a way to go back in time and restore the “most recent known good” state of data. Normally, it means restoring from the last (or some previous, if you are extremely unlucky) generation of backup.

Now, there is a better way to achieve the same goal: it is to have data storage capable for “persistent snapshots”. While (usually non-persistent) snapshots have been used for a long time, on high-end storage systems, in conjunction with regular tape backup, they where not considered as an alternative to tape backup. On the other hand, Apple’s Time Machine is specifically designed as an alternative to traditional backup, remedying for “second class” of errors, but it is not, in typical configuration, backed by anything that could protect from the errors of the “first class”.

So, I see the data storage system for today and tomorrow use as a decent RAID (e.g., RAID1 over TCP/IP via DRBD worked really well for me, and it even allows for geographically distributed configurations), and a filesystem (or volume manager) capable for reliable persistent snapshots on top of it. Throw in a policy manager that would create new and remove old unneeded snapshot generations, and a monitor that would alert the user in case of component failure, and you get a tape-less system that is as reliable as a system with a tape backup, and can recover from disasters much faster.