Projections tutorial

Follow the steps below to learn how to create a user defined projection.

Add sample data

Download the following files that contain sample data used throughout this step of the getting started guide.

• shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1164.json
• shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1165.json
• shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1166.json
• shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1167.json

Add the sample data to four different streams:

curl -i -d "@shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1164.json" -u "admin:changeit" "http://127.0.0.1:2113/streams/shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1164" -H "Content-Type:application/vnd.kurrent.events+json"

curl -i -d "@shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1165.json" -u "admin:changeit" "http://127.0.0.1:2113/streams/shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1165" -H "Content-Type:application/vnd.kurrent.events+json"

curl -i -d "@shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1166.json" -u "admin:changeit" "http://127.0.0.1:2113/streams/shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1166" -H "Content-Type:application/vnd.kurrent.events+json"

curl -i -d "@shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1167.json" -u "admin:changeit" "http://127.0.0.1:2113/streams/shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1167" -H "Content-Type:application/vnd.kurrent.events+json"

Create your first projection

The projection counts the number of 'XBox One S's that customers added to their shopping carts.

A projection starts with a selector, in this case fromAll(). Another possibility is fromCategory({category}) which this step discusses later, but for now, fromAll should do.

The second part of a projection is a set of filters. There is a special filter called $init that sets up an initial state. You want to start a counter from 0 and each time KurrentDB observes an ItemAdded event for an 'Xbox One S,' increment the counter.

Here is the projection code:

fromAll()
.when({
    $init: function(){
        return {
            count: 0
        }
    },
    ItemAdded: function(s,e){
        if(e.body.Description.indexOf("Xbox One S") >= 0){
            s.count += 1;
        }
    }
})

You create a projection by calling the projection API and providing it with the definition of the projection. Here you decide how to run the projection, declaring that you want the projection to start from the beginning and keep running. You can create a projection using the Admin UI by opening the Projections tab, clicking the New Projection button and filling in the details of your projection.

You can also create projections programmatically. Pass the projection JSON file as a parameter of your request, along with any other settings:

curl -i --data-binary "@xbox-one-s-counter.js" \
    http://localhost:2113/projections/continuous?name=xbox-one-s-counter%26type=js%26enabled=true%26emit=true%26trackemittedstreams=true \
    -u admin:changeit

Query for the projection state

Now the projection is running, you can query the state of the projection. As this projection has a single state, query it with the following request:

curl -i http://localhost:2113/projection/xbox-one-s-counter/state -u "admin:changeit"

The server will send a response similar to this:

{
    "count": 3
}

Append to streams from projections

The above gives you the correct result but requires you to poll for the state of a projection. What if you wanted KurrentDB to notify you about state updates via subscriptions?

Output state

Update the projection to output the state to a stream by calling the outputState() method on the projection which by default produces a $projections-{projection-name}-result stream.

Below is the updated projection:

fromAll()
.when({
    $init: function(){
        return {
            count: 0
        }
    },
    ItemAdded: function(s,e){
        if(e.body.Description.indexOf("Xbox One S") >= 0){
            s.count += 1;
        }
    }
}).outputState()

To update the projection, edit the projection definition in the Admin UI, or issue the following request:

curl -i -X PUT --data-binary @"xbox-one-s-counter-outputState.js" http://localhost:2113/projection/xbox-one-s-counter/query?emit=yes -u admin:changeit

Then reset the projection you created above:

curl -i -X POST "http://localhost:2113/projection/xbox-one-s-counter/command/reset" -H "accept:application/json" -H "Content-Length:0" -u admin:changeit

You should get a response similar to the one below:

{
    "msgTypeId": 293,
    "name": "xbox-one-s-counter"
}

You can now read the events in the result stream by issuing a read request.

curl -i "http://localhost:2113/streams/\$projections-xbox-one-s-counter-result?embed=body" -H "accept:application/json" -u admin:changeit | grep data

And you'll get a response like this:

{
    "msgTypeId": 293,
    "name": "xbox-one-s-counter"
}

The number of items per shopping cart

The example in this step so far relied on a global state for the projection, but what if you wanted a count of the number of items in the shopping cart per shopping cart.

KurrentDB has a built-in $by_category projection that lets you select events from a particular list of streams. Enable this projection with the following command.

curl -i -d{} http://localhost:2113/projection/%24by_category/command/enable -u admin:changeit

The projection links events from existing streams to new streams by splitting the stream name by a separator. You can configure the projection to specify the position of where to split the stream id and provide a separator.

By default, the category splits the stream id by a dash. The category is the first string.

You want to define a projection that produces a count per stream for a category, but the state needs to be per stream. To do so, use $by_category and its fromCategory API method.

Below is the projection:

fromCategory('shoppingCart')
.foreachStream()
.when({
    $init: function(){
        return {
            count: 0
        }
    },
    ItemAdded: function(s,e){
        s.count += 1;
    }
})

Create the projection with the following request:

curl -i --data-binary "@shopping-cart-counter.js" http://localhost:2113/projections/continuous?name=shopping-cart-item-counter%26type=js%26enabled=true%26emit=true%26trackemittedstreams=true -u admin:changeit

Querying for the state of the projection by partition

Querying for the state of the projection is different due to the partitioning of the projection. You have to specify the partition and the name of the stream.

curl -i http://localhost:2113/projection/shopping-cart-item-counter/state?partition=shoppingCart-b989fe21-9469-4017-8d71-9820b8dd1164 -u "admin:changeit"

The server then returns the state for the partition:

{
    "count": 2
}

Configure projection properties

You can configure properties of the projection by updating values of the options object. For example, the following projection changes the name of the results stream:

options({
  resultStreamName: "xboxes"
})
fromAll()
.when({
    $init: function(){
        return {
            count: 0
        }
    },
    ItemAdded: function(s,e){
        if(e.body.Description.indexOf("Xbox One S") >= 0){
            s.count += 1;
        }
    }
}).outputState()

Then send the update to the projection:

curl -i -X PUT -d "@update-projection-options.js" http://localhost:2113/projection/xbox-one-s-counter/query?emit=yes -u admin:changeit

Now you can read the result as above, but use the new stream name:

curl -i "http://localhost:2113/streams/xboxes?embed=body" -H "accept:application/json" -u admin:changeit | grep data

Query running projections

You can also query the state of all projections using the HTTP API.

curl -i http://localhost:2113/projections/any \
    -H "accept:application/json" -u "admin:changeit" \
     | grep -E 'effectiveName|status'

The response is a list of all known projections and useful information about them.

"effectiveName": "$streams",
"status": "Running",
"statusUrl": "http://localhost:2113/projection/$streams",
"effectiveName": "$stream_by_category",
"status": "Running",
"statusUrl": "http://localhost:2113/projection/$stream_by_category",
"effectiveName": "$by_category",
"status": "Running",
"statusUrl": "http://localhost:2113/projection/$by_category",
"effectiveName": "$by_event_type",
"status": "Running",
"statusUrl": "http://localhost:2113/projection/$by_event_type",
"effectiveName": "$by_correlation_id",
"status": "Running",
"statusUrl": "http://localhost:2113/projection/$by_correlation_id",