Skip to content
January 4, 2013 / jphoward

End to end web app in under an hour–Part 1

Here is the video tutorial that goes with this post:

Here’s how to create a complete web app in under an hour. We will (naturally!) create a todo app. The features:

  • Create, update, delete a todo
  • Sort (and show sort order)
  • Search
  • Pagination

We’ll try to make it somewhat user friendly, e.g.

  • Date-picker widget for todo due date
  • Enable/disable buttons as appropriate for context

Our infrastructure will be:

  • Backend Language: C#
  • Backend Framework: WebApi
  • Database: SQL Server (including SQL Express LocalDB during development)
  • ORM: Entity Framework Code First + Migrations
  • Frontend Framework: AngularJS
  • CSS styles: Bootstrap
  • IDE: Visual Studio + Resharper
  • Hosting: Appharbor

As you’ll see, this particular choice of tools is well suited to rapid application development, and is also very flexible.

The goal is not just to throw together the minimal necessary to have something working, but to create a really flexible infrastructure that we can use as a foundation for many future applications.  OK, let’s get started.

The Backend

In Visual Studio, create a new document and choose to create an MVC web application project.

image

Use the web API template.

image

Web API is a framework which makes it easier to create REST APIs. It is very similar to ASP.net MVC.

Delete HomeController.cs, everything in the Content folder, and everything except Web.config in the Views folder. (These are all for using ASP.Net MVC views and default styles, none of which we’ll need).

By default the API will use XML, but we would prefer JSON (this makes it a little easier to debug), therefore add this to the end of WebApiConfig.Register():

config.Formatters.XmlFormatter.SupportedMediaTypes.Remove(
                config.Formatters.XmlFormatter.SupportedMediaTypes.FirstOrDefault(t => t.MediaType == "application/xml"));

Now, create a new class for your Todo Items:

public class TodoItem {
        public int TodoItemId { get; set; }
        public String Todo { get; set; }
        public byte Priority { get; set; }
        public DateTime? DueDate { get; set; }
    }

And now we’re ready to create our REST API! After compiling, right-click the Solution Explorer, and choose Add->Controller, and create a controller called TodoController.

image

You’ll need to choose the option to create a new data context.

You should now have a working REST API! Press F5 to run your solution, and it should open in a browser. You will get a “not found” error since we don’t have any pages set up yet, so you’ll need to modify the URL path to ‘/api/todo’.

image

Of course, at this stage all we get is an empty array. We need to put some items into our database! First, check out SQL Server Object Browser to see that Visual Studio has already created a DB for us:

image

To add items, we are going to use Entity Framework Migrations. We will go to Tools->Library Package Manager in order to open the Package Manager Console, which is where we can enter commands to work with Entity Framework. In the console, type “Enable-Migrations”.

image

This has created a file for me called Configuration.cs, which allows me to specify data to seed my DB with. Let’s edit that now to seed some data.

protected override void Seed(AmazingTodoContext context) {
    var r = new Random();
    var items = Enumerable.Range(1, 50).Select(o => new TodoItem {
        DueDate = new DateTime(2012, r.Next(1, 12), r.Next(1, 28)),
        Priority = (byte)r.Next(10),
        Todo = o.ToString()
    }).ToArray();
    context.TodoItems.AddOrUpdate(item => new { item.Todo }, items);
}

Any time I change my model or edit Seed(), I’ll need to run Update-Database in Package Manager Console to have the DB show my changes to the code.

image

Now I’ll refresh my browser to see the data via the REST API:

image

The Basic AngularJS setup

Now that the API is working, we can create a simple page to show a list of todos. We will use AngularJS as our Javascript MVC framework, so let’s install that: simply type “Install-Package angularjs” at the package manager console. We’ll be using Bootstrap to style things up, so install that too: “Install-Package Twitter.Bootstrap”. In the root of your project, create index.html, with references to the css and js files we’ll be using.

<!DOCTYPE html>
<html ng-app="TodoApp" xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <script src="Scripts/jquery-1.7.1.js"></script>
        <script src="Scripts/angular.js"></script>
        <script src="Scripts/angular-resource.js"></script>
        <link rel="stylesheet" type="text/css" href="Content/bootstrap.css" />
        <title>Amazing Todo</title>
    </head>
    <body>
        <div class="container">
            <div ng-view></div>
        </div>
    </body>
</html>

You will need to change the ng-app attribute in the html element, and the title, for each of your projects. Other than that, your index.html will be the same for most of your projects (other that having some different js and css links, of course). All the actual work will occur in AngularJS templates. In order for AngularJS To know what template and controller to use we need to set up some routes. To do this, we use the config method of the AngularJS module class. Let’s create a new JavaScript file for our AngularJS code; the convention is to call this app.js.

var TodoApp = angular.module("TodoApp", ["ngResource"]).
    config(function($routeProvider) {
        $routeProvider.
            when('/', { controller: ListCtrl, templateUrl: 'list.html' }).
            otherwise({ redirectTo: '/' });
    });

This creates a route for ‘/’ that uses a controller called ListCtrl and a template called list.html.

Let’s create an empty controller in app.js.

var ListCtrl = function($scope, $location) {
    $scope.test = "testing";
};

Let’s create a basic list.html as well.

<h1>Test {{test}}</h1>

Any properties and methods of $scope are made available automatically in the template. In the template, use ‘handlebars’ (double braces) to indicate where AngularJS expressions should be placed. Don’t forget to add a script element to index.html pointing at your app.js. Now try going to http://localhost:5127/index.html in your browser (you’ll need to change the port of course). If it’s working, you’ll see “Test testing”.

The List Page

Let’s edit our list.html to do something more useful!

<table class="table table-striped table-condensed table-hover">
    <thead>
        <th>Todo</th>
        <th>Priority</th>
        <th>Due</th>
    </thead>
    <tbody>
        <tr ng-repeat="item in items">
            <td>{{item.Todo}}</td>
            <td>{{item.Priority}}</td>
            <td>{{item.DueDate}}</td>
        </tr>
    </tbody>
</table>

For this to do anything, we will need $scope.items to contain our todos. We can define in app.js an AngularJS $resource that we can use to query our REST API.

TodoApp.factory('Todo', function($resource) {
    return $resource('/api/todo/:id', { id: '@id' }, { update: { method: 'PUT' } });
});

var ListCtrl = function ($scope, $location, Todo) {
    $scope.items = Todo.query();
};

$resource is a function provided by AngularJS that creates an object for accessing a REST API. We use a factory method so that we can reuse the object without it getting recreated every time. Note that we also had to add a parameter to ListCtrl in order to have access to this object.

At this point, you should be able to view the list in your browser.

image

It’s interesting to note that all the html files are entirely static – the only thing that’s dynamic is the JSON sent by the web API.

That’s the end of Part 1 of this walkthrough. In the next part, we’ll add sorting, searching, and pagination to the list.

Continued in Part 2.

Advertisements
November 8, 2011 / jphoward

$40/month to send my email–are you serious?

The Kaggle web site needs to send emails from time to time – for example when confirming new users’ email addresses. Sending directly from a web server is not generally a good idea; even if you’ve taken the steps to set up DKIM, senderid, etc, you still have the problem that your IP isn’t a reputed email sender. It’s also likely that at some point some bad apple on your IP block will ruin the reputation for all their neighbors.

So instead, I decided to use a mail sending service. Here’s some examples, along with their pricing for their cheapest account:

Wo. That seems like a lot. Is it really so hard to send mail? Actually, no. Here’s a crazy option:

  • FastMail: $3/month (max 2,000 messages per hour –That’s up to 1,500,000 per month!)

And BTW that FastMail option also gives you a bunch of other stuff you may find useful (e.g. host and manage 50 domains, 2GB file storage, 10GB IMAP storage…)

I founded FastMail back in 1999 (I sold it to Opera a couple of years back and don’t work there any more) and worked hard to make the infrastructure efficient. However I’m surprised that folks building much more focussed tools today aren’t able to do it much cheaper. I know the focussed tools I’ve listed have some extra features (such as an HTTP API), but I don’t see why that should increase the unit price substantially.

What do you think is the reason for this difference?

February 21, 2011 / jphoward

Kaggle’s Nick Gruen on ABC Radio

I enjoyed listening to Kaggle’s chairman, Nick Gruen, interview by James O’Loughlin on ABC radio. Here’s the full interview, in which Nick discusses the power of predictive modelling, the amazing results achieved by data mining competitions, and the upcoming $3m Heritage Health Prize.

Click the triangle below to listen.

October 18, 2010 / jphoward

Visualising Time Series

Over at Kaggle there’s an interesting competition involving time series prediction. Since I’ve never done much with time series before, I figured I’d give it a go. It’s a good chance to learn something new, and have some fun in the process.

I decided to try a new (for me) approach to the analysis, which is to use general purpose programming tools for all the data analysis, including import/export, visualization, modelling, etc. My hypothesis was that with powerful languages which strong functional capabilities, I would be able to achieve results just as quickly as using a dedicated tool (like R), plus have the benefits of a “proper” programming language (e.g. strong language design, excellent IDE, speed, etc).

My first approach was to use Javascript to chart the 400-odd time series in each category (quarterly, and monthly). It turns out that it’s only about 10 lines of code, plus a cut-and-paste of a function from Google Charts docs:

var content = $("#content");
for (var i = 0; i < qdata.length; i++) {
    var url = chartUrl(qdata[i], i);
    content.append('' + '<img src="' + url + '" /> ');
}

function chartUrl(data, i) {
    var res = "http://chart.apis.google.com/chart?chs=440x220&cht=lc&chtt=" + i +    "&chd=";
    var maxval = Math.max.apply(Math, data);
    return res + extendedEncode(data, maxval);
}

The result is this page, which is a fast and easy way to see all the time series at once (click one of the buttons on that page to see the data). If you’re interested in seeing how it works, feel free to look at the JavaScript linked from that page.

Next, I moved to C#, and found that the functional capabilities added in .Net 3.5 (LINQ et al), and the automatic parallelization added in .Net 4, made it a real pleasure to work with. I also used GlowCode to profile my algorithms as I went, which made it easy to keep them running fast. I used the free Microsoft Chart components, plus a FlowLayoutPanel, to easily generate visualizations. For example, here’s a (subset of a) visualization showing in-sample predictions (blue) vs actual data (orange):

TsMetrics

(click image to view full size)

In this example, it’s easy to see some models that aren’t ideal: series 2 shows that the underlying trend is not matching closely enough in this instance, and series 5 shows the problem of using additive seasonality in appropriately. You can see that adding the series number and a fitness metric to each chart makes it easier to work with.

Here’s a visualization showing out-of-sample predictions for a different model:

TsPredict

(click image to view full size)

In this case we can confirm visually that the models have reasonable-looking predictions.

August 10, 2008 / jphoward

Learning from Dan Bernstein

Dan Bernstein (DJB) is someone who you’ve probably never heard of – yet his work has covered an astonishing amount of territory and benefited everyone who today uses the Internet or e-commerce. His achievements include forcing the US government to allow import/export of cryptography, working out how to protect the Internet from debilitating attacks, writing what may be the world’s most secure email server (used on over 700,000 servers), and discovering faster algorithms in key areas of mathematics.

Unfortunately, despite all this, most of the world decided to ignore him when he claimed 8 years ago that DNS can be forged. In order to counteract these problems he even wrote his own DNS server (djbdns). Most people didn’t use it, instead using the far less secure BIND server. (I don’t know why – I’ve been using djbdns for 8 years and it’s faster, easier to use, and more reliable than BIND.)

So, to those who know the history, it wasn’t so surprising to learn that the huge security hole that hit most of the Internet last month  would have been nearly entirely avoided if only they had used DJB’s DNS software.

I’ve been going back over DJB’s writings on the topic of DNS to see what else he’s been trying to tell us. One of the pages that came up (from 1999!) was his description of how even SSL-secured web sites are vulnerable to DNS attacks. SSL "security" has not improved at all since then – it’s still incredibly easy to create an effective phishing site.

This all goes to show that, if there’s anything we can learn from history, it’s that we very rarely learn from history…

July 18, 2008 / jphoward

Scheduled tasks using ASP.Net

When using a shared hosting service, there’s generally no way to create scheduled tasks. This can be a big problem, since it provides no way to run scheduled maintenance tasks, automated email systems, and so forth.

A very clever workaround which uses nothing but ASP.Net, and will work on any shared hosting service, is described in this Code Project article. The basic trick is to create an expiring cache object when the application first starts, and to then attach to the event that fires each time the object expires which creates a new expiring cache object.

July 10, 2008 / jphoward

Styling Forms with CSS

Here’s a couple of nice approaches to styling HTML forms using CSS, to automate layout and formatting of forms: