Embedthis Appweb 4.3.4
Home >Programmer Reference> ESP Architecture

Quick Nav

ESP Architecture

The ESP Web Framework was designed to make it dramatically easier to create dynamic web applications. The web framework provides an application generator, templating engine, a powerful Model/View/Controller framework and a library of Ajax view controls to take the tedium out of creating web applications. Via this framework, ESP drastically reduces the number of lines of code you need to write for a compelling, dynamic web application.

This document describes the ESP Web Framework and how ESP is embedded in web servers to run web applications and respond to web requests. It describes the flow of execution and the main processing components.

ESP Web Framework Architecture

The Web Framework embeds the ESP language into leading web servers such as Appweb and Apache to provide a hosting environment for web applications. ESP learns from other web frameworks like Rails and PHP, to deliver a web framework that uses the "C" language for server side web programming.

The Web Framework builds upon the ESP core language and provides a web request handler, a Model/View/Controller application framework and an edit-and-continue development model.

ESP Web Framework Architecture

The main components of the ESP Web Framework:

Web Server Interface

The ESP Web Server Interface (or web interface) provides the interface and coupling between ESP and the web server. It accepts incoming web requests from the web server, routes the request to the appropriate application or ejscript web page, processes the output and passes it back to the web server for transmission to the client.

Hosting Alternatives

ESP offers three possible hosting paradigms:

The Common Gateway Interface (CGI) is a standard protocol for hosting applications in a web server. CGI offers good stability but low performance.

FastCGI is a replacement for CGI. It provides higher performance than CGI by supporting multiple Http requests per FastCGI instance. It also offers improved stability by isolating application instances.

Custom in-memory modules offer the highest performance. ESP provides modules for Appweb and Apache. The custom module, mod_ejs, provides excellent performance but less application isolation than FastCGI.

When using a custom in-memory module, each ESP application will typically define a URL alias using the EjsAppAlias directive. For example:

EjsAppAlias /myAppName/ "/Directory/to/myApp"

This instructs the web server to send all requests with the URL prefix "myAppName" to ESP for routing and processing.

Form Data

The web interface converts incoming post and form data into a format suitable for easy processing by ESP applications. Traditionally, form data is passed to web applications by environment variables. But this format is often clumsy, doesn't scale, and does not map well onto an object-oriented language such as Javascript.

ESP replaces environment variables with a params[] collection object and it transparently converts form data into a nested collection of objects. It automatically converts dot separated form names into objects that are addressable by dot notation.

Request State

The web interface will construct request, response, and host objects that store request state. ESP avoids the normal overhead in creating and populating these objects by using virtual, lazy construction. Consequently, ESP applications incur minimal overhead when accessing request state.

Interpreter Hosting

A high performance web server must accept incoming requests, route to the appropriate request handler, initialize the handler including necessary interpreters, gather response data, transmit to the client, and then tear down the handler. And this must be done many times per second, ideally hundreds of times per second for dynamic content.

To do this effectively, ESP pre-creates a master interpreter instance replete with all the requisite system classes and web framework. When incoming requests arrive, the web interface quickly clones this master interpreter to create a light-weight copy that is dedicated to the new request. The clone leverages the system types and web framework in the master, and so needs minimal initialization before it can start servicing the web request.

Clone interpreters are not completely isolated. Controlled session state sharing is accomplished by sharing session[] and application[] objects.

Application Generator

The web framework supports two kinds of ESP web applications.

Stand-alone web pages use embedded ESP code but do not use the full Model View Controller paradigm, nor do they use the application generator. They typically use a "Post-Back" paradigm where form data is posted back to the same page. The do not have controllers, models or view layouts.

MVC applications are more powerful. They have database models, controllers, views and layouts and typically start life by using the application generator. This creates an application directory and populates it with the necessary directories, configuration files and stubs to begin the application.

The application generator is a program named ejsweb. To create a new application called test:

ejsweb -v generate app test

This will create the following directories:

It will also create the following files:

The ejsweb program can also generate controllers. It is currently being enhanced to generate models, views, and entire scaffolds. To compile your application, use the ejsweb program:

ejsweb -v compile

During development time, the web framework uses ejsweb behind the scenes to transparently recompile all, or portions, of the application if the developer modifies any model, view, or controller.

Model, View, Controller Framework

A Model View Controller framework, also known as MVC, is a proven paradigm for organizing web applications. The model manages the state of the application including the database. The controller manages the application, responding to inputs and invoking the relevant views to generate the user interface.

Originally developed in the '70s, it has been more recently adapted for web applications and been popularized by frameworks such as Ruby on Rails. ESP uses the same paradigm with a "C" language and embedded spin.

Models

For object-oriented languages such as ESP, there is sometimes a mismatch between relational database data and the JavaScript objects that are used to represent and manage that data. To solve this problem, ESP provides an Object Relational Mapping (ORM) layer to make it easy to read, modify, and write database data. ORM layers map database tables to classes, rows to object instances, and columns to properties of the classes and objects. If a database has a table named Users, then there would be a model class named User.

When ESP reads records from a database table it constructs an instance of the model class and dynamically creates properties for each of the columns in the table. It also maps the SQL data types into equivalent JavaScript types. This allows a natural access and update paradigm where the application can deal with JavaScript objects while the ORM takes care of reading, querying and updating the database.

The web framework provides a powerful set of access methods to query, join, update, and deal with SQL tables. You can use higher level functions where the ORM generates appropriate SQL statements or you can construct and issue your own SQL.

The ORM Methods include: belongsTo, find, findAll, findOneWhere, findWhere, getColumnNames, getColumnTitles, getDb, getKeyNme, getNumRows, getTableName, hasAndBelongsToMany, hasMany, remove, save, saveUpdate, setKeyName, setTableName, sql and trace.

The ESP ORM currently only provides one database connector for SQLite. It is expected that a future release will add support for other major database engines.

Views

The View part of the ESP web framework is responsible for generating the user interface. It typically takes an input HTML web page with embedded ESP and generates the HTML response to send back to the client. However, views can also be generated by controllers and thus require no HTML page.

The ESP View framework provides:

View Pipeline

The View mechanism consists of a processing pipeline that progressively transforms the web page. Views start with a partial HTML view page which typically contains embedded ESP. This is parsed by the Web Page Parser and combined with layout views to create a composite web page. It is then parsed and converted to pure ESP code which is compiled and cached in a module file (.mod). The module file can be run repeatedly by the VM to generate the required HTML for the user view. View Pipeline

The time-intensive part of this pipeline processing occurs once, at development time. This is comprised of the pipeline stages up to and including compilation. The often less costly run-time processing for each request consists of reading the module file and executing it in the VM.

Web Page Parser

The Web Page Parser is a templating engine and embedded script parser. It understands and processes embedded javascript and layout directives to expand the view content pages and create a composite page that represents an entire web page that the user will see.

A templating engine is an important part of a view framework. It allows you to specify the "look and feel" and standard components of a web application in one place, and then reuse the "look and feel" in content pages by simply referencing a layout page. The layout page typically contains the top level HTML structure, style sheets, and graphic content that is standard on every page. It also specifies the location to insert content from content pages. The content pages focus on the content and data unique to that page. They do not replicate the standard content specified in the master page. In this way, changing the layout page in one place will automatically change every web page in the application.

The web page parser reads the content view pages that contain embedded ejscript code (with .ejs extension). It blends the content page with the layout pages and converts it to a pure ESP program (with .es extension) which represents the composite web page that the user will see.

Here is a simple layout page

<html>
<head>
    <title>@@title</title>
    <link rel="stylesheet" type="text/css" href="@@style"/>
</head>
<body>
    <img src="banner.jpg">
    <%@ content %>
</body>
</html>

The <%@ content %> directive instructs the web page parser to insert the content page at this location. The @@title and @@style directives access ESP variables set to the required content title and style sheet at run-time.

Here is a simple content page:

<h1>Hello World</h1>
<p>Today is <%= new Date %>

The web parser supports the following web page directives. These can be used in layout or content pages.

The <%= ejscript expression %> directive will evaluate the expression and substitute the resulting expression value.

The @@variable directive is a shortcut for <%= expression %>.

The <% ejscript code %> directive will invoke the given code. No automatic substitution of output occurs. However, the code can call "write()" to generate output in the place of the directive.

The <% include "filename" %> directive will include the given file name at the location of the directive when parsing the web page. The Include directives can be nested.

The <%@ layout "file" %> directive specifies the name of the layout page. By using this directive in layout pages, you can build up the web page layout by nesting layout pages. If omitted in content pages, which is usually the case, the default layout is views/layouts/default.ejs. If omitted in layout pages, it is assumed the layout page is the top level layout page.

The web parser supports the following web page directive in layout pages:

This specifies the location for the content page data.

NOTE: you do not have to use layout pages. Simple stand-alone web pages without layouts code are supported. To disable templating, use a <%@ layout="" %> directive.

View Controls

The web framework provides a suite of view controls for common UI elements. These include button, chart, checkbox, form, image, label, link, list, progress bar, radio button, table, tabs, text, textarea and tree. The purpose of the controls is to be a themeable and skinnable set of UI controls that provides a higher level of functionality than bare HTML. For example, the table control allows the easy display of database data with sortable rows and selectable columns.

The view controls are implemented via View connector modules that communicate with the web framework's View connector interface. This is a modular interface for industry Ajax and UI toolkits. The ESP web framework provides a HTML view connector, and a Google Ajax Visualizer connector. Users can select which connector module to use for a specific control on a control by control basis. Additional Ajax connector libraries will be supported in the future.

The controls are bindable to data stored in models and many controls can dynamically refresh their content using Ajax techniques without redisplaying the entire page.

Views can also use a library of view methods to generate and manage HTML output and responses. These include: redirect, render, setCookie, setHeader, write, writeHtml and writeRaw.

Controllers - Responding to Requests

The role of the controllers is to manage the application and respond to inputs so controllers form the heart of the web application. Controllers are typically bound to various URLs so that user input is routed to specific actions within the controls. When a controller receives an incoming Http message, the requested action is run. The action will typically mutate the state of the application in some manner and then render a response view to the user.

Routing

The ESP web framework routes incoming URL requests by tokenizing the request URL. The format used is:

http://site/APP/CONTROLLER/ACTION

Where APP is the name of the application, CONTROLLER the name of the controller and ACTION is the name of an action method within the controller. When such a URL request is received by ESP, it routes the request to the specified controller and invokes the action method. The routing format is currently fixed and not configurable but it is anticipated that user controlled routing will be added in a future release.

Actions

When a controller receives a request to service, it is dispatched to the action method specified in the URL. The action is a simple function defined with an "action" namespace qualifier. The job of the action method is:

  1. Respond to the request
  2. Render a response view back to the client

The action responds to the request by examining the request form parameters, query string, and other Http and application state information.

Here is a sample action method that updates a database table based on user submitted form data in "params".

action function update() {
    /* Update */
    portfolio = Portfolio.find(params.id)
    if (portfolio.saveUpdate(params.portfolio)) {
        inform("Portfolio updated successfully.")
        redirect("list")
    }
}

An action method can explicitly render data by calling one of the render() methods. It can redirect the client to a new URL via the redirect() method. Manual rendering is ideal for "RESTful" web services, particularly when coupled with the XML/E4X capabilities of ESP.

If the action method does not explicitly render any data, the web framework will automatically render a view page of the same name as the action method. That view has access to the controller and request state.

Other methods used by Controller actions include: cache, createSession, destroySession, escapeHtml, html, inform, keepAlive, makeUrl, render, renderFile, renderXml, reportError, setCookie, setHeader, setHttpCode, setMimeType, warn and write.

Development and Deployment

There are two phases of use for ESP: Development and Deployment. When developing, developers need quick turn-around with rapid test-modify-build cycles. Interpreted languages have a distinct advantage over traditional compiled languages when used for such iterative or Agile Web Development. ESP offers the best of both. When developing, ESP runs as an interpreted environment. Changes made to web pages or scripts cause the files to be parsed and compiled without restarting the web server. Simply reloading the web pages in the browser will trigger the changed page to be rebuilt.

When ready for deployment, the entire application can be compiled into a single module file for easy deployment.

© Embedthis Software LLC, 2003-2013. All rights reserved. Embedthis, Appweb, ESP, Ejscript and Embedthis GoAhead are trademarks of Embedthis Software LLC.