Google

Running the Quixote Demo

Quixote comes with a tiny demonstration application that you can install and run on your web server. In a few dozen lines of Python and PTL code, it demonstrates most of Quixote's basic capabilities. It's also an easy way to make sure that your Python installation and web server configuration are cooperating so that Quixote applications can work.

Installation

The demo is included in the quixote.demo package, which is installed along with the rest of Quixote when you run python setup.py install. The driver script (demo.cgi) and associated configuration file (demo.conf) are not installed automatically -- you'll have to copy them from the demo/ subdirectory to your web server's CGI directory. Eg., if you happen to use the same web server tree as we do:

cp -p demo/demo.cgi demo/demo.conf /www/cgi-bin

You'll almost certainly need to edit the #! line of demo.cgi to ensure that it points to the correct Python interpreter -- it should be the same interpreter that you used to run setup.py install.

Verifying the installation

Before we try to access the demo via your web server, let's make sure that the quixote and quixote.demo packages are installed on your system:

$ python
Python 2.1.1 (#2, Jul 30 2001, 12:04:51) 
[GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import quixote
>>> quixote.enable_ptl()
>>> import quixote.demo

(Quixote requires Python 2.0 or greater; you might have to name an explicit Python interpreter, eg. /usr/local/bin/python2.1. Make sure that the Python interpreter you use here is the same as you put in the #! line of demo.cgi, and the same that you used to install Quixote.)

If this runs without errors, then Quixote (and its demo) are installed such that you can import them. It remains to be seen if the user that will run the driver script -- usually nobody -- can import them.

Running the demo directly

Assuming that

  • your web server is running on the current host
  • your web server is configured to handle requests to /cgi-bin/demo.cgi by running the demo.cgi script that you just installed (eg. to /www/cgi-bin/demo.cgi)

then you should now be able to run the Quixote demo by directly referring to the demo.cgi script.

Start a web browser and load

http://localhost/cgi-bin/demo.cgi/

You should see a page titled "Quixote Demo" with the headline "Hello, world!". Feel free to poke around; you can't break anything through the demo. (That's not to say you can't break things with Quixote in general; since Quixote gives you the full power of Python for your web applications, you have the power to create stupid security holes.)

If you don't get the "Quixote Demo" page, go look in your web server's error log. Some things that might go wrong:

  • your web server is not configured to run CGI scripts, or it might use a different base URL for them. If you're running Apache, look for something like

    ScriptAlias /cgi-bin/ /www/cgi-bin/

    in your httpd.conf (for some value of "/www/cgi-bin").

    (This is not a problem with Quixote or the Quixote demo; this is a problem with your web server's configuration.)

  • your web server was unable to execute the script. Make sure its permissions are correct:

    chmod 755 /www/cgi-bin/demo.cgi

    (This shouldn't happen if you installed demo.cgi with cp -p as illustrated above.)

  • demo.cgi started, but was unable to import the Quixote modules. In this case, there should be a short Python traceback in your web server's error log ending with a message like

    ImportError: No module named quixote

    Remember, just because you can "import quixote" in a Python interpreter doesn't mean the user that runs CGI scripts (usually "nobody") can. You might have installed Quixote in a non-standard location, in which case you should either install it in the standard location (your Python interpreter's "site-packages" directory) or instruct your web server to set the PYTHONPATH environment variable. Or you might be using the wrong Python interpreter -- check the #! line of demo.cgi.

  • demo.cgi started and imported Quixote, but was unable to read its config file. There should be a short Python traceback in your web server's error log ending with a message like

    IOError: [Errno 2] No such file or directory: 'demo.conf'

    in this case.

    Make sure you copied demo.conf to the same directory as demo.cgi, and make sure it is readable:

    chmod 644 /www/cgi-bin/demo.conf

    (This shouldn't happen if you install demo.conf with cp -p as illustrated above.)

Running the demo indirectly

One of the main tenets of Quixote's design is that, in a web application, the URL is part of the user interface. We consider it undesirable to expose implementation details -- such as "/cgi-bin/demo.cgi" -- to users. That sort of thing should be tucked away out of sight. Depending on your web server, this should be easy to do with a simple tweak to its configuration.

For example, say you want the "/qdemo" URL to be the location of the Quixote demo. If you're using Apache with the rewrite engine loaded and enabled, all you need to do is add this to your httpd.conf:

RewriteRule ^/qdemo(/.*) /www/cgi-bin/demo.cgi$1 [last]

With this rule in effect (don't forget to restart your server!), accesses to "/qdemo/" are the same as accesses to "/cgi-bin/demo.cgi/" -- except they're a lot easier for the user to understand and don't expose implementation details of your application.

Try it out. In your web browser, visit http://localhost/qdemo/.

You should get exactly the same page as you got visiting "/cgi-bin/demo.cgi/" earlier, and all the links should work exactly the same.

You can use any URL prefix you like -- there's nothing special about "/qdemo".

One small but important detail here is "/qdemo" versus "/qdemo/". In the above configuration, requests for "/qdemo" will fail, and requests for "/qdemo/" will succeed. See the "URL rewriting" section of web-server.txt for details and how to fix this.

Understanding the demo

Now that you've gotten the demo to run successfully, let's look under the hood and see how it works. Before we start following links in the demo (don't worry if you already have, you can't hurt anything), make sure you're watching all the relevant log files. As with any web application, log files are essential for debugging Quixote applications.

Assuming that your web server's error log is in /www/log/error_log, and that you haven't changed the DEBUG_LOG and ERROR_LOG settings in demo.conf:

$ tail -f /www/log/error_log & \
  tail -f /tmp/quixote-demo-debug.log & \
  tail -f /tmp/quixote-demo-error.log 

(Note that recent versions of GNU tail let you tail multiple files with the same command. Cool!)

Lesson 1: the top page

Reload the top of the demo, presumably http://localhost/qdemo/. You should see "debug message from the index page" in the debug log file.

Where is this message coming from? To find out, we need to delve into the source code for the demo. Load up demo/__init__.py and let's take a look. In the process, we'll learn how to explore a Quixote application and find the source code that corresponds to a given URL.

First, why are we loading demo/__init__.py? Because that's where some of the names in the "quixote.demo" namespace are defined, and it's where the list of names that may be "exported" by Quixote from this namespace to the web is given. Recall that under Quixote, every URL boils down to a callable Python object -- usually a function or method. The root of this application is a Python package ("quixote.demo"), which is just a special kind of module. But modules aren't callable -- so what does the "/qdemo/" URL boil down to? That's what _q_index() is for -- you can define a special function that is called by default when Quixote resolves a URL to a namespace rather than a callable. That is, "/qdemo/" resolves to the "quixote.demo" package; a package is a namespace, so it can't be called; therefore Quixote looks for a function called _q_index() in that namespace and calls it.

In this case, _q_index() is not defined in demo/__init__.py -- but it is imported there from the quixote.demo.pages module. This is actually a PTL module -- demo/pages.py does not exist, but demo/pages.ptl does. So load it up and take a look:

template _q_index(request):
    print "debug message from the index page"
    """
    <html>
    <head><title>Quixote Demo</title></head>
    <body>
    <h1>Hello, world!</h1>
    [...]
    </body>
    </html>
    """

A-ha! There's the PTL code that generates the "Quixote Demo" page. This _q_index() template is quite simple PTL -- it's mostly an HTML document with a single debug print thrown in to demonstrate Quixote's debug logging facility.

Outcome of lesson 1:

  • a URL maps to either a namespace (package, module, class instance) or a callable (function, method, PTL template)
  • if a URL maps to a namespace, Quixote looks for a callable _q_index() in that namespace and calls it
  • _q_index() doesn't have to be explicitly exported by your namespace; if it exists, it will be used
  • anything your application prints to standard output goes to Quixote's debug log. (If you didn't specify a debug log in your config file, debug messages are discarded.)

Lesson 3: error-handling

The next link in the "Quixote Demo" page is to the "error" document, which is handled by the error() function in demo/__init__.py. All this function does is raise an exception:

def error (request):
    raise ValueError, "this is a Python exception"

Follow the link, and you should see a Python traceback followed by a dump of the CGI environment for this request (along with other request data, such as a list of cookies).

This is extremely useful when developing, testing, and debugging. In a production environment, though, it reveals way too much about your implementation to hapless users who should happen to hit an error, and it also reveals internal details to attackers who might use it to crack your site. (It's just as easy to write an insecure web application with Quixote as with any other tool.)

Thus, Quixote offers the DISPLAY_EXCEPTIONS config variable. This is false by default, but the demo.conf file enables it. To see what happens with DISPLAY_EXCEPTIONS off, edit demo.conf and reload the "error" page. You should see a bland, generic error message that reveals very little about your implementation. (This error page is deliberately very similar, but not identical, to Apache's "Internal Server Error" page.)

Unhandled exceptions raised by application code (aka "application bugs") are only one kind of error you're likely to encounter when developing a Quixote application. The other ones are:

  • driver script crashes or doesn't run (eg. can't import quixote modules, can't load config file). This is covered under "Running the demo directly" above
  • publishing errors, such as a request for "/simpel" that should have been "/simple", or a request for a resource that exists but is denied to the current user. Quixote has a family of exception classes for dealing with these; such exceptions may be raised by Quixote itself or by your application. They are handled by Quixote and turned into HTTP error responses (4xx status code).
  • bugs in Quixote itself; hopefully this won't happen, but you never know. These usually look a lot like problems in the driver script: the script crashes and prints a traceback to stderr, which most likely winds up in your web server's error log. The length of the traceback is generally a clue as to whether there's a problem with your driver script or a bug in Quixote.

Publishing errors result in a 4xx HTTP response code, and are entirely handled by Quixote -- that is, your web server just returns the HTTP response that Quixote prepares.

Application bugs result in a 5xx HTTP response code, and are similarly entirely handled by Quixote. Don't get confused by the fact that Quixote's and Apache's "Internal Server Error" pages are quite similar!

Driver script crashes and Quixote bugs (which are essentially the same thing; the main difference is who to blame) are handled by your web server. (In the first case, Quixote doesn't even enter into it; in the second case, Quixote dies horribly and is no longer in control.) Under Apache, the Python traceback resulting from the crash is written to Apache's error log, and a 5xx response is returned to the client with Apache's "Internal Server Error" error page.

Lesson 4: object publishing

Publishing Python callables on the web -- i.e., translating URLs to Python functions/methods/PTL templates and calling them to determine the HTTP response -- is a very powerful way of writing web applications. However, Quixote has one more trick up its sleeve: object publishing. You can translate arbitrary names to arbitrary objects which are then published on the web, and you can create URLs that call methods on those objects.

This is all accomplished with the _q_getname() function. Every namespace that Quixote encounters may have a _q_getname(), just like it may have a _q_index(). _q_index() is used to handle requests for the empty name -- as we saw in Lesson 1, a request for "/qdemo/", maps to the "quixote.demo" namespace; the empty string after the last slash means that Quixote will call _q_index() in this namespace to handle the request.

_q_getname() is for requests that aren't handled by a Python callable in the namespace. As seen in Lessons 2 and 3, requests for "/qdemo/simple" and "/qdemo/error" are handled by the simple() and error() functions in the "quixote.demo" namespace. What if someone requests "/qdemo/foo"? There's no function foo() in the "quixote.demo" namespace, so normally this would be an error. (Specifically, it would be a publishing error: Quixote would raise TraversalError, which is the error used for non-existent or non-exported names. Another part of Quixote then turns this into an HTTP 404 response.)

However, this particular namespace also defines a _q_getname() function. That means that the application wants a chance to handle unknown names before Quixote gives up entirely. Let's take a look at the implementation of _q_getname():

from quixote.demo.integer_ui import IntegerUI
[...]
def _q_getname(request, component):
    return IntegerUI(request, component)

Pretty simple: we just construct an IntegerUI object and return it. So what is IntegerUI? Take a look in the demo/integer_ui.py file to see; it's just a web interface to integers. (Normally, you would write a wrapper class that provides a web interface to something more interesting than integers. This just demonstrates how simple an object published by Quixote can be.)

So, what is an IntegerUI object? From Quixote's point of view, it's just another namespace to publish: like modules and packages, class instances have attributes, some of which (methods) are callable. In the case of IntegerUI, two of those attributes are _q_exports and _q_index -- every namespace published by Quixote must have an export list, and an index function is almost always advisable.

What this means is that any name that the IntegerUI constructor accepts is a valid name to tack onto the "/qdemo/" URL. Take a look at the IntegerUI constructor; you'll see that it works fine when passed something that can be converted to an integer (eg. "12" or 1.0), and raises Quixote's TraversalError if not. As it happens, Quixote always passes in a string -- URLs are just strings, after all -- so we only have to worry about things like "12" or "foo".

The error case is actually easier to understand, so try to access http://localhost/qdemo/foo/. You should get an error page that complains about an "invalid literal for int()".

Now let's build a real IntegerUI object and see the results. Follow the third link in the "Quixote Demo" page, or just go to http://localhost/qdemo/12/. You should see a web page titled "The Number 12".

This web page is generated by the _q_index() method of IntegerUI: after all, you've selected a namespace (the IntegerUI object corresponding to the number 12) with no explicit callable, so Quixote falls back on the _q_index() attribute of that namespace.

IntegerUI only exports one interesting method, factorial(). You can call this method by following the "factorial" link, or just by accessing http://localhost/qdemo/12/factorial.

Remember how I said the URL is part of the user interface? Here's a great example: edit the current URL to point to a different integer. A fun one to try is 2147483646. If you follow the "next" link, you'll get an OverflowError traceback (unless you're using a 64-bit Python!), because the web page for 2147483647 attempts to generate its own "next" link to the web page for 2147483648 -- but that fails because current versions of Python on 32-bit platforms can't handle regular integers larger than 2147483647.

Now go back to the page for 2147483646 and hit the "factorial" link. Run "top" on the web server. Get yourself a coffee. Await the heat death of the universe. (Actually, your browser will probably timeout first.) This doesn't overflow, because the factorial() function uses Python long integers, which can handle any integer -- they just take a while to get there. However, it illustrates another interesting vulnerability: an attacker could use this to launch a denial-of-service attack on the server running the Quixote demo. (Hey, it's just a demo!)

Rather than fix the DoS vulnerability, I decided to use it to illustrate another Quixote feature: if you write to stderr, the message winds up in the Quixote error log for this application (/tmp/quixote-demo-error.log by default). The IntegerUI.factorial() method uses this to log a warning of an apparent denial-of-service attack:

def factorial (self, request):
    if self.n > 10000:
        sys.stderr.write("warning: possible denial-of-service attack "
                         "(request for factorial(%d))\n" % self.n)
    return "%d! = %d" % (self.n, fact(self.n))

Since the Quixote error log is where application tracebacks are recorded, you should be watching this log file regularly, so you would presumably notice these messages.

In real life, you'd probably just deny such a ludicrous request. You could do this by raising a Quixote publishing error. For example:

def factorial (self, request):
    from quixote.errors import AccessError
    if self.n > 10000:
        raise AccessError("ridiculous request denied")
    return "%d! = %d" % (self.n, fact(self.n))

Lesson 5: widgets

You can't get very far writing web applications without writing forms, and the building blocks of web forms are generally called "form elements": string input, checkboxes, radiobuttons, select lists, and so forth. Quixote provides an abstraction for all of these form elements: the Widget class hierarchy. The widget classes are explained in detail in widget.txt; I'm going to give a brief description of the "Quixote Widget Demo" page and the code behind it here.

If you follow the "widgets" link from the main page, you'll see a fairly ordinary-looking web form -- the sort of thing you might have to fill out to order a pizza on-line, with the oddity that this pizza shop is asking for your eye colour. (Hey, I had to demonstrate radiobuttons somehow!) This form demonstrates most of HTML's basic form capabilities: a simple string, a password, a checkbox, a set of radiobuttons, a single-select list, and a multiple-select list.

Whenever you implement a web form, there are two things you have to worry about: generating the form elements and processing the client-submitted form data. There are as many ways of dividing up this work as there are web programmers. (Possibly more: every time I tackle this problem, I seem to come up with a different way of solving it.) The form in the Quixote widget demo is implemented in three parts, all of them in demo/widgets.ptl:

  • widgets() is the callable that handles the "/qdemo/widgets" URL. This template creates all the widget objects needed for the form and then calls either render_widgets() or process_widgets() as appropriate.
  • render_widgets() is called by widgets() when there is no form data to process, eg. on the first access to the "/qdemo/widgets" URL. It generates the form elements and returns an HTML document consisting of a table that lays them out in an attractive form.
  • process_widgets() is called by widgets() when there is form data to process, ie. when the form generated by render_widgets() is submitted by the user. It processes the form data, ie. it looks up the user-submitted form values and returns an HTML document listing those values.

This division of labour works well with Quixote's widget classes, since you need a collection of Widget objects whether you are generating the form elements or processing form data. For generating the form, we (1) create all the widget objects, and (2) generate an HTML document that includes the output of each widget object's render() method. (Laying out the form is the responsibility of render_widgets(), which is why it's littered with table tags.) For processing the form, we (1) create all the widget objects, and (2) generate an HTML document incorporating the user-submitted form values. In both cases, step (1) is handled by the widgets() template, which then calls either render_widgets() or process_widgets() for step (2).

Thus, there are three things you have to understand about widget objects: how to create them, how to render them, and how to use them to parse form values. Widget creation is the only step that's very interesting, since each widget class has different constructor arguments. For example, here's how we create the "name" widget in the pizza shop form:

widgets['name'] = widget.StringWidget('name', size=20)

When rendered, this widget will produce the following HTML:

<input size="20" name="name" type="text">

A more complex example is the "pizza size" widget:

widgets['size'] = widget.SingleSelectWidget(
    'size', value='medium',
    allowed_values=['tiny', 'small', 'medium', 'large', 'enormous'],
    descriptions=['Tiny (4")', 'Small (6")', 'Medium (10")',
                  'Large (14")', 'Enormous (18")'],
    size=5)

which will generate the following HTML when rendered:

<select size="5" name="size">
  <option value="0">Tiny (4")
  <option value="1">Small (6")
  <option selected value="2">Medium (10")
  <option value="3">Large (14")
  <option value="4">Enormous (18")
</select>

Some things you might need to know about widget creation:

  • the standard widget classes are in the quixote.form.widget module; see widget.txt and/or the source code for the complete list
  • every widget class constructor has exactly one required argument: the widget name. This is used as the form element name in the generated HTML. (Things are a bit different for compound widgets, but I'm not covering them in this document.)
  • every widget class supports a number of keyword arguments that generally correspond to attributes of some HTML tag. The one argument common to all widget classes is value, the current value for this widget.

Rendering widgets is easy: just call the render() method, passing in the current HTTPRequest object. (It's currently not used by the standard widget classes, but could be used by derived or compound widget classes for context-sensitive widget rendering.)

Parsing form values is just as easy: call the parse() method, again passing in the current HTTPRequest object. The return value depends on the nature of the widget, eg.:

  • StringWidget and PasswordWidget return a string
  • CheckboxWidget returns a boolean
  • RadiobuttonsWidget, SingleSelectWidget, and MultipleSelectWidget return one of the values supplied in allowed_values -- in the demo these are all strings, but they can be any Python value. (If the client submits bogus data, the widget will return None.)

$Id: demo.txt,v 1.13 2002/10/01 21:27:31 gward Exp $