Django 1.4.5 in Wheezy

Django 1.4.5 is the version that comes from the debian packages for Wheezy. Following the standard tutorial (version 1.6) does not contain the complete set of instructions to set up the Admin site. To do this you will need to follow the set of instructions for 1.4.x, which contains additional instructions to edit urls.py and also set up a site id in the django database.

Hello World Plugin for Spyder 2

The following is a set of instructions for writing a simple Hello World plugin for Spyder. Hopefully it will serve as a template for future plugins.
Note that this is for Spyder 2, which is not available on squeeze, but is available on wheezy. Be very careful if you have to upgrade your distribution. Alternatively, you can try to use the latest source code, which is what I ended up doing.
Note that some files also include directory structure; most files referred to here are all relative to the <source path>/spyderlib/spyderplugins directory.

Specification

The plugin will provide a plain window (that can be docked and resized as per the other windows), simply with a button and a label. The label will be blank until the button is pressed.

The plugin architecture will consist of three files helloengine, hellogui and p_hello, corresponding roughly to model, gui and plugin (interface to spyder).

Brief Walk-through

After Spyder has loaded its core components such Sift, Project Explorer, Variable Explorer, Object Inspector, Find in Files, etc,by the function setup in the MainWindow class (spyder.py), it then creates a list of third-party plugins

# this is an annotated extract from Mainwindow.setup()

# Third-party plugins
for mod in get_spyderplugins_mods(prefix='p_', extension='.py'):
    # get list of the mods in the plugins directory, 
    # which is found in the otherplugins.py file
    # each mod variable is a module, for example, spyderplugins.p_pylint
    try:
        # the module spyderplugins.p_pylint has a special variable, PLUGIN_CLASS
        # which is literally a reference the class you want to use as the Plugin
        plugin = mod.PLUGIN_CLASS(self)
        # note that plugin variable, which gets updated every time through the loop is 
        # is an initialisation of the 


        self.thirdparty_plugins.append(plugin)
        plugin.register_plugin()
    except AttributeError, error:
        print >>STDERR, "%s: %s" % (mod, str(error))

Setup

First of all I don't want to mess with the standard Debian settings, so I'll be getting the recommended packages for Spyder  and setting up a play directory for Spyder in my home directory.
Using aptitude:

• Initially install spyder, include all the recommended packages.
• Uninstall spyder and spyderlib. Aptitude may make a list of other packages that are going to be uninstalled (docutils, pygments, pep8, etc) as they aren't used by any other packages (I have very bare bones system). Manually mark them for installation, and then get the packages - which should uninstall spyder and spyderlib, but leave all the other packages.

I should now have mercurial installed on my system because it was one of the recommended packages for Spyder.

Make a new directory to play with and clone the repository:

mkdir ~/usr/spyder2
hg clone https://code.google.com/p/spyderlib/ 

Hello Plugin for Spyder

And now, we get to the walk-through for the Hello Plugin for Spyder. Spyder, as noted above, loops through all the plugins in the spyder plugin directory, and tries loading them.

Spyder Loading our Plugin

In the Hello plugin case, it will try:

1. loading the module spyderplugins.p_hello
2. finding the class name to be used as the plugin in the module variable PLUGIN_CLASS, which will be class HelloPlugin.
3. Initializing the HelloPlugin (__init__(self, Parent))class with the Spyder MainWindow class being the parent.

p_hello.py

This file provides the final class definition that is used by Spyder for a plugin. Our plugin class (HelloPlugin) has the following variable and methods:

• CONF_SECTION (= 'hello'): Not sure where this is used.
• __init__(self, parent=None): the initialization method for the class. Our example calls a secondary initialisation method, but I don't think that this is strictly necessary.
• get_plugin_title(self): <SpyderPluginWidget API> Only basic functionality defined.
• get_plugin_icon(self): <SpyderPluginWidget API> Only basic functionality defined
• get_focus_widget(self):<SpyderPluginWidget API> Only basic functionality defined
• get_plugin_actions(self): <SpyderPluginWidget API> Only basic functionality defined
• register_plugin(self): <SpyderPluginWidget API> I did not change any of the code in this method, except to change all the relevant parts to "hello". Note also, that one of registered actions is the method run_hello.
• refresh_plugin(self): <SpyderPluginWidget API> Only basic functionality defined
• closing_plugin(self, cancelable=False): <SpyderPluginWidget API> Only basic functionality defined
• apply_plugin_settings(self, options): <SpyderPluginWidget API> Only basic functionality defined
• initialize_plugin(self): <Public API> This method did not actually do anything, but does provide a hook to separate some of the initialisation code from the __init__method.
• run_hello(self):  <Public API> This is the method that actually prints out the greetings. It calls the  method say_greetings()  inherited from HelloWidget.

widgets/hellogui.py

This file will provide the user interface description that is used by Spyder. By separating out the user interface from the main work module we create the option of obtaining workings of the hello module from other user interfaces.The main part of this file is the class HelloWidget which contains:

• __init__(self, parent): basic initialisation of the inherited classes and creates the layout of GUI 
• say_greetings(self): the triggered function for the button.

widgets/helloengine.py

The first file (model) simply provides the backend of the plugin, where all the work is actually done, (in this case providing the string "Hello"). In a useful plugin it might provide a database access, API connection to web resource or some kind of data transformation.

 # This is the entire working part of the file.
class Hello():
    def __init__(self):
        self.greetings = "hello"

The greetings variable is passed into the HelloWidget as the text to display.

---

---

Summary Checks

The following is a list of checks required to ensure that a plugin will be correctly implemented:

• Spyder looks for plugins in the spyderplugins directory with filenames that start with "p_" and end with ".py". Ensure that the plugin that interfaces with Spyder meets these requirements.
• In your plugin file, there is a variable PLUGIN_CLASS at the module level. This variable is the class that you wish to use as the plugin. I haven't tried it, but I assume that you could wrap that in a conditional, so that different plugin could be loaded depending on the conditional.
• The plugin class __init__ function takes exactly two arguments: self and Parent. The parent is used by Qt to keep track of the widget hierarchy, i.e. the plugin becomes a widget of Spyder. Additional arguments could theoretically be used when initializing the plugin class, however, Spyder won't call them during the plugin plugging in.

Passing Functions in D

So, the problem is to create a function (foo) that:

• takes two values x₁ and x₂.
• returns a a function that 
• takes as argument a single number
• return the  interpolated value between the two intial values

For example:
x₁ = 10
x₂ = 15
f(x₁, x₂) = (x₂ - x₁)*t + x₁ = F(t)
F(0.5) = 12.5

---

import std.stdio;

auto interp_proc_maker(float x1, float x2){
    float interp(float s) {
        return (x2-x1)*s+x1;}
    return  &interp;}

void main()
{
  auto f1 = interp_proc_maker(10, 15);
  writefln("f1(0.5) = %f", f1(0.5));
}

---

$ gdc example.d
$ ./a.out
f1(0.5) = 12.500000

Call Graph in Xfoil

Use the following procedure to visualise the logic through xfoil:

1. Make sure you have the correct dependencies
1. gprof (or one of the alternative profilers installed).
2. Make sure you have graphviz 
3. Download gprof2dot.py
4. Make sure that you can compile xfoil properly
2. Modify the flags in the Makefile (${srcdir}/Xfoil/bin/Makefile) to allow the profiling of the xfoil code.
1. Add the -pg option to the FFLAGS and FFLOPT makefile variables.
2. Add the -pg option to the make xfoil command line ("    xfoil: $(XFOILOBJ) $(XUTILOBJ)... $(FC) -o xfoil $(XFOILOBJ) .... -pg"
3. Note it may be necessary to do a "make clean" to be able to recompile xfoil.
3. Now create a script file that touches all the xfoil commands that your interested in. 
1. Example below does an viscous, (slightly) compressible analysis analysis of a NACA 23012 airfoil section at 10 degrees AoA.
4. Now run the script.
1. ~/bin/xfoil < naca23012.xfoilscript
2. Note that I have a bin directory in my home folder (~) that has a symbolic link to the particular xfoil program I want to run. Hence ~/bin/xfoil command.
5. If you've compiled xfoil correctly you'll get profiler file (gmon.out for gprof) file in the same directory that you ran the xfoil script in.
6. Now run the gprof2dot script to generate a dot file.
1. gprof ~/bin/xfoil | ./gprof2dot.py -n0 -e0 | dot -Tpng -o xfoil.png
7. Enjoy the call graph

Xfoil call graph for the example script

---

Xfoil Script

$ cat naca23012.xfoilscript
plop
g


naca
23012

oper
v
3e5
m
0.3
a
10


quit

Designing a System for Technical Skills Progression

Found in drafts file, Started Jan-2011

First some background about me: I work two part-time jobs, Mondays and Tuesdays I'm an engineer, Wednesday through to Friday I'm a draftee, although at the moment I'm doing some excel/multiframe programming/engineering analysis design.

The situation at my engineering company is this: foreign company required by government contract to provide certain percentage of work in country. That's us. Effectively our client is the government, and we represent the foreign company. Most of our (maintenance, very important point) work uses a streamlined process, where there is the minimum legal amount of oversight on the work that we provide in order to improve the overall efficiency / effectiveness/ turn-around time; and that is the work that I have been mostly doing for the past year or so. However, recently I was tasked with a modification instead of maintenance.

My manager decided to step with me through how our company handles on-going multi-week projects (most of my projects are expected to be completed within a week). I estimated about 60 hours to complete (about 4 weeks). He laughed. After going through the project times with admin, checking, writing, drafting, etc. I came up with 144 hours (about 10 weeks) which includes a 10% risk (I like to think of this as a training allowance). Now compared with my previous 15 hour project turn-around time I felt this was hideously over budgeted. As it was, the project ended up around the 135 hour mark, smack bang in the middle of the risk margin.

So how was I so very wrong in my estimation for the budgeting?

My first 'proper' job was doing similar kind of engineering but for the commercial sector. The turn-around times and budgets are much tighter than for government work. Also, access to OEM data is restricted. As a result, a higher level of engineering risk is accepted and use of engineering judgement was favoured (and often was the only engineering justification). My boss at the time was an engineer of about 35 years experience, running his own business (there were only two of us; I think the peak employment for his company was four employees). On occasion, I would not be able to understand why a particular solution was acceptable, to which he would reply that "based on my experience and knowledge - it will be okay".

This annoyed me.

I rebelled by making forms. And checklists. And cheat sheets. I even started on a manual. Of course I was trying to systematise the process for the engineering we were doing; trying to capture the skills and abilities of my boss, by without having to go through the time taken to gain that experience. This strategy is not uncommon. Although there was resistance to this, some forms later proved useful in the form of third-party audits. One thing I was not able to solve was the "don't know what you don't know" problem.

As an aside, it seems to me that the solution the pro-systems camp adopt is one of oversight, review and approval. However, at a proto-level, oversight, review and approval is still being performed by people; and so you are still vulnerable to the "you don't know what you don't know" problem. The systems' solution is, of course, to add oversight, review and approval to the people doing the overseeing, reviewing and approving...

Adding these meta-levels of process to a problem at the proto-level can only ever dilute the problem, not eliminate it.

The authority (e.g. Civil Aviation Safety Autority) has the problem of how much authority to delegate, to little and the delegation has no purpose. Too much and delegatee is vulnerable to the Dunning-Kruger effect. Myself, I go for the personal education and training as a defense to this.

Now my draftee work has had problems with their quality. Jobs have gone out that are straight out unfixable. This has meant massive back charge to us. Recently, our manager was promoted across the country, and our final checker was promoted into his position. What this has meant is that the new final checkers were now checking work for which they were not skilled enough to do. In addition the only training is on-the-job training. The person most recently able to be productive is tasked with the training new hires (an effort to solidify the training of the productive person). Any problems or questions are forwarded to one of the more senior draftees for resolution. What this has bought about is draftees who have been taught to obey the instructions of the engineers and engineers who have not been trained to an adequate level.

Another symptom is the sudden push at the end of every month to get jobs out. Ideally, there should be no push for jobs out as the budget should be comfortably passed near the end of the month with no change of pace. But there is: so what does that mean?

We now have three situations in mind. The first is my preference to for systems when first starting out. The second is a poor estimate of project time budget. And the third is the significant drop in quality of the drafting work. To provide a paradigm on which to analyse this I adopt the Dreyfus Model of Skills Aquisition. This model sets out 5 levels at which a person's skill can be categorised. Quoting Wikipedia:

1. Novice:
   • "rigid adherence to taught rules or plans"
   • no exercise of "discretionary judgment"
2. Advanced beginner
   • limited "situational perception"
   • all aspects of work treated separately with equal importance
3. Competent
   • "coping with crowdedness" (multiple activities, accumulation of information)
   • some perception of actions in relation to goals
   • deliberate planning
   • formulates routines
4. Proficient
   • holistic view of situation
   • prioritizes importance of aspects
   • "perceives deviations from the normal pattern"
   • employs maxims for guidance, with meanings that adapt to the situation at hand
5. Expert
   • transcends reliance on rules, guidelines, and maxims
   • "intuitive grasp of situations based on deep, tacit understanding"
   • has "vision of what is possible"
   • uses "analytical approaches" in new situations or in case of problem

Now when I first started, by predilection for systems was for precipitated by being a novice in an environment of an expert. In retrospect, this seems reasonable. Effectively, the forms and checklists are being used as training wheels.

In the second situation, I feel I am about the level of advanced beginner / competent. For maintenance projects. What happened was that I based initial estimate.

So what does all this have to with the title? I propose that the system for technical jobs be flexible enough to accommodate the people who are working on it. In the specific example of the my modification project, if I had recognised that my level of skills was closer to novice than to competent, I spend some time with the manager to create some forms and checklists to go through job.

On checklists: The effectiveness of a checklist is equal to the comprehensiveness of the checklist multiplied by the probability that it is used. There is a tendency at my drafting job to fix problems with jobs by adding more items to the checklist. What this results in is a checklist that is 80% not applicable. However, hidden in amongst the "n/a" is a checklist item required that gets overlooked. Or you 'check' the same way that you do create, in which case a mistake is 'checked' OK. You can't check your own work, especially if you've just finished. I'm always surprised at the number of mistakes that sneak in overnight.

TLDR; Customise checklists based on the job and the person doing it.

ProgEx: Python Ex1 rev 1 comments

1. The code won't run from the command line under linux, nor does it run from the Python command line. Feature request?
2. I made a change to how the sqrt function works (it was even more 'hacky'). Actually the whole check for negative numbers the using the complex math library seems a bit clunky. Options? Major sin: I didn't check to see if it works. I'll post it anyway.*
3. On that note, I have a feeling that the sqrt function only returns the first value. Fails functional spec. D'oh.A comment on the spec: three of the functions expect a single return value, however the fourth (sqrt), expects a list to be returned. Something that should have considered in the 'architecture'.
4. Should the doit() have been called main(), and the output in that routine, instead of printing the result of doit()?
5. The shebang line to make it run on Linux was expected. It was a bit of easter egg to try and get people to check further...
6. I hadn't got up to the chapter on list comprehensions, or I've skipped it.
7. sys.exit instead of exit. That appears to be a bit of a trap.
8. It should have unit testing in the module specified. 
9. Docstrings are absent.

ProgEx: Python Ex rev 1 Code

# Number functions

# A program written for Exercise 1 of 15 programming exercises.
# As specified in knowing.net
# The aim of this exercise is to familiarise with basic code constructs,
# basic operators, standard libraries, simple maths, arrays, functional
# programming concepts (map and apply?), parsing input, error handling, etc.
#
# The aim of this module is to create a simple program that will
# take as its first argument one of the following arguments:
# sum, product, mean, sqrt,
# The remainder of the arguments will be a list numbers that the
# functions apply to.
# This module is inteded to be called from shell of the OS., eg
#
# c:\python\exercises\numberfun.py sum 2 3 4 5
#  14
#

# Extensions and maintenance guide:
# To add extra functions:
#   - Define the function in the Function Definitions
#   - Add the function to the function table with its 'call'
#   - Implement any exceptional behaviour in the main program and
#         document it.

from __future__ import division
import math
import cmath
import sys

# The module is divided into the following sections:
# 1) Function Definitions
# 2) Function Lookup table
# 3) Main program definition
# 4) The command to run

### Function Definitions
## sum
#def sum(lst):
#    return reduce ((lambda x,y: x+y), lst, 0)
#
# sum is an inbuit function

## product
def product(lst):
    return reduce ((lambda x,y: x*y), lst, 1)

## mean
def mean(lst):
    return (reduce ((lambda x,y: x+y), lst, 1)) / len(lst)

## sqrt
def sqrt(lst):
    return map (math.sqrt, lst)

def csqrt(lst):
    return map (cmath.sqrt, lst)

## help
def numberfunhelp():
    print """"
    Help for numberfun: an exercise in learning."""
## Function table:
functiontable = {'sum': sum,
'product': product,
'mean': mean,
'sqrt': sqrt,
'help': numberfunhelp}
#########################################################
### Program Starts Here.

## Error check the arguments
## and run the program
def doit(lst):    # lst is command line argument strings
    numlst = []   # the numeric equivalent of the argument strings.
    for el in lst[2:]:
        try:
            numlst.append(float(el))
        except ValueError:
            print '%s is not a valid number' % el
            exit
    # final chance for exceptional functions:
    if any(map((lambda x: x&lt;0), numlst)):
        functiontable["sqrt"] = csqrt
    if lst[1] in functiontable:
        return map(functiontable[lst[1]], [numlst])[0]
    else:
        print "Function %s not implemented" % lst[1]
        exit

print doit(sys.argv)