RSS feed for AIChE ChEnected

If you are looking for the RSS feed of AIChE ChEnected (the online community of young professional chemical engineers hosted by the American Institute of Chemical Engineers), here it is: http://libpf.com/chenected2.rss.

ChEnected has plenty of social connectivity options (at the moment: LinkedIn, Twitter, Facebook, YouTube, Flickr and Slide Share …) but no official RSS feed.

If you’re a not-so-young professional chemical engineer like me and use RSS aggregation to monitor many news sources, you’ll like this experimental service that scraps hourly the ChEnected homepage and generates a non-official RSS 2.0.1 feed.

Under the hood, it’s a python script which relies on these technologies:

Here is the source of the script: https://gitlab.com/simevo/html2rss.

Disclaimer: we do not alter in any way the contents of ChEnected; we do not claim any rights on their content; we have no responsibility of their content; the service may break in the future; etc etc.

Posted in Chemeng | Tagged | Leave a comment

Automatic generation of C++ code for process models – 3 of 3

Here we are at the last installment of this series of posts on automatic code generation, which started out as a journey to make the workflow for coding process models in C++ with the LIBPF™ library easier and less boring.

In part one we covered the JSON representation of process models, its JSON schema and C++ code generation with a command-line tool using the jinja2 template engine.

In part two we made the code generation interactive with the LIBPF™ model wizard, a free and open source client-side, browser-based interactive JSON editing + code generation tool that can be used online or run locally.

In this last part of the series we will present an example of the complete model development workflow using the LIBPF™ model wizard, and close up with some philosophical considerations.

An example

To demonstrate the complete model development workflow with the LIBPF™ model wizard, we’ll pick up an integrated distillation / pervaporation process to purify a methylbutynol (MBI) / water mixture, as laid out in a recent United States Patent Application by Stefan Ottiger, Thomas Scholl, Stefan Stoffel, Klaus Kalbermatter, Andreas Klein and Kishore Nedungadi all at Lonza Ltd.

We refer to the left-hand part of figure 1 of the application (let’s neglect the subsequent purification steps):
Figure1

This is the legend:

  • 1: feed composition comprising methylbutynol and water
  • 2: distillation column
  • 3: column bottom
  • 4: column head
  • 5: sidestream collection tank
  • 6: pervaporation device
  • 7: sidestream removal
  • 8: sidestream to pervaporation
  • 9: permeate enriched in water
  • 10: retentate dewatered and enriched in methylbutynol
  • 11: low-boiling components which are liquid at room temperature (acetone) removal
  • 12: column bottom comprising a high ratio of methylbutynol
  • 19: incondensables (mostly ammonia) removal in gaseous form

Before we can create a model for this process in LIBPF™, we need to “translate” this block diagram in a process flowsheet proper.

This requires to break down some units and add other units.

In fact we need break down the column in three sections (C1, C2 and C3) to insert the feed and extract the side-draw.

Furthermore we need to add the following units:

  • a feed mixer M to mix the fresh feed with the retentate recycle; we can use the same to mix the total feed with the upper tray liquid)
  • a tee T1 to partition the condensate between reflux and distillate product
  • a tee T2 to partition the tray liquid between side-draw and the liquid that is allowed to proceed to the next stage
  • a tee T3 to partition the column last tray liquid between bottom product and boil-up

This is the resulting process flowsheet diagram as a paper & pencil sketch:

PervaDist

LIBPF™ Model Wizard to the rescue

We can now use the LIBPF™ Model Wizard to define this flowsheet.

This 3-minute video shows the procedure (some parts of the procedure are fast-forwarded – in real life it takes about 20 minutes):

Here are the four main required steps:

  1. clear the form
  2. type the front-matter, in particular the model name which will become the name of the class and of the generated files
  3. define the unit operations, for each of them you need to assign a name, an optional description and the type; to decide which types to use, check the LIBPF SDK documentation for the available unit operations, or use “Other” if you’re uncertain
  4. create the connections between the units, by defining the material streams; for each stream besides name, optional description, and type you also need to specify units and ports for the source and destination of the stream; for the source and destination units you pick them from the list of units (based on the names you previously assigned) but there are also two special units: “source” (use as source unit to indicate that a stream is a feed stream) and “sink” (use as destination unit to indicate that a stream is a product from the flowsheet); for the ports, you should check the unit documentation since each unit has different available ports – often inlet ports are called “in” and outlet ports are called “out” or some specific name such as “vapor”, “liquid”, “permeate” or “retentate”; for the available stream types check the LIBPF SDK documentation

In this example we are using 5 different unit types:

  • three “ColumnSections”, which are counter-current multi-stage, adiabatic, vapor-liquid separations
  • a “Mixer”, which does just that: mix the inlet streams;
  • three “Dividers” for the tees which split their inlet without changing its state and composition;
  • “Other” for the pervaporation unit;
  • and “FlashDrum” for two single-stream heat exchangers;

and 3 stream types:

  • StreamLiquid” for liquid-only streams;
  • StreamVapor” for vapor-only streams;
  • and “StreamNrtl1LiquidVapor” for two-phase, vapor-liquid streams with the Non Random Two Liquids (NRTL) activity coefficient model for the phase equilibrium.

The end product of the LIBPF™ model wizard is a JSON description of the PervaDist process, but a more intuitive view is offered by the generated Process Flowsheet Diagram preview:
PFD

To move forward we will use the generated C++ code: interface, implementation and driver.

Show me the code, Luke

So let’s get the C++ code generated by the LIBPF™ Model Wizard and use it set up a model in the LIBPF™ SDK.

Open a terminal window, (on Windows, that would be a Git Bash shell), move to the location of the LIBPF SDK (we assume it has been installed in the Desktop), then invoke the create_model_skeleton script with the name of the model we want to create as argument:

cd Desktop/LIBPF_sdk
 ./scripts/create_model_skeleton.sh PervaDist

Be careful to use here the very same name you used in the model wizard (“PervaDist” in this example).

This command will create a PervaDist directory, with src and include sub-directories in accordance with the source code directory structure standard.

In those directories you will find three files:

  • include/PervaDist.h where you will copy-paste the interface from the LIBPF™ Model Wizard, completely overwriting the file content;
  • src/PervaDist.cc where you will copy-paste the implementation from the LIBPF™ Model Wizard, completely overwriting the file content;
  • src/PervaDistDriver.cc where you will copy-paste the driver from the LIBPF™ Model Wizard, placing it in the body of the “initializeKernel” function, replacing the placeholder text under the comment line “register the provided types”.

To perform the copy-paste, it’s easier to use an Integrated Development Environment (IDE) that will also make things easier during further manipulation of the code.

The supported IDEs are:

  • Qt Creator (all platforms): to open Qt Creator, just open the PervaDist.pro project file in the “PervaDist” directory by double-clicking;
  • Xcode (on Apple OS X): enter the “PervaDist” directory then generate the project file with qmake:
    cd PervaDist
    /Qt/5.5/clang_64/bin/qmake -spec macx-xcode

    this command will generate the “pervadist.xcodeproj” Xcode project file file in the “PervaDist” directory. You can open that with Xcode by double-clicking;

  • and Microsoft Visual Studio (on Microsoft Windows): enter the “PervaDist” directory then generate the project file with qmake:
    cd PervaDist
    /Qt/Qt5.5.5/5.5/msvc2013/bin/qmake -tp vc

    this command will generate the “pervadist.vcproj” Visual Studio project file in the “PervaDist” directory. You can open that with Visual Studio by double-clicking.

These three videos show this procedure in practice on OS X with Qt Creator or with Xcode, and on Windows with Microsoft Visual Studio:

Next steps

The generated C++ code will compile and link, but it not work out of the box: consider it a starting point !

From here you still need to do some work to complete your model – these are the minimum required steps:

  1. in the driver:
    1. define the components
    2. provide additional details for the service / kernel in the instantiation of the KernelImplementation object, replacing the ellipsis (…)
    3. if you are using any string enumerator, populate them
  2. if you used the “Other” type placeholder, replace that with whatever model you need
  3. if you have defined any integer or string option, make use of their values to trigger logic or calculations as required
  4. complete the implementation of the setup function with:
    1. settings for  input variables: feed streams
    2. setting input variables: operating conditions for the unit operations
    3. initializing cut streams (if any)
  5. supply the model icon, by default it should be named PervaDist.svg (where PervaDist is your model name).

Finally there are some additional, optional steps that are sometimes required:

  • provide a non-dummy implementation for the mandatory member functions makeUserEquations, pre and post
  • implement the non-mandatory functions supportsSimultaneous, maximumIterations and sequential
  • the videos cover the basic use case where you only want your service / kernel to support a single model, but actually it is possible to pack several models inside a kernel; the workflow to add new models and the associated files to a LIBPF project is covered by this old (but still relevant !) blog post.

Finale: some philosophy

In this video:

you can see Bret Victor, a famous user interface designer performing as a young 1973 egghead in a white shirt and black tie, talking about the future of programming in 2013 as if it were 1973. The presentation (done with an overhead projector) pushes the key message that already back in 1973 you could tell the future of programming would be setting concurrent goals and constraints by direct manipulation of data as spatial representations. Victor regrets it’s 2013 (now it’s 2015!) and we’re still coding sequential procedures as sequences of symbols.

As much as I agree with Bret Victor that creative processes are visual, that does not necessary mean that the user interface should be visual in all cases: a textual representation is more distant from the way we create, but it can be formally compared, exchanged and processed. The move from symbolic to visual manipulation has been the marketing mantra of the IT industry for decades: it goes in the direction of user friedliness and increased usability. While easy to use interfaces can be a blessing for commoditized, consumer-oriented software, history has shown that there are domains (music, mathematics, and – we argue – process modeling) where visual interfaces can be of help, but do not solve all the problems !

When you use LIBPF™ to develop process models, you declaratively specify your goals and constraints using a symbolic, textual programming language (C++), but without specifying the procedure and the sequence to solve them. Additionally, with the LIBPF™ Model Wizard you now have a tool which can make it easier and less boring to set up the process models in C++ !

 

 

 

Posted in C++, Chemeng, Howtos | Leave a comment

Automatic generation of C++ code for process models – 2 of 3

In this part two of the series of three posts we will leverage the techniques discussed in part one (command-line C++ code generation from the JSON representation of process models with the jinja2 template engine) to make the code generation interactive.

JSON, JSON schema and Jinja2 templates are de-facto standards and there are quite a few tools that understand these formats.

One such tool is Jeremy Dorn’s json-editor (JSON Schema Based Editor), a JavaScript library which takes a JSON Schema and uses it to generate an HTML form. Here it is in a graphical depiction of the idea behind json-editor (Copyright (c) 2013 Jeremy Dorn):

jsbe

To get the best out of json-editor, we’ll need to expand the basic JSON schema presented in part one, adding a few additional, optional fields (title, format, propertyOrder, template, watch, enumSource). The expanded schema is quite verbose so if you really want to see it here is the link.

Another tool is Mozilla’s Nunjucks, a JavaScript implementation of the Jinja2 templating language.

With the help of these two and a few more JavaScript libraries, it is possible to create a client-side, browser-based interactive JSON editing + code generation tool: the LIBPF™ model wizard.

Here is a screenshot of the LIBPF™ model wizard with the MicroTurbine model from the cogeneration demo:

Model Wizard (1)The wizard behind the scenes uses the very same JSON representation we have shown in part one, and you can see it for the currently loaded model by clicking on the JSON button at the top of the Model tab. For example here is the link to the JSON representation for the MicroTurbine model.

From the JSON representation, the LIBPF™ model wizard can generate the Process Flow Diagram preview, which gives you a rough idea of how the connectivity you have defined in the streams section of the JSON translates:

Model Wizard (2)The graph is rendered as a SVG file by Mike Daines’s  excellent Viz.js library, a port of the graphviz dot tool from C to JavaScript, based on emscripten (a compiler from C and C++ into JavaScript that lets you run C and C++ on the web at near-native speed, without plugins). Notice that cut streams are displayed in blue.

Finally thanks to the Nunjucks library the LIBPF™ model wizard can perform the C++ code generation directly in the browser, without the need to invoke the command-line commands described in part one (but if you prefer them, you can still use them !).

Here is the generated interface for the MicroTurbine model (.h file):

Model Wizard (3)and here is the generated implementation (.cc file):

Model Wizard (4) The syntax highlighting is also performed in the browser thanks to the highlight.js JavaScript library.

The LIBPF™ model wizard can be used online at this address: http://libpf.com/sdk/model_wizard.html.

This tutorial video will guide you in your first steps:

Finally, we choose to release the source code for the LIBPF™ model wizard with an open source license so you can also run it locally: to do so, clone the gitlab repo and have a look at the README.

Watch out for part three in this series, where we’ll present the complete model development workflow using the automatic code generation tool presented here !

Posted in C++, Chemeng, Howtos | Tagged | Leave a comment

Automatic generation of C++ code for process models – 1 of 3

Coding process models in C++ with the help of LIBPF is easy (try one of our tutorials !), but it’s fair to say that it’s somewhat boring and error prone.

For example if you add an integer parameter to your model you need to make 4 distinct changes:

  1. declare the parameter in the model interface:
    Integer n; ///< number of edges of the base

    (notice the optional but useful doxygen-style comment that described the parameter)

  2. initialize it before the constructor body:
    DEFINE(n, "number of edges of the base", 4)

    (notice how the description quotes the doxygen-style comment above)

  3. register it for reflection in the constructor body:
    addVariable(n);
  4. and finally add the command to retrieve it and define the default, max and min values:
    n = retrieveInteger(defaults, id, persistency, 4, 999, 3); // number of edges of the base

    (again, the comment quotes the doxygen-style comment in the header).

We want to make this workflow easier and less boring, so here is a series of three posts.

In this part one we will present a new way to represent process models as JSON files, and how to generate the required C++ code (interface and implementation) automatically from this representation.

The JSON representation

If you haven’t heard about JSON, that’s an open standard format that uses human-readable text to represent objects consisting of attribute-value pairs, other sub-objects and arrays thereof; think of it as a leaner alternative to XML.

Here is an example JSON file to describe a car:

{
  "wheels": 4,
  "max_speed": 165.0,
  "color": "blue"
}

and here is another one to describe an integer parameter of a LIBPF process model:

{
  "name": "n",
  "description": "number of edges of the base",
  "min": 3,
  "max": 999,
  "value": 4
}

Notice how these two JSON files are made of of attribute-value pairs each separated by a colon, arranged in a list where each pair is separated by a comma, and wrapped in curly brackets: this is a JSON object !

In general a process model can have several integer parameters, so we could define an array attribute integerOptions (arrays are enclosed in square brackets and the elements are separated by commas):

integerOptions: [{ ... }, { ... } ... , { ... } ]

Each of the { ... } sections could be a different integer parameter object, complete with name, description, min, max and value attributes.

For a complete model representation we need quite a few attibutes:

  • name: model name (will be the class name);
  • description: a one-liner description of the model;
  • author: for author and copyright;
  • icon: used for visualization;
  • category: one of flowsheet, phase, stream, unit or option; actually will be set to flowsheet in most cases for user defined models !
  • instantiable: normally set to true;
  • integerOptions: array of integer parameter objects, each complete with name, description, min, max and value fields;
  • stringOptions: array of string parameter objects, each complete with name, description, enumerator and value fields;
  • variables: array of variable objects, each complete with name, description and value fields, with optional unit, input and output fields;
  • units: array of unit operations object, each complete with name, description and type, plus optional arrays of integer and string parameters for configuration;
  • streams: array of material stream objects, each complete with name, description, type, from /portFrom and to/portTo fields, plus optional arrays of integer and string parameters for configuration;
  • cuts: array of cut material streams.

Here is an example of a complete JSON file for a dummy model “Simple”:

{
  "author": "(C) Copyright 2015 Paolo Greppi simevo s.r.l.",
  "name": "Simple",
  "icon": "Simple.svg",
  "description": "Simple process model in JSON format",
  "category": "flowsheet",
  "instantiable": true,
  "integerOptions": [{
    "name": "n",
    "description": "number of stages",
    "min": 3,
    "max": 999,
    "value": 4
  }],
  "stringOptions": [ {
    "enumerator": "processType",
    "name": "processType",
    "description": "adjusts temperature and holding time",
    "value": "HTST15"
  }, {
    "enumerator": "feedType",
    "name": "feedType",
    "description": "sets a predefined composition for the fluid to be processed",
    "value": "chocolateIceCream"
  } ],
  "variables": [
    {
      "input": true,
      "name": "Qin",
      "description": "Thermal power input",
      "value": 0.0,
      "unit": "W"
    },
    {
      "output": true,
      "name": "Qout",
      "description": "Thermal power output",
      "value": 0.0,
      "unit": "W"
    }
  ],
  "streams": [
    { "description": "Feed1", "from": "source", "name": "S01", "portFrom": "out", "portTo": "in", "to": "MIX", "type": "StreamVapor" },
    { "description": "Feed2", "from": "source", "name": "S02", "portFrom": "out", "portTo": "in", "to": "MIX", "type": "StreamLiquid" },
    { "description": "Product", "from": "MIX", "name": "S03", "portFrom": "out", "portTo": "in", "to": "SPL", "type": "StreamIdealLiquidVapor" },
    { "description": "Recycle", "from": "SPL", "name": "S04", "portFrom": "out", "portTo": "in", "to": "MIX", "type": "StreamIdealLiquidVapor" },
    { "description": "Discharge", "from": "SPL", "name": "S05", "portFrom": "out", "portTo": "in", "to": "sink", "type": "StreamIdealLiquidVapor" }
  ],
  "units": [
    { "description": "Mixer", "name": "MIX", "type": "Mixer" },
    { "description": "Three-way valve", "name": "SPL", "type": "Divider",
      "integerOptions": [{"name": "nOutlets", "value": 2}]}
  ],
  "cuts": ["S04"]
}

Formalization of the JSON schema

We can formally define a JSON format to represent a process model using a JSON schema, another JSON file which is kind of “meta” in that it describes how the other JSON files should look like. If you come from the XML world, the JSON schema is similar to the XSD (XML Schema Definition).

A schema is useful for validation but also has other advantages as we’ll se in part 2.

Here is the basic JSON schema for our process models:

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "minLength": 0,
      "maxLength": 50
    },
    "description": {
      "type": "string",
      "minLength": 0,
      "maxLength": 255
    },
    "author": {
      "type": "string"
    },
    "icon": {
      "type": "string",
      "minLength": 0,
      "maxLength": 50
    },
    "category": {
      "type": "string",
      "enum": ["flowsheet", "phase", "stream", "unit", "option"]
    },
    "instantiable": {
      "type": "boolean"
    },
    "time_out": {
      "type": "number"
    },
    "integerOptions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 0,
            "maxLength": 50
          },
          "description": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          },
          "min": {
            "type": "integer"
          },
          "max": {
            "type": "integer"
          },
          "value": {
            "type": "integer"
          }
        },
        "additionalProperties": false,
        "required": ["name", "description", "min", "max", "value"]
      }
    },
    "stringOptions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 0,
            "maxLength": 50
          },
          "description": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          },
          "enumerator": {
            "type": "string"
          },
          "value": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          }
        },
        "additionalProperties": false,
        "required": ["name", "description", "enumerator", "value"]
      }
    },
    "variables": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 0,
            "maxLength": 50
          },
          "description": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          },
          "input": {
            "type": "boolean"
          },
          "output": {
            "type": "boolean"
          },
          "value": {
            "type": "number"
          },
          "unit": {
            "type": "string",
            "enum": ["",
                      "°C",
                      "°F",
                      "$",
                      "$/J",
                      "$/mmBTU",
                      "€",
                      "€/J",
                      "€/kcal",
                      "€/kWh",
                      "€/mmBTU",
                      "€/MWh",
                      "1/K",
                      "1/m",
                      "A",
                      "A/cm2",
                      "A/dm2",
                      "A/ft2",
                      "A/m2",
                      "A/mm2",
                      "atm",
                      "bar",
                      "barg",
                      "bbl",
                      "BTU",
                      "BTU/(hft2F)",
                      "BTU/ft3",
                      "BTU/h",
                      "BTU/h/°F",
                      "BTU/lb",
                      "BTU/lb/°F",
                      "BTU/lbmol",
                      "C",
                      "C/kmol",
                      "cd",
                      "cm",
                      "cm2",
                      "cm3",
                      "cP",
                      "d",
                      "dm",
                      "dm2",
                      "dm3",
                      "F",
                      "ft",
                      "ft2",
                      "ft3",
                      "ft3/h",
                      "g",
                      "g/d",
                      "g/m3",
                      "g/Nm3",
                      "GPU",
                      "GW",
                      "H",
                      "h",
                      "hPa",
                      "Hz",
                      "in",
                      "J",
                      "J/(kg*K)",
                      "J/(kg*K2)",
                      "J/(kg*K3)",
                      "J/(kmol*K)",
                      "J/K",
                      "J/kg",
                      "J/kg/K",
                      "J/kmol",
                      "J/kmol/K",
                      "J/l",
                      "J/m3",
                      "K",
                      "K/Pa",
                      "K2/Pa",
                      "kcal",
                      "kcal/(hm2K)",
                      "kcal/(kg*K)",
                      "kcal/h",
                      "kcal/kg",
                      "kcal/kg/K",
                      "kcal/kmol",
                      "kcal/l",
                      "kcal/m3",
                      "kcal/s",
                      "kg",
                      "kg/(m3*K)",
                      "kg/(m3*K2)",
                      "kg/cm2",
                      "kg/d",
                      "kg/h",
                      "kg/J",
                      "kg/kmol",
                      "kg/m3",
                      "kg/m7",
                      "kg/s",
                      "kJ",
                      "kJ/g",
                      "kJ/kg",
                      "kJ/kg/K",
                      "kJ/kmol",
                      "kJ/kmol/K",
                      "kJ/m3",
                      "km",
                      "kmol/h",
                      "kmol/kg",
                      "kmol/kg/m",
                      "kmol/m3",
                      "kmol/s",
                      "kPa",
                      "kW",
                      "kWh",
                      "kWh/m3",
                      "l",
                      "l/h",
                      "lb",
                      "lb/ft3",
                      "lb/h",
                      "lb/lbmol",
                      "lbmol/ft3",
                      "lbmol/h",
                      "m",
                      "m*K/W",
                      "m*s",
                      "m/s",
                      "m/s2",
                      "m2",
                      "m2*K/W",
                      "m2*K2/W2",
                      "m2/s",
                      "m2/s2",
                      "m3",
                      "m3/h",
                      "m3/kg",
                      "m3/kmol",
                      "m3/s",
                      "mA/cm2",
                      "mbar",
                      "mg",
                      "mg/m3",
                      "mg/Nm3",
                      "min",
                      "MJ/kg",
                      "ml",
                      "mm",
                      "mm2",
                      "mm3",
                      "mmBTU",
                      "mmH2O",
                      "mmHg",
                      "mmol/dm3",
                      "mol",
                      "mol/dm3",
                      "mol/h",
                      "mol/kg",
                      "mol/m3",
                      "mol/s",
                      "MPa",
                      "mPa*s",
                      "mV",
                      "mW",
                      "MW",
                      "MWh",
                      "N",
                      "N*m4",
                      "N/m",
                      "Nm3/d",
                      "Nm3/h",
                      "Nm4kmol-2",
                      "ohm",
                      "ohm*cm",
                      "ohm*cm2",
                      "ohm*m",
                      "ohm*m2",
                      "Pa",
                      "Pa*m6/kmol2",
                      "Pa*m6/kmol2/K",
                      "Pa*m6/kmol2/K2",
                      "Pa*s",
                      "Pa/K",
                      "Pa/K2",
                      "psi",
                      "rad",
                      "rpm",
                      "S",
                      "s",
                      "T",
                      "t",
                      "t/d",
                      "t/h",
                      "t/yr",
                      "Torr",
                      "TW",
                      "ul",
                      "um",
                      "uPa*s",
                      "V",
                      "W",
                      "W/(m*K)",
                      "W/(m2*K)",
                      "W/K",
                      "W/m/K",
                      "W/m2",
                      "W/m2/K",
                      "Wb",
                      "yr"]
          }
        },
        "additionalProperties": false,
        "required": ["name", "description", "value"]
      }
    },
    "units": {
      "type": "array",
      "items": {
        "type": "object",
        "headerTemplate": "{{ i1 }} - {{ self.name }} (type: {{ self.type }})",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 0,
            "maxLength": 50
          },
          "description": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          },
          "type": {
            "type": "string",
            "enum": ["Column",
                      "Compressor",
                      "Decanter",
                      "Degasser",
                      "Divider",
                      "Exchanger",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDegasser",
                      "FlashDrum",
                      "FlashSplitter",
                      "FlashSplitter",
                      "FlashSplitter",
                      "FlashSplitter",
                      "FlashSplitter",
                      "FlashSplitter",
                      "HtuNtu",
                      "LiquidRingVacuumPump",
                      "Mixer",
                      "MultiCompressorIntercooled1",
                      "MultiCompressorIntercooled2",
                      "MultiCompressorIntercooled3",
                      "MultiExchanger",
                      "Multiplier",
                      "Other",
                      "Pipe",
                      "PressureSwingAbsorption",
                      "Pump",
                      "Selector",
                      "Separator",
                      "Separator",
                      "Splitter",
                      "Terminator"]
          },
          "integerOptions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "name": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 50
                },
                "value": {
                  "type": "integer"
                }
              },
              "additionalProperties": false,
              "required": ["name", "value"]
            }
          },
          "stringOptions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "name": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 50
                },
                "value": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 255
                }
              },
              "additionalProperties": false,
              "required": ["name", "value"]
            }
          }
        },
        "additionalProperties": false,
        "required": ["name", "description", "type"]
      }
    },
    "streams": {
      "type": "array",
      "items": {
        "type": "object",
        "headerTemplate": "{{ i1 }} - {{ self.name }} (type: {{ self.type }})",
        "properties": {
          "name": {
            "type": "string",
            "minLength": 0,
            "maxLength": 50
          },
          "description": {
            "type": "string",
            "minLength": 0,
            "maxLength": 255
          },
          "type": {
            "type": "string",
            "enum": ["Other",
                      "StreamEosLiquid",
                      "StreamEosLiquidSolidVapor",
                      "StreamEosLiquidVapor",
                      "StreamEosSolidVapor",
                      "StreamEosVapor",
                      "StreamGerg2004Liquid",
                      "StreamGerg2004LiquidVapor",
                      "StreamGerg2004Vapor",
                      "StreamIapwsLiquid",
                      "StreamIapwsLiquidVapor",
                      "StreamIapwsVapor",
                      "StreamIdealLiquidSolid",
                      "StreamIdealLiquidSolidVapor",
                      "StreamIdealLiquidVapor",
                      "StreamIdealSolidVapor",
                      "StreamLiquid",
                      "StreamNrtl1Liquid",
                      "StreamNrtl1LiquidSolid",
                      "StreamNrtl1LiquidSolidVapor",
                      "StreamNrtl1LiquidVapor",
                      "StreamNrtl2Liquid",
                      "StreamNrtl2LiquidSolid",
                      "StreamNrtl2LiquidSolidVapor",
                      "StreamNrtl2LiquidVapor",
                      "StreamPcsaftLiquid",
                      "StreamPcsaftLiquidVapor",
                      "StreamPcsaftVapor",
                      "StreamSimpleLiquid",
                      "StreamSimpleLiquidSolid",
                      "StreamSimpleLiquidSolidVapor",
                      "StreamSimpleSolid",
                      "StreamSimpleVapor",
                      "StreamSolid",
                      "StreamTabularLiquidLiquid",
                      "StreamTabularLiquidVapor",
                      "StreamVapor"]
          },
          "from": {
            "type": "string"
          },
          "portFrom": {
            "type": "string"
          },
          "to": {
            "type": "string"
          },
          "portTo": {
            "type": "string"
          },
          "integerOptions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "name": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 50
                },
                "value": {
                  "type": "integer"
                }
              },
              "additionalProperties": false,
              "required": ["name", "value"]
            }
          },
          "stringOptions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "name": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 50
                },
                "value": {
                  "type": "string",
                  "minLength": 0,
                  "maxLength": 255
                }
              },
              "additionalProperties": false,
              "required": ["name", "value"]
            }
          }
        },
        "additionalProperties": false,
        "required": ["name", "description", "type", "from", "portFrom", "to", "portTo"]
      }
    },
    "cuts": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  },
  "additionalProperties": false,
  "required": ["name", "description", "author", "category", "instantiable", "integerOptions", "stringOptions", "variables", "units", "streams", "cuts"]
}

Code generation

Now we can use the JSON representation of the process model to generate the required C++ code. We’ll use Python and jinja2, a full-featured template engine available as a Python extension.

Jinja2 templates are plain text files which closely resemble the files we want to generate. They contain magic commands enclosed in double curly brackets {{ ... }} (variables and/or expressions) which get replaced with values retrieved from a JSON file, and tags enclosed in curly-percent brackets {% ... %} which control the logic of the template.

Here is an example template:

Car{% if female %}a{% else %}o{% endif %} {{ name }}, ciao !

When rendered against this JSON:

{ "name": "Cristina", "female": true }

the template will give you:

Cara Cristina, ciao !

Here it is in a graphical depiction of the process:
jinjaBut let’s try something closer to our application; suppose we are bored of repeating the class name in the constructor interface and implementation. This template:

class {{ name }} {
  public:
  {{ name }}(void);
}; // {{ name }}

{{ name }}::{{ name }}(void) {
  // init stuff here
}

will render like this with the same JSON as above:

class Cristina {
public:
  Cristina(void);
}; // Cristina

Cristina::Cristina(void) {
  // init stuff here
}

Go have a look at the full jinja2 Template Designer Documentation for going deeper in the syntax.

To generate the C++ code we will need two separate templates:

  • interface file (the header file, .h):
    /** @file {{ name }}.h
     *  @brief Interface for the {{ name }} class
     *  @author {{ author }}
     */
    
    #ifndef {{ name | upper }}_H
    #define {{ name | upper }}_H
    
    /* SYSTEM INCLUDES */
    
    /* PROJECT INCLUDES */
    #include <libpf/Model.h>
    #include <libpf/FlowSheet.h>
    
    /* LOCAL INCLUDES */
    
    /* FORWARD REFERENCES */
    
    /** @class {{ name }}
     * @brief {{ description }}
     * 
    {% if stringOptions | length > 0 %} * string options:
    {% for option in stringOptions %} * - {{ option.name }}: {{ option.description }}
    {% endfor %}{% endif %}{% if integerOptions | length > 0 %} * integer options:
    {% for option in integerOptions %} * - {{ option.name }}: {{ option.description }}
    {% endfor %}{% endif %} */
    class {{ name }} : public FlowSheet {
    private:
      const static std::string type_;
    public:
      // LIFECYCLE
      {{ name }}(Libpf::User::Defaults defaults, uint32_t id=0, Persistency *persistency=nullptr, Persistent *parent=nullptr, Persistent *root=nullptr);
    
      // CUSTOM variables
    {% for variable in variables %}  Quantity {{ variable.name }}; ///< {{ variable.description }}
    {% endfor %}
    {% for option in stringOptions %}  String {{ option.name }}; ///< {{ option.description }}
    {% endfor %}
    {% for option in integerOptions %}  Integer {{ option.name }}; ///< {{ option.description }}
    {% endfor %}
      // CUSTOM function
    
      // MANDATORY
      const std::string &type(void) const { return type_; }
      void makeUserEquations(std::list::iterator &p) { }
      void setup(void);
      void pre(SolutionMode solutionMode, int level) { }
      void post(SolutionMode solutionMode, int level) { }
    
      // NON-MANDATORY
    }; // {{ name }}
    
    #endif // {{ name | upper }}_H
  • implementation file (the actual C++ file, .cc):
    /** @file {{ name }}.cc
     *  @brief Implementation of {{ name }} class
     *  @author {{ author }}
     */
    
    /* SYSTEM INCLUDES */
    //
    
    /* PROJECT INCLUDES */
    //
    #include <libpf/utility/diagnostic.h>
    
    /* LOCAL INCLUDES */
    //
    #include "{{ name }}.h"
    
    /* LOCAL VARIABLES */
    //
    static const int verbositySubSystem = 0;
    
    /* FUNCTIONS */
    //
    
    const std::string {{ name }}::type_("{{ name }}");
    
    {{ name }}::{{ name }}(Libpf::User::Defaults defaults, uint32_t id, Persistency *persistency, Persistent *parent, Persistent *root) :
      Model(defaults, id, persistency, parent, root),
      VertexBase(defaults, id, persistency, parent, root),
      FlowSheet(defaults, id, persistency, parent, root),
    {% for variable in variables %}{% if variable.unit %}  DEFINE({{ variable.name }}, "{{ variable.description }}", {{ variable.value }}, "{{ variable.unit }}"),{% else %}  DEFINE({{ variable.name }}, "{{ variable.description }}", {{ variable.value }}),{% endif %}
    {% endfor %}
    {% for option in stringOptions %}  DEFINE({{ option.name }}, "{{ option.description }}", "{{ option.value }}"),
    {% endfor %}
    {% for option in integerOptions %}  DEFINE({{ option.name }}, "{{ option.description }}", {{ option.value }}),
    {% endfor %}{
      static const int verbosityLocal = 0;
    
      diagnostic(2, "Entered");
    {% for variable in variables %}  addVariable({{ variable.name }});
    {% endfor %}
    {% for option in stringOptions %}  addVariable({{ option.name }});
    {% endfor %}
    {% for option in integerOptions %}  addVariable({{ option.name }});
    {% endfor %}
      diagnostic(3, "Retrieve string options");
    {% for option in stringOptions %}  {{ option.name }} = retrieveString(defaults, id, persistency, "{{ option.enumerator }}", "{{ option.value }}"); // {{ option.description }}
    {% endfor %}
      diagnostic(3, "Retrieve integer options");
    {% for option in integerOptions %}  {{ option.name }} = retrieveInteger(defaults, id, persistency, {{ option.value }}, {{ option.max }}, {{ option.min }}); // {{ option.description }}
    {% endfor %}
      if (!persistency) {
        diagnostic(2, "Define unit operations");
    {% for unit in units %}    addUnit("{{ unit.type }}", defaults.relay("{{ unit.name }}", "{{ unit.description }}"){% for option in unit.integerOptions %}, ("{{ option.name }}", "{{ option.value }}"){% endfor %}{% for option in unit.stringOptions %}, ("{{ option.name }}", "{{ option.value }}"){% endfor %});
    {% endfor %}
        diagnostic(2, "Define stream objects and connect");
    {% for stream in streams %}    addStream("{{ stream.type }}", defaults.relay("{{ stream.name }}", "{{ stream.description }}"){% for option in stream.integerOptions %}, ("{{ option.name }}", "{{ option.value }}"){% endfor %}{% for option in stream.stringOptions %}, ("{{ option.name }}", "{{ option.value }}"){% endfor %}, "{{ stream.from }}", "{{ stream.portFrom }}", "{{ stream.to }}", "{{ stream.portTo }}");
    {% endfor %}  }
    } // {{ name }}::{{ name }}
    
    void {{ name }}::setup(void) {
      static const int verbosityLocal = 0;
      try {
        diagnostic(2, "Entered for " << tag());     diagnostic(3, "Calling flowsheet::setup to initialize any sub-flowsheet");     FlowSheet::setup();     diagnostic(3, "Setting input variables: feed streams");     /// @TODO     diagnostic(3, "Setting input variables: operating conditions for the unit operations");     /// @TODO     diagnostic(3, "Initializing cut streams");     /// @TODO {% if cuts | length > 0 %}    diagnostic(3, "Defining cut streams");
    {% for cut in cuts %}    addCut("{{ cut }}");
    {% endfor %}{% endif %}
        diagnostic(3, "Making selected inputs/outputs visible in GUI");
    {% for variable in variables %}{% if variable.input %}    {{ variable.name }}.setInput();
    {% endif %}{% if variable.output %}    {{ variable.name }}.setOutput();{% endif %}
    {% endfor %}  } // try
      catch(Error &e) {
        e.append(CURRENT_FUNCTION);
        throw;
      } // catch
    } // {{ name }}::setup

Now we need to feed these templates through Jinja2, and save the results as the cc and h files. There are CLI interfaces to jinja2 around, but we can reach out goal with a simple script j2.py:

#!/usr/bin/env python
# coding=utf-8

# ***** BEGIN LICENSE BLOCK *****
# This file is part of LIBPF
# (C) Copyright 2015 Paolo Greppi simevo s.r.l.
# ***** END LICENSE BLOCK ***** */

""" j2.py: generate cc / h file from JSON DSL """

import sys
import jinja2
import json
import jsonschema

if len(sys.argv) != 4:
    print "usage: %s template.tmpl schema.json variables.json"
    sys.exit(-1)

templateLoader = jinja2.FileSystemLoader(searchpath='.')
templateEnv = jinja2.Environment(loader=templateLoader)
template = templateEnv.get_template(sys.argv[1])

model_file = open(sys.argv[3], 'r')
model_json = model_file.read()
model_file.close()
model = json.loads(model_json)

model_schema_file = open(sys.argv[2], 'r')
model_schema_json = model_schema_file.read()
model_schema_file.close()
model_schema = json.loads(model_schema_json)
jsonschema.validate(model, model_schema)

print template.render(model)

Here is a sample invocation for the j2 script:

./j2.py views/model_h.tmpl model_schema_basic.json Simple.json > Simple.h
./j2.py views/model_cc.tmpl model_schema_basic.json Simple.json > Simple.cc

And here is the generated code, interface:

/** @file Simple.h
 *  @brief Interface for the Simple class
 *  @author (C) Copyright 2015 Paolo Greppi simevo s.r.l.
 */

#ifndef SIMPLE_H
#define SIMPLE_H

/* SYSTEM INCLUDES */

/* PROJECT INCLUDES */
#include <libpf/Model.h>
#include <libpf/FlowSheet.h>

/* LOCAL INCLUDES */

/* FORWARD REFERENCES */

/** @class Simple
 * @brief Simple process model in JSON format
 * 
 * string options:
 * - processType: adjusts temperature and holding time
 * - feedType: sets a predefined composition for the fluid to be processed
 * integer options:
 * - n: number of stages
 */
class Simple : public FlowSheet {
private:
  const static std::string type_;
public:
  // LIFECYCLE
  Simple(Libpf::User::Defaults defaults, uint32_t id=0, Persistency *persistency=nullptr, Persistent *parent=nullptr, Persistent *root=nullptr);

  // CUSTOM variables
  Quantity Qin; ///< Thermal power input
  Quantity Qout; ///< Thermal power output

  String processType; ///< adjusts temperature and holding time
  String feedType; ///< sets a predefined composition for the fluid to be processed

  Integer n; ///< number of stages

  // CUSTOM function

  // MANDATORY
  const std::string &type(void) const { return type_; }
  void makeUserEquations(std::list::iterator &p) { }
  void setup(void);
  void pre(SolutionMode solutionMode, int level) { }
  void post(SolutionMode solutionMode, int level) { }

  // NON-MANDATORY
}; // Simple

#endif // SIMPLE_H

and implementation:

/** @file Simple.cc
 *  @brief Implementation of Simple class
 *  @author (C) Copyright 2015 Paolo Greppi simevo s.r.l.
 */

/* SYSTEM INCLUDES */
//

/* PROJECT INCLUDES */
//
#include <libpf/utility/diagnostic.h>

/* LOCAL INCLUDES */
//
#include "Simple.h"

/* LOCAL VARIABLES */
//
static const int verbositySubSystem = 0;

/* FUNCTIONS */
//

const std::string Simple::type_("Simple");

Simple::Simple(Libpf::User::Defaults defaults, uint32_t id, Persistency *persistency, Persistent *parent, Persistent *root) :
  Model(defaults, id, persistency, parent, root),
  VertexBase(defaults, id, persistency, parent, root),
  FlowSheet(defaults, id, persistency, parent, root),
  DEFINE(Qin, "Thermal power input", 0.0, "W"),
  DEFINE(Qout, "Thermal power output", 0.0, "W"),

  DEFINE(processType, "adjusts temperature and holding time", "HTST15"),
  DEFINE(feedType, "sets a predefined composition for the fluid to be processed", "chocolateIceCream"),

  DEFINE(n, "number of stages", 4),
{
  static const int verbosityLocal = 0;

  diagnostic(2, "Entered");
  addVariable(Qin);
  addVariable(Qout);

  addVariable(processType);
  addVariable(feedType);

  addVariable(n);

  diagnostic(3, "Retrieve string options");
  processType = retrieveString(defaults, id, persistency, "processType", "HTST15"); // adjusts temperature and holding time
  feedType = retrieveString(defaults, id, persistency, "feedType", "chocolateIceCream"); // sets a predefined composition for the fluid to be processed

  diagnostic(3, "Retrieve integer options");
  n = retrieveInteger(defaults, id, persistency, 4, 999, 3); // number of stages

  if (!persistency) {
    diagnostic(2, "Define unit operations");
    addUnit("Mixer", defaults.relay("MIX", "Mixer"));
    addUnit("Divider", defaults.relay("SPL", "Three-way valve"), ("nOutlets", "2"));

    diagnostic(2, "Define stream objects and connect");
    addStream("StreamVapor", defaults.relay("S01", "Feed1"), "source", "out", "MIX", "in");
    addStream("StreamLiquid", defaults.relay("S02", "Feed2"), "source", "out", "MIX", "in");
    addStream("StreamIdealLiquidVapor", defaults.relay("S03", "Product"), "MIX", "out", "SPL", "in");
    addStream("StreamIdealLiquidVapor", defaults.relay("S04", "Recycle"), "SPL", "out", "MIX", "in");
    addStream("StreamIdealLiquidVapor", defaults.relay("S05", "Discharge"), "SPL", "out", "sink", "in");
  }
} // Simple::Simple

void Simple::setup(void) {
  static const int verbosityLocal = 0;
  try {
    diagnostic(2, "Entered for " << tag());

    diagnostic(3, "Calling flowsheet::setup to initialize any sub-flowsheet");
    FlowSheet::setup();

    diagnostic(3, "Setting input variables: feed streams");
    /// @TODO

    diagnostic(3, "Setting input variables: operating conditions for the unit operations");
    /// @TODO

    diagnostic(3, "Initializing cut streams");
    /// @TODO

    diagnostic(3, "Defining cut streams");
    addCut("S04");

    diagnostic(3, "Making selected inputs/outputs visible in GUI");
    Qin.setInput();

    Qout.setOutput();
  } // try
  catch(Error &e) {
    e.append(CURRENT_FUNCTION);
    throw;
  } // catch
} // Simple::setup

Limitations

The C++ code generated with this automatic technique will not work out of the box: consider it a starting point !

The technique has a few limitations, mainly caused by the shortcomings of the JSON format; for example JSON does not support multiline strings and requires manual escaping of reserved characters such as quote signs (xml’s CDATA is much better !).

So here are the limitations in detail:

  • it does not fill these sections of the implementation (you’ll find TODO reminders there):
    • Setting input variables: feed streams;
    • Setting input variables: operating conditions for the unit operations;
    • Initializing cut streams;
  • it generates just the basic mandatory member functions (constructor, type and setup); there is no way to generate the advanced mandatory member functions (makeUserEquations, pre and post – for these a default empty implementation is provided) and any optional member function;
  • it is rather rigid w.r.t. unit operations and stream types; if you need a unit and stream type outside the list, use the Other type and then choose the correct type by manually overriding the generated Implementation (.cc file).

In part one of this series of posts we have generated C++ code from a compact representation of process models, using a command-line tool. Watch out for part two, where we’ll show how to make the code generation interactive !

Posted in C++, Chemeng, Howtos | Tagged | Leave a comment

Set up Android Studio on debian 7 wheezy 64-bit

Now that Android Studio is the official IDE for Android, you can set it up on your development box running debian 7 wheezy 64-bit as follows:

  1. Get Android studio from here
    cd /tmp
    wget https://dl.google.com/dl/android/studio/ide-zips/1.1.0/android-studio-ide-135.1740770-linux.zip
  2. Unzip it to /opt:
    cd /tmp
    unzip android-studio-ide-135.1740770-linux.zip
    sudo mv android-studio /opt
  3. Install OpenJDK 7 JDK:
    sudo apt-get install openjdk-7-jdk
  4. Set up the required environmental variable
    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64
  5. Set up 32-bit libraries to avoid the error:
    Downloading android-sdk_r22.6.2-linux.tgz from https://dl.google.com/android/android-sdk_r22.6.2-linux.tgz
    Unpacking android-sdk_r22.6.2-linux.tgz
    tar xzvfC /tmp/AndroidStudio1254025343410050340FirstRun/android-sdk_r22.6.2-linux.tgz /tmp/AndroidStudio1254025343410050340FirstRun/android-sdk_r22.6.2-linux.tgz-unpacked
    Unable to run mksdcard SDK tool.
    

    as follows:

    sudo dpkg –add-architecture i386
    sudo apt-get update
    sudo apt-get install i32-libs
    
  6. Start Android studio:
    /opt/android-studio/bin/studio.sh
Posted in Uncategorized | Tagged , | Leave a comment

Interpreting “The program can’t start errors” on Windows

Sometimes when you start an executable named say xxx.exe on Windows Desktop a modal pop-up may appear:

cantstartwith the message: The program can’t start because xxx.exe is missing from your computer. Try reinstalling the program to fix this problem.

As often happens, this error message is incomplete, misleading and unhelpful. What is missing is not xxx.exe, which likely is a freestanding application with no installer.

If this error occurs during a debug session inside Microsoft Visual Studio, you may examine the Debug tab of the Output windows, just to find the laconic message:

The program '[10080] xxx.exe' has exited with code -1073741515 (0xc0000135) 'A dependent dll was not found'.

which is not correct, but still incomplete and unhelpful.

The way to know which DLL is missing is to use GFlags to set a particular Windows feature – no need to install the DependencyWalker utility (mentioned in many places on the interweb). gflags.exe is the Global Flags Editor, an utility that enables and disables advanced debugging, diagnostic, and troubleshooting features. Get it from here, look for  the “Standalone Debugging Tools for Windows (WinDbg)” section.

Once you got that installed, you may wonder where it did install – well, on my 64-bit Windows it ends up in C:\Program Files (x86)\Windows Kits\8.1\Debuggers\x64\.

To set the option we need, do this in a Command Prompt:

"C:\Program Files (x86)\Windows Kits\8.1\Debuggers\x64\gflags.exe" /i xxx.exe +sls

Note 0: the option is sls, an acronym that stands for Show Loader Snaps – a flag that captures detailed information about the loading and unloading of executable images and their supporting library modules and displays the data in the kernel debugger console

Note 1: Replace xxx.exe with the name of your executable without the path.

Note 2: the UAC confirmation should come up; if not, launch the command from a Command Prompt window that you started by right-mouse-button clicking on the Command promp icon and selecting the “Run as Administrator” context menu command.

If you then launch again the xxx.exe debug session inside Microsoft Visual Studio, and examine the Debug tab of the Output windows after the pop up comes, you’ll find an extensive diagnostic that should allow you to identify the missing DLL:

Another option is to directly launch the executable by double-clicking on it. In this case you need to access the kernel debugger console, which is most easily done via the excellent sysinternals DebugView utility.

Anyway the diagnostic you get is complete and leads in the right direction, but unhelpful because difficult to interpret. Here are some tips to help yourself:

  1. read the Dynamic-Link Library Search Order for Desktop Applications documentation
  2. look for DLL names immediately before the lines with the messages “LdrpSearchPath – RETURN: Status: 0xc0000135″

Good luck !

Post Scriptum 1: In practice, the easiest thing to do once you find the missing DLLs, is copying them to the executable location. The second option is to change the working directory (in Visual Studio, do that in Project properties, Configuration Properties, Debugging, Working Directory (be careful to set that for the Configuration / Platform combination you’re actually trying to debug !)

Post Scriptum 2: I’d suggest disabling the sls option when you’re done.

 

Posted in Howtos, Rants | Leave a comment

wordpress POST xmlrpc.php WTF ?

WordPress is widespread because it is such a good blogging platform, CMS, DMS and you-name-it. Unfortunately it means that it becomes the target of many spam and DoS attacks.

The one that is steaming right now it the one that fills your logs with this:
93.174.93.204 wp.libpf.com - [04/Sep/2014:07:33:30 +0000] "POST /xmlrpc.php HTTP/1.0" 403 345 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
212.79.108.47 wp.libpf.com - [04/Sep/2014:07:34:02 +0000] "POST /xmlrpc.php HTTP/1.1" 403 345 "-" "-"
...

To block it, add this to your lighttpd.conf:
url.access-deny = ( "~", ".inc", "xmlrpc.php" )

and restart it:
service lighttpd restart

Posted in Uncategorized | Leave a comment

Automated bug finding with git bisect and boost::test

Here we use boost::test to write test programs for our C++ code, and git as a distributed version control system to track revisions.

One day a test that used to pass stopped working so I decided to try git-bisect to identify the commit  that broke the test. This took me along a journey of shell scripting and git magic that I’d like to share.

First I chdir’d to /tmp and cloned the repo:
cd /tmp
git clone ~/my_repo
cd my_repo

Then I created a shell script that builds the test and runs it, and placed it in the root of the git repository. This is the script, aptly called my_script:
#!/bin/sh
rm -rf bin
cd src
bjam release -j3
cd ../test
bjam release TestModel -j3
cd ..
bin/test/gcc-4.9/release/TestModel --build_info=yes --log_level=test_suite --report_level=detailed
OUT=$?
if [ $OUT -eq 0 ];then
exit 0
else
exit 1
fi

The shell script checks the exit code of the boost::test executable, and if that is different from zero (some tests failed) it returns 1, or 0 if all is OK. This is based on the magic shell variable $? that returns the exit code of the last run command – a trick I always forget and that seems so difficult to find with a search engine.

Now I make the script executable:
chmod u+x my_script

Then I test it here where I am (the HEAD):
./my_script
echo $?

This returns 1, so the test is failing. Now we can start the git-bisect journey:
git bisect reset
git bisect start
git bisect bad

Next we look for a previous commit where the test used to pass; let’s rewind by 20 commits and try there:
git checkout HEAD~20
./my_script
echo $?

Now this passes, so let’s get back to HEAD and find the corresponding commit:
git checkout master
git log --pretty=oneline | head -n 21

The HEAD~20 has a hash of f6aabbe80766a9745447214a89c513de29cbbe70 so let’s mark that as a good one:
git bisect good f6aabbe80766a9745447214a89c513de29cbbe70

and now we’re ready to start:
git bisect run ./my_script

after some time git-bisect returns with:
3c9b0231c2dbef5a7c6a490c877116e2c5bd89c2 is the first bad commit

Done !

Posted in Howtos | Leave a comment

Third LIBPF app preview

This time we have dynamic pages and you can switch between 4 pseudo-servers (switching server changes the app skin). But of course no server is running yet so it’s all fakes (there is a fruit salat configurator you can play around with).

Here is a group photo of the app on our test gizmos:

app on gizmosFor a hands-on session straight into your own device, try this link.

Posted in UI | Leave a comment

Using jshint on debian testing

tl;dr: If you wish to use jshint on debian testing (jessie), do this:

sudo apt-get update
sudo apt-get install nodejs nodejs-legacy
sudo npm install jshint -g
jshint

Long story: this is difficult to get right if you read random posts around. There are flamewars from 2012 about node-js package being obsolete in Debian (not true as of today) and WNPP for jshint is stuck because the package contains some code licensed with a variant of the MIT license considered un-kosher by debian (still true).

Posted in Howtos | Leave a comment