To troubleshoot, sometimes you need to inspect the settings.xml file, and to do that you can use a few good tools.

The first thing you might want to do is looking at which groups are defined. To do that, I use the standard UNIX utilities grep and sed. These are available from any UNIX shell, but also on Windows if you installed Git for Windows (just open the Git bash shell). The command to list the defined groups is:

grep '<group' settings.xml | sed 's/.* id="\([^"]*\)" .*$/\1/g'

If you want to detect duplicates, you can pipe the output of the sed command to sort:

grep '<group' settings.xml | sed 's/.* id="\([^"]*\)" .*$/\1/g' | sort

But for more complex analyses, grep and sed soon hit their limits. The good news is that we can exploit the structure of XML, and use some more XML-specific tool, such as one for running XQuery queries in Nokia Qt SDK, called xmlpatterns.

If you create a XQuery file called groups_id.xq with this code:

<groups>
  {doc("settings.xml")//group[@id=$id]}
</groups>

then this command will filter your settings.xml and print only the group with id equal to R14AR4:

xmlpatterns -param id="R14AR4" group_id.xq

This other XQuery “program” (save as input_tag.xq) will print all input variables linked to a specific LIBPF variable, defined within groups that are enabled:

<inputs>
  {doc("settings.xml")//group[@enabled="true"]/input[TAG=$tag]}
</inputs>

One example of use for the input_tag XQuery is finding the input variables linked to the LIBPF variable S01.Tphase.mdot:

xmlpatterns -param tag="S01.Tphase.mdot" input_tag.xq

You can find more hints on XQuery here.

Here is one such step-by-step guide for adding a new header/source file pair to LIBPF. This applies to the standard Microsoft Visual Studio 2008 project file (kernel.vcproj) shipped with the LIBPF SDK.

• Add the header file to the project, selecting the command Add → New element from the context menu (RMB click) on the Header files folder of the project:
• make sure you select Header file (.h), change the name, and make sure you put it in the include path (so that the compiler will find it):
• In the new, empty header file, copy-paste the standard skeleton for a LIBPF header file:

  /** @file newfile.h
  @brief Contains .....
  @author (C) Copyright 2012 .....
  */
  
  #ifndef LIBPF_NEWFILE_H
  #define LIBPF_NEWFILE_H
  
  // SYSTEM INCLUDES
  
  // PROJECT INCLUDES
  
  // LOCAL INCLUDES
  
  // FORWARD REFERENCES
  
  // code ....
  
  #endif // LIBPF_NEWFILE_H

• Do a search and replace on the file to change the file name from newfile to whatever your file name is;
• Do a search and replace on the file to change the header guard from LIBPF_NEWFILE_H to LIBPF_YOURFILENAME_H;
• Add the source file to the project, selecting the command Add → New element from the context menu (RMB click) on the Source files folder of the project:
• make sure you select C++ file (.cpp), change the name, and make sure you put in the src path where source files belong:
• Copy-paste the standard skeleton for a LIBPF source file:

  /** @file newfile.cc
  @brief Contains .....
  
  @author (C) Copyright 2012 .....
  */
  
  // SYSTEM INCLUDES
  
  // PROJECT INCLUDES
  
  // LOCAL INCLUDES
  #include "newfile.h"
  
  // FORWARD REFERENCES
  
  // code ....

• Finally, do a search and replace on the file to change the file name from newfile to whatever your file name is.

For example, the interactive user sees:

CommonProgramFiles=C:\Program Files\Common Files

but the service sees:

COMMONPROGRAMFILES=C:\Program Files\Common Files

This behavior is consistent across Windows XP, Windows 7 and Windows 8 Consumer Preview. The behavior is likely to have no effects in most cases, since in Windows pretty much everything case-insensitive … but, but there are exceptions to this rule, for example the Registry Redirector is case sensitive. And (what hit us) bjam is case-sensitive when accessing the operating system environment variables with the os.environ command, as some users already found out.

Since the buildslaves run as a service impersonated by a non-privileged user as usual on Windows, so bjam was unable to find the required headers.

The solution was to add this code to out Jamroot.jam:

local commonProgramFiles = [ os.environ "CommonProgramFiles(x86)" ] ;
commonProgramFiles ?= [ os.environ "CommonProgramFiles" ] ;
commonProgramFiles ?= [ os.environ "COMMONPROGRAMFILES(X86)" ] ;
commonProgramFiles ?= [ os.environ "COMMONPROGRAMFILES" ] ;

We tested that in Debian wheezy; install some additional packages:

sudo apt-get install libqt4-dbg libqtscript4-qtbindings

Then clone the git repo:

git clone git://gitorious.org/hooq/hooq.git

enter it, apply the patch which includes the fix mentioned here and here, then qmake and make:

cd hooq
wget http://wp.libpf.com/files/2012/03/0001-patch-GdbInjector-add-a-couple-of-asserts.patch
git apply 0001-patch-GdbInjector-add-a-couple-of-asserts.patch
qmake hooq.pro
make

We have tried it out with UIPF, the User Interface for Process Flowsheeting.

To run, it start hooq/hooq. The user interface appears:

For organizing your tests, Hooq uses the concept of a ‘test set’ – this can be one per application, tests for different areas of the same application, or whatever you choose. Select File -> New test set to create one, fill in the stuff including full path to your debug-mode application:

Now you can type the description for the first test case in the textbox at the bottom and click the “Record” button to open up your application and perform the required actions:

We end recording by closing the application. At the end of the recording session, the test set contains our first test entry, with run and edit buttons:

Running the test now is of little use because it will execute till the end at max speed, possibly missing any asynchronous event and closing the application before we can actually test anything. The idea is that the script obtained by recording should be edited, adding verifications, diagnostic etc. – for the test scripts Hooq uses QtScript, a scripting language based on ECMAScript i.e. JavaScript + a few Qt-specific extensions. By clicking on the edit button you’ll get a script editor with nice synthax highlighting:

If you then click in the margin on the line 17, there’ll then be a breakpoint marker on that line:

If you then click the run button in the toolbar, the script will stop running at that line; at this point, if you click ‘pick property’ in the toolbar, a pop-up “Click on a widget to retrieve its porperties” appears, the cursor becomes a cross and you can then click on any widget in the application being tested, to get a list of its Q_PROPERTYs; for example for the message box in the lower part of UIPF we get:

For example, scroll down to “plainText”, and click fectch. This this will insert the JavaScript code required to fetch the variable from the live application during tests:

objectFromPath("WindowMain-0::QSplitter-1::QTextEdit-1").property("plainText")

This requires you to do something with this value. If on the other hand you click “compare”, it will automagically insert a check into the JavaScript code in the editor:

compare(objectFromPath("WindowMain-0::QSplitter-1::QTextEdit-1").property("plainText"), "* ****************** LIBPF 01.00.0868 [2012/02/26 16:09:12] ******************\n* All rights reserved; do not distribute without permission.\n* Paolo Greppi (C) Copyright 2004-2012 Paolo Greppi libpf.com\n* ****************** Jasper 00.01.0192 [2012/03/08 10:03:37] *****************\n* (C) Copyright 2010-2012 Paolo Greppi, Daniele Bernocco Universita' di Genova\nmain * Define components\nmain * Clean up persistent storage database\nmain * Program exits correctly");

To make sure the variable actually has that value at that point, I also insert a wait statement before:

msleep(1000)
compare(objectFromPath("WindowMain-0::QSplitter-1::QTextEdit-1").property("plainText"), "* ****************** LIBPF 01.00.0868 [2012/02/26 16:09:12] ******************\n* All rights reserved; do not distribute without permission.\n* Paolo Greppi (C) Copyright 2004-2012 Paolo Greppi libpf.com\n* ****************** Jasper 00.01.0192 [2012/03/08 10:03:37] *****************\n* (C) Copyright 2010-2012 Paolo Greppi, Daniele Bernocco Universita' di Genova\nmain * Define components\nmain * Clean up persistent storage database\nmain * Program exits correctly");

Now remove the breakpoint, and run it a few times; it should repeat your input and quit each time, without an error; however, if you change the reference string in the compare statement to something else, then click run, the test will produce an error:

This is more useful when the test hasn’t changed, but when the application itself has (i.e. a regression). Hooq also has a “Run All” feature, to run all the tests in a set unattended, and show a summary at the end.

To turn this tool in an effective automated UI test tool, the following is required:

• fix a few rough spots
• add a CLI command to run selected tests without any access to a display (hidden UI) for running in daemon mode
• add XML output similar to boost::test
• get some help from a JavaScript guru and prepare a few example scripts with wait-on-event conditions (i.e. wait for table to be populated from remote database, wait for disk operation to complete…), identifying controls etc.

D=1-exp(-(d/d’)^n)

and has this appearance:

The name RRSB is used in the tecnical norm DIN 66145 , “Darstellung von Korn-(Teilchen-)größenverteilungen – RRSB-Netz” (“Graphical representation of particle size distributions; RRSB-grid“) which presents a curious grid:

The grid is based on log vs. log-log scales and is such that if the experimental measurements from a sieve analysis are drawn on it, and a linear interpolation is used, the resulting curve has the form of the RRSB equation above.

Indeed that can be rewritten:

ln(1-D)= -(d/d’)^n

now calling Y=ln(1-D) it becomes:

Y= -(d/d’)^n

that’s why it appears like a straight line on the log vs. log-log plot (y = x^n curves are straight lines on a log vs. log plot).

At first sight it is easy to oversee the fact that the y axis is on a log-log scale rather than just a log-scale. If you look at low D values (right axis in the figure above, it looks like a plain log-scale (because for D small ln(1-D) is almost the same as D). But the distance between the decades increases at values of D approaching 1, which hints at its log-log nature.

In non-German-speaking parts of the world and outside of the concrete and coal processing industries, the RRSB curve is more widely known as Weibull distribution. In Microsoft Excel for example you have the function:

WEIBULL(X;Alfa;Beta;Cumulative)

where:

• X is the diameter
• Alfa is the form factor and is the same as n in the expression above
• Beta is the scale factor and is the same as d’ above
• Cumulative can be set to FALSE to obtain the differential probability distribution curve, which is the same as the PSD in particle jargon.

The RRBS i.e. Weibull PSD has been attributed some physical meaning for powders resulting from crushing or grinding processes. It is also used for crystallizates though.

This was fixed in the latest versions of buildbot (0.8.6), but that does not help if one of your testing machines (one of the buildslaves) has an unsupported combination of buildbot / git. Here is how to fix that in Debian 7.0 (Wheezy) and in Apple Mac OSX 10.7 (Lion) with Macports.

Debian 7.0 (Wheezy)

You need to pin the git package to a version supported by the current buildbot, as follows:

apt.get purge git
cd /etc/apt/preferences.d
cat > git-pinning
Package: git
Pin: version 1:1.7.2.5-3
Pin-Priority: 1001
apt-get install git

Apple Mac OSX 10.7 (Lion) with macports

You need to apply the required patch from here to buildbot:

cd /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/buildslave/commands
sudo vi git.py
...

Now force compile the modified code as root (the user impersonating buildbot will not have the privileges required to write in this directory) :

sudo /opt/local/bin/python2.7
<enter password>
import git
^D

Finally stop / restart the buildbot slave to pick up the updated code

cd ~/buildbot_home
buildslave stop .
buildslave start .

Highlights:

1. Change the uipf.pro file adding this (we don’t use QtDeclarative):

   greaterThan(QT_MAJOR_VERSION, 4) {
     QT += widgets
   }

2. Fixing #includes with the provided fixqt4headers.pl Perl script resulted in compilation failures under Qt 4 since the QtWidgets header does not exist; trick:

   #if (QT_VERSION < 0x050000)
     #include <QtGui>
   #else
     #include <QtWidgets>
   #endif

   Note: The best practice would be to get rid of the Qt* headers and just include the really required classes; chase them before you start with:

   grep 'include <Qt' src/*cc include/*h

3. One specific issue with UIPF was using the Q_WS_XX macros to conditionally compile for Windows (QT_WS_WIN), OSX (QT_WS_MAC) and Linux (QT_WS_X11);  these do not exist anymore (everything is Q_WS_QPA after the Lighthouse merge)  so one needs to switch on the Q_OS_XX macros:  QT_OS_WIN for Windows, QT_OS_MAC for OSX and QT_OS_LINUX for Linux, see this post. The solution was:

   sed -i 's/Q_WS_X11/Q_OS_LINUX/g' *cc
   sed -i 's/Q_WS/Q_OS/g' *cc

Mandatory screenshot (the fonts look ugly because it did not pick up the Oxygen theme for some reason):

1. Get OpenSSL 1.0.1
2. Get the latest package of git for windows (this includes perl which we need)
3. Open the Visual Studio 2011 Developer Command Prompt
4. Launch sh from git-msys (look for the your correct command by inspecting the Git bash link):

   C:\Windows\SysWOW64\cmd.exe /c ""C:\Program Files (x86)\Git\bin\sh.exe" --login -i"

5. Move to the folder where you have extracted the openssl source archive

   cd
   cd Desktop/openssl-1.0.1/

6. Run autoconfig and prepare make files; ignore the prefix; the sample below only builds suffort for a few ciphers: aes md5 rc4 rsa sha mac md2

   perl Configure VC-WIN32 -DOPENSSL_NO_HW no-asm no-static-engine no-bf no-cast no-des no-dh no-dsa no-mdc2 no-rc2 no-rc5 no-capieng --prefix=c:/some/openssl/dir

7. Create a shell script to emulate the ms/do_ms.bat batch file:

   cat > ms/do_ms.sh
   perl util/mkfiles.pl >MINFO
   perl util/mk1mf.pl no-asm VC-WIN32 > ms/nt.mak
   perl util/mk1mf.pl dll no-asm VC-WIN32 > ms/ntdll.mak
   perl util/mkdef.pl 32 libeay > ms/libeay32.def
   perl util/mkdef.pl 32 ssleay > ms/ssleay32.def
   ^D

8. Launch the shell script you just created

   ms/do_ms.sh

9. Build the static and dynamic libraries:

   nmake -f ms/nt.mak

10. At the end of the process copy the include files openssl-1.0.1/inc32/openssl as openssl to your preferred include path, then copy the static library out32/libeay32.lib to your preferred library path.

The correlation is presented in the form of a plot, which makes its application in a computer system unpractical:

In Perry cit. an analytical expression for these curves is quoted from Lygeros and Magoulas, Hydrocarbon Proc. 65(12), 43 (1986), but the fit is not accurate; also the derivative of the capacity parameter w.r.t. the flow parameter, which in the plot is positive at low values of the latter, when computed with the analytical expression is negative.

The Fair plot can be digitized using the excellent utility ScanIt. The resulting digitized plot can be fitted using gnuplot to this expression:

h(x,y) = (((a1*y+a2)*log(x) + (b1*y+b2))*log(x) + (c1*y+c2))*log(x) + (d1*y+d2)

which returns the capacity parameter (m/s) if it is supplied x, the flow parameter (adimensional) and y, the tray spacing (mm). Log is the natural logarithm.

The numerical values for the coefficients are:

The expression is smooth and reproduces the original plot both qualitatively and quantitatively:

The deviation in term of graph abscissa are on average 2.9% on all the sampled points (384). The curve at tray spacing = 600 mm has a marger error (7.1%); if that curve is excluded, the average error on the remaining 330 sample points is 2.2%.

= d^2*π/8

= (d^2*π/8) / (p^2*sqrt(3)/2) = d^2/p^2 * (π/2/sqrt(3)).

The constant π/2/sqrt(3) = 0.906899682119 so Q.E.D.