Last update:
Tue Dec 24 23:30:29 2024
•
•
•
•
Below is data from Randall Monroe’s Radiation Dose chart normalized to a year:
The spreadsheet with the data can be had here (in open document format (open office)) or in Excel format (don’t know if it renders well in Excel and others, since it was exported from open office).
Tomáš Pospíšek
Below is data from Randall Monroe’s Radiation Dose chart normalized to a year:
The spreadsheet with the data can be had here (in open document format (open office)) or in Excel format (don’t know if it renders well in Excel and others, since it was exported from open office).
Tomáš Pospíšek
A while ago I’ve compiled PostGIS 1.5.2 for PostgreSQL 9.0.
To install it on Ubuntu the following steps are required:
apt-get install python-software-properties
add-apt-repository ppa:ubuntugis/ubuntugis-unstable
add-apt-repository ppa:pitti/postgresql
add-apt-repository ppa:pi-deb/gis
apt-get update
apt-get install postgresql-9.0-postgis
A basic template database can be created with the following commands:
sudo su - postgres
createdb template_postgis
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/postgis.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/spatial_ref_sys.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis_comments.sql
cat <<EOS | psql -d template_postgis
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
REVOKE ALL ON SCHEMA public FROM public;
GRANT USAGE ON SCHEMA public TO public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE
ON TABLE public.geometry_columns TO PUBLIC;
GRANT SELECT, UPDATE, INSERT, DELETE
ON TABLE public.spatial_ref_sys TO PUBLIC;
EOS
To test database creation you can do the following:
createdb --template template_postgis test_gis
psql -d test_gis -c "select postgis_lib_version();"
Ubuntu supports parallel installations of different PostgreSQL versions. So if you have already PostgreSQL 8.x installed, PostgreSQL 9.0 will probably be configured to listen on port 5433. So you have to add the option -p 5433 to each command and to specify the full path for the executables. For example:
/usr/lib/postgresql/9.0/bin/psql -p 5433 -d test_gis -c "select postgis_lib_version();"
A while ago I’ve compiled PostGIS 1.5.2 for PostgreSQL 9.0.
To install it on Ubuntu the following steps are required:
apt-get install python-software-properties
add-apt-repository ppa:ubuntugis/ubuntugis-unstable
add-apt-repository ppa:pitti/postgresql
add-apt-repository ppa:pi-deb/gis
apt-get update
apt-get install postgresql-9.0-postgis
A basic template database can be created with the following commands:
sudo su - postgres
createdb template_postgis
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/postgis.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/spatial_ref_sys.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis_comments.sql
cat <<EOS | psql -d template_postgis
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
REVOKE ALL ON SCHEMA public FROM public;
GRANT USAGE ON SCHEMA public TO public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE
ON TABLE public.geometry_columns TO PUBLIC;
GRANT SELECT, UPDATE, INSERT, DELETE
ON TABLE public.spatial_ref_sys TO PUBLIC;
EOS
To test database creation you can do the following:
createdb --template template_postgis test_gis
psql -d test_gis -c "select postgis_lib_version();"
Ubuntu supports parallel installations of different PostgreSQL versions. So if you have already PostgreSQL 8.x installed, PostgreSQL 9.0 will probably be configured to listen on port 5433. So you have to add the option -p 5433 to each command and to specify the full path for the executables. For example:
/usr/lib/postgresql/9.0/bin/psql -p 5433 -d test_gis -c "select postgis_lib_version();"
A short visit and 7 hours train ride to the OpenLayers code sprint mainly for a presentation at the Swiss MapFish user group meeting in Lausanne, resulted in a new release of the QGIS OpenLayers plugin. The OpenLayers plugin adds WebKit based layers to QGIS and ships with OpenStreetMap-, Google- and Yahoo-Layers.
Changes in this release:
The next planned step is integrating this plugin with the very nice Openlayers Overview plugin from Luiz Motta.
Information for adding your own layers and a bug tracker is now available at hub.qgis.org/projects/openlayers
QGIS server is already known as a full featured, WMS 1.3 compliant map server (see e.g. ETHZ, Linfiniti or 3LIZ).
For the city of Uster, Switzerland, Sourcepole recently extended QGIS server with the possibility to use the print composer via WMS in order to offer printing functionality for web maps. A very nice GeoExt based client can be found at http://gis.uster.ch/webgis/. Andreas Neumann used and extended the GeoExt PrintProvider and PrintExtent classes which allows the user to intuitively select a layout, extent, scale, rotation and resolution for printing. The widget then sends a GetPrint request to QGIS server and a printable PDF is returned.
Now to the technical details: in order to use the print capability of QGIS server, the published project needs to contain one or more print compositions. QGIS server includes information about the available composer templates in the GetCapabilities response:
<WMS_Capabilities>
...
<ComposerTemplates xsi:type="wms:_ExtendedCapabilities">
<ComposerTemplate width="297" height="210" name="Composer 1">
<ComposerMap width="102" height="95" name="map0"/>
<ComposerMap width="133" height="79.87912087912088" name="map1"/>
<ComposerLabel name="hello_world"/>
</ComposerTemplate>
</ComposerTemplates>
...
With a 'GetPrint' request, the client may now request printable output in png or pdf format. The client needs to select an available composer template and needs to pass the selected extents for the composer maps. Additionally, there is the possibility to replace the text of composer labels. An example for a 'GetPrint' request is (note the new 'TEMPLATE' and 'map0:extent' Parameters):
http://localhost/fcgi-bin/qgis_mapserver/qgis_mapserv.fcgi?MAP=/home/marco/geodaten/projekte/composertest.qgs&SERVICE;=WMS&VERSION;=1.3.0&REQUEST;=GetPrint&TEMPLATE;=Composer 1&map0;:extent=693457.466131,227122.338236,700476.845177,230609.807051&BBOX;=693457.466131,227122.338236,700476.845177,230609.807051&CRS;=EPSG:21781&WIDTH;=1467&HEIGHT;=729&LAYERS;=layer0,layer1&STYLES;=,&FORMAT;=pdf&DPI;=300&TRANSPARENT;=true
In detail, the following parameters can be used to set properties for composer maps:To replace text in composer labels, the label needs to have a text id (can be assigned in the print composer user interface of the composer label). The text to insert into the label can be passed in the GetPrint request with an additional parameter =text (of course, the label id should be different to the standard WMS parameters...).
Besides printing, an additional nice feature in the most recent developer version is the possibility to also use relative pathes in published project files. Like this, it's possible to conveniently export a project file recursively together with the corresponding data.
Finally, I'd like to thank the city of Uster and Andreas Neumann for the generous support and the creative ideas during the implementation of the printing functionality.
This is a short series of howtos for and a critique of JMeter v2.4.
The following articles have been done:
Tomáš Pospíšek, 4.1.2011
(This article is part of the JMeter Series)
JMeter’s “HTTP Request Sampler” will read the whole input (the downloaded page, pdf, movie etc.) into its main memory. That means that working with larger (f.ex. movies) files will bring JMeter, the JVM and/or many machines to their memory limits.
If you do not not need to do more than making sure that something gets downloaded, then you can “offload” the download work to an outside process, f.ex. to curl or wget:
A few things to note:
I used the “BeanShell Sampler” module, since both the “BeanShell Pre-” and the “PostProcessors” are only executed when there’s something to process after another sampler produced some data.
As you can see, we are passing a JMeter variable to the script
Aparently JMeter or the BeanShell do not care much about the output of the exec’ed script, thus you won’t see anything inside the “Response data” in the “View Results Tree Sampler”. I don’t know whether that behaveour is a missing feature, a bug or for some unknown reason intended as such.
You can download this JMeter test from here
Tomáš Pospíšek, 30.12.2010
(This article is part of the JMeter Series)
While testing a Rails application there was a situation where a background worker (DJB or background task) would get an order to execute and eventually the completed order would appear on a page.
Thus we needed to wait for the page to be updated and continue the test after.
I’m predefining used variables here - see “We set up our variables” in the Extracting text from a page and using it somewhere else in JMeter article for an exaplanation why.
This step is not strictly necessary in our example here, however if you want to reuse this sequence of steps more than one time - that is call it from different places, then you need to clear the loop variable first.
Since creating these steps was not easy, I’ve put a debug statement in so that I see, when JMeter calls into these steps and to see how the variables are set.
Here we really start the loop.
I’m not sure whether if would be better to use BeanShell commands here instead - using JavaScript to test the loop condition works in any case…
We get the page we want to check.
It’s better - as far as we can - to be sure we force the web application to recreate the page we’re waiting for to change. I’m not sure I’ve done it right here, however it works:
We wait two seconds here before retrying.
You can download this JMeter test from here
Tomáš Pospíšek, 4.1.2011
(This article is part of the JMeter Series)
In the following we’ll do these things:
In more detail:
The idea here being that we predefine the variables, so they would show up in the “Debug Sampler” which makes debugging easier, because you see at each step, whether the variable has the correct value or not. I do predefine the variables with a “non-value” so that I can see immediately whether the variable has been already assigned something during the execution of the test or not.
(The auth token is being used by the web application to prevent cross-site-scripting)
Notice that the extractor is set to extract the string from the HTTP reply-body.
This is because we want to pass the cookie to an outside application, so that it can call the web app from within the same session.
Notice that this time we told the extractor to extract the string from the HTTP reply-headers.
Note that:
Submitting the order will redirect us to the page that’ll show us the resulting order.
The headers contain the location of the resulting order page where we would be redirected to.
Note that:
we are passing the SESSION_COOKIE to curl to be able to download the artifact in the same (potentially authenticated) session.
we also construct the download URL from the ORDER_ID
You can download this JMeter test from here
Tomáš Pospíšek, 4.1.2011
(This article is part of the JMeter Series)
Upon starting JMeter you’ll see two branches: “Test Plan” and “WorkBench”.
“Test Plan” is the place where your tests will live.
What the purpose of “WorkBench” is, is not really clear. It seems to be meant to be a place to do your throw-away experimentation.
The really crucial “trap” of the “WorkBench” is however, that JMeter will throw away whatever you put into the “WorkBench” upon exit. JMeter will not save the contents of the “WorkBench” if you tell it to “Save” your work and you’ll loose whatever is in there. It will only save the contents of the “Test Plan” branch.
So be ware of putting anything in the “Work Bench”. You’re bound to get burned.
Tomáš Pospíšek, 27.12.2010
(This article is part of the JMeter Series)
As tests get larger, or as steps need to be repeated you’ll want to structure your tests into distinct entities - these seem to be called “Modules” in JMeter.
However, there is no “Module” element JMeter. As a “Module” you can however use the “Simple Controller”. It allows you to drop other elements into it and to name them as a whole. I don’t know whether it has additional features such as providing scoping of any kind.
Thus if you need to “call” the same set of steps from different places, then you can place them in a “Simple Controller”. However you’ll need to place that controller somewhere and as such it will get executed in that place and order. You can prevent it being executed by disabling it:
even though being disabled, components inside a Controller still can be called and executed. Therefore, you can use a disabled “Simple Controller” as a repository for reusable components or modules:
On the picture you can see a “module repository” and a call to a specific “Module” inside it.
Tomáš Pospíšek, 30.12.2010
(This article is part of the JMeter Series)
Useful ways to debug JMeter as far as I know:
the “Debug Sampler” will emit “everything that’s known” to JMeter. That output can be displayed in the “View Results Tree Listener”:
have a look at the JMeter log file (which is usually dumped from where you’ve started JMeter)
check the output of one of the listeners such as the “View Results Tree Listener”.
if you need to debug regexes of a “Regular Expression Extractor” then you can have a look at the source code of the page you want to extract a value from inside the “View Results Tree Llistener” and enter the same regular expression in the “Search” field (searching is only implemented in JMeter > v2.4).
In that picture you can see JMeter matching and displaying the regex entered in the search field inside an HTML page retrieved though the “HTTP Client Sampler”.
Tomáš Pospíšek, 1.1.2011
(This article is part of the JMeter Series)
JMeter is a testing tool. It comes accross as a graphical tool, where you can half visually half through text define your test. Its roots seem to lie in web testing - that means testing a website on how long it takes to return pages, how well it does under stress, how well it scales with increasing numbers of parallel requests etc.
However JMeter can be both used non-visually and has extended far beyond web testing.
While working on a test with JMeter 2.3.4 it has had a multitude of stability problems:
That means that you can expect JMeter to hang or crash at any point while developing, executing or saving your tests.
That means that you should really be saving (CTRL-S or Apple-S) very often. On the other hand saving often will increase the likeness of your save file itself being destroyed and your work lost (see last point above). Thus saving is not enough - you need to version your work too, in order to be able to access older versions, that are not damaged.
Running something like this from a Unix command line should be saving revisions of your current work file - you’ll need to be saving your work continuously though:
$ revision=99 $ save_interval=300 # seconds $ work_file="Load Test.jmx" $ while true; do > cp "$work_file" "$work_file.$revision" > echo "Saved "$work_file" under "$work_file.$revision" > sleep $save_interval > if [ "$?" != 0 ]; then break; fi > done
One problem that does not seem to be resolved within JMeter at all is “big” requests. The various Samplers seem allways to put whatever the get from the test target into memory. Thus doing tests on multimedia content with gigabyte sized files will make JMeter run out of memory. There does not seem to be a way to tell JMeter to throw away downloaded content and/or to treat it in flight and not to save it in its entirety.
The online JMeter documentation is brief and is rather just mentioning features than describing them in depth. Also, it’s not easy at all to find ready made examples that demonstrate syntax and finer points of how to use JMeter. When accessing the help coming with JMeter itself the JMeter instance would just hang.
I did a lot of searching the web, without much success and a lot of trying.
What do you do when tests don’t run the way you want or do unexpected things? Then you need a way to debug them. There are three debugging facilities of JMeter, that are not hard to access:
Here’s a snapshot of the Debug Sampler:
And a snapshot of the Debug Sampler displaying JMeter properties inside the “View Results Tree” Listener.
These facilities are very useful, however the various JMeter elements themselves are blackboxes - there seems to be no way to introspect them or “step through” them at runtime. When they “don’t work”, then either they might display something useful inside the JMeter Log or possibly you’ll be able to collect hints of why they might not work after they’ve run through the Debug Sampler results. If both approaches fail, then it’s not clear what else can be done other than pluging into the source code of JMeter itself.
I’d describe working JMeter as “to do easy things is non-trivial and doing hard things is extremely demanding”
The various testing elements, such as “Logic Controllers”, “Listeners” etc. seem to have different scopes and different orders of execution.
“Listeners” for example seem to be global in scope, in that they “see” and can react on whatever happens in the whole of JMeter. They might be possibly limited to a “Thread Group” though, depending on where they are placed - I did not try to find out.
Doing slightly more complex things - such as a loop to repeat a row of steps multiple times, or executing external scripts does not seem to be supported within JMeter itself - one needs to ressort to nearby tools such as the Bean Shell and accordingly to learn the syntax and the working of that tool as well.
Same goes for “Extractors” which refer to a nearby tools that do Perl regexes or XPath matching.
In short to do even rather easy tests one needs to get to know a vast array of tools and syntaxes and how those various tools interact with each other.
All those negative points seem to weight quite heavily and they do. However after quite a lengthy and steep learning curve JMeter as a tool starts showing its strength. Once one has resolved a larger number of problems, adding more steps and functionality to JMeter starts getting easier and one productivity starts to increase.
It’s also fun working with such a complex, versatile and powerful tool and it’s fun to work with a tool that starts from a fresh and different perspective how to solve (visually) problems.
One thing that one gets for “free” with JMeter is the nice reports generated by the various included Listeners. It’s something that will certainly impress management and is useful for gaining a high level understanding of the performance of a web-site.
Other things - like traversing web sites, extracting and matching information might be just as well done with specialised tools coming along with Python, Ruby or command line tools available under Unix.
Ease of automation when using proper scripting languages and their effectiveness and efficiency will be hard to beat given that one allready has a good working knowledge of them.
To me it’s not really clear whether using a scripting language for the task of testing and profiling is not more efficient and more flexible than using JMeter.
However I do reccomend JMeter: it’s fun to work with and it does give a new perspective on how things can be done.
Tomáš Pospíšek, 28.12.2010
For data collection, it is a common situation to work with a laptop or a phone offline in the field. Upon returning to the network, the changes need to be synchronized with the master data source, e.g. a PostGIS database. If several persons are working simultaneously on the same datasets, it is difficult to merge the edits by hand, even if people don’t change the same features.
Therefore, Mathias Walker implemented an offline plugin for QGIS. This plugin automates the synchronisation by copying the content of a datasource (usually PostGIS or WFS-T) to a spatialite database and storing the offline edits to dedicated tables. After being connected to the network again, it is possible to apply the offline edits to the master dataset.
To give the plugin a try, unpack the sources, apply the patch ‘qgissvn.diff’ to a current svn version of QGIS. Then copy the offlineediting folder to $PREFIX/src/plugins and recompile QGIS.
The usage of the plugin is straightforward:
Presumably, the offline editing plugin will be part of the next QGIS version (1.6)
In cartography, it is a frequent operation to set labels to fixed positions, together with the position of the fix point (left/middle/right, Top, Half, Bottom) that is kept constant in case of font change, rotation or zoom. Therefore, three new editing tools to manipulate text labels are now in the QGIS developer version:
All three tools work on the new labeling engine and data defined labeling needs to be enabled for the layer (e.g. x coordinate attribute / y coordinate attribute for the move tool, rotation for the rotate tool). Additionally, the layer needs to be in edit mode. The new tools are well suited to mix fixed label positions and automated label opsitioning in the same or among several layers. If the x- or y attribute value is NULL, the position is set automatically by the pal library (http://pal.heig-vd.ch/). As soon as a position is manipulated by the move label tool, the position is written into the attribute field and the label position for this feature is fixed. So if a layer does not yet have attribute fields for x, y, you could create two new fields of type double (using the buttons in the vector properties dialog or in the attribute table). Initially, all values will be ‘NULL’ and all the label positions set automatically.
There are further plans to improve the user interface. It could be handy to have the properties dialog always open (non-modal), which would allow faster edits of a large number of labels. And a live text rotation preview is planned too. And yes, if someone likes making real icons, this would be highly appreciated (my graphical skills are somewhat limited…).
Finally I’d like to thank the city of Thun (Switzerland) for funding these tools and sharing it with the rest of the FOSSGIS world.
The Sunday night session of the QGIS hackfest resulted in a new release of the Mapfile Tools plugin.
This QGIS plugin allows you to display an UMN Mapserver mapfile in QGIS without running a Mapserver instance. It depends only on Mapscript (apt-get install python-mapscript on Debian/Ubuntu) and allows you to zoom and pan on the mapfile layer.
In release 0.6, an output window has been added, which shows error messages and detailed layer information. This makes it a convenient tool to test your mapfiles.
Marco, Matthias and me spent three days at the QGIS hackfest in Wroclaw (pictures). There I got the time to work on the QGIS globe plugin and made a presentation of the current state.
As soon as the threading branch (Martin Dobias’ Google of Summer project) is merged into trunk, the globe should make its way into trunk as well. In the meantime you can compile the QGIS branch from guthub to test the globe. Thanks Vincent for writing step by step build instructions.
Recently I had to pre-select a Node inside a TreePanel ExtJS widget. I tried many
ways but failed because most of the time when I tried to:
node.select();
that node would not yet be rendered into the browser's DOM and so the
select would fail somewhere inside the extjs.js blob with something like
"this ... undefined ...". What I needed was a "rendered" event, but there
doesn't seem to be such an event for neither TreePanel not TreeNode
or any of their superclasses. Diving into ExtJS code was not really very
helpful because it's a framework that does things through layers and
layers and as such is not trivial to understand quickly.
Thus, the same ole problem with JavaScript as ever: "show me all the events there are". However, surprisingly, in contrast to standard JavaScript, ExtJS has an easy standard way to accomplish this:
Ext.util.Observable.capture( myTree, function(event) {
console.log("got an event in myTree");
console.log(event); });
And quickly I discovered that there indeed is an obscure event that I can,
out of alternatives, missuse to do what I want, which is expandnode.
Aparently, after a node is exapanded, all its children are put into the DOM and seem to be manipulable then. Thus:
/*
* select node with real_id - this is only called once
* after the tree is rendered for the first time.
* After that the listener itself is unregistered.
*/
function select_node(node) {
node.eachChild( function(child) {
if(child.attributes.real_id == real_id ) {
child.select();
categories_panel.un('expandnode', select_node);
}
});
};
myTree.on('expandnode', select_node);
Tomáš Pospíšek
Problem statement:
Which of our strings are not translated yet in our Ruby on Rails app?
Unfortunately there’s no easy way to know for sure. One solution is to log untranslated strings as soon as they appear - the following solution applies to the spree-i18n extension but should be easily adaptable to other contexts.
What we do here is monkey patching I18n#t, to check whether the original I18n#t told us that there’s no translation and log it in that case. Then we return whatever the original I18n#t gave us:
I18n.class_eval do
class << self
alias_method :alias_for_t, :t
end
# make t log missing translations
def self.t( keyz, options = {})
translation = self.alias_for_t( keyz, options )
if translation =~ /translation missing:/
MISSING_TRANSLATION.info translation
end
return translation
end
end
end
To initialize the logger:
class MissingTranslationLogger < Logger
def format_message(severity, timestamp, progname, msg)
"#{msg}\n"
end
end
logfile = File.open('log/missing_translations.log', 'a')
# optional: # logfile.sync = true
MISSING_TRANSLATION = MissingTranslationLogger.new(logfile)
You’ll need to place those two code snipplets in apropriate places. I’ve put the first one in vendor/extensions/site/site_extension.rb and the second one into config/initializers/missing_translation_logger.rb.
Hope somebody’ll find this snippet useful.
Tomáš Pospíšek
