Monday, January 27, 2014

#Reasonator - the need for speed

If Reasonator is to work well and has to continue to work well, it has to scale and be as efficient as possible. Magnus has done a lot of work recently to improve its responsiveness. As it is, it does get its data from elsewhere and it is all about efficiently handling requests for information.

These are three results of the recent improvement in performance. 
Johann Sebastian Bach (Q1339)Wikidata : 18 sec (no images, no related items)
Old Reasonator : 12 sec (14 with images)
New Reasonator : 6 sec (9 sec with images)
Mammillaria (Q311120)Wikidata : 5 sec (no images, no taxon tree, no related items)
Old Reasonator : 8 sec (11 with images)
New Reasonator : 7 sec (10 sec with many more images)
Cambridge (Q350)Wikidata : 7 sec (no images, no location tree, no maps, no related items)
Old Reasonator : 8 sec (9 with images)
New Reasonator : 7 sec (10 sec with many more images)
The data is retrieved from Wikidata using the available APIs. There are two ways to look at that; you either accept what you are given OR you know that WDQ is able to provide the same data but faster. WDQ is as you read on Magnus's blog all of Wikidata but running completely in memory. At start time it is only 868MB RAM in size.

Help is welcome to make Reasonator and WDQ scale and to improve its user experience. Its page view statistics indicate a bight future and therefore an urgent need to consider all of this. Things that are considered are:

  • using puppet for better management and multiplexing the services provided
  • fixing memory leaks
  • packaging WDQ and Reasonator
We expect that Wikidata will be able to cope with us in bringing the best information available to us.
Thanks,
      GerardM


No comments: