Execution (Speed) Tests
Finally we get to the interesting part of these series of blog posts. In the previous post, we looked at compilation speed and EXE size for the Hello World project from the introduction. In this section, we are going to look at the execution speed for the Hello World project in Delphi 2010 to Delphi XE6, for Win32, Win64 (Delphi XE2-XE6 only). Next week, we will finish the Win64 FMX tests and then also show OSX (Delphi XE2-XE6 only), iOS (Delphi XE4-XE6 only), and Android (Delphi XE5-XE6 only). The Hello World project fills a TListBox or a TMemo with a number of strings with every click of a button. In our speed tests, we tested adding the string “Hello World” with a number appended after it. The button click adds 10, 100, 1000, or 10000 strings at a time. We are also going to test adding strings inside a BeginUpdate/EndUpdate, which should be significantly faster unless something has been broken. Note that the charts display the execution time on the y-axis in milliseconds. The number of strings added as well as if BeginUpdate/EndUpdate were used are on the x-axis. Except for the VCL charts, the y-axis is logarithmic as the amount of time to add strings goes up exponentially with FMX applications.
Hello World, VCL, Win32
The first test is with the VCL version of Hello World in Win32. Both the TListBox and TMemo pass their control handling to the underlying Windows control so unless Embarcadero broke something, we should see little change between the different versions of Delphi. And that is what we see. Every version of Delphi is able to fill a TListBox with up to 10000 strings in under a second. Note that using BeginUpdate and EndUpdate methods speed up the execution by over 30x! Every version of Delphi is able to fill a TMemo with up to 10000 strings in just around 6 seconds with the BeginUpdate/EndUpdate tests finishing in a sixth of the time.
Hello World, VCL, Win64
For the 64-bit Hello World application, we can only test with Delphi XE2 through XE6. There are also no significant execution speed differences between the Delphi versions in Win64. It is, however, gratifying to see that the promise of Win64 Delphi applications running native in a Windows 64-bit OS and 64-bit CPU is realized as the Win64 versions run around 10-25% faster. Even though 64-bit applications are larger and consume more memory, it can be worth it in order to see these speed increases.
Hello World, FMX, Win32
From here on out, we are going to be testing FMX applications. Everything… every pixel, every string, every button… is drawn by the FMX library and does not use OS controls. It is expected that the FMX applications will be, not only bigger (as we saw in the last post as all the drawing and interaction code must be compiled into the application), but may also draw GUI controls slower. Not only can Delphi applications not count on using OS native controls, but in general FMX controls are much, much richer than their VCL counterparts. The FMX TListBox implementation will be slower than the VCL implementation for this reason. However, this blog post is about exploring the execution differences between the various Delphi versions. What we hope to see is that the execution speed will improve with each new version of Delphi, especially with the latest Delphi XE6 and its mantra of “Quality, Performance, and Stability.”
(Update: Note the FMX charts have a logarithmic y-axis for execution time. We used this because the execution times vary so greatly between a small number of items and a large number of items. Filling a TListBox or a TMemo with 10000 items would swamp out the other results and make them appear tiny and indistinguishable.)
With the Win32 version of the FMX Hello World application, at small number of items, the XE2 version is fastest using the TListBox. Surprisingly, at higher numbers of items, Delphi XE3 is fastest. Delphi XE6 manages to be faster than Delphi XE4 and XE5 when not using the BeginUpdate/EndUpdate methods. Using BeginUpdate and EndUpdate, Delphi XE6 is slower than every prior version until we get up to 10000 items where Delphi XE2 is the slowest. (Update: I forgot to mention an important point about Delphi XE2 TListBox performance: the Clear method is broken in XE2. It doesn’t seem to use BeginUpdate/EndUpdate internally so emptying the list goes exponential as well causing any use of Delphi XE2’s TListBox to be very painful)
Comparing FMX execution time to their VCL counterparts, the TListBox is much slower (there is a reason that Embarcadero recommends using TListView for lots of items). At its best (100 items using BeginUpdate and EndUpdate), the FMX TListBox is 3x (XE5) to 6.76x (XE6) times slower. Even using the BeginUpdate and EndUpdate methods, the FMX TListBox is much slower when adding 10000 items, from 72x (XE3) to a staggering 2258x (XE2, without BeginUpdate/EndUpdate) slower.
Filling a TMemo, the story is completely different: Delphi XE6 is faster than every other version of Delphi except XE4 when not using BeginUpdate/EndUpdate, and faster than all other versions of Delphi when using BeginUpdate/EndUpdate (even the VCL versions)! Delphi XE5 is the big loser when the BeginUpdate/EndUpdate methods are not used. If they are used, Delphi XE3 has the slowest FMX TListBox.
As stated, Delphi XE6 FMX TMemo is even faster than the VCL TMemo, even without using BeginUpdate/EndUpdate; however, the comparison is not fair as the VCL TMemo draws every string as it is added while the FMX TMemo does not. As anyone who has used the FMX TMemo can attest, the FMX TMemo draws very slowly, especially compared to the VCL TMemo.
Coming up next week…
Next week, we will look at the Win64, OSX, iOS, and Android speed comparisons. I apologize that you cannot look at them this week, but the FMX versions of the applications can take a frustratingly long time as has been shown. I am still running speed tests and will report more when I have finished. For now, Happy CodeSmithing!