Sunday, June 12, 2005

qsort madness

qsort() (redhat glibc 2.3.2-95.30) attempts to be clever with memory by switching to a different algorithm for large inputs. On my computer, the threshold is at 131459071-131459072 for 4-byte integers. Performance abruptly degrades at the threshold. It also has the unfortunate bug-hiding effect that programs can work properly with the small algorithm and fail when inputs get large enough. Finally, at large inputs, the C++ STL implementation becomes comparable. Here are some counts to the number of calls to the comparison function for an array initialized by a[0]=1 a[n+1]= a[n]*1664525+1013904223

qsort
 131459070 easy  0        hard 3333417997
 131459071 easy  0        hard 3333417454
 131459072 easy  50294519 hard 4063553331
 268435456 easy 102684870 hard 8680917338

C++ STL sort()
 131459070 easy 15139169 hard 4345836000
 131459071 easy 15141479 hard 4316599909
 131459072 easy 15139606 hard 4470743742
 268435456 easy 30917419 hard 9444799480

The columns give the number of "easy" comparisons (pa==pb for qsort; a==b for sort) and "hard". The PRNG has period 2^32 so every number is unique.

2 comments :

Ken said...

for qsort():

0 easy 0 hard 0
10000000 easy 0 hard 216266939
20000000 easy 0 hard 452530964
30000000 easy 0 hard 696576195
40000000 easy 0 hard 945069519
50000000 easy 0 hard 1197223512
60000000 easy 0 hard 1453143663
70000000 easy 0 hard 1711135054
80000000 easy 0 hard 1970131119
90000000 easy 0 hard 2231382748
100000000 easy 0 hard 2494462575
110000000 easy 0 hard 2759641755
120000000 easy 0 hard 3026284797
130000000 easy 0 hard 3294263116
140000000 easy 53557409 hard 4398792114
150000000 easy 57386544 hard 4749438860
160000000 easy 61209168 hard 5061160284
170000000 easy 65023816 hard 5347417554
180000000 easy 68860782 hard 5712033995
190000000 easy 72677313 hard 5970629940
200000000 easy 76511510 hard 6316372409
210000000 easy 80324822 hard 6654795019
220000000 easy 84159849 hard 6990509557
230000000 easy 87983753 hard 7369057633
240000000 easy 91804339 hard 7716429314
250000000 easy 95630083 hard 8118864948
260000000 easy 99469465 hard 8367556145
270000000 easy 103284265 hard 8691906444
280000000 easy 107107561 hard 8991239135
290000000 easy 110932808 hard 9264248398
300000000 easy 114756199 hard 9775734853
310000000 easy 118586519 hard 10163754332
320000000 easy 122398915 hard 10483599861
330000000 easy 126226608 hard 10842680309
340000000 easy 130058013 hard 11107292579
350000000 easy 133877166 hard 11526705300
360000000 easy 137707433 hard 11963182352
370000000 easy 141534941 hard 12088337808
380000000 easy 145358067 hard 12412918837
390000000 easy 149188828 hard 12798650514
400000000 easy 153009195 hard 13104420799

Ken said...

For sort:

0 easy 0 hard 0
10000000 easy 1151767 hard 282554726
20000000 easy 2304748 hard 589526013
30000000 easy 3455731 hard 924275231
40000000 easy 4606066 hard 1226293151
50000000 easy 5758404 hard 1564652361
60000000 easy 6912340 hard 1869122367
70000000 easy 8060471 hard 2282802918
80000000 easy 9214346 hard 2543726744
90000000 easy 10366564 hard 2996897127
100000000 easy 11520599 hard 3200915600
110000000 easy 12670059 hard 3635538614
120000000 easy 13820781 hard 3903521335
130000000 easy 14975352 hard 4312778135
140000000 easy 16125803 hard 4746579416
150000000 easy 17277596 hard 5004956704
160000000 easy 18428672 hard 5358716541
170000000 easy 19584667 hard 5717276066
180000000 easy 20733252 hard 6030676079
190000000 easy 21887525 hard 6484111044
200000000 easy 23036937 hard 6691532070
210000000 easy 24186719 hard 7070777566
220000000 easy 25337157 hard 7436042572
230000000 easy 26488940 hard 7749227942
240000000 easy 27644096 hard 8335895866
250000000 easy 28790194 hard 8742045259
260000000 easy 29940197 hard 8851983289
270000000 easy 31097838 hard 9218981717
280000000 easy 32250860 hard 9582046792
290000000 easy 33403274 hard 10070285904
300000000 easy 34552422 hard 10248699922
310000000 easy 35702344 hard 10566278981
320000000 easy 36854509 hard 11051827825
330000000 easy 38008769 hard 11679951941
340000000 easy 39157546 hard 11771088420
350000000 easy 40312709 hard 12059960318
360000000 easy 41463815 hard 12359345426
370000000 easy 42616997 hard 12944667851
380000000 easy 43767031 hard 13676569358
390000000 easy 44911934 hard 13855111796
400000000 easy 46069745 hard 14302871543