Changes between Version 7 and Version 8 of Processing/lagdevelopersfaq


Ignore:
Timestamp:
Jul 5, 2012, 6:51:02 PM (12 years ago)
Author:
jaho
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Processing/lagdevelopersfaq

    v7 v8  
    11= LAG Developers FAQ =
    2 [#q1 1. What is LAG?] \\
    3 [#q2 2. What are major components of LAG?] \\
    4 [#q3 3. How do I edit the GUI?] \\
    5 [#q4 4. What are LAS files?] \\
    6 
    7 [#q4.1 4.1 What's the deal with LAS 1.3 files?] \\
    8 
    9 [#q5 5. What do I need to know about laslib?] \\
    10 [#q6 6. How do I make laslib a shared library?] \\
    11 [#q7 7. Why is LAG's code so messy?] \\
    12 [#q8 8. How can I make LAG's code better?] \\
    13 [#q9 9. Where can I learn good C++ coding practices?] \\
    14 [#q10 10. Who should I ask for help?] \\
    15 [#q11 11. How does Quadtree work?] \\
    16 [#q12 12. How is Quadtree structured?] \\
    17 [#q13 13. What is the reason behind subbuckets?] \\
    18 [#q14 14. How does caching work?] \\
    19 [#q15 15. What is the reason behind data compression?] \\
    20 [#q16 16. What are LasSaver and LasLoader classes in the quadtree?] \\
    21 
    22 [#q17 17. How is LAG's code structured?] \\
    23 [#q18 18. How does loading of files work?] \\
    24 [#q19 19. How does saving of files work?] \\
    25 [#q21 21. How is LAS 1.3 point format handled?] \\
    26 [#q22 22. How does rendering work?] \\
    27 
    28 [#q23 23. What are main ideas for further LAG development?] \\
    29 [#q24 24. What are the unfinished development stuff?] \\
    30 [#q25 25. What is lag memory issue?] \\
    31 [#q26 26. What are some major issues with LAG that need fixing?] \\
    32 [#q27 27. What are some additional features that can be added to LAG?] \\
    33 [#q28 28. What tools are there to help me with LAG development?] \\
    34 [\q29 29. How do I use profiling tools to optimise code?] \\
    35 
    36 [#q5 Question X?] \\
    37 [#q5 Question X?] \\
    38 [#q5 Question X?] \\
    39 [#q5 Question X?] \\
    40 
    41 == 1. What is LAG? [=#q1] ==
     2[#q1 1. General] \\
     3[#q1.1 1.1 What is LAG?] \\
     4[#q1.2 1.2 What are major components of LAG?] \\
     5[#q1.3 1.3 Why is LAG's code so messy?] \\
     6[#q1.4 1.4 How can I make LAG's code better?] \\
     7[#q1.5 1.5 Where can I learn good C++?] \\
     8[#q1.6 1.6 Who can I ask for help?] \\
     9
     10[#q2 2. Laslib] \\
     11[#q2.1 2.1 What is laslib?] \\
     12[#q2.2 2.2 What are LAS files?] \\
     13[#q2.3 2.3 What's the deal with LAS 1.3 files?] \\
     14[#q2.4 2.4 What do I need to know about laslib?] \\
     15[#q2.5 2.5 How do I make laslib a shared library?] \\
     16
     17[#q3 3. Quadtree] \\
     18[#q3.1 3.1 How does Quadtree work?] \\
     19[#q3.2 3.2 How is Quadtree structured?] \\
     20[#q3.3 3.3 What is the reason behind subbuckets?] \\
     21[#q3.4 3.4 How does caching work?] \\
     22[#q3.5 3.5 What is the reason behind data compression?] \\
     23[#q3.6 3.6 What are LasSaver and LasLoader classes in the quadtree?] \\
     24
     25[#q4 4. LAG] \\
     26[#q4.1 4.1 How is LAG's code structured?] \\
     27[#q4.2 4.2 How do I edit the GUI?] \\
     28[#q4.3 4.3 How does loading of files work?] \\
     29[#q4.4 4.4 How does saving of files work?] \\
     30[#q4.5 4.5 How is LAS 1.3 point format handled?] \\
     31[#q4.6 4.6 How does rendering work?] \\
     32
     33[#q5 5. Further Development] \\
     34[#q5.1 5.1 What are main ideas for further LAG development?] \\
     35[#q5.2 5.2 What are the unfinished development stuff?] \\
     36[#q5.3 5.3 What is lag memory issue?] \\
     37[#q5.4 5.4 What are some major issues with LAG that need fixing?] \\
     38[#q5.5 5.5 What are some additional features that can be added to LAG?] \\
     39[#q5.6 5.6 What tools are there to help me with LAG development?] \\
     40[#q5.7 5.7 How do I use profiling tools to optimise my code?] \\
     41
     42== 1. General [=#q1] ==
     43
     44=== 1.1 What is LAG? [=#q1.1] ===
    4245
    4346LAG stands for [http://en.wikipedia.org/wiki/LIDAR LiDAR] Analysis GUI. It is a software which provides graphical user interface for processing and visualisation of LiDAR [http://en.wikipedia.org/wiki/Point_cloud point cloud] data. LAG is written in C++ and uses Gtk ([http://www.gtkmm.org/en/ gtkmm]) for its GUI, and [http://www.opengl.org/ OpenGL] for rendering.
    4447
    45 == 2. What are major components of LAG [=#q2] ==
    46 
    47 LAG consist of the main GUI application and a Lidar Quadtree library used for storing and querying the data.
    48 
    49 == 3. How do I edit the GUI? [=#q3] ==
    50 
    51 The GUI layout is stored in //lag.ui// [http://glade.gnome.org/ Glade] file which is in plain xml. This allows editing the interface without the need for recompiling the source code every time a change is made. The //lag.ui// file can be opened in Glade User Interface Designer which can by run with //glade-3// command.
    52 
    53 The interface of the application is created from xml using Gtk::Builder object, which is instantiated in lag.cpp and then passed by Glib::RefPtr to each of the classes that handle the GUI. All of these classes are located under src/ui folder and each of them has two common methods: //void load_xml(builder)// and //void connect_signals()// which are called in their constructors.
    54 
    55 The //load_xml(const Glib::RefPtr<Gtk::Builder>& builder)// method gets widgets from the builder file and assigns them to the class members (pointers). Widgets instantiated by Gtk::Builder have to be deleted manually. In fact, according to documentation, this only applies to top-level widgets (windows and dialogs), but LAG's authors seem to prefer to delete every single one anyway.
    56 
    57 The //connect_signals()// method is responsible for connecting various input signals to class methods. An example lifetime of a Gtk Widget then looks like this:
    58 
    59 {{{
    60 #!div style="font-size: 100%"
    61   {{{#!c++
    62   // Declare a pointer to the widget (as class member)
    63   class MyClass {
    64     Gtk::Button* my_button;
    65     void on_my_button_clicked();
    66   }
    67 
    68   // Load the widget from xml (in MyClass::load_xml(const Glib::RefPtr<Gtk::Builder>& builder))
    69   builder->get_widget("mybutton", my_button);
    70 
    71   // Connect some method to it (in MyClass::connect_signals())
    72   my_button->signal_clicked().connect(sigc::mem_fun(*this, &MyClass::on_my_button_clicked));
    73 
    74   // Delete the widget (in the class destructor)
    75   delete my_button;
    76   }}}
    77 }}}
    78 
    79 Modifying the GUI is just as simple as adding new widgets to the Glade file and then handling them in the ui classes.
    80 
    81 == 4. What are LAS files? [=#q4] ==
    82 
    83 LAS is a binary, public file format designed to hold LiDAR point data. An alternative to LAS are ASCII files, which are however much less efficient in terms of both processing time and file size. \\
     48=== 1.2 What are major components of LAG [=#q1.2] ===
     49
     50LAG consist of the main GUI application and the Lidar Quadtree library used for storing and querying the data. It also uses laslib library for handling LAS files.
     51
     52=== 1.3 Why is LAG's code so messy? [=#q1.3] ===
     53
     54The lag has been in development since 2009 and there were at least four different people working on it, each with different coding style, ideas and barely any supervision. It is now much better then it used to be, but it's easy to see that different parts have been written by different people. There also still seems to be a lot of temporary solutions which were never implemented properly, since they worked at the time. //
     55
     56=== 1.4 How can I make LAG's code better? [=#q1.4] ===
     57
     58Think before you code. Take users of your classes into consideration and try to design easy to use interfaces. Do some refactoring if you think it's going to make things easier to maintain, and to understand the code better. Use consistent coding style and comments. Try to learn good coding practices before you start writing the code (eg. implementing const correctness from the start is much easier then adding it once everything has been written). Profile your code instead of relying on what seems faster.
     59
     60=== 1.5 Where can I learn good C++? [=#q1.5] ===
     61
     62Whether you already know C++, come from C or are just starting to learn, it is a good idea to familiarise yourself with good programming practice before writing production code. Assuming you already know the basics of the language I highly recommend these two sources:
     63
     64[http://www.parashift.com/c++-faq-lite/ C++ FAQ] - every C++ programmer should read this at least once. It's well written, to the point and covers a lot of things from OO design to freestore management.
     65
     66[http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml Google C++ Style Guide] - you don't have to blindly follow it, but it should give you an idea of what a good coding style looks like.
     67
     68=== 1.6 Who can I ask for help? [=#q1.6] ===
     69
     70One problem with LAG is that it has been entirely developed by students, so the only people that know how it really works are most likely gone (thus this FAQ). For general help with programming you should probably ask Mike Grant (he knows a bit about LAG) or Mark Warren as they seem to be the best programmers in ARSF. They also tend to be quite busy though so if you don't feel like bugging them [http://stackoverflow.com/ Stack Overflow] is always a good place to look for help.
     71
     72== 2. Laslib [=#q2] ==
     73
     74=== 2.1 What is Laslib [=#q2.1]
     75
     76[http://www.cs.unc.edu/~isenburg/lastools/ Laslib] is a library for handling various LiDAR file formats including LAS. LAG makes use of laslib for loading and saving points.
     77
     78=== 2.2 What are LAS files? [=#q2.2] ===
     79
     80LAS is a binary, public file format designed to hold LiDAR points data. An alternative to LAS are ASCII files, which are however less efficient in terms of both processing time and file size. \\
    8481
    8582LAS files consist of several parts: Public Header Block, Variable Length Records, Point Records and in format 1.3 and later Waveform Records. \\
    … …  
    9087[http://www.asprs.org/Committee-General/LASer-LAS-File-Format-Exchange-Activities.html http://www.asprs.org/Committee-General/LASer-LAS-File-Format-Exchange-Activities.html]
    9188
    92 == 5. What do I need to know about laslib? [=#q5] ==
    93 
    94 [http://www.cs.unc.edu/~isenburg/lastools/ Laslib] is a library for handling various LiDAR file formats including LAS. LAG makes use of laslib for loading and saving points.
     89=== 2.3 What's the deal with LAS 1.3 files [=#q2.3] ===
     90
     91In addition to discrete point records LAS 1.3 files also hold full waveform data which is quite huge in volume. The point records have also been modified to include a number of new attributes related to waveform data. \\ \\
     92
     93=== 2.4 What do I need to know about laslib? [=#q2.4] ===
    9594
    9695There is no official documentation for laslib API, however it is a fairly well written C++ which provides an easy to use interface. The best way to learn laslib is to study the source code of programs included in lastools.
    … …  
    158157}}}
    159158
    160 == 6. How do I make laslib a shared library? [=#q6] ==
     159=== 2.5 How do I make laslib a shared library? [=#q2.5] ===
    161160
    162161It seems that laslib is mainly developed for Windows users so there are no targets for shared libraries in the Makefiles by default. At the same time shared libraries are needed by LAG to work correctly. To fix this you're going to need to modify the Makefiles and add -fPIC option to the compiler. You'll also have to change the name of the library from laslib to //liblaslib// for the linker to detect it.
    … …  
    198197You're not likely to have to alter this part and if a newly downloaded version of laslib fails to build with modified Makefiles, the first thing to check is if OBJ_LAS (the list of object files) hasn't changed since the previous release.
    199198
    200 == 7. Why is LAG's code so messy? [=#q7] ==
    201 
    202 The lag has been in development since 2009 and there were at least four different people working on it, each with different coding style, ideas and barely any supervision. It is now much better then it used to be, but it's easy to see that different parts have been written by different people. There also still seems to be a lot of temporary solutions which were never implemented properly, since they worked at the time. //
    203 
    204 == 8. How can I make LAG's code better? [=#q8] ==
    205 
    206 Think before you code. Take users of your classes into consideration and try to design easy to use interfaces. Do some refactoring if you think it's going to make things easier to maintain, and to understand the code better. Use consistent coding style and comments. Try to learn good coding practices before you start writing the code (eg. implementing const correctness from the start is much easier then adding it once everything has been written).
    207 
    208 == 9. Where can I learn good C++? [=#q9] ==
    209 
    210 Whether you already know C++, come from C or are just starting to learn, it is a good idea to familiarise yourself with good programming practice before writing production code. Assuming you already know the basics of the language I highly recommend these two sources:
    211 
    212 [http://www.parashift.com/c++-faq-lite/ C++ FAQ] - every C++ programmer should read this at least once. It's well written, to the point and covers a lot of things from OO design to freestore management.
    213 
    214 [http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml Google C++ Style Guide] - you don't have to blindly follow it, but it should give you an idea of what a good coding style looks like.
    215 
    216 == 10. Who can I ask for help? [=#q10] ==
    217 
    218 One problem with LAG is that it has been entirely developed by students, so the only people that know how it really works are most likely gone (thus this FAQ). For general help with programming you should probably ask Mike Grant (he knows a bit about LAG) or Mark Warren as they seem to be the best programmers in ARSF. They also tend to be quite busy though so if you don't feel like bugging them [http://stackoverflow.com/ Stack Overflow] is always a good place to look for help.
    219 
    220 == 11. How does Quadtree work? [=#q11] ==
     199
     200== 3. Quadtreee [#q3] ==
     201
     202== 3.1 How does Quadtree work? [=#q3.1] ==
    221203
    222204The Lidar Quadtree library provides a [http://en.wikipedia.org/wiki/Quadtree quadtree] data structure for storing and indexing points. It also contains a caching mechanism for loading and unloading points from memory.
    … …  
    226208When retrieving points a bounding box is passed to the quadtree and all nodes which intersect with it are queried for the points. It's quite simple really.
    227209
    228 == 12. How is quadtree's code structured? [=#q12] ==
     210== 3.2 How is quadtree's code structured? [=#q3.2] ==
    229211
    230212Quadtree.cpp
    … …  
    243225  A class to keep track of how many points are currently loaded into memory.
    244226
    245 === 13. What is the reason behind subbuckets? [=#q13] ===
     227=== 3.3 What is the reason behind subbuckets? [=#q3.3] ===
    246228
    247229The PointBucket class may hold copies of the same point in several arrays called sub-buckets. There are two parameters which control of how many sub-buckets are created: Resolution Depth and Resolution Base. The Resolution Depth determines the number of sub-buckets for each bucket and Resolution Base determines the number of points at each level by specifying the interval between included points based on a formula of Resolution Base^(Level - 1)^. For example the default LAG values of 5 and 4 create 4 sub-buckets per bucket: one containing every point (5^0^), second containing every 5th point (5^1^), third containing every 25th point (5^2^) and fourth with every 125th point (5^3^). //
    … …  
    249231Because each resolution level effectively slowers the quadtree (some points need to be inserted n times into n arrays) for applications other then LAG, which make no use of sub-buckets Resolution Depth and Resolution Base should be set to 1.
    250232
    251 == 14. How does caching work? [=#q14] ==
     233=== 3.4 How does caching work? [=#q3.4] ===
    252234
    253235Upon creation of the quadtree the user specifies a maximum number of points to be held in cache.
    254236
    255 == 15. What is the reason behind data compression? [=#q15] ==
     237=== 3.5 What is the reason behind data compression? [=#q3.5] ===
    256238
    257239To make caching faster. //
    258240PointBucket uses [http://www.oberhumer.com/opensource/lzo/ lzo compression] to compress buckets before writing them to disk. It may seem like a sub-optimal solution but, since quadtree's performance is mainly IO bound, the compression time is still much lower then reading/writing to the hard drive. Since the volume of data gets smaller thanks to compression in the end it speeds up IO operations which are the major bottleneck.
    259241
    260 == 16. What are LasSaver and LasLoader classes in the quadtree? [=#q16] ==
    261 
    262 These should be removed at some point together with geoprojectionconverter. They are currently there only for backwards compatibility with other programs that use the quadtree (classify_las).
    263 From design point of view these classes should not be a part of the quadtree which should only serve as a data container. In practice, when these classes were used, making any changes in LAG regarding saving or loading files required alterations to the quadtree which was less then convenient. Now that loading and saving has been moved to separate classes inside LAG it is much more maintainable.
    264 
    265 === 17. How is LAG's code structured? [=#q17] ===
     242=== 3.6 What are LasSaver and LasLoader classes in the quadtree? [=#q3.6] ===
     243
     244These should be removed at some point together with geoprojectionconverter. They are currently there only for backwards compatibility with other programs that use the quadtree (classify_las). \\ \\
     245
     246From design point of view these classes should not be a part of the quadtree which should only serve as a data container. In practice, when these classes were used, making any changes in LAG regarding saving or loading files required alterations to the quadtree which was less then convenient. Now that loading and saving has been moved to separate classes inside LAG it is much more maintainable. Points inspection. \\
     247
     248== 4. LAG [=#q4] ==
     249
     250=== 4.1 How is LAG's code structured? [=#q4.1] ===
    266251
    267252lag.cpp
    … …  
    298283  These classes represent top-level interface elements (windows and dialogs) and are responsible for connecting the UI to signal handlers.
    299284
    300 === 18. How does loading of files work? [=#q18] ===
     285=== 4.2 How do I edit the GUI? [=#q4.2] ===
     286
     287The GUI layout is stored in //lag.ui// [http://glade.gnome.org/ Glade] file which is in plain xml. This allows editing the interface without the need for recompiling the source code every time a change is made. The //lag.ui// file can be opened in Glade User Interface Designer which can by run with //glade-3// command.
     288
     289The interface of the application is created from xml using Gtk::Builder object, which is instantiated in lag.cpp and then passed by Glib::RefPtr to each of the classes that handle the GUI. All of these classes are located under src/ui folder and each of them has two common methods: //void load_xml(builder)// and //void connect_signals()// which are called in their constructors.
     290
     291The //load_xml(const Glib::RefPtr<Gtk::Builder>& builder)// method gets widgets from the builder file and assigns them to the class members (pointers). Widgets instantiated by Gtk::Builder have to be deleted manually. In fact, according to documentation, this only applies to top-level widgets (windows and dialogs), but LAG's authors seem to prefer to delete every single one anyway.
     292
     293The //connect_signals()// method is responsible for connecting various input signals to class methods. An example lifetime of a Gtk Widget then looks like this:
     294
     295{{{
     296#!div style="font-size: 100%"
     297  {{{#!c++
     298  // Declare a pointer to the widget (as class member)
     299  class MyClass {
     300    Gtk::Button* my_button;
     301    void on_my_button_clicked();
     302  }
     303
     304  // Load the widget from xml (in MyClass::load_xml(const Glib::RefPtr<Gtk::Builder>& builder))
     305  builder->get_widget("mybutton", my_button);
     306
     307  // Connect some method to it (in MyClass::connect_signals())
     308  my_button->signal_clicked().connect(sigc::mem_fun(*this, &MyClass::on_my_button_clicked));
     309
     310  // Delete the widget (in the class destructor)
     311  delete my_button;
     312  }}}
     313}}}
     314
     315Modifying the GUI is just as simple as adding new widgets to the Glade file and then handling them in the ui classes.
     316
     317=== 4.3 How does loading of files work? [=#q4.3] ===
    301318
    302319Upon pressing //Add// or //Refresh// button an instance of //LoadWorker// is created in the //FileOpener// class with all the parameters from the file opening dialog (like filenames, ASCII parse string, filters etc) passed to its constructor. Then its //run()// method is called which actually does all the loading and then sends a signal back to the GUI thread through //Glib::Dispatcher// when the loading has finished.
    … …  
    305322If the file is in latlong projection a GeoProjectionConverter class (which comes from lastools) is used to first adjust scale factors and offsets in the header, and then transform point coordinates. The points stored inside the quadtree are always in UTM projection as it is much easier to handle (in latlong x, y are in degrees and z is in metres which causes some difficulties with rendering).
    306323
    307 === 20. How does saving of files work? [=#q20] ===
     324=== 4.4 How does saving of files work? [=#q4.4] ===
    308325
    309326Upon pressing //Save// button an instance of //SaveWorker// is created with all necessary parameters from the save dialog passed to its constructor. Then its //run()// method is called where the saving actually happens. \\
    … …  
    311328If the output or the input file is in loatlong projecion two GeoProjectionConverter objects are used to get correct scale factor and offset values in the header and then convert points' coordinates. This is because the points inside the quadtree are in UTM projection and if the input file is in latlong then its scale factors and offsets need to be converted to UTM. At the same time if the output file is in latlong point coordinates need to be converted.
    312329
    313 === 21. How is LAS 1.3 point format and waveform data handled? [=#q21] ===
     330=== 4.5 How is LAS 1.3 point format and waveform data handled? [=#q4.5] ===
    314331
    315332The problem with points in LAS 1.3+ files is that they contain a number of additional attributes which are used to describe corresponding waveform data, but which are not needed by lag. These values are all 32 or 64 bit and adding them to the //LidarPoint// class would effectively double its size. In turn they would considerably slower the quadtree and make it occupy additional memory. Therefore all these attributes are stored in a temporary file on the hard drive and then retrieved when points are being saved with help of //LidarPoint::dataindex// variable. (If you have any doubts it is much faster add several double values to //LidarPoint// class and profile some quadtree operations. The performance impact is huge and maybe it would be a good idea at some point to try to store coordinates as scaled integers and have the Quadtree unscale them whenever they're requested. It would make get_x() operations a bit slower but the overall quadtree quite faster.) \\
    … …  
    346363Finally we copy over the waveform data manually  and attach it to the end of the output file. Note that this requires for LAS 1.3 files to be saved to a separate file after modification. Unfortunately there is no way around this with how laslib library currently works.
    347364
    348 === 22. How does rendering work?] [=#q22] ===
     365=== 4.6 How does rendering work?] [=#q4.6] ===
    349366
    350367It's difficult to tell. It's quite a mess and it definitely needs some refactoring for maintainability. It does work however so I haven't changed it, focusing on other things. \\ \\
    … …  
    357374Simple as that.
    358375
    359 === 23. What are main ideas for further LAG development? [=#q23] ===
     376== 5. Further Development [=#q5] ==
     377
     378=== 5.1 What are main ideas for further LAG development? [=#q5.1] ===
    360379
    361380There are few categories of things that need to be done:
    … …  
    366385To add to that there's still plenty of refactoring to be done (possibly a good thing to start with to understand the code better). Particularly the rendering, which works, but is kind of scattered between different classes and methods and its threading which is just bad. Ideally there should be a separate Renderer/RenderingWorker class with all the code in one place.
    367386
    368 === 24. What is the unfinished development stuff? [=#q24] ===
     387=== 5.2 What is the unfinished development stuff? [=#q5.2] ===
    369388
    370389That's mainly sorting out the threading. While LoadWorker and SaveWorker have been fully implemented, the ProfileWorker and ClassifyWorker need some improvement (so the job is actually done inside these classes). To add to that a progress indication for classifying points and loading a profile could be added. \\ \\
    … …  
    372391Add more things here.
    373392
    374 === 25. What are some major issues with LAG that need fixing? [=#q25] ===
     393=== 5.3 What are some major issues with LAG that need fixing? [=#q5.3] ===
    375394
    376395Luckily there's not so many any more, however the major one is the memory consumption. This is quite a difficult problem, but it has to be looked at before lag can be released to the public. \\
    … …  
    378397Also there is an issue with rendering/loading thread where forcing ON_EXPOSE event while loading a file at the same time may sometimes cause a segfault. For example moving some window on top of the TwoDeeOverview window while file is being loaded may cause this. This is most likely cause by both loading and rendering thread trying to access the quadtree at the same time and I believe sorting out the rendering thread should solve this problem.
    379398
    380 === 26. What is lag memory issue? [=#q26] ===
     399=== 5.4 What is lag memory issue? [=#q5.4] ===
    381400
    382401It's an unknown source of high memory consumption of LAG. It looks like either a memory leak, memory fragmentation or both. The reason to suspect memory fragmentation is that the memory usage keeps growing to some point then stays constant. It's easy to see when loading the same file multiple times. Also when using a heap profiler it will show correct and constant memory usage and will not grow as high, possibly because of custom memory pool from the profiler.
    … …  
    384403Probably the best way to fix it would be to create a [http://en.wikipedia.org/wiki/Allocator_(C%2B%2B)#Custom_allocators custom memory allocator] which is an interesting project on its own.
    385404
    386 === 27. What are some additional features that can be added to LAG? [=#q27] ===
     405=== 5.5 What are some additional features that can be added to LAG? [=#q5.5] ===
    387406
    388407Here are some ideas:
    … …  
    394413  * Multi-platform support.
    395414
    396 === 28. What tools are there to help me with LAG development? [=#q28] ===
     415=== 5.6 What tools are there to help me with LAG development? [=#q5.6] ===
    397416
    398417I'd recommend the following: \\
    … …  
    401420[http://code.google.com/p/gperftools/ gperftools] - a set of performance analysis tools from Google. \\
    402421
    403 === 29. How do I use profiling tools to optimise code? [=q29] ===
     422=== 5.7 How do I use profiling tools to optimise my code? [=q5.7] ===
    404423
    405424Profiling is better then your intuition when it comes to optimising code and you should do it often. It lets you easily identify bottlenecks and see how your changes actually affect the performance. It also lets you pinpoint functions that cause high memory consumption and find possible memory leaks. \\ \\
    … …  
    409428valgrind --tool=callgrind lag
    410429}}}
    411 The lag will start as usual however it will run extremely slow. Run some operations (possibly such that you can repeat with no variations, like loading a single file) and close the program. The callgrind will have generated a call-graph file called callgrind.out.xxxxx which you can open in {{{kcachegrind}}} for visualisation.
    412 
     430The lag will start as usual however it will run extremely slow. Run some operations (possibly such that you can repeat with no variations, like loading a single file) and close the program. The callgrind will have generated a call-graph file called callgrind.out.xxxxx which you can open in [http://kcachegrind.sourceforge.net/html/Home.html kcachegrind] for visualisation.
     431[[Image(kcachegrind.png,align=center)]]
     432{{{
     433#!div style="font-size: 80%; text-align: center"
     434Callgrind output visualised in kcachegrind.
     435}}}
     436From the output you will be able to see an inclusive and self cost of each function as well as go down into the hierarchy of its calees and their costs. To make the results comparable you should try to eliminate various external factors, like network lag, in between the runs, for example by placing the files you load on a local hard drive.
     437Valgrind also offers a tool called massif for heap profiling which shows program's memory consumption. You run it in a similar way:
     438{{{#!sh
     439valgrind --tool=massif lag
     440}}}
     441To visualise the output you can use [http://gitorious.org/massif-visualizer massif-visualiser].