DISQLite3_2.4.5.7z

Introduction This collection of projects demonstrates that DISQLite3 is well capable to handle 20 million in very reasonable time. DISQLite3 by far outperforms other Delphi database systems and even rivals big database players like MySql. The individual projects in this folder show how to generate data, export to CSV, and import from CSV. All projects handle the full number of 20 million records. Requirements * DISQLite3, DISQLite3 Pro required for Mem_Helper projects. * DIUnicode for CSV import. How to Run the Tests 1. Compile and run one of the Generate projects. They will create a 20_million_generate.db3 database file filled with random data. 2. Compile and run the Export project. This will export all 20 million records from the generated table to a newly created CSV file. 3. Compile and run one of the Import projects. They will import all 20 million records from the CSV file to a newly generated 20_million_import.db3 database. Further Performance Improvements The projects are set up to use no more than about 450 MB of memory for database operations. Using more memory will speed up insertions and index creation. Refer to the source code comments for details. About 20% improvement can be achieved by using a fast memory manager like FastMM. Approximate Timings Generate with memory helper table: 11 Minutes CSV export: 8 Minutes Import with memory helper table: 13 Minutes All timings measured on a Intel Core Duo 1.8 MHz, 1 GB memory, 5200 rpm hard disk notebook system. Systems with faster hard disks and server hardware are expected to show even better performance.