Billion Records + Complex Queries = Sub-second Responses

TRY IT FOR YOURSELF AT www.whamtechdev.com/billion

WhamTech developed two high performance extensions to its real-time data and information integration and sharing product, EIQ Server™, namely:

  • Massive real-time data inserts, updates and deletes

  • Complex query processing accelerator

The first extension is demonstrated through the www.whamtechdev.com Web site - REAL-TIME UPDATE AND QUERY DATABASE.

The second extension is demonstrated through this Web site - COMPLEX QUERY PROCESSING ACCELERATOR, which uses part of the server's memory to process Boolean operations. In the past, WhamTech's database technology, Thunderbolt, processed Boolean operations using intermediate disk I/O. This did not cause a significant performance degradation with smaller databases - up to 100 million records or so, but did become significant as Thunderbolt databases grew to 100s of millions and billions of records. WhamTech's focus has always been on high performance - always striving to achieve sub-second responses to queries, in fact, we own the domain name www.sub-second.com (which currently redirects to www.whamtech.com). Our technology development, system architecture, and system tuning, are all directed towards exceptionally high performance and advanced capabilities. The complex query processing accelerator is an important part of that effort.

Introduction to the hands-on demonstration

WhamTech created a billion records database to test and demonstrate its unique, high performance relational index and query management system (RIQMS).  The database is the legacy WhamTech product, Thunderbolt, that uses the RIQMS in the EIQ Server product and is live with data in tables - no special indexes nor data warehouse techniques - and is relatively simple - a single table - there are plans to extend the database and types of queries and some of these plans are discussed below. 

The database consists of various fields that can be queried independently or used in combination to form complex and range queries. Queries are resolved "virtually" with Boolean operations on result sets and using result sets to virtually join tables and databases - no interim or temporary tables are needed. Only after arriving at the final result set is any contact with the data in the database made for reading and writing.

Other techniques such as using bitmaps to join databases and tables are practiced by other database technologies, most to a lesser extent, and usually as an adjunct option rather than a core, fundamental operation which has been used by Thunderbolt for over 25 years. Also, while WhamTech supports SQL, it also extended integer list and bitmap result set technology to enable two unique commands SQL does not cover that allow heuristic data mining and virtual table join orders.

Link to the billion records database: www.whamtechdev.com/billion

Link to more information on WhamTech's database and search technologies: www.whamtech.com/technologies.htm

Link to more information on WhamTech: www.whamtech.com

Back to top



  Index and Query Other Databases  
  External Index and Query (EIQ) Server under development. Capability to externally index and process complex queries on highly normalized third-party databases without going through native indexes and query processing. See EIQ Server for more...  

Copyright © 1998 - 2003 WhamTech, Inc.  www.whamtech.com  +1 972-380-4545  info@whamtech.com