HP Integrity rx1620 Server 1.60 GHz 267 MHz FSB Base System AB431A Prospecto

Increased storage requirements 
To support the commit/rollback protocol, RDBMSs maintain a transaction log that will allow for the 

potential rollback of non-committed updates. To optimize precision searches for specific applications, 
RDBMSs support the creation of special access data structures, such as b-tree indices and hash tables. 

Some RDBMSs support isolation concurrency through the storage of lock information along with the 

data. To support data versioning, multiple versions of the data are stored. 
This is just a partial list of the RDBMS features that result in increased storage requirements. The net 
result is the volume of RDBMS storage overhead may be up to three times larger than that of the 

original data, resulting in a four to one expansion of storage requirements. 

Increased CPU requirements 
Row-level concurrency control requires that RDBMS check a lock/transaction table for every row 

access. Most RDBMSs use a paging system to allow multiple transactions to share data and reduce 

I/O requirements. These and other internal structures use additional CPU cycles. 

Increased I/O requirements 
Increases storage requirements result in increased I/O requirements. Indices must me maintained in 

tandem with every row update. 

Geometrically decreasing load performance 
As the volume of data increases, the cost and time to load the next row of data increases 

geometrically. This is due to the maintaining of indices. RDBMSs are optimized for precision 

searching of data, which loads one transaction at a time; and not meant for bulk data loading. Due 

to RDBMS indices, this loading-rate degradation is permanent. Although the absolute numbers may 

vary based on hardware configuration, the shape of the curve is the same. 

Search specific optimization 
Search is optimized in RDBMSs by creating indices which anticipate a specific set of search criteria. 

For example, if it is anticipated that many searches will involve a user name, then an index will be 

created on the user name column. Searches which fall within the bounds of the anticipated search 

criteria will execute quickly. Searches that do not fall within the bounds of the anticipated criteria will 
cause a complete table scan. 
Some common event data searches cannot be optimized by an RDBMS at all. These include substring 

and pattern-matching searches. These also result in a complete table scan. For large databases with 

billions of rows, these searches can become multi-day queries rendering a result of no practical value. 

Example: A customer attempted to perform a search against one billion web proxy events that 

would return timestamp, IP address, user ID and URL for all records where the URL contained a 

specific string. As this is a pattern search, the RDBMS was required to conduct a full table scan 

and could take advantage of any indices. This resulted in a search that took days to complete 

using high-cost, massive multi-processor, and high-performance servers. 

Search optimization and load rate trade-off 
To optimize a greater number of search scenarios, more indices must be created. The more indices 

that are created, the slower the event data load rate becomes. This forces the database administrator 

to constantly maintain a balance between search performance and load performance. To do so, 

indices have to be created and dropped. Consequently, search scenarios must be anticipated and 
prioritized. If the event data load rate becomes unacceptable, then indices must be dropped. 

Dropping an index can reduce search performance in ways that are hard to predict. For large 

volumes of event data, creating and dropping indices are a major undertaking that can temporarily 

shut down database operations and take hours or days to complete. 

8