Designing the AdLeaf system

Thought it was a good idea to share with you the design of the AdLeaf system. In our design, it is always important to remember that disk I/O operations are much more expensive than operations in memory. Given that the AdLeaf system has to be designed to serve millions of targeted advertising and to keep track of all the earnings, hits and impression credits shown, it was not ideal to just use the database as one big storage system. When you need to serve ads at thousands or ten thousand transactions per second, it was important to just hit on the memory versus the disk and avoid relatively expensive database reads/writes.

 

With that in mind, we store data into memory as much as it would allow us and use queue throttling to smooth the writes or recording into the databse. Reads are essentially all from memory thus allowing us to optimize the database for writes. Depending on whether there were more inserts or updates, the tables were indexed appropriately. When the server goes down and comes back up, it just sucks everything from the database back into memory again.

 

When an AdLeaf is served up, the AdLeaf system queries the memory for the text advertising. It then records the impressions earned for the website that display the hits and records the impressions shown for the AdLeaf that was shown all in memory. The data then get persisted either when a certain time has passed. For example, if website A shows an AdLeaf, A will get an impression credit. Assume the time delay to write to the database is more 10 seconds. Suppose website A is a really popular website and generates 1 hit per second. This means that we only write to the database once in 10 seconds versus 10 writes in 10 seconds. This is really important because as the number of members of the AdLeaf community increases, all these writes can translate to millions. Moreover, we can adjust the timing so that when the system is heavily loaded, the time lag can increase. The disadvantage of such an approach is that if the system dies in between the next update, the updates in hits will be lost. However, we felt that this was acceptable since for a site like A, it would have lost an average of 5 impressions earned when the server goes down. We can compensate by crediting everyone in the community with a computed average credit refund. By designing the system that way, each server can serve up free advertising at thousands of transactions per secon. Architectural choices can sometimes influence the performance by a magnitue of hundred times.

Advertisement