The Hoard: Paged Virtual Memory for the Cell BE SPE

The hoard is a paged virtual memory system for program data designed for the Cell Broad- band Engine processor (Cell BE). The hoard makes it easier to utilise the performance available from the processor’s Synergistic Processor Elements (SPEs). Traditional high- performance hardware cache designs are not well-suited to being implemented in soft- ware on the Cell BE architecture, so the hoard takes advantage of the SPE’s wide registers to implement a multi-level page table. This permits fast, fully associative lookup into a dynamically-sized collection of data pages. Write-back of all pages is performed (not just those modified) and a first-in first-out page replacement method is used. This design is compared with the software cache provided with the IBM SDK for Multicore Acceleration — a fixed-size, four-way set associative design, using a traditional hardware approach — through extensive experiments with a diverse range of benchmarks. An in-depth analysis is made of the performance of the hoard design, addressing its strengths and shortcomings. From this analysis, potential improvements to the hoard are considered, implemented, and their relative benefits measured. Changes include the tracking of page modification, pre-writing of pages, use of a write-through policy, pre- and partial-fetching of pages, and alternative replacement algorithms. Results show that simple and speculative methods are often the most beneficial, and that performance varies greatly depending on the memory access patterns of a particular program. It was found that programs with high locality of access will perform better when using the hoard, in some cases exceeding the performance of the same program running on the Power Processor Element of the Cell BE. For programs with lower locality of reference, performance of the hoard and SDK cache are within a few percent of one another. An additional design goal for the hoard is to make the access and management of data outside of the SPE’s local store as transparent as possible to the programmer. Not imple- mented as part of a particular compiler, the hoard is a compile- and run-time library that makes use of C++ language features to provide a natural pointer interface, requiring a minimal amount of changes to existing programs. Careful implementation of the interface ensures that program semantics are well preserved and accidental misuse is discouraged, while taking advantage of the features of the Cell BE to provide high-speed lookup. The result is a tool that facilitates the reuse of a broad range of existing code-bases on the SPE.