Overview
High-performance and energy-efficient data management applications are a necessity for high performance computing systems due to the extreme scale of data produced by high fidelity scientific simulations that these systems support. Data layout in memory hugely impacts the performance. For better performance, most simulations interleave variables in memory during their calculation phase, but deinterleave the data for subsequent storage and analysis. As a result, efficient data deinterleaving is critical; yet, common deinterleaving methods provide inefficient throughput and energy performance. To address this problem, we propose an out-of-place deinterleaving (OPD) method that is high performance, energy efficient, and generic to any data type. This deinterleaving method exploits data cache prefetching, reduces memory accesses, and optimizes the use of complete cache line writes.
Publications
- Schendel, E. R., Harenberg, S., Tang, H., Vishwanath, V., Papka, M. E., and Samatova, N. F. "A generic high-performance method for deinterleaving scientific data." In Euro-Par 2013 Parallel Processing, pp. 571-582. Springer Berlin Heidelberg, 2013. [pdf]
News
September, 2013:
Initial OPD library release (low-level C functions): [OPD-0.3.0.tar.gz]
