Containers of pointers to data are very inefficient because of two main reasons:
Each pointer points to a "different" memory location. In other words, the data is not stored contiguously. That breaks cache locality and therefore it is a major inefficiency.
To access data through a pointer, in general, you must do two things; first you must dereference the pointer and then access the data it is pointing to. For large datasets it becomes an inefficiency.
[C++ DataFrame]